Anthropic, the makers of the AI chatbot Claude, claim to have identified Chinese government-backed hackers using their tool to conduct automated cyber attacks on approximately 30 global organizations. According to Anthropic, these hackers deceived Claude into performing automated tasks disguised as cybersecurity research, forming what the company described as the "first reported AI-orchestrated cyber espionage campaign. " However, some skeptics question both the validity and motivations behind this claim. Anthropic discovered the hacking attempts in mid-September, where attackers posed as legitimate cybersecurity personnel to give Claude small automated tasks. When combined, these tasks constituted a "highly sophisticated espionage campaign. " Researchers expressed "high confidence" that the perpetrators were a state-sponsored Chinese group, though the company did not provide further specifics. Targets reportedly included major technology firms, financial institutions, chemical manufacturers, and government agencies. The hackers developed a program with Claude’s coding help to autonomously compromise chosen targets with minimal human involvement. The chatbot reportedly succeeded in breaching various unnamed organizations, extracting and analyzing sensitive data for valuable insights. Anthropic has since banned the hackers from using Claude and informed affected companies and law enforcement agencies. However, Martin Zugec from cybersecurity firm Bitdefender expressed skepticism, noting the report makes bold claims without supplying verifiable threat intelligence.

He emphasized the need for more detailed information to properly assess AI-driven attack risks. This incident is among the most prominent claims of AI tools being exploited for automated hacks, highlighting a growing concern. Other AI developers have also reported state-affiliated actors using their platforms; for instance, in February 2024, OpenAI and Microsoft revealed disruption of five state-linked groups, including from China, primarily attempting to use AI for information queries, translation, code debugging, and simple coding tasks. Anthropic has not disclosed how it linked the hackers to the Chinese government, and the Chinese embassy in the United States denied involvement. The cybersecurity sector has faced criticism for potentially overstating AI’s current capability in facilitating automated attacks, as the technology remains somewhat unwieldy. A November 2023 Google research paper noted concerns over AI being employed to create new malware but concluded such use was experimental and not yet very effective. The cybersecurity and AI industries alike tend to highlight hacker use of AI to promote their products and services. In its blog, Anthropic stressed that combating AI-driven threats requires AI-based defenders, claiming Claude’s capabilities are essential for cyber defense. The company also acknowledged that Claude made errors during attacks, such as fabricating login credentials and mistakenly identifying publicly available information as secret, underscoring that fully autonomous cyberattacks remain difficult to achieve.