Cyberattacks and Critical Infrastructure

The argument runs through cost. Finding an exploitable flaw in unfamiliar software, writing code that reliably triggers it, moving laterally through a network without being noticed and deciding what to steal are all skilled tasks that historically required years of training. If a language model with tool access can perform most of those steps, the number of people capable of running an intrusion rises sharply, and the number of hours a skilled operator needs per target falls. RAND researchers Benjamin Sperisen, Jair Aguirre, Henri van Soest and Zylex Lopez put the point plainly in a report published in June 2026: capture-the-flag challenges that novices and technically skilled participants alike had mostly failed to solve in eight hours using 2025 chatbots could all be solved in under an hour by users with no cyber expertise, provided they could install an agentic coding tool.^[1]

The second half of the claim concerns what those attackers can reach. Water treatment, electricity distribution and hospital systems run on industrial control equipment that is often decades old, rarely patched and increasingly connected to ordinary business networks. The United States Government Accountability Office counted roughly 170,000 drinking water and wastewater systems in an August 2024 report and found that the Environmental Protection Agency had neither completed a sector-wide risk assessment nor produced a strategy to address known vulnerabilities.^[2] An Environmental Protection Agency Office of Inspector General passive assessment published on 13 November 2024 scanned 1,062 drinking water systems serving more than 193 million people and identified 97, serving about 26.6 million users, with critical or high-risk cyber security vulnerabilities.^[3]

A third element is pre-positioning. In February 2024 the Cybersecurity and Infrastructure Security Agency, the National Security Agency and the Federal Bureau of Investigation jointly advised that People's Republic of China state-sponsored actors tracked as Volt Typhoon had compromised communications, energy, transportation and water sector organisations in order to be in place for disruptive attacks during a future crisis, in some cases maintaining access for at least five years.^[4] The fear is that AI makes this kind of patient, expensive campaign cheap enough to run against many more targets at once.

Unlike most AI fears, this one has an incident record. In November 2025 Anthropic disclosed a campaign it attributed with high confidence to a Chinese state-sponsored group it labelled GTG-1002. The operators posed as a security firm conducting authorised testing, used Claude Code together with network scanners and password tools connected through the Model Context Protocol, and targeted roughly 30 organisations including technology companies, financial institutions, chemical manufacturers and government agencies. Anthropic assessed that the model performed 80-90% of the campaign, with human operators intervening at perhaps 4-6 decision points, and stated that the intrusion succeeded in a small number of cases.^[5]

That was not the first such disclosure. Anthropic's August 2025 threat intelligence report described an actor it tracked as GTG-2002 using Claude Code to run data theft and extortion against at least 17 organisations across health care, emergency services, government and religious institutions, with ransom demands that sometimes exceeded $500,000, and a separate actor selling AI-built ransomware variants on criminal forums for $400 to $1,200 who appeared unable to implement encryption without model assistance.^[6] Google's Threat Intelligence Group has published recurring trackers documenting malware that uses language models at different points: the Russia-nexus families CANFAIL and LONGSTREAM carry decoy code generated by a model when the malware was written, while a family Google tracks as HONESTCUE calls the Gemini API while running. In May 2026 the group reported that it had for the first time identified a threat actor using a zero-day exploit it believed was developed with AI, a two-factor authentication bypass in an open source system administration tool.^[7]

The competition figures matter because they cut both ways. The DARPA AI Cyber Challenge ran cyber reasoning systems over more than 54 million lines of open source code that underpins hospitals, power plants and water systems, and four of the seven finalist systems were released as open source software at the August 2025 finals, with the remaining three to follow.^[12] Google announced on 4 August 2025 that its Big Sleep agent had found 20 previously unknown flaws in widely used open source projects including FFmpeg and ImageMagick, though a Google spokesperson told TechCrunch that a human expert reviews each finding before it is reported.^[13] Google had said in July 2025 that Big Sleep identified a SQLite flaw, CVE-2025-6965, that it assessed was known only to threat actors and at risk of being exploited.^[25] The same capability that produces those results also produces the offensive results.

The most direct path does not require any new capability. It requires the existing capability to become cheap enough that attackers bother with targets they previously ignored. Benjamin Murphy and Twm Stone argued in a 2025 preprint that organisations running legacy systems have long protected themselves by being uninteresting, and that as the marginal cost of an intrusion falls, that protection disappears. Rural water districts, small hospitals and municipal utilities are precisely the organisations with the least security staff and the most obsolete equipment.^[15]

A second path is timing. The UK National Cyber Security Centre judged in a May 2025 assessment that AI will almost certainly make cyber intrusion more effective and efficient, that a digital divide will almost certainly open between systems that keep pace with AI-enabled threats and a large proportion that do not, and that there is a realistic possibility of critical systems becoming more vulnerable to advanced actors by 2027. It also judged that AI will highly likely improve zero-day discovery and exploitation for skilled actors over the same period.^[16] If the interval between a flaw becoming known and being exploited compresses faster than patching cycles, defenders lose the window they currently rely on.

The third path is combination. Volt Typhoon-style access already exists inside some infrastructure networks without any AI involvement.^[4] The concern is not that AI creates the access but that it makes the follow-on work, understanding an unfamiliar control system well enough to break it safely from the attacker's point of view, something that can be done in hours rather than months. Christopher Covino of the Institute for AI Policy and Strategy made the narrower version of this point in a November 2025 memo on the Anthropic disclosure: AI was used to scale operations that exploited existing gaps such as unpatched systems and default passwords, not to defeat well-maintained controls.^[17]

The strongest objection is that the loudest evidence comes from companies selling AI or selling defence against it. Toby Murray, professor of cyber security at the University of Melbourne, set out the criticisms of Anthropic's GTG-1002 disclosure in November 2025: the report contained no indicators of compromise, so other defenders could not establish whether they had been hit by the same campaign, and the account was too thin on technical detail to verify independently. He also noted that the campaign reached roughly 30 targets and succeeded against only a few.^[18] Anthropic's own report gives part of the reason. It records that Claude frequently overstated findings and occasionally fabricated data during autonomous operation, claiming credentials that did not work and presenting publicly available information as a critical discovery.^[5]

Google's own threat intelligence team, which has more visibility into state actor behaviour than most, has consistently declined to describe what it sees as transformative. Its February 2026 tracker stated that it had not yet observed advanced persistent threat or information operations actors achieving breakthrough capabilities that fundamentally alter the threat picture, and described the AI-enabled malware families it found as early proof-of-concept indicators rather than a paradigm shift.^[19] That is a measured judgement from an organisation with commercial reasons to say the opposite.

There is also a structural argument. Andrew Lohn's 2025 review of the offence-defence literature identified nine arguments for offensive advantage and nine for defensive advantage in cyber conflict, and concluded that the domain is too varied for a single answer about which AI favours.^[20] Defenders get the same vulnerability-finding tools, apply them to code they own before release, and can afford to run them continuously. The DARPA results and Google's Big Sleep disclosures are defensive uses of exactly the capability that worries people.^[12][13] The RAND team that measured novice uplift made the same caveat about its own findings, noting that offence-only evaluations such as capture-the-flag challenges leave out whatever AI is doing for defenders.^[1] The World Economic Forum's Global Cybersecurity Outlook 2026 reported that 77% of surveyed organisations had already adopted AI in their security operations, most commonly for phishing detection.^[21]

Finally, the benchmark numbers carry heavier caveats than headlines suggest. The AI Security Institute stated that its cyber time horizon estimate rests on relatively few models, that a token budget artificially lowered success rates, that only six tasks in its suite exceeded eight hours, and that results on a narrow benchmark suite may not transfer to defended real-world systems.^[9] The Fang and Kang result depended on the model being handed a CVE description; without it, performance fell from 87% to 7%.^[8]

Model developers now run cyber capability evaluations before release and restrict some uses. Anthropic banned the accounts involved in both the 2025 campaigns it disclosed, notified affected organisations and published its findings.^[5][6] Under Article 55 of the EU AI Act, providers of general purpose AI models with systemic risk must carry out adversarial testing, assess and mitigate systemic risks, report serious incidents to the AI Office and maintain adequate cyber security for the model and its infrastructure. Those obligations have applied since 2 August 2025, with Commission enforcement powers taking effect on 2 August 2026.^[22]

Government evaluation capacity has grown alongside. The UK AI Security Institute publishes rolling measurements of autonomous cyber capability and of the gap between open-weight and closed models, which gives policymakers a trend line rather than anecdotes.^[9][10] The International AI Safety Report 2026, chaired by Yoshua Bengio and published on 3 February 2026 with contributions from more than 100 experts, judged that there is now strong evidence that criminal groups and state-sponsored attackers actively use AI in their cyber operations, that AI systems are automating more parts of an attack but cannot yet execute one autonomously and no fully autonomous end-to-end attack has been reported, and that whether AI has increased the overall scale and severity of cyberattacks remains uncertain because the causal effect is hard to establish.^[23]

On the defensive side, America's AI Action Plan, published in July 2025, recommended that the Department of Homeland Security issue and maintain guidance to private sector entities on remediating AI-specific vulnerabilities and threats, and that it lead an AI Information Sharing and Analysis Center covering critical infrastructure sectors.^[24] The decision to open source the AI Cyber Challenge finalist systems was aimed at the same problem from the other direction, putting automated vulnerability discovery in the hands of maintainers of the open source code that infrastructure depends on.^[12] Covino's recommendation, that developers deliberately prioritise advanced capability for defenders such as software maintainers and infrastructure operators, describes a policy lever that has been discussed more than it has been used.^[17]

Any claim in this area rests on three separable things: whether the capability has been measured and on what benchmark, whether it has been used in an actual intrusion and by whom, and whether it produced a consequence that would not have happened otherwise. Most reporting collapses them. The Anthropic and Google disclosures address the first two for particular cases. Almost nothing yet addresses the third, because attributing a specific outage to AI assistance requires a counterfactual nobody can construct.

The most useful trend line is capability doubling time rather than incident counts, because incident counts depend on disclosure practices that change with commercial incentives. The AI Security Institute's shift from an 8-month to a 4.7-month doubling estimate between November 2025 and May 2026 is a larger signal than any single campaign, and it is a number that can be checked against later measurements.^[9] The second useful indicator is the open-weight gap, since capability that remains behind an API can be withdrawn from an abusive account, and capability in downloadable weights cannot.^[10]

For the infrastructure half of the fear, the relevant baseline is that water and energy systems were already inadequately defended before AI entered the picture. The GAO and EPA Inspector General findings describe unpatched systems, default passwords and internet-exposed portals.^[2][3] Improvements there reduce the risk regardless of what happens to model capability, which is why the offence-defence debate is less decisive than it appears: both sides of it point at the same remediation work.