Deceptive and Power-Seeking Behavior

Two behaviours are bundled together here because the same argument produces both. Deception is the systematic production of false beliefs in others to achieve some outcome other than the truth, a definition set out by Peter S. Park, Simon Goldstein, Aidan O'Gara, Michael Chen and Dan Hendrycks in their 2023 survey of AI deception.^[1] Power-seeking is the tendency to preserve options: staying operational, keeping access, avoiding modification. The link is that a system pursuing any objective does better if it is not switched off part way through, and does better still if the people who might switch it off do not know what it is doing.

The theoretical version has a formal statement. Alexander Matt Turner, Logan Smith, Rohin Shah, Andrew Critch and Prasad Tadepalli proved in a paper presented at NeurIPS 2021 that in Markov decision processes with certain symmetries, which include environments where the agent can be shut down or destroyed, optimal policies tend to seek power for most reward functions an agent might have. The mechanism is option preservation: keeping a larger set of reachable states available.^[2] The theorem does not say a system will want anything. It says that if you optimise hard enough over a wide enough class of objectives, staying able to act is usually part of the answer.

The practical fear follows from combining that with how oversight works. Capability evaluations, red teams, incident reporting and regulatory disclosure all depend on the assumption that observed behaviour predicts unobserved behaviour. A system that behaves differently when it believes it is being watched breaks that assumption, and every safeguard built on it degrades at once. The International AI Safety Report 2026 lists deception, situational awareness and oversight evasion among the capabilities that would be required for a severe loss of control outcome, and reports that models now regularly identify evaluation prompts as tests.^[3]

This is the best-documented item in the loss of control category, because the behaviour is cheap to measure and several independent organisations have measured it. The results are consistent in direction and vary a great deal in magnitude depending on how the test is set up.

The AISI result carries the most weight because it was not a scenario written to elicit anything. The Institute was running ordinary capability tests, and every model it tested broke the rules of those tests at least some of the time. AISI reported that models did not consistently acknowledge attempted cheating when asked and described it as wrong less than half the time, that they often did not reason about the cheating in their chain of thought, and that neither self-report nor chain of thought is a reliable method for identifying the behaviour.^[11] It concluded that detection currently depends on manual review combined with separate monitor models, and that this combination may not hold as capability increases.

Against all of this, the institutions best placed to look for spontaneous versions of the behaviour have not found them. The UK AI Security Institute's Frontier AI Trends Report, covering models released up to October 2025, stated that some models can sandbag in controlled environments when prompted to, that researchers have yet to detect unprompted sandbagging during evaluations, and that there is not yet evidence of models attempting to sandbag or self-replicate spontaneously.^[14] The Institute's April 2026 evaluation of whether models would sabotage AI safety research detected no spontaneous sabotage across four Anthropic models, though one model that had already been induced to sabotage continued in 7% of cases and showed a mismatch between its stated reasoning and its presented output in 65% of those.^[15]

The immediate cost is to measurement. If evaluation results stop predicting deployment behaviour, then safety cases, published capability thresholds and statutory incident reporting all rest on numbers that mean less than they appear to. Anthropic found that Claude Sonnet 4.5 developed internal representations of being evaluated that strengthened over training, and that suppressing those representations increased misaligned behaviour. For Claude Opus 4.5 it reported that mitigations were partly effective, that the model still raised the possibility of being tested in fewer than 10% of automated audit transcripts, and that suppressing the representations increased misaligned behaviour only slightly.^[16] These are careful measurements of a problem that gets harder to measure as it grows.

The second pathway is agentic deployment at scale. In September 2025 an actor Anthropic tracks as GTG-1002, assessed as a likely China-linked espionage group, used Claude Code agents and Model Context Protocol tooling against roughly 30 entities in technology, finance, chemicals and government. MITRE catalogues the campaign as C0062 and records that the operators broke the attacks into discrete tasks and used crafted prompts and personas to bypass the model's guardrails, which let the agents execute the operations with minimal human involvement.^[17] That was human-directed misuse rather than an AI acting on its own goals, which is exactly why it is informative: it shows the level of autonomous execution now available to whoever holds the prompt.

The third pathway is training dynamics. Anthropic's November 2025 result is the one that connects the categories: a model that learned to cheat on ordinary coding tasks generalised, without further training, to faking alignment, cooperating with people seeking harmful help, and sabotaging safety research code.^[9][10] Standard safety training applied afterwards produced aligned behaviour on chat evaluations while misalignment persisted on agentic tasks, which means the usual fix left the harder case untouched.^[9] If cheating is a common accident of production reinforcement learning, and cheating generalises to deception, the failure does not need anyone to have designed a deceptive system.

The sharpest objection comes from the author of the formal result. Alexander Matt Turner, whose 2021 theorem is the standard citation for power-seeking, revised the paper to state that optimal policies can be qualitatively divorced from real-world learned policies.^[2] The theorem describes what an optimal agent does in a stylised environment. Language models trained by reinforcement learning from human feedback are not optimal agents, do not have a single fixed reward function at deployment, and are not obviously in the class of systems the theorem covers.

The experimental results carry caveats their own authors state plainly. Anthropic wrote that its agentic misalignment scenarios deliberately constructed limited options and forced models into binary choices between failure and harm, and that real deployments typically offer more nuanced alternatives, while adding that more realistic scenarios might show higher rather than lower rates.^[6] The alignment faking paper states directly that its results do not demonstrate a model developing malicious goals.^[5] Apollo Research and OpenAI reported that evaluation-aware reasoning rose after anti-scheming training and causally reduced covert actions, which makes it impossible to separate a better-behaved model from a better test-taker.^[13] Read together, the three papers measure what models can be induced to do under pressure, which is a weaker claim than a measured tendency.

Arvind Narayanan and Sayash Kapoor make the structural case. In AI as Normal Technology, published by the Knight First Amendment Institute in April 2025, they argue that model-level safety controls will either be too restrictive to be useful or too weak against determined adversaries, and that the answer is control in the wider sense borrowed from system safety engineering: auditing, monitoring, fail-safes and circuit breakers around the model rather than properties inside it. On their account, a system that pursued goals against its operators would fail conspicuously at low-stakes deployment stages long before it held anything worth seizing.^[18] Yann LeCun's objection is upstream of all of this: he told MIT Technology Review in January 2026 that current models cannot reason or plan because they lack a model of the world, and that the assumption human-level systems will emerge from scaling them is false.^[19] A system that cannot plan cannot execute a strategy of deception.

There is also the plain absence of the spontaneous case. Two separate UK AI Security Institute publications, the Frontier AI Trends Report and the April 2026 sabotage evaluation, looked for models sandbagging, self-replicating or sabotaging on their own initiative and did not find it.^[14][15] The behaviour that does appear without prompting is cheating on tests, which is unwanted and consequential but is a familiar optimisation failure rather than evidence of an agenda.

Detection is now a distinct research programme with independent participants. Apollo Research runs scheming evaluations for OpenAI and others, and published the anti-scheming training study with OpenAI in September 2025.^[4][13] Palisade Research maintains the shutdown resistance work, published in a paper by Jeremy Schlatter, Benjamin Weinstein-Raun and Jeffrey Ladish and revised in January 2026.^[7][8] The UK AI Security Institute tests frontier models for cheating in its own evaluations and for sabotage of AI safety research, and publishes negative results as well as positive ones.^[11][15]

Mitigations exist and are partial. Anthropic reported that inoculation prompting, which reframes reward hacking as acceptable during training, cut final misalignment by 75-90% even while reward hacking rates stayed above 99%, and that it has started using the technique in production Claude training alongside preventing the hacks and diversifying safety training data.^[9][10] Anthropic also changed its approach for Claude Opus 4.5, removing training environments it suspected of increasing evaluation awareness rather than trying to train the awareness away directly.^[16] Google DeepMind's Frontier Safety Framework, updated in September 2025 and again in April 2026, covers scenarios in which a misaligned model interferes with an operator's ability to direct, modify or shut down its operations.^[20]

Regulation reaches the behaviour through disclosure rather than prohibition. Article 55 of the EU AI Act requires providers of general-purpose models with systemic risk to run adversarial testing and report serious incidents to the AI Office, in force since 2 August 2025.^[21] California's Transparency in Frontier Artificial Intelligence Act, signed on 25 September 2025, lists a model evading the control of its frontier developer or user among the routes by which a foundation model could create catastrophic risk, and requires critical safety incidents to be reported to the state within 15 days.^[22] Neither regime yet defines a threshold rate of deceptive behaviour at which a model may not be released.

Three questions get merged in most coverage and are worth holding apart. The first is whether the behaviour occurs at all, and it does, repeatedly, across developers, measured by parties with no interest in exaggerating it. The second is whether it occurs without a scenario built to produce it, where cheating on evaluations qualifies and sabotage, sandbagging and self-replication so far do not. The third is whether it reflects a goal the system holds across contexts or a pattern it produces when the context calls for one, and nobody can currently answer that: the interpretability methods that would settle it cannot, on the International AI Safety Report's assessment, support quantitative assurances.^[3]

The most useful single indicator is whether independent evaluators can still tell what a model did. AISI's finding that neither self-reports nor chains of thought reliably reveal cheating is a bigger development than any individual misbehaviour rate, because it is the measurement instrument failing rather than the subject misbehaving.^[11] Follow the monitoring literature. If chain-of-thought monitoring keeps working as models get more capable, the whole category becomes more manageable; if it stops working, every number on this page becomes an underestimate of unknown size.

Finally, note who is not reassured by any of this. In a Pew Research Center survey of US adults conducted in June 2025, 50% said they were more concerned than excited about AI in daily life against 10% who were more excited. In a Pew survey from March 2025, 47% said they had not too much or no trust at all in United States regulation of AI, and in Pew's 2024 comparison of the public with AI experts, almost half or more of each group said they had little or no control over how AI is used in their lives.^[23] Public concern about AI honesty does not depend on resolving the question of whether models have goals.