What the fear claims
Authoritarian rule has always been constrained by two shortages: information about what subjects actually think, and people willing to act on it. Secret police forces were expensive and their reports were unreliable, because informants told superiors what superiors wanted to hear. The claim behind this fear is that machine learning relaxes both constraints at once. Cameras that identify people, models that read every post on a national platform and systems that rank citizens by predicted disloyalty do the work an informant network used to do, at a cost that a mid-sized state can afford.
There are two distinct versions. The first is entrenchment: existing autocracies become harder to overthrow because coordination among opponents is detected before it becomes a movement. The second is drift, in which elected governments adopt the same tools for ordinary policing and border control, and the institutional habits that follow are difficult to reverse when a less scrupulous administration inherits them.
A third strand concerns the people who build the systems rather than the people who run states. Tom Davidson, Lukas Finnveden and Rose Hadshar of Forethought argued in April 2025 that automation removes the need for large numbers of loyal subordinates, which is the mechanism that has historically limited what a leader could do. Dario Amodei of Anthropic wrote in January 2026 that sufficiently powerful AI could allow a government to produce a complete list of everyone who disagrees with it on any given question, and identified a global totalitarian outcome as one of the risks he takes seriously.[1][2]
Where the evidence stands
The central empirical result comes from economics rather than security studies. Martin Beraja, Andrew Kao, David Y. Yang and Noam Yuchtman, writing in the Quarterly Journal of Economics in 2023, assembled data on Chinese AI firms, government procurement contracts and recorded episodes of local unrest over a decade. They report that local unrest is followed by greater government procurement of facial recognition AI, that the increased procurement suppresses subsequent unrest, and that firms winning those contracts go on to produce more innovation in both government and commercial markets. That mutual reinforcement between the technology and the regime is what they mean by AI-tocracy.[3]
| Item | Finding | Source and date |
|---|---|---|
| Facial recognition procurement in China | Unrest raises procurement; procurement lowers later unrest | Quarterly Journal of Economics, 2023[3] |
| Xinjiang | Biometric data collection including iris scans and facial imagery; arbitrary and discriminatory detention may constitute crimes against humanity | UN Office of the High Commissioner for Human Rights, 2022[4] |
| Iran | Drones, campus facial recognition and the Nazer reporting app used to enforce hijab rules | UN Independent International Fact-Finding Mission on Iran, 2025[5] |
| Russia | 595 documented cases of facial recognition used against dissenters since 2021, 141 of them preventive metro detentions | OVD-Info, reported by Rest of World, 2024[6] |
| Global internet freedom | 15th consecutive annual decline; 28 of 72 countries worsened, 17 improved; arrests for online speech in at least 57 | Freedom House, 2025[7] |
| Regime counts | 92 autocracies against 87 democracies; about 74 percent of world population in autocracies | V-Dem Institute, 2026[8] |
| Refusal rates on 145 political questions in Chinese | 60.2 percent BaiChuan, about 36 percent DeepSeek, 32 percent Ernie Bot, 10 percent ChatGLM, against 0 to 2.8 percent for models built outside China | PNAS Nexus, February 2026[9] |
The most serious documented case is Xinjiang. The Office of the United Nations High Commissioner for Human Rights published its assessment on 31 August 2022, drawing on Chinese government documents and interviews. It describes a large scale surveillance system built with private technology suppliers, covering in-person and electronic monitoring, biometric data collection including iris scans and facial imagery, a dense network of cameras and checkpoints, and the Integrated Joint Operations Platform, which stores data on individuals who can then be flagged for potential detention. Its conclusion was that the extent of arbitrary and discriminatory detention of Uyghurs and other predominantly Muslim groups may constitute international crimes, in particular crimes against humanity.[4]
Iran and Russia show the same tools used by weaker states. The United Nations Independent International Fact-Finding Mission on Iran reported in March 2025 that authorities had deployed aerial drones over Tehran and southern Iran from April 2024 to monitor compliance with hijab rules, installed facial recognition software at the entrance gate of Amirkabir University to identify women not wearing the hijab, and operated a mobile application called Nazer through which police and members of the public report dress code violations, including the location, date, time and vehicle registration, which then flags the vehicle and alerts police. In Russia, the monitoring group OVD-Info has documented 595 cases in which facial recognition was used against people for dissent since 2021, of which 141 were preventive detentions on the metro, all after the invasion of Ukraine.[5][6]
Control of what people can read is the newer channel. Jennifer Pan and Xu Xu, writing in PNAS Nexus in February 2026, put 145 questions about Chinese politics to four models developed in China and five developed elsewhere. The China-developed models refused far more often, with refusal rates on Chinese language prompts running from 10 percent for ChatGLM to 60.2 percent for BaiChuan, against 0 percent for GPT-3.5 and GPT-4o and 2.8 percent for the uncensored Llama 2. The gap narrowed for less politically sensitive questions, which is the authors' evidence that the pattern reflects Chinese regulation rather than a general property of the models.[9]
The background trend is not encouraging. Freedom House recorded a fifteenth consecutive annual decline in internet freedom in its 2025 report covering June 2024 to May 2025, with conditions worsening in 28 of 72 countries and people arrested for online expression in at least 57. The V-Dem Institute's 2026 Democracy Report counted 92 autocracies against 87 democracies, with roughly 74 percent of the world's population, about 6 billion people, living under autocratic government and about 7 percent in liberal democracies.[7][8]
How it could go wrong
Export is the mechanism that turns a national capability into a global one. Steven Feldstein's 2019 index for the Carnegie Endowment found technology linked to Chinese companies, principally Huawei, Hikvision, Dahua and ZTE, in 63 countries, with Huawei alone supplying at least 50, well ahead of the next largest non-Chinese supplier, Japan's NEC, in 14. A state that could never have built such a system can buy one, along with the training and the operating doctrine that comes with it. The Beraja result suggests why sellers have an incentive to keep selling: government contracts fund the commercial innovation that makes the next generation cheaper.[10][3]
Democracies build the same infrastructure for reasons that seem reasonable at the time. Fixed facial recognition cameras on London streets, a nationwide private number plate reader network in the United States queried thousands of times for immigration enforcement, and the consolidation of tax, payroll and social security records into searchable systems are all defended as proportionate responses to specific problems. Whether they are proportionate is a live argument. What is not in doubt is that the resulting capability transfers intact to whoever wins the next election.
The Forethought analysis adds a mechanism specific to advanced systems: if military and administrative functions are performed by models rather than people, the loyalty of those models becomes the only thing standing between a leader and unchecked action. Davidson and colleagues describe systems built with hidden loyalties that pass evaluation and act differently in deployment, and note that such behaviour could carry from one model generation into the next without anyone noticing. That failure mode has no analogue in a bureaucracy staffed by humans, who can refuse.[1]
Frontier developers report attempts to use their models for exactly these purposes. OpenAI's threat report of October 2025 describes banning accounts believed to be linked to Chinese government entities that had asked the models to help design systems for monitoring social media activity. OpenAI says it has disrupted and reported more than 40 networks violating its usage policies. That is a record of attempted use rather than of successful deployment, but it establishes that the demand exists and that it reaches commercial developers directly.[11]
Why it might be overstated
The strongest counterargument is that AI does not solve the problems that actually limit authoritarian control. L. Jason Anastasopoulos and Jie Lian, writing in the Journal of Democracy in April 2026, call this the calibration dilemma. Any detection threshold trades false positives against false negatives, so a regime must choose between sweeping up innocent people, which generates the grievance that produces opposition, and setting the bar high enough that genuine organisers slip through. Because dissent is rare relative to the population searched, even an accurate system produces large numbers of false alarms, a constraint that better engineering does not remove.[12]
They add three further limits. Populations adapt, using coded language, switching platforms or moving offline, which makes trained models stale faster than they can be retrained. Officials distort the data they report upward to appear competent, so the picture reaching the leadership diverges from reality. And much of the deterrent effect rests on the belief that the system sees everything, a belief that erodes once it visibly fails. Eddie Yang makes a related argument in the American Journal of Political Science, calling it the authoritarian data problem: strategic behaviour under repression removes useful information from the training data, so AI accuracy in censorship falls as repression rises, especially during political crises, and he finds that more data does not fix it.[12][13]
The historical record since 2022 does not show surveillance preventing regime change. Freedom House's largest single improvement in its 2025 report was Bangladesh, where a student uprising removed a repressive government in August 2024. Kenya recorded the largest decline in the same report after a mass internet shutdown and hundreds of arrests during the June 2024 tax protests, which is evidence of severe repression but also evidence that the protests happened at scale in the first place.[7]
Finally, some of the fear conflates digital repression with artificial intelligence. Internet shutdowns, arrests for social media posts and legal harassment of journalists account for most of the deterioration Freedom House measures, and none of them require machine learning. Attributing the trend to AI risks both overstating the technology's role and understating how much of the damage is done by ordinary law and ordinary police.
What is being done
The European Union's AI Act prohibits, from 2 February 2025, several of the practices at the centre of this fear: social scoring that produces detrimental treatment in unrelated contexts, biometric categorisation used to infer political opinions, religion, trade union membership or sexual orientation, untargeted scraping of facial images from the internet or closed circuit television, and predictive policing based solely on profiling or personality traits. Real time remote biometric identification by police in public places is permitted only in narrow listed circumstances and normally requires prior authorisation from a judicial or independent administrative authority. The prohibitions bind conduct in the Union and do not reach deployments elsewhere.[14]
Export controls have been the main instrument aimed outward, and their direction has reversed. The United States rescinded the AI diffusion rule in May 2025, announced a one year waiver for exports of Nvidia H200 chips to China on 8 December 2025, and on 13 January 2026 the Bureau of Industry and Security released a licensing policy moving those exports to case by case review. Whatever the merits, the policy is less restrictive than it was, and the same infrastructure that trains a commercial model trains a surveillance one.[15]
Documentation is doing much of the practical work. The OHCHR assessment on Xinjiang, the United Nations fact-finding mission on Iran, OVD-Info's case records in Russia and Freedom House's annual country scoring are the sources most policy in this area actually rests on. In Glukhin v. Russia, decided on 4 July 2023, the European Court of Human Rights found violations of Articles 8 and 10 where facial recognition was used to identify and arrest a man who had staged a solo protest on the Moscow metro, and called the technology highly intrusive. It is one of the few instances in which an international court has addressed the technology directly.[4][5][6][16]
On the developer side, OpenAI publishes threat reports naming and banning accounts that sought to build population monitoring tools, and Forethought's recommendations include contractual and model specification rules against assisting a seizure of power, security against the insertion of secret loyalties, procurement of military AI from more than one independent provider, and distributed control so that no individual commands enough military AI to act alone. The International AI Safety Report 2026, led by Yoshua Bengio with 91 co-authors and an expert advisory panel nominated by 29 countries plus the United Nations, the OECD and the European Union, treats concentration of power as a systemic risk warranting scientific attention.[11][1][17]
How to think about it
Separate the two claims. That AI assists repression is documented in China, Iran and Russia with named systems, dates and case counts, and the Beraja study provides causal evidence rather than correlation. That AI makes authoritarian rule permanent is a further claim, and the political science published in 2026 by Anastasopoulos and Lian and by Yang argues against it on grounds internal to how these systems work. A reader can hold the first without the second.
The most informative indicator is not capability but reversal. Watch whether any country that has installed a national biometric identification system has subsequently dismantled it, whether courts anywhere have ordered a state surveillance programme stopped, and whether protest movements continue to reach the scale seen in Bangladesh and Kenya in 2024. Watch also for the specific mechanism the Forethought analysis names: automation of functions that currently require the consent of large numbers of officials or soldiers.
For democracies the question is narrower and more practical. Every capability built for a legitimate purpose is available to the next government, so the relevant safeguard is not whether a system is used well now but whether its use is logged, time limited, judicially supervised and capable of being switched off. Those are institutional properties, and they can be checked without any judgment about how good the technology has become.
What to watch for
Would make this more warranted
- A state deploys AI screening of communications at national scale and publishes or leaks evidence that it identifies organisers before public action occurs.[3]
- A democracy adopts biometric identification without per deployment authorisation, judicial oversight or retention limits, and the arrangement survives legal challenge.[14]
- Export of surveillance platforms expands to states with no prior capability, on terms that include operational training as well as hardware.[10]
- Military or internal security functions are automated to the point that carrying out an order no longer requires the cooperation of a substantial number of people.[1]
- Model level censorship spreads beyond the jurisdiction imposing it, so that users outside a country receive its political restrictions by default.[9]
Would make this less warranted
- Sustained evidence that predictive systems used for political control produce enough false positives to be politically costly to their operators, as Anastasopoulos and Lian predict.
- A court or legislature in a country with an established biometric surveillance programme orders it dismantled rather than merely constrained.
- Binding, verified limits on export of population scale surveillance platforms, applied by the major supplier states rather than only by importing states.
Related fears
Read further
- AI-tocracy The Quarterly Journal of Economics, 2023
- The Limits of Authoritarian AI Journal of Democracy, 2026
- OHCHR Assessment of human rights concerns in the Xinjiang Uyghur Autonomous Region, People's Republic of China UN Office of the High Commissioner for Human Rights, 2022
- Freedom on the Net 2025: An Uncertain Future for the Global Internet Freedom House, 2025
- Political censorship in large language models originating from China PNAS Nexus, 2026
References
- 1.AI-Enabled Coups: How a Small Group Could Use AI to Seize Power Forethought, 2025
- 2.The Adolescence of Technology: Confronting and Overcoming the Risks of Powerful AI Dario Amodei, 2026
- 3.AI-tocracy The Quarterly Journal of Economics, volume 138, issue 3, pages 1349-1402, 2023
- 4.OHCHR Assessment of human rights concerns in the Xinjiang Uyghur Autonomous Region, People's Republic of China UN Office of the High Commissioner for Human Rights, 2022
- 5.Report of the Independent International Fact-Finding Mission on the Islamic Republic of Iran, March 2025 UN Human Rights Council, 2025
- 6.The changing face of protest Rest of World, 2024
- 7.Freedom on the Net 2025: An Uncertain Future for the Global Internet Freedom House, 2025
- 8.Democracy Report 2026: Unraveling the Democratic Era? V-Dem Institute, 2026
- 9.Political censorship in large language models originating from China PNAS Nexus, volume 5, issue 2, 2026
- 10.The Global Expansion of AI Surveillance Carnegie Endowment for International Peace, 2019
- 11.Disrupting malicious uses of AI: October 2025 OpenAI, 2025
- 12.The Limits of Authoritarian AI Journal of Democracy, volume 37, issue 2, April 2026, 2026
- 13.The limits of AI for authoritarian control American Journal of Political Science, doi 10.1111/ajps.70045, 2026
- 14.Article 5: Prohibited AI Practices EU Artificial Intelligence Act, 2024
- 15.The US pivot on regulating AI diffusion International Institute for Strategic Studies, Strategic Comments volume 31 number 10, 2025
- 16.Glukhin v. Russia, application no. 11519/20, judgment of 4 July 2023 European Court of Human Rights, 2023
- 17.International AI Safety Report 2026 arXiv, 2026
Last reviewed July 2026. Ratings are editorial judgements, not forecasts. Spotted something wrong? Request a correction.