Epistemic Collapse

The term for what is said to be at risk is epistemic security. Elizabeth Seger, Shahar Avin, Gavin Pearson, Mark Briers and colleagues, writing for the Alan Turing Institute in 2020, defined an epistemically secure society as one that reliably averts threats to the processes by which reliable information is produced, distributed, acquired and assessed. They grouped the technologically exacerbated threats under four headings: action by adversaries and blunderers, attention scarcity and bounded rationality, insular communities and group polarisation, and fabrication and erosion of trust.^[1] The AI version of the fear is that generative systems act on several of those at once.

Four mechanisms are usually named. Generation costs for plausible content fall to near zero. A machine intermediary is inserted between people and primary sources, so that most first contact with a claim is a summary rather than the source. The economics of the institutions that do verification weaken, because summaries substitute for visits. And the range of what people encounter narrows, because models produce output near the centre of their training distribution. Andrew Peterson gave the last mechanism the name knowledge collapse in AI and Society in January 2025, arguing from a simulation model that cheap access to a compressed middle of human knowledge can leave the specialist tails unvisited.^[2]

What separates this from ordinary worry about misinformation is the level at which the damage is claimed. The claim is not that individuals hold more false beliefs. It is that the procedures for resolving disagreement stop being available: a court cannot establish what a recording shows, a newsroom cannot fund the reporting that would settle a dispute, and a citation trail cannot be followed because the intermediary did not preserve it. A January 2026 Demos report by Seger, Sam Stockwell, Tyreese Calnan, Henry Ajder, Jamie Hancock and Hannah Perry works through hypothetical crisis scenarios that follow from a degraded information ecosystem, among them a bank run and economic crash and an AI driven breakdown of the legal system.^[3]

The trust series are the most cited evidence and the most ambiguous. The Reuters Institute Digital News Report 2026, published on 16 June 2026 and covering 48 markets with about 2,000 respondents each, put overall trust in news at 37%, the lowest since the series began in 2015 and three points down on the previous year. Concern about telling real from fake online reached 62%, up four points. Weekly use of AI chatbots for news reached 10%, up from 7%, and 16% among people under 35, while trust in the answers those chatbots give was 20%.^[4] Gallup, asking a different question in the United States between 2 and 16 September 2025, found 28% of Americans expressing a great deal or fair amount of trust in mass media to report the news fully, accurately and fairly, against 40% five years earlier and between 68% and 72% when the series started in the 1970s.^[5]

The reliability of the new intermediary has been audited once at scale. A study coordinated by the European Broadcasting Union and led by the BBC had journalists from 22 public service media organisations in 18 countries, working in 14 languages, assess more than 3,000 answers from ChatGPT, Copilot, Gemini and Perplexity against criteria including accuracy, sourcing, separation of fact from opinion, and context. Forty-five percent of answers contained at least one significant issue. Sourcing was the largest single cause, at 31%, and 20% had major accuracy problems including invented detail and stale information. Gemini was the worst performer, with significant issues in 76% of its answers, more than double the other assistants, largely because of sourcing.^[6]

The economic mechanism has direct measurements. Pew Research Center analysed the browsing records of 900 US adults across 68,879 Google searches in March 2025 and found that when an AI summary appeared, users clicked a traditional result 8% of the time against 15% when it did not, clicked a link inside the summary in 1% of visits, and ended the browsing session entirely 26% of the time against 16%.^[7] The Wikimedia Foundation reported in October 2025 that human pageviews for May and June 2025 were about 8% below the same months in 2024, a figure it identified only after updating its bot detection and reclassifying traffic that had been counted as human. It attributed the decline to search engines answering directly, often from Wikipedia's own content, and to younger readers using social video platforms instead of the open web.^[8]

Two further strands are technical rather than social. Ilia Shumailov and co-authors showed in Nature in 2024 that generative models degrade when trained recursively on their own outputs, losing the tails of the distribution first.^[9] Matthias Gerstgrasser, Rylan Schaeffer and colleagues showed the same year that this depends on synthetic data replacing real data, and that accumulating synthetic alongside real data keeps test error bounded.^[10] Separately, Hao-Ping Lee and colleagues at Microsoft Research and Carnegie Mellon surveyed 319 knowledge workers about 936 real uses of generative AI for CHI 2025 and found that higher confidence in the tool was associated with less critical engagement with its output, while higher confidence in one's own expertise was associated with more.^[11]

The failure mode that concerns auditors is not random error but systematic error at the point of first contact. If a growing share of people meet claims through assistants that misattribute or omit sources in roughly a third of news answers, the resulting mistakes are correlated rather than scattered, and they are invisible to the reader because a fluent summary carries no signal of its own unreliability.^[6] The European Broadcasting Union framed the risk in terms of participation: an audience that does not know what to trust may end up trusting nothing.

The economic loop is the part that would be hard to reverse. Verification is expensive and has always been cross-subsidised by attention. If summaries capture the attention and the underlying organisations lose the visits, the material being summarised gets thinner over time, and the summaries get worse with it. Wikipedia is the cleanest case because it sells no advertising: its dependence on human visitors runs through editor recruitment and donations rather than page revenue, so a sustained fall in human pageviews affects the supply of the content itself.^[8]

Narrowing is the slowest and least visible mechanism. Peterson's argument is that models generate toward the centre of their training distribution, that people who rely on them therefore sample a compressed version of what is known, and that if enough of the record is regenerated this way the specialist tails stop being maintained.^[2] The model-training version of the same problem is Shumailov's: distributional tails disappear first when training data is recycled.^[9] Neither has been demonstrated at the level of a whole knowledge system, and both are arguments from mechanism rather than measurements of an outcome.

The institutional version is what the Demos report models. Courts, insurers, elections and emergency response all depend on being able to authenticate records quickly and cheaply, and each has procedures built for a world in which fabrication was expensive. The report identifies content and data provenance, media literacy, digital infrastructure security, platform crisis protocols, government preparedness, AI risk management and rebuilding local news as the seven areas where intervention would matter most for the United Kingdom.^[3]

The single strongest objection is chronology. Gallup's US trust series fell from around 70% in the 1970s to 40% by 2020, before any generative model was in public use, and reached 28% in 2025.^[5] Reuters Institute trust has been declining across its series as well. A causal story that assigns the decline to AI has to explain a curve that was already falling steeply for fifty years, and has to separate the AI contribution from partisan polarisation, platform substitution and the collapse of local advertising revenue that preceded it.

Sacha Altay, Manon Berriche and Alberto Acerbi argued in Social Media and Society in 2023 that alarmist narratives about misinformation persist despite evidence that its prevalence and impact are overstated, that the internet is mostly not news, and that how researchers define misinformation largely determines the size of the problem they find.^[12] Felix Simon and Altay applied the same reasoning to generative AI for the Knight First Amendment Institute in 2025 and concluded that AI's influence on the 2024 national elections was small compared with other factors.^[13] The Federal Judicial Center survey of US federal trial judges in January 2026 supplies a concrete negative result: of 931 respondents, 15 said a litigant in one of their cases had challenged audiovisual evidence as a deepfake.^[14]

Several of the technical claims are weaker than their headlines. Model collapse in Shumailov's Nature paper is conditional on synthetic data replacing real data; Gerstgrasser and colleagues showed that accumulating both keeps error bounded, which is closer to how training corpora are actually assembled.^[9][10] The 45% figure from the European Broadcasting Union study is the share of answers with at least one significant issue against journalistic criteria dominated by sourcing, not a rate of factual falsehood, and 20% is the relevant accuracy number.^[6] The web is also not drowning as fast as claimed: Graphite sampled 43,000 English-language articles from Common Crawl and found that AI-written articles passed human-written ones in November 2024, then sat at roughly parity rather than continuing to climb. That count rests on a single AI detector applied to 500-word chunks, so it carries the error rate of the detector as well as the trend.^[15]

There is also a positive case that gets little attention. The International AI Safety Report 2026 records that a combination of provenance techniques within a wider system of standards can help people identify synthetic content more reliably than any one method, and that persuasive AI can move beliefs toward accurate positions as well as away from them.^[16] A reference system that answers questions in a hundred languages, including for people with no access to a library, is not obviously a net loss to public understanding, and the studies measuring harm have not been paired with studies measuring that benefit.

Provenance is the main technical response. The transparency rules in Article 50 of the EU AI Act apply from 2 August 2026 and require providers of generative systems to mark synthetic output so it can be detected by machine, deployers to label deepfakes visibly, and AI-generated text on matters of public interest to be labelled where no human editor reviewed it. A grace period runs to 2 December 2026 for systems already on the market.^[17] The International AI Safety Report 2026 is candid about the limits: watermarks can be removed or spoofed, particularly where model weights are public, content logs raise privacy problems, and warning labels raised viewers' accuracy at spotting AI video only from 10.7% to 21.6% in one study.^[16]

Evidentiary institutions are adapting more slowly than the technology. In the United States, the Judicial Conference Advisory Committee on Evidence Rules commissioned the Federal Judicial Center survey specifically to decide whether Rule 901 on authentication needs amending, and has a separate proposed Rule 707 covering machine-generated evidence offered without a human expert.^[14] The survey result, that almost no federal judges have yet met a deepfake challenge, is itself the main argument against moving quickly.

Measurement of the intermediary is beginning to be institutionalised. The European Broadcasting Union study was designed to run across languages and territories rather than as a single-market snapshot, which makes repetition possible and creates a baseline against which assistant providers can be held.^[6] Pew's browsing-panel method does the same for the traffic question.^[7] The Demos report argues that the missing piece in the United Kingdom is not measurement but crisis preparedness: protocols agreed in advance for what platforms, regulators and government do when an information system fails during an emergency.^[3]

Treat this as four separate trends rather than one event, because they have different evidence and different remedies. Falling trust is real, long-running and mostly not caused by AI. Assistant unreliability is measured, recent and in principle fixable by the companies concerned. Traffic diversion is measured and structural. Narrowing of the knowledge base is an argument from mechanism with no outcome measurement yet. Anyone who presents the four as a single accelerating curve is combining a fifty-year trend with a two-year one.

For your own use, the habit that matters most is checking whether a claim arrived with a source you can open. The sourcing failure rate found by the European Broadcasting Union study was higher than the accuracy failure rate, which means the most common defect in an AI answer is not that it is wrong but that you cannot check it.^[6] Treat assistant output as a lead rather than a citation, and notice when a question you would once have researched has become a question you now only ask a model. The Microsoft and Carnegie Mellon finding is that confidence in the tool, not the difficulty of the task, predicts how little you will check.^[11]