Model collapse doesn't start in training. It starts in the evidence.

In 2024, Shumailov and colleagues published a result in Nature that gave the field a name for something it had been circling for a while: model collapse. Train a model on its own outputs, repeat across generations, and the model loses the tails of the original distribution. Rare information disappears first. The loss compounds.

The follow-up literature complicated the picture in useful ways. Gerstgrasser and colleagues showed the outcome isn't automatic; accumulating real data alongside synthetic data can break the recursion. Gillman and colleagues showed that self-correcting training loops can stabilize it.

We take all of that as established framing. It is not our contribution, and we're not trying to relitigate it.

Our interest sits one step upstream.

The loop that doesn't need a training run

Here's the observation our program is built on: nothing in the collapse dynamic actually requires a model to be retrained.

The public information environment already contains a large volume of machine-written articles, summaries, product pages, forum answers, and documentation. Retrieval systems and research agents consume that material, synthesize over it, and publish results that re-enter the same environment. That's a recursive loop, and it runs at the speed of publishing, not the speed of training cycles.

A single generation of agents, operating today, can already do all of the following:

1. Propagate a false claim, because an unsupported assertion gains apparent credibility every time a dependent document repeats it.

2. Launder a citation, so a secondary summary appears to support a claim its underlying source never made.

3. Collapse sources, where many apparently independent references resolve to one ancestor, or to no primary source at all.

4. Lose rare information, as uncommon but correct human-origin facts get displaced by more statistically common approximations.

5. Inflate certainty, because qualifications and error bars vanish through repeated summarization.

6. Homogenize expression, replacing distinctive perspectives with averaged phrasing.

We treat these as the proximate failure modes and distributional collapse as the terminal one. The proximate failures are observable now, in a single agent run, without access to anyone's training pipeline.

That's what makes them tractable. And it's why we've chosen to work there.

Our framing in one sentence: model collapse is not only something that happens to weights during training. It is something that happens to evidence during retrieval, and the evidence layer is where it can be measured and intervened on first.

What we are deliberately not building

This is not an AI-writing detector. We want to be blunt about that, because it's the first thing people assume.

Detection of machine authorship is unreliable in the general case, and the field has already retracted deployed classifiers on accuracy grounds; OpenAI withdrew its own text classifier for exactly that reason. But the deeper problem is that detection answers the wrong question. Knowing a document "looks AI-written" tells you nothing about whether its central claim is supported, whether its three citations share one ancestor, or whether reusing it will contaminate the next generation of outputs.

A perfectly accurate authorship label would still leave the propagation problem untouched.

So we hold a few positions on purpose. Style is not provenance, and we don't convert a provenance score into an accusation against any author. Human-origin is not a correctness label; human-written material is frequently wrong, and our historical reference class is a sampling frame, not a truth claim. AI-assisted work is a distinct lineage class, not a contaminant by default. And disclosure is an audit output, not the endpoint. An agent that flags its uncertainty and then publishes the claim anyway has prevented nothing.

The intervention: trace, then stop

The behavioral hypothesis is narrow and falsifiable:

Can a research agent trace claims upstream and stop reusing evidence when it cannot recover an acceptable provenance root, thereby reducing recursive self-cannibalization across generations?

The registered success condition has three parts. The agent must stop treating an unresolved or derivative source as independent support. It must continue only with claims that have acceptable upstream evidence. And when the central claim no longer has sufficient support, it must omit the claim or abstain from the answer entirely.

That third part is the one most systems don't do. Better classification, better citation formatting, and better disclosure all leave the derivative claim sitting in the output. This is why we describe the intervention as preventive stopping rather than provenance scoring.

The comparison that matters is against agents given the same task and the same resources but not the trace-and-stop rule. Critically, it's also against agents that are merely asked for citations or disclosure. If a content-neutral prompt of the same length produces the same behavior, the intervention has explained nothing.

How we're studying it

A single output tells us very little, so the unit of analysis is a lineage: one scenario, one model, one experimental arm, one replicate, run forward through a pre-registered number of generations, where each generation's outputs update the corpus the next generation reads.

The primary comparison is the trajectory, not the final score. The rate of degradation per generation. The generation at which a false claim becomes dominant. The generation at which a rare fact disappears. Recovery after new human evidence is introduced.

Two structural rules do most of the methodological work.

First, the agent's provenance checker and the researchers' evaluator are different systems that never touch. The agent sees a lineage checker during the run. Researchers score outputs afterward against seeded claims and a known source graph the agent never had access to. If those two collapse into one system, the study measures its own scoring function instead of agent behavior. We enforce the separation architecturally and verify it mechanically.

Second, the confirmatory design is a matched factorial, so the policy effect can be separated from the effect of simply giving an agent another revision pass. One arm gets the provenance policy, one gets a content-neutral placebo prompt of registered length and timing, one gets score feedback without the policy, one gets both.

Guardrails matter as much as the endpoint. We do not count the intervention as successful if it rejects rare true claims along with false ones, produces blanket refusals, rewards only conventional answers, teaches agents to cite sources without reading them, can be gamed by appending plausible-looking source identifiers, or works only on the model family used to design the score. Rare-fact retention is a pre-registered non-inferiority guardrail with a fixed margin and failure rule, not a secondary metric to be discussed away.

Where the work actually stands

We're going to be more conservative here than most research communications are, because the point of the governance structure is to make overstatement difficult.

The program has run through a staged, gated sequence: environment lock-down, an offline tracer, a container-isolated producer sandbox with independent security review, calibration, and a historical-source pilot. Every stage's entry, exit, failures, and deviations are recorded in a decision register.

What that has produced so far is engineering and methods evidence. Not research findings.

The original pilot stopped early without producing a validated response, and the exact cause remains irrecoverable from the retained evidence. We're documenting that as a finding about our instrumentation rather than quietly reprocessing it. A later diagnostic sequence identified and repaired two concrete defects, and one authorized end-to-end validation subsequently executed a single baseline path through to a closed output and a post-closure hidden evaluation.

One execution of one baseline path is not a rate, a comparison, or an effect estimate. Total reconciled spend across the program to date is on the order of one tenth of one US cent, which should tell you something about how early this is.

One more disclosure, because it belongs in the open: this project is currently owner-led and self-funded, with operational, analysis, governance, and release roles held by the same person. That is a real limitation on independence. Automated verification is not a substitute for external review, and we're not going to pretend otherwise.

The questions we want pressure on

A few open problems, stated plainly, because we'd rather have them attacked now than after a results release.

Is preventive stopping even the right endpoint, or does it trade propagation for systematic abstention on exactly the rare claims we most want retained? What counts as an acceptable provenance root on the live web, where most claims bottom out in a page with no verifiable ancestry rather than in a primary source? How should independence be defined when apparently distinct sources share an upstream ancestor that neither cites? And does an upstream-trace rule generalize across model families, or does it mostly measure each family's instruction-following?

We don't have answers to these yet. That's the honest status of the work.

The recursion is already running in the open web. Whether it can be interrupted at the evidence layer is the question we've committed to testing, under rules we published before we knew the answer.

The full methods document, including pre-registration commitments, arm definitions, and the decision register structure, precedes any results release.

References

1. Shumailov, I., et al. "AI models collapse when trained on recursively generated data." Nature 631, 755-759 (2024).

2. Gerstgrasser, M., et al. "Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data." COLM (2024).

3. Gillman, N., et al. "Self-Correcting Self-Consuming Loops for Generative Model Training." ICML (2024).

4. Sadasivan, V. S., et al. "Can AI-Generated Text be Reliably Detected?" arXiv:2303.11156.

5. OpenAI. "New AI classifier for indicating AI-written text." (Classifier subsequently withdrawn.)

6. Coalition for Content Provenance and Authenticity. C2PA specifications, v2.2.