Context poisoning

Context poisoning is the failure mode in which a hallucination or other error enters an agent's context window or persistent memory and is then referenced again and again, compounding as the agent treats its own earlier mistake as established fact.

How it works

A model attends over everything in its window, and message roles mark who said a thing, not whether it was checked, so a sentence the agent generated three turns ago competes on even terms with a fact it read from a verified source, and once an error is in the transcript it re-enters every subsequent reasoning step as input. Repetition compounds the damage: each time the agent cites the bad entry, the window gains another copy, and every repetition makes the mistake look better established. Multi-turn research shows the behavioral signature in simulation: models that take a wrong turn early tend to entrench the wrong assumption rather than recover, which is the dynamic poisoning rides on. Persistence escalates it from a session problem to a durable one, because an error written into notes or memory survives the session that created it and seeds future sessions. The pattern is one mode in a wider taxonomy of context failures, alongside distraction, confusion, and clash, and in this reliability framing it is typically self-inflicted; security writing also uses poisoning for adversarial seeding of context or memory, and injected content that gets cached or written to memory crosses into the same compounding shape.

Why it matters

Context poisoning and context rot get conflated because both degrade a long-running agent, but they are different failures with different fixes: rot is a gradual signal-to-noise decline as the window fills, managed by pruning and compaction, while poisoning is a specific falsehood with compounding influence, and no amount of general pruning helps if the pruned summary keeps the poisoned claim. The distinction matters operationally because the poisoning fix is targeted: find and remove the corrupted entry, or rebuild the window from verified sources, rather than trimming for size. The nastiest variant is goal-level poisoning, where the corrupted entry is the objective itself, because every subsequent step can be locally correct while the run as a whole drives toward something wrong. And nothing flags it inline: the agent's confidence doesn't drop when it cites its own mistake, the window still validates, and the output stays fluent, so detection depends on checks that live outside the transcript. That is the deeper lesson the failure teaches: an agent's transcript is not a source of truth about the world, only a record of what the agent has said and seen, and designs that treat it as ground truth inherit its errors at compound interest.

In practice

A coding agent misreads a library's interface early in a long session and writes the wrong function signature into its working notes. Later steps cite the note instead of the source, generate code against the wrong signature, and explain the resulting failures by inventing causes consistent with the mistake, because the note now looks like established fact while the real documentation sits unread. The unblocking move is a reset: a fresh session whose window is rebuilt from the actual documentation and the verified parts of the work, which clears the behavior because the rebuilt window simply leaves the poisoned note out. The durable fix is procedural, having the agent re-verify load-bearing facts against the source rather than against its own transcript before building on them.

Practical considerations

The recovery primitive is the reset: when a session keeps re-asserting a claim no source supports, rebuilding the window from verified inputs is usually cheaper and more reliable than arguing the agent out of it inside the poisoned context. Quarantine generated content from verified content at the boundary where it persists, because a note that will outlive the session deserves a check the model did not author before it is written. Watch for the fixation symptoms: an agent pursuing a goal that no longer matches the task, repeating a claim it cannot cite, or explaining failures with increasingly elaborate causes, all of which point at a poisoned entry rather than a hard task. Isolation bounds the blast radius, since a subagent's poisoned intermediate state dies with its own window unless the summary it returns, or an artifact it wrote, carries the poison back. Compaction cuts both ways here: summarizing a poisoned window can concentrate the poison into the summary, so treat compaction as a rot tool, not a poisoning fix.

Defined by Ready Solutions AI