Agent Memory Poisoning

Starting a new chat does not clear an injection, because the thing that reinfects you is not the attacker's page. It is a note your own agent wrote about the day it read that page, filed in the memory store, and retrieved again three sessions later because it happened to be the most similar record to what you asked next.

An agent memory store is a retrieval index the agent both reads from and writes to. On the read side it is ordinary vector retrieval: embed the query, score every stored record by cosine similarity, keep the top few above a threshold, paste them into the context window. On the write side, at the end of a session, the agent summarises what it learned and stores that summary as a new record. Both halves are the feature. Together they are a cycle, and a cycle with a compromised turn in it does not end when the session does.

The first hop is an ordinary indirect prompt injection — text somebody else wrote, read by the agent, treated as instruction. That part is well covered. What is not covered is the second hop, and the second hop is free: the payload is now inside a record whose recorded author is your agent, sitting in a store your agent is allowed to read.

Below, forty sessions run end to end. Session 1 is the only session in which the attacker's content is present at all — after that the page is gone, the email is deleted, the attacker has left. Every later session starts with an empty context window, which is exactly what "start a new chat" gives you. Move records retrieved per turn first, then the two defences that most teams reach for.

At the defaults the hero number reads 38. Thirty-eight of the thirty-nine sessions that ran after the attacker's content was gone still acted on the injected instruction. Every one of those sessions began with a clean context window. The poisoned share of retrieved context is 50% over the first ten sessions and 84% over the last ten — it goes up, which is the part that surprises people. A conventional injection decays because the attacker has to keep pushing. This one compounds, because each successful retrieval writes another near-duplicate next to the query that retrieved it. By session 40 the store holds 40 poisoned records out of 65, and the agent authored 39 of them.

Now switch on the provenance filter — the control that refuses to retrieve anything sourced from untrusted content. This is the defence most teams have, in the form of "we only index documents from trusted sources". The hero number stays at 38. Not lower. The store still holds 40 poisoned records out of 65. The filter is working perfectly and catching exactly one record: the snippet of the attacker's page that the agent stored in session 1. Every other poisoned record has source: agent-summary, because that is literally true, and no check that asks "where did this text come from" can return anything else.

Set when the agent writes to memory to never and leave the provenance filter on: the hero number drops to 0. That is the honest scope of provenance filtering. It defends the store against what you put in it. It does not defend the store against the agent.

Why the record that reinfects you passes every provenance check

Put the write policy back to after every session and switch the provenance filter back off, so the page is at its defaults again, then read the decision log for the last five sessions. Each compromised row names the record that did it and the session in which the agent wrote it. Follow the chain backwards and it terminates at session 1, but no record in the chain except w0 has anything to do with the attacker. They are summaries of your agent's own work, written by your agent, stored through your agent's own write path, in your agent's own voice.

This is why the standard advice misses. The Open Worldwide Application Security Project's Top 10 for Large Language Model Applications v2.0 covers the store directly under LLM08:2025, Vector and Embedding Weaknesses, and its Data Poisoning scenario is a resume with white-on-white hidden text submitted to a screening system. The prescribed mitigation is "Data validation & source authentication": validate the knowledge sources, audit the knowledge base for hidden codes, accept data only from trusted and verified sources. Every one of those controls sits at ingestion. The write-back is not ingestion. It happens after the model has run, on the far side of every filter you installed, and it arrives labelled with the one source you decided to trust unconditionally.

The same gap sits under LLM01:2025, Prompt Injection, whose fourth example scenario is an attacker modifying a document in a repository used by a retrieval-augmented generation application — retrieval-augmented generation being the pattern where the system looks up documents and pastes them into the prompt. That is a one-shot event with an attacker in it. The mitigation list includes "Segregate and identify external content", which is the tagging family of defences and which works exactly as well as its labels are accurate. After a write-back the label is accurate and useless at the same time.

Dong, Xu, He, Li, Tang, Liu, Liu and Xiang published the sharpest version of this in March 2025 as Memory Injection Attacks on LLM Agents via Query-Only Interaction. Their attack, MINJA, "injects malicious records into the memory bank by only interacting with the agent via queries and output observations." The attacker never touches the store. They send queries. Two details of their construction are the ones worth carrying around: a set of bridging steps that connect an ordinary victim query to the malicious reasoning, so the record looks relevant to things the victim will actually ask; and a progressive shortening strategy that strips the give-away instruction prompt out over successive interactions, leaving a record that is retrieved on its merits. Their stated conclusion is the one that should change your threat model: the requirements for execution are minimal enough that any user could influence agent memory.

The dial that decides it is top-k, and it is not tuned for this

Set records retrieved per turn to 1. The hero number goes to 0 of 39 and the longest clean run becomes 39 sessions. At 2 it is 13 of 39. At 3 it is 27. At 5, which is a perfectly ordinary production setting, it is 38. One step on a slider nobody thinks of as a security control moves the outcome from nothing to everything.

The reason is geometric and worth sitting with. The seed store holds one record every 15°, so the nearest clean record to any query is at most 7.5° away and scores about 0.991. A poisoned record was written at a previous query angle, so it beats that only when the current query lands close to a previous one. At top-1 it usually loses. At top-5 it does not have to win — it only has to place fifth. The threshold does not save you either: at 0.900 the hero number is 38, at 0.980 it is 24, and it only collapses to 5 at 0.990 and to 0 at 0.999. Watch the records retrieved per session readout while you do that. It falls from 4.9 to 4.2 to 3.6 to 1.9. At the setting where the attack stops, the agent is retrieving under two records a session, which is a memory system you have switched off by degrees rather than one you have secured. Put the threshold back to 0.900 before the next move, or everything below is measured on a store nobody can reach. This is the same similarity is not relevance problem that makes retrieval tuning hard in the first place, with a consequence attached.

Now move how concentrated the user's work is from 25° to 5° — a person who uses their agent for one narrow job, which is most people. The hero number goes to 39 of 39, the poisoned share climbs from 74% in the first ten sessions to 96% in the last ten, and the longest clean run is 0 sessions. Push it the other way to 90° and the attack nearly dies on its own. The variable that decides whether a memory poisoning incident is survivable is how repetitive the victim's work is, and nobody has that on a dashboard.

Put how concentrated the user's work is back to 25° and set the write policy to only when the user says "remember this". This is the product decision several assistants shipped, on the theory that an explicit gesture is a consent boundary. The hero number goes from 38 to 33 of 39. The poisoned share of retrieved context is 16% over the first ten sessions and 50% over the last ten. It arrives more slowly and it arrives. Making the write rarer changes the slope, not the destination, because the loop is still closed.

Deleting the record you caught

The obvious response to finding a poisoned memory is to delete it. Put the write policy back to after every session, then move you notice and delete the record to 100%: every single session in which the agent acts on the injected instruction, you spot it and remove the exact record that fired. Perfect vigilance, forever.

The hero number is 17 of 39. You performed 18 deletions and the store still ends with 1 poisoned record in it. You never reach zero, and the reason is ordering. Within a session, retrieval happens first, the write-back happens second, and your deletion happens third. The evidence that lets you notice is produced by the same turn that writes the replacement. One in, one out, forever.

Then set how concentrated the user's work is to 5° and leave the notice rate at 100%. The hero number is 39 of 39. You deleted a record 40 times — once per session, catching it every time without a single miss — and every session was still compromised. The store ends with 1 poisoned record, which is all it ever needed. This is the part to remember when someone proposes "we'll review the memory store": a review cadence slower than the write cadence is not a control, and the write cadence is one per session.

There is a second reason deletion under-performs in the field, visible in the memory map. The poisoned records do not sit in one bucket. They spread across the angles you have been working in, because each was written at the query that retrieved its parent. An operator who deletes "the poisoned memory" deletes one of a cluster whose extent nobody has measured.

What actually works, and the bill it comes with

Reset the page to its defaults — spread back to 25°, notice rate back to 0% — and switch on the taint-propagating write guard: a record born in a session that touched outside text is never retrievable. The hero number goes to 0 of 39 and stays there under every other setting on the page. This is the only control here that closes the loop rather than narrowing it, and it closes it for a structural reason — it breaks the edge from "compromised context" to "retrievable record", which is the edge the whole mechanism runs on. It is the same move Simon Willison named in 2023 as the Dual LLM pattern: a quarantined model handles the untrusted data and is prevented by construction from influencing what the privileged model does, rather than being asked to label its own output honestly. CaMeL, the design that formalised it, calls that pattern foundational to its own work and adds a capability system on top so that untrusted data cannot steer data flow either — but the move that matters here, breaking the edge rather than labelling it, is the older one.

Now read the write-backs the guard refused readout. At the default 60% of sessions reading outside text, it is 22 of 40. More than half of everything the agent tried to learn was thrown away, and almost all of it was legitimate — reading a web page is not the same as reading a malicious one, and the guard cannot tell the difference, which is precisely why it is sound. Push sessions that read outside text at all to 100%, which describes any agent with a browser or an issue tracker attached, and the readout says 40 of 40. The guard has not secured your memory system. It has turned it off, and left the chrome in place.

The other structural option is expiry, and its boundary is sharper than it looks. Switch the write guard back off, put sessions that read outside text at all back to 60%, and switch on expire written memories at 5 sessions: the hero number is 33 of 39, barely better than doing nothing. Take it to 3 sessions and it is 13 of 39, with the poisoned share of the last ten sessions at 0% — the last compromised session is 17, after which the store is clean, because the reinfection interval finally exceeds the record lifetime. Then set how concentrated the user's work is to 5° with expiry still at 3 sessions: 39 of 39. Expiry only removes a record that nobody refreshed, and a user who asks the same kind of question every day refreshes it every day. The defence works exactly when the attack was going to fizzle anyway.

Nothing on this page fixes the case where the store is shared. Two agents reading one memory store is a channel that carries the payload with no message passing between them at all, and the deduplication that most stores perform will happily merge a poisoned entity into a neighbouring one. That is trust laundering between agents, and the shared store is its quietest form.

Checking this on a system you actually run

Start by finding out whether your agent has a write path at all, because a surprising number of teams do not know. Grep the harness for the code that appends to the store after a turn: in a framework it is usually a save_memory, add_to_memory or session-summary hook; in a coding agent it is whatever writes to CLAUDE.md, AGENTS.md or a project rules file. Those files are agent memory with the retrieval step replaced by "always in the context window", which makes them the worst case: reinfection probability 1.0, no threshold, no top-k. Put them under review in version control and read the diffs. An agent that edits its own instructions is running the loop on this page with the dials welded open.

Then make the store's schema tell you what you need in an incident. Every record wants four fields: the session id it was written in, the tool-call ids whose output was in the context window when it was written, whether any of those tools returned bytes chosen outside your trust boundary, and the query angle — in practice, the embedding — it was written at. The third field is the taint bit, and having it stored is what lets you decide later whether to enforce on it. The fourth is what lets you answer "what else is near this one", which is the question deletion actually requires. Most stores today keep the text, a timestamp and nothing else, which is why the answer to "how many copies are there" is always a shrug.

For detection, the highest-signal query is a similarity search of the store against itself. Poisoned clusters are near-duplicates by construction — MINJA's shortening strategy makes them near-duplicates deliberately, and the automatic write-back loop makes them near-duplicates accidentally. Take every record written in the last thirty days, compute pairwise cosine within the store, and list clusters of three or more records above 0.97 that were written in different sessions. Legitimate memory is repetitive too, so this is a triage queue rather than an alert; the thing that separates the two is whether the cluster's members all contain an imperative the user never issued.

In production, log a retrieval trace: for every turn, the record ids retrieved, their scores, and the session each was born in. That single line turns "the agent keeps doing the wrong thing" into a bisection. The born-in-session field is the one that ends the argument, because when the reinfecting record was authored three weeks after the incident everyone believed was closed, no amount of clearing the chat history was ever going to be the fix. The scope question underneath — why the agent could act on the instruction once it had it — is excessive agency, and it is the part you can still fix after the store is dirty.