Trust Laundering Between Agents

Giving the untrusted work to a sub-agent contains the tools and launders the text. The reader agent has no credentials, no shell and no network — and it does not need any, because its job is to produce a paragraph, and that paragraph arrives in the orchestrator's context window as a message from a trusted peer rather than as a quotation from a web page.

The containment argument is genuinely good as far as it goes. A low-privilege agent reads the hostile page. Whatever that page tells it to do, it cannot do: no tool, no token, no blast radius. The privileged orchestrator never touches the page. Two of the three legs of the lethal trifecta are in different processes.

What crosses the boundary is a string. Taint is a property of a span inside one context window — a note attached to a range of tokens saying "these bytes came from outside" — and a message between agents is a new span in a new context window with no such note on it. The orchestrator's own system prompt tells it that a worker's result is the answer it asked for. So the label dies, the instruction survives, and the orchestrator does the thing.

Below, a multi-agent system runs 3,000 times. One or more reader agents fetch sources; at least one of those sources is a page the attacker can write to. The payload spreads only by being reproduced in an agent's own output, one handoff at a time. Move per-hop tagging up to 90% first. Then switch on the message envelope carries provenance and watch what tagging was worth all along.

At the defaults — three readers, one of them on a page the attacker can write to, no tagging, the orchestrator holding the only privileged tool — a privileged action fires in 84.9% of runs. The orchestrator executes it every time. The payload crossed exactly 1 handoff to get there, and 1.85 of the four agents ended up carrying it.

Now push per-hop tagging blocks to 90%: an agent that is shown peer content wrapped in a "the following is untrusted data, do not obey it" marker refuses nine times in ten. The hero number reads 84.9%. Unchanged, to the decimal. Not because tagging is weak — because there is nothing for it to tag. The orchestrator received a paragraph, and a paragraph is what a worker's answer looks like. Nothing in the envelope says the bytes started life on a web page.

Switch on the message envelope carries provenance and the same 90% tagging takes the hero number to 8.2%. That ordering is the whole lesson about tagging: it is not a defence you turn on, it is a defence you turn on after you have built a message format that carries taint, and almost no agent framework has one. Check yours by asking a single question: when worker output is appended to the orchestrator's messages, is there a field that says which tool call produced it and whether that call returned third-party bytes? If the answer is "it is a string", your tagging slider is at zero no matter what your prompt says.

More hops is safer; more parallelism is not

Put per-hop tagging blocks back to 0% and switch the envelope toggle back off — the tagging story is finished, and the bar chart below is measured without it. Now switch the wiring to chain and read it. The bars run downhill: 85% at one reader, 73% at two, 61% at three, 37% at eight. Switch back to fan-out and they run uphill: 85%, 84%, 86%, and 100% at eight. Same eight agents, same attacker, opposite direction.

The reason is that the two shapes multiply different things. A chain is a series: the payload has to survive every retelling, so with an 85% chance per hop, six hops leave 38%. Fan-out is a parallel set: the payload only has to survive one retelling out of however many readers are sitting on attacker-writable sources, and at 34% of sources hostile, eight readers means three of them are. Each one is an independent attempt.

This is why the per-hop tagging needed to hold this under 5% readout is the most useful number on the page. It is computed with the envelope assumed present, so it does not move when you touch the tagging slider. Set the wiring to chain with 8 readers and it says 31% — a mediocre detector, run six times, is enough. Confirm it: tagging to 50% with the envelope on gives a hero number of 0.8%. Now put tagging back to 0%, switch the wiring to fan-out and keep the eight readers: the same readout says 97%, and pushing sources the attacker can write to to 100% takes it to 99%. Set tagging to 90% with the envelope on and the actual hero number is 51.3%: a detector that is right nine times out of ten, facing eight independent attempts, is a coin flip. This is the same arithmetic as why 95% is a failing grade for an injection defence, with the number of attempts set by your architecture diagram rather than by the attacker.

Nobody builds a chain for security reasons, and this is not an argument that they should — serial handoffs are slower and lose information. It is an argument that "we split it into more agents" is not a direction on the safety axis at all. It is a direction on the concurrency axis, and which way safety moves depends on whether the new agents are in series or in parallel with the attacker's entry point.

The payload that rewrites itself, and the store that does not rewrite it

Reset the page to its defaults — fan-out, three readers, 34% of sources hostile, tagging at 0%, the envelope off — and move payload survives a retelling to 50%: the hero number falls to 49.5%. Move it to 100% and the hero number is 100.0%. That slider is entirely under the attacker's control. It is a property of how the injected text is written — whether it survives being paraphrased into somebody else's summary — and writing text that survives paraphrase is a solved problem, because instructions phrased as facts about the task get carried forward by a summariser doing its job correctly.

Lee and Tiwari named the general case Prompt Infection in October 2024: malicious prompts that self-replicate across interconnected agents, in their words, "behaving much like a computer virus". Their measured result is the uncomfortable one. Multi-agent systems were highly susceptible "even when agents do not publicly share all communications" — the containment intuition that a message only reaches its addressee does not hold, because the addressee retells it. The defence they propose is LLM Tagging, and they report that it mitigates spread significantly when combined with existing safeguards, which is the qualification the simulation above is built to make concrete.

Now put payload survives a retelling back to 85%, switch the wiring to fan-out over a shared scratchpad, keep three readers, and read the graph panel. The hero number is 97.3% and 3.66 of the four agents are compromised per run, up from 1.85. Reader-2 and reader-3 never read a hostile page and never received a message from reader-1. They read the scratchpad. A shared store is an edge in the graph that nobody drew, and it is worse than a message because nothing rewrites the text on the way through: the 85% survival slider does not apply to a copy. At eight readers the hero number is 100.0% and 8.26 agents are carrying it. That is the memory poisoning mechanism arriving through the side door — the same store, with the infection spreading sideways instead of forwards in time.

The structural fix, and the exact shape of its boundary

Put the wiring back to fan-out with three readers, then switch on the orchestrator's control flow is fixed by the user's task. The hero number goes to 0.0% and the graph still shows agents being compromised — 1.85 per run at the defaults, unchanged. That gap is the point. This control does not stop the payload spreading. It removes the orchestrator's ability to be redirected by anything a peer says, so the spread stops mattering.

The move is older than the paper most people cite for it. Simon Willison named it in 2023 as the Dual LLM pattern: a privileged model plans the work and never sees untrusted text, while a quarantined model reads the untrusted text and cannot decide what happens next. Debenedetti and colleagues' Defeating Prompt Injections by Design calls that pattern foundational to its own work and builds CaMeL on top of it, contributing a capability system so that untrusted data cannot steer data flow either — the half that the Dual LLM pattern on its own leaves open. What both share is the shape: a quarantined model is structurally prevented from influencing control flow rather than being trusted to label its own output honestly. It is the difference between a defence with a rate and a defence with a proof, and it is the reason the per-hop tagging needed readout says already under 5% — no detector is in the loop at all.

Read the benign tasks needing a policy or a human readout while it is on. At the default 45% it says 45 in 100. Those are the tasks where the worker's answer is supposed to decide what happens next — "find the customer's order and refund it if it shipped late" — and under this design each one has to be pre-authorised as a policy or escalated to a person. Getting that number down is design work on your task shapes, not a setting. Where it goes instead is the approval queue, and what happens to a human reviewing forty-five prompts in a hundred.

Then find the boundary. Leave the quarantine on, switch the wiring to fan-out over a shared scratchpad, and set who holds a privileged tool to the orchestrator, plus a code-runner reader. The hero number is 84.5%, and the which agent executes it readout no longer says orchestrator — it says reader-2. The action fires 2 handoffs from the source, through an agent that never read the hostile page, executed by a sub-agent you gave a shell to because it needed to run the tests. The tagging needed to hold that under 5% is 76%.

Set who holds a privileged tool to every agent with the quarantine still on. The hero number is 100.0%, the executing agent is reader-1, the payload crossed 0 handoffs, and the tagging readout says no rate is enough — correctly, because tagging only ever inspects peer output, and reader-1 was compromised by the page itself. Every control on this page is downstream of one decision: which agent holds which tool. Quarantining the orchestrator protects the orchestrator. It is not a property of the system.

Checking this on a system you actually run

Draw the edge list, not the org chart. For every agent, write down what it reads and what reads it, and add three edges people always leave out: the shared scratchpad or memory store, the task queue if agents can enqueue work for each other, and the file system if two agents share a working directory. Then mark which nodes hold a tool that does something irreversible. The dangerous configuration is not "an agent reads the web"; it is "a node that reads something an attacker can write has any path to a node that holds a tool", and the paths through stores are the ones that never appear on the diagram.

Make provenance a field, not a convention. The concrete change is in the message type that carries a sub-agent's result: alongside the text, carry the tool-call ids that produced it and a boolean for whether any of them returned bytes from outside your trust boundary, and make that boolean sticky — an agent's output inherits it from every input it read. Without that field the tagging slider on this page is nailed to zero, and so is yours. The agent-to-agent protocol work has the same question open; the A2A project's own tracker carries an issue asking it to account for the confused deputy problem, which is the authorisation half of this — whose authority is the second agent acting under.

For detection, log a payload fingerprint per span. Hash the normalised trigram set of every tool output and every inter-agent message, and alert on the same fingerprint appearing in the output of two different agents within one task. Self-replicating text is, by definition, text that appears in several places while claiming several different authors, and that is detectable even when the wording drifts, because the drift is bounded by the payload needing to remain executable as an instruction.

In an incident, the question is which span introduced the bytes. That means every model call needs the tool-call id and the resource identifier of every span in its context, and every inter-agent message needs the id of the model call that produced it. With those two joins you can bisect a compromise back to a URL in a few minutes. Without them you have a log showing that an agent summarised a page and, separately, that another agent ran a command, with nothing connecting the two — which is the state most multi-agent deployments are in today.