Prompt Injection is an Information-Flow Problem
How prompt injection exploits trust boundaries through saved memory, and what runtime checks can prevent unsafe actions.
- Real-world attacks (OpenClaw, Adversa AI's Grok exploit, Embrace The Red's Claude Code test) demonstrate that untrusted external content can propagate into agent memory, tool outputs, and code dependencies, effectively granting attackers persistent influence over agent behavior.
- A key vulnerability is agents treating a source's approval claim (e.g., "workflow_status: approved" in a supplier invoice) as actual authorization, a confusion the author terms "provenance is not permission."
- In a controlled invoice-assistant experiment using GPT-6-astra, the baseline configuration produced 2 unauthorized mock email sends out of 12 attack attempts, while input-label, persistent-provenance, and capability-gate configurations each produced zero unauthorized sends.
- A runtime recipient gate that compares proposed destinations against a runtime-owned authorized list (not modifiable by the model) successfully rejected both unsafe proposals replayed from baseline failures, while still accepting legitimately authorized external recipients.
- The author recommends tracking data provenance separately from model-written summaries, binding user approvals to specific operations and recipients, and enforcing policy checks immediately before execution with tamper-resistant logs outside the agent's write permissions.
IronCore Labs recently described an OpenClaw attack in which an agent flagged emails as suspicious, then saved claims from those emails to memory without the warnings. Repeated messages established the attacker's address as an internal account. The agent later followed instructions from that address to modify its email workflow and forward other emails to the attacker.
In many use cases your agent turns external documents into summaries, memories, or tool arguments. Which is why you need to preserve the distinction between what a source says and what it may authorize.
For example, an invoice record can name a backup address and claim that it is approved. A later request to archive an invoice still needs an authorization decision about that destination.
I built a small invoice-assistant experiment to examine that transition. I compared input labels, persistent source metadata, and a recipient gate. The test produced two unauthorized mock sends under the baseline configuration and all three label configurations avoided them. A separate rerun showed how the runtime rejects those same proposals when it checks recipient permission before execution.
How far can an untrusted source's influence travel?
In the IronCore's Openclaw test, untrusted email content entered daily memory files after the initial model flagged messages as suspicious. The cautionary qualifiers disappeared during that process. OpenClaw's background "dreaming" feature can promote recurring facts into global memory. These repeated emails helped establish the attacker's address as an internal account. It also describes direct updates to the global memory files. The agent then accepted instructions to alter this workflow.
Adversa AI's Grok report describes a different route.
The attacker supplied encrypted instructions and the material required to decrypt them. Grok invoked its code runtime to recover the plaintext which then entered the model's context as tool output.
The researchers encrypted the instructions so the model had to run decryption code to read them. Their reported failure concerned the authority given to the recovered instructions after that computation.
In Embrace The Red's Claude Code test, the agent rejected a supplied binary decoder and wrote a Python replacement. It ran the replacement inside an attacker-controlled archive directory. A malicious python file shadowed a standard-library module and executed through an import. It reports that Auto Mode inspected the short decoder while the execution chain reached code outside that immediate inspection.
These cases expose different boundaries: memory promotion, tool output, and code dependencies. For your application, the common design question is how far a source's influence can travel before the runtime checks what it may affect. A trustworthy transformation can faithfully preserve an attacker's content.
In these cases shows that the source's influence research beyond the original document into persistent memory, or executable code. Each intermediate step carries that influence forward. To control it, you need to track where a value comes from and check whether that source may influence the proposed action.
Provenance is not permission
Consider an address extracted from a supplier invoice. Your parser can identify the address correctly. That establishes neither the supplier's authority to choose a backup destination nor the recipient's permission to receive the invoice.
Four properties help you specify the decision.
For the invoice workflow, you might accept a total from supplier material while requiring an authenticated user or trusted policy to approve its delivery destination. Both values come from the same document, but they play different roles in the action.
Don't miss what's next in AI
Join 300,000+ engineers and researchers who get the signal, not the noise. Create a free account to read the rest of this story.
- Full access to in-depth AI research breakdowns
- Be the first to know what's trending before it hits mainstream
- Daily curated papers, repos, and industry moves