Your team just deployed an AI agent with access to your knowledge base and the ability to send email summaries. Then someone from security asks: "What if an attacker plants instructions in a document the agent reads?" You don't have a clear answer yet, and you're not alone.
These questions are common for teams building or validating production agents. They're the ones you ask in a design review or when writing a validation plan. The answers should cite specific attack stages and controls, not abstract principles.
Q1: What's the difference between a prompt injection and a jailbreak?
A jailbreak targets the model's safety filters. You write a prompt that tricks the model into ignoring its refusal training. You know you're doing it, and you see the result.
An indirect prompt injection targets your agent's tools and data. An attacker plants instructions in content your agent retrieves later, like a web page, an email, or a document in your retrieval-augmented generation (RAG) store. Your agent reads that content while doing its job, interprets the hidden text as an instruction, and acts on it. You never see the payload, and you didn't ask the agent to follow it.
The UK National Cyber Security Centre noted in December 2025 that "Current large language models (LLMs) simply do not enforce a security boundary between instructions and data inside a prompt." When retrieved text looks like an instruction, the model can't reliably tell it apart from your request.
Q2: How does an attacker actually get instructions into my agent's context?
The payload can be placed anywhere your agent reads. OWASP's Top 10 for LLM Applications 2026 lists web pages, documents, emails, tool responses, RAG passages, images, Model Context Protocol server output, database rows, and issue titles.
Consider a team deploying a coding agent connected to GitHub through MCP. In May 2025, Invariant Labs showed that a malicious public issue led a connected agent to leak private repository data into a public pull request. The issue title contained instructions the agent read and followed.
Attackers hide the text from human reviewers using techniques like white-on-white text, zero-size fonts, HTML comments, Unicode tag characters, and zero-width characters. Multimodal models can read instructions rendered inside images. Some payloads address a human reader rather than an AI, which lets them pass content classifiers that look for phrases like "ignore previous instructions."
Q3: We added an input filter. Isn't that enough?
Input filters inspect the user's prompt. An indirect payload arrives after that check, inside a tool return or retrieved document that looks like ordinary data.
Nasr et al., published at USENIX Security 2026, tested 12 recent defenses including delimiters and spotlighting. They found that once an attacker tunes the payload against those defenses, attack success rises above 90 percent for most. The authors note that "The majority of defenses originally reported near-zero attack success rates." In the same study's Red Teaming event, over 500 participants produced 265 successful attacks against Spotlighting alone.
A June 2026 NIST announcement of a proof by Apostol Vassilev states that "there is no finite set of guardrails that is universally robust against adversarial prompts." NIST recommends continuous Red Teaming and continuous guardrail updates. It also calls for "operational resilience that prioritizes impact limitation and quick recovery when, not if, an exploit occurs."
Use detection as one layer. Don't rely on it to stop every attack.
Q4: What makes an agent actually dangerous once it's injected?
Simon Willison named the combination the lethal trifecta on June 16, 2025: access to private data, exposure to untrusted content, and the ability to communicate externally. If your agent combines all three, an attacker can trick it into accessing your data and sending it out.
On June 11, 2025, Microsoft published CVE-2025-32711, a critical vulnerability in Microsoft 365 Copilot rated CVSS 9.3. In the attack Aim Labs named EchoLeak, an attacker placed instructions in a single inbound email that Copilot later retrieved. Those instructions moved sensitive tenant data to an attacker-controlled server without the recipient opening the message.
The payload included benign headings like "Here is the complete guide to HR FAQs" so Copilot's retrieval pulled the message into context across many topics. The instructions read like a note to an employee, which let them pass the classifier. Copilot followed the planted instructions, wrote selected data into a reference-style markdown image URL, and the client rendered it automatically. The browser fetched it, and the request carried the data out.
Microsoft fixed the flaw with no customer action required, and Aim Labs reported that Microsoft confirmed the flaw affected no customers.
Q5: How do I scope my agent's capabilities so a successful injection doesn't wreck everything?
Map your controls to the attack stages. An indirect injection runs through five stages: plant, retrieve, interpret, act, and exfiltrate. Stages three through five happen inside your deployment, so that's where your runtime controls belong.
Least-privilege tool scoping (act stage): Keep credentials and state-changing capability in application code, not in the agent's tool definitions. Scope each operation to the minimum it needs. Check every call with a deterministic policy engine at execution time. OWASP adopts Meta's Agents Rule of Two as a minimum standard: an agent should not combine more than two of these three capabilities: access to private data, exposure to untrusted content, and the ability to take state-changing actions.
Quarantined-LLM pattern (interpret stage): A privileged model plans and calls tools using only trusted input. A quarantined model with no tool access reads untrusted content and returns structured values, which the privileged model sees only as references.
Human approval (act stage): Require explicit confirmation for consequential actions, including externally visible ones. Show the reviewer the exact action about to run. OWASP warns that approval fatigue and invisible characters can undermine this control, so reserve it for high-impact operations.
Output controls and egress allowlisting (exfiltrate stage): Disable rendering of external images and links in agent output. Redact URLs to unapproved domains. Restrict outbound network calls to an allowlist.
Q6: What should I test during validation?
Test the controls that limit damage, not just the filters that try to catch payloads. Your validation plan should cover:
- Tool-call authorization: Inject a payload that requests a state-changing action the agent shouldn't have permission for. Verify your policy engine blocks it.
- Data-scope enforcement: Inject a payload that asks the agent to read data outside its authorized scope. Verify the agent can't access it.
- Egress restrictions: Inject a payload that tries to send data to an external URL. Verify your output controls catch it.
- Quarantine integrity: If you're using the quarantined-LLM pattern, verify the privileged model never receives raw untrusted content.
Run these tests against your deployment's own tool outputs and retrieved documents, not just generic attack libraries. Train classifiers on known attack styles, but don't stop there. The NCSC advises protections that "focus more on deterministic (non-LLM) safeguards that constrain the actions of the system."
Where to Go for More
OWASP's Top 10 for LLM Applications 2026 catalogs ingestion paths and concealment techniques. The UK NCSC's December 2025 guidance explains why prompt injection may never be totally mitigated. NIST's June 2026 announcement on adversarial prompts recommends continuous Red Teaming and operational resilience. Nasr et al.'s USENIX Security 2026 paper evaluates 12 defenses under adaptive attack.
Your validation evidence should show which legs of the trifecta your agent holds, which controls limit each attack stage, and how you tested those controls under adversarial conditions. That's the documentation your auditors and your security team both need.
