Skip to main content
Bureau of AI
Category: Adversarial Security

Indirect Prompt Injection

Also known as: IDPI, indirect prompt injection attack
Simply put

Indirect prompt injection is a type of attack against AI systems that use large language models to read outside content, such as web pages, documents, or emails. Instead of typing a malicious instruction directly into the AI, the attacker hides instructions inside content the AI will later pull in and process, so the system may follow those hidden instructions without the user realizing it. Because the attacker never interacts with the AI directly, this route can be harder to detect than a direct prompt.

Formal definition

Indirect prompt injection (IDPI) is an adversarial technique in which malicious or manipulated instructions are embedded within untrusted external data sources that an LLM-based system ingests and processes, rather than being supplied directly by the user. As described in the evidence, such payloads may be hidden in text, code, emojis, website content, or other retrieved material, and are typically activated when the model incorporates that content into its context (for example, during retrieval, browsing, or document processing). The attack exploits the model's difficulty in distinguishing trusted system or user instructions from instructions contained in the data it is asked to handle. Note: the evidence describes the attack pattern and its risk surface but does not establish a single authoritative definition or standardized taxonomy; detection, mitigation efficacy, and scope may vary by system architecture and are not fully characterized in the sources provided.

Why it matters

Indirect prompt injection matters because it targets a structural weakness in how large language model systems operate: as described in the evidence, these systems often ingest and process untrusted external data, and the model can struggle to distinguish trusted user or system instructions from instructions hidden inside that data. As AI systems are increasingly connected to web browsing, document processing, email, and retrieval pipelines, the attack surface expands to include any content the system might pull in. This shifts the threat model away from what a user types directly and toward whatever untrusted material the system is asked to handle.

A distinguishing risk is that the attacker never interacts with the AI directly. Because the malicious instructions arrive through content the system later ingests, this route can be harder to detect than a direct prompt, and the affected user may be unaware that hidden instructions were followed. This has implications for organizations deploying LLM-based agents or assistants with access to sensitive data or downstream actions, since the pathway for manipulation lies in the data sources rather than in a monitored input field.

It is worth noting the limits of current understanding. The sources describe the attack pattern and its risk surface but do not establish a single authoritative definition or standardized taxonomy. Detection and mitigation efficacy may vary substantially by system architecture and are not fully characterized in the evidence provided, so organizations should treat controls as measures that reduce and manage this risk rather than eliminate it.

Who it's relevant to

AI Security Teams
Teams responsible for securing LLM-based deployments need to account for untrusted external data as an attack vector, not only direct user input. Because the attacker never interacts with the AI directly, monitoring approaches focused solely on user prompts may not surface this route, and controls should be treated as risk-reducing rather than fully protective given the uncertain and architecture-dependent efficacy noted in the evidence.
Developers of LLM-Integrated Applications
Engineers building systems that browse the web, process documents, or retrieve external material are directly exposed to this risk, since any ingested content could carry hidden instructions. Design decisions about how untrusted data is incorporated into the model's context are central, as the attack exploits the model's difficulty in distinguishing trusted instructions from instructions embedded in data.
Model Risk and Governance Functions
Those overseeing AI systems should recognize indirect prompt injection as a risk arising from model use in connected pipelines, relevant to how inherent and residual risk are assessed for deployed LLM applications. Given that the evidence does not establish a standardized taxonomy or fully characterized mitigations, governance treatment should use qualified language and avoid presenting any control as eliminating the risk.
Compliance and Audit Professionals
Professionals evaluating AI deployments that handle external content should understand that manipulation can occur through data sources rather than monitored inputs. This affects how controls, data provenance, and system boundaries are reviewed, while acknowledging that the scope and detectability of the attack vary by system and are not fully settled in current sources.

Inside IDPI

Untrusted content channel
The external data source—such as a web page, document, email, or retrieved database record—that the model ingests and that carries the attacker-controlled instructions. In indirect prompt injection, the malicious payload is embedded in data the model processes rather than typed directly by the user.
Instruction-data conflation
The underlying condition that enables the attack: a language model typically does not maintain a reliable boundary between instructions it should follow and content it should merely process. Injected text within ingested data can therefore be interpreted as a command.
Payload delivery vector
The mechanism by which attacker content reaches the model, commonly through retrieval-augmented generation pipelines, tool or plugin outputs, or automated agents that fetch and summarize third-party material.
Triggered behavior
The unintended model action that results, which may include disclosing sensitive context, overriding prior system instructions, invoking downstream tools, or producing manipulated output. The severity typically depends on the model's permissions and connected capabilities.
Contrast with direct prompt injection
Direct prompt injection originates from the interacting user's own input, whereas indirect prompt injection originates from third-party content the system consumes on the user's behalf, often without the user's awareness.

Common questions

Answers to the questions practitioners most commonly ask about IDPI.

Is indirect prompt injection the same as a user directly typing malicious instructions into a chatbot?
No. Direct prompt injection involves a user who interacts with the model supplying adversarial instructions themselves. Indirect prompt injection, as commonly defined, occurs when the malicious instructions are embedded in external content the model later ingests—such as a web page, document, or email—so that the injecting party and the interacting user are typically distinct. Conflating the two obscures the fact that indirect injection can compromise a user who has no intent to attack the system and may not see the injected content at all.
Does adding a strong system prompt or instruction to 'ignore untrusted content' fully prevent indirect prompt injection?
Not reliably. Instructing a model to disregard untrusted instructions can reduce the likelihood of some attacks but should not be treated as a complete control. Because many models do not maintain a hard boundary between trusted instructions and ingested data, injected content can still influence behavior. Framing a system prompt as eliminating the risk overstates its effect; it is better understood as one mitigating measure among several rather than a solution that removes the underlying vulnerability.
How should indirect prompt injection risk be reflected in a model risk assessment?
It is typically treated as a source of inherent risk arising from the model's exposure to untrusted external inputs, evaluated before controls are applied. Assessments commonly consider the pathways through which external content reaches the model, the actions the model can take on that content's behalf, and the potential impact of manipulated outputs. Residual risk is then assessed after mitigations. The distinction between inherent and residual risk should be preserved rather than collapsed into a single rating.
What controls are commonly used to reduce indirect prompt injection risk in a retrieval-augmented or tool-using system?
Frequently discussed measures include limiting the model's ability to take consequential actions without human confirmation, constraining or sandboxing tool and API permissions, separating and labeling untrusted content where the architecture allows, filtering or sanitizing ingested inputs, and monitoring outputs for anomalous instructions. No single control is generally regarded as sufficient on its own, so these are typically layered. The effectiveness of any specific control varies by system design and threat model.
How does responsibility for managing this risk map to lines of defense?
In many organizations, the first line—development and engineering teams that build and operate the system—implements input handling, tool permissioning, and monitoring controls. The second line, such as model risk or security functions, independently evaluates whether those controls are adequate relative to the assessed risk. The third line, internal audit, provides independent assurance over the overall process. These roles are complementary; assigning testing or validation solely to the team that built the system undermines the independence the model of defense is intended to provide.
How can indirect prompt injection be tested or validated as part of ongoing monitoring?
Common approaches include adversarial testing that seeds untrusted content sources with crafted instructions to observe whether the model acts on them, red-teaming of tool-enabled workflows, and monitoring production outputs for signs of manipulated behavior. Because the threat surface can change as new content sources, tools, or model versions are introduced, testing is generally treated as a recurring activity rather than a one-time exercise. Validation should distinguish whether a control works as intended (verification) from whether it adequately addresses the intended risk in operation (validation).

Common misconceptions

Indirect prompt injection is the same as a user trying to jailbreak the model.
It is commonly distinguished from direct injection or jailbreaking because the malicious instructions come from external ingested content rather than the user's own prompt. The user may be an unwitting victim rather than the attacker.
Input filtering or a well-written system prompt fully prevents the attack.
Because models typically do not reliably separate instructions from data, such controls generally reduce rather than eliminate exposure. They are commonly described as mitigations, not guarantees, and can be bypassed by novel payloads.
The risk is limited to the text the model generates back to the user.
Where a model has tool access, retrieval capability, or agentic permissions, injected instructions can influence downstream actions. The potential impact typically scales with the system's connected capabilities and privileges.

Best practices

Treat all externally retrieved or third-party content as untrusted, and design pipelines so that ingested data is not automatically granted instruction-level authority.
Constrain the model's connected capabilities using least-privilege principles, limiting tool access, permissions, and downstream actions to reduce the potential impact of a successful injection.
Apply content sanitization and structural separation between system instructions and ingested data where feasible, while recognizing these measures typically reduce rather than eliminate risk.
Add human review or confirmation steps for high-consequence actions triggered by content the model has processed from external sources.
Monitor and log model interactions with external content and tool invocations to support detection, investigation, and post-incident analysis.
Test systems against indirect injection scenarios as part of ongoing evaluation, since defenses can be bypassed by payloads not anticipated at design time.