Prompt injection

The root cause is architectural. An LLM receives one token sequence in which the system prompt, user turns, retrieved documents and tool results are separated only by chat-template markers and formatting conventions; attention operates over all of it, and there is no equivalent of a parameterised query that forces a span to be treated as inert data. Role tokens and training give privileged segments more weight, but the separation is statistical, which is why the analogy with SQL injection is instructive but misleading: SQL injection has a complete fix (prepared statements), prompt injection currently does not. Greshake et al. (2023) formalised indirect prompt injection, in which the payload arrives through content the application retrieves - web pages, emails, PDFs, tool descriptions - so the attacker never interacts with the system directly.

Payloads pursue a small set of goals: goal hijacking (do something else), prompt leaking (reveal the system prompt, see OWASP LLM07), data exfiltration, and unauthorised tool use. A well-known exfiltration channel is output rendering: the model is told to emit a Markdown image whose URL contains conversation data in the query string, and the client leaks it when it fetches the image; blocking external image rendering or enforcing a strict content security policy closes that path. Agents with memory add persistence, since an injected instruction can be written into long-term memory and replayed in later sessions.

Mitigations at the model level include instruction-hierarchy training (Wallace et al., 2024), which teaches models to privilege system and developer messages over tool output, and spotlighting (Hines et al., 2024), which marks untrusted text with delimiters, interleaved datamarks or encoding. Classifier-based detectors screen retrieved content. All of these reduce success rates but are bypassed by adaptive attacks, as benchmarks such as AgentDojo show. Stronger guarantees come from system design: the dual-LLM pattern, where a privileged model never sees untrusted text and a quarantined model processes it without tool access; CaMeL (Debenedetti et al., 2025), which extracts a control flow from the trusted query and tracks capabilities on data derived from untrusted sources; and plain least privilege, human confirmation and egress restrictions, which limit what a successful injection can do.

OWASP ranks prompt injection as LLM01:2025 and includes jailbreaking as a subtype; NIST AI 100-2 classifies it as an attack on generative AI with direct and indirect variants. It differs from data poisoning, which alters the model's weights during training, and from excessive agency, which is the design weakness that determines the blast radius. For an EU deployer, a successful injection that exposes personal data is a personal data breach under GDPR Art. 33 with the usual 72-hour notification clock, and for high-risk AI systems, EU AI Act Art. 15(5) requires resilience against attempts by unauthorised third parties to alter use or outputs by exploiting system vulnerabilities.