Prompt injection: attacks, detection and defense

Prompt injection is an attack where text an LLM reads, whether typed by a user or hidden inside content it retrieves, overrides the instructions the application gave it. It is LLM01:2025, the top risk in the OWASP Top 10 for LLM applications, and MITRE ATLAS technique AML.T0051. It remains unsolved because the cause is structural: a model receives instructions and data in the same token stream and cannot reliably tell them apart. If your LLM application reads anything an outsider can write, you are exposed today; the practical question is how fast you detect an attempt and how little it can reach.

How do prompt injection attacks work?

Two paths, catalogued as ATLAS sub-techniques.

Direct injection (AML.T0051.000): the attacker is the user. They type instructions crafted to override the system prompt: role-play framing, encoded payloads, or instructions to ignore prior rules. Annoying for chatbots, dangerous the moment the model holds privileges the user does not.

Indirect injection (AML.T0051.001): the attacker plants instructions in content the model will eventually read: an email, a web page, a shared document, a calendar invite, a code comment. The user never sees the payload; the model does.

Indirect is the one that produces incidents, and EchoLeak (CVE-2025-32711) is the reference case. Aim Security showed in June 2025 that a single crafted email, with no click from the victim, could steer Microsoft 365 Copilot into reading internal content and exfiltrating it to an attacker’s server. CVSS 9.3, patched server-side by Microsoft the same month, no known in-the-wild exploitation. The pattern generalizes: the HOUYI study compromised 31 of 36 production LLM applications with black-box injections, an 86 percent success rate.

The risk compounds in agents. Simon Willison’s “lethal trifecta” names the condition to watch for: one system that combines access to private data, exposure to untrusted content, and a way to communicate externally. All three together mean an injection can become exfiltration, exactly the EchoLeak shape.

Why filters alone do not stop it

Input classifiers are useful and insufficient. Microsoft Prompt Shields flags direct jailbreaks and injected instructions in documents; Meta’s Llama Prompt Guard 2 does the same as small open models you can run in-line. Deploy one of them. Then assume it will be beaten: evaluations of guardrail-style defenses keep showing adaptive attacks that bypass every tested filter, because the attacker iterates against a static judge while the underlying ambiguity between instructions and data stays.

That is why current engineering, like Google DeepMind’s CaMeL, moves the defense out of the prompt: separate the control flow from untrusted data, and attach capabilities to data so a poisoned document cannot authorize an action the plan never intended. You do not need CaMeL itself to apply the principle. Decide what the model may do before it reads anything untrusted, and make the surrounding system enforce it.

How to detect prompt injection

Detection is a tripwire, not a boundary. Two working layers on a Microsoft estate; the concepts port to Splunk unchanged.

Platform alerts. Microsoft Defender for Cloud AI threat protection raises security alerts, including suspected prompt injection, on Azure OpenAI workloads.

Data source: SecurityAlert (Microsoft Sentinel, Defender for Cloud connector)

SecurityAlert
| where TimeGenerated > ago(7d)
| where AlertName has_any ("prompt injection", "jailbreak")
| summarize attempts = count(), first_seen = min(TimeGenerated), last_seen = max(TimeGenerated)
    by CompromisedEntity, AlertName
| order by attempts desc

Gateway verdicts. Route every model call through a gateway that calls a classifier (Prompt Shields returns an attackDetected verdict per analyzed input) and logs the result. With verdicts in a custom table, hunt for the sequence that matters: a flagged input followed by a privileged tool call from the same session.

Data source: LLMGateway_CL (custom table, your gateway’s ingestion into Log Analytics)

LLMGateway_CL
| where TimeGenerated > ago(7d) and attack_detected_b == true
| join kind=inner (LLMGateway_CL | where event_type_s == "tool_call") on session_id_s
| where TimeGenerated1 between (TimeGenerated .. TimeGenerated + 5m)
| project session_id_s, source_s, tool_name_s, TimeGenerated, TimeGenerated1

Data source note: LLMGateway_CL is your own ingestion; field names here are illustrative of that pattern and must match your schema. Tuning: security teams quoting attacks and red-team traffic are the main false positives; scope by application identity before alerting. Honest limits: a bypassed classifier logs nothing, so pair these with egress monitoring on the model’s outbound destinations. Investigation window on first deployment: search back as far as your gateway logs exist; injections predating the tripwire are invisible to it.

Mitigation: reduce the blast radius

Ordered by return on effort, aligned with the OWASP LLM01 mitigations:

1. Least-privilege tools. The model gets the narrowest tool set the task needs, per session, not per product.
2. Approval gates. A human confirms every state-changing or data-leaving action an agent takes. This single control turns most injection outcomes from incident into log entry.
3. Egress allowlists. If the model’s runtime can only reach named destinations, the trifecta loses its third leg and exfiltration dies at the network.
4. Segregate untrusted content. Mark retrieved content as data in the prompt structure, and never let it introduce new tools or goals.
5. Constrain output. Validate model output against expected formats before anything downstream consumes it.

This is the architecture work we do in LLM and agent security engagements. It is also where a written AI policy either becomes a control or stays a document, a distinction we tested in How safe are your organization’s AI policies?

Prompt injection testing checklist

Before an application ships, and again on every material change:

• Indirect payloads in every content channel the model reads: documents, email, web, tickets, calendar entries, code comments.
• Payloads that instruct tool use, not just “ignore instructions”: the goal is proving what an injected instruction can reach.
• The trifecta review: list private data access, untrusted inputs and external channels per agent; any agent holding all three gets redesigned or gated.
• Classifier regression: your last bypassed payloads, replayed after every model or filter update.
• Detection proof: a test injection must appear in your alert pipeline end to end, not just in the classifier’s own logs.

Frequently asked questions

Can prompt injection be fully prevented? No. OWASP states the root cause is intrinsic to how models process language, so the goal is bounded impact: an injection that cannot reach data or actions is a curiosity, not an incident.

What is the difference between direct and indirect prompt injection? Direct comes from the user typing at the model (AML.T0051.000). Indirect hides in content the model retrieves, so the victim never sees it (AML.T0051.001). Indirect drives the serious cases, EchoLeak included.

Do Prompt Shields or Prompt Guard make an LLM application safe? They raise the attacker’s cost and give you telemetry, and adaptive attacks still get through in testing. Treat classifiers as sensors that feed detection, with architecture controls carrying the real weight.

Which agents should be reviewed first? The ones matching the lethal trifecta: private data, untrusted content, external communication. That combination is where injection converts to exfiltration.

Bottom line

Inventory every LLM application against the trifecta this week, gate the state-changing actions, and wire classifier verdicts and platform alerts into the SOC queue. Assume the filter fails and design so that failure is boring.