A customer's support agent read a ticket containing hidden instructions and emailed out internal data. How do you respond and prevent it?
Answer it out loud first, then compare. That's what makes it stick.
This is indirect prompt injection. I contain first: disable the email tool or require human approval, rotate anything exposed, and preserve logs. Then I assess what leaked and notify the customer's security team. The durable fix is architectural: assume the model can be tricked, so give it least privilege, separate untrusted content from instructions, require approval for outbound actions, and filter outputs. I add the attack as a regression test and red-team similar paths.
- Contain: cut the tool, preserve logs
- Scope the damage; tell security
- Root cause: untrusted text plus powerful tool
- Least privilege, approvals, output controls
- Red-team and add regression tests
Treat it as a security incident, not a prompt bug.
1. Contain (minutes)
- Disable or gate the outbound tool (email, webhook, ticket update). Switch to human approval for external sends.
- Preserve traces, the offending ticket and tool-call logs. Do not scrub evidence.
2. Scope
- From logs: which tickets contained the payload, which tool calls fired, what data left, to whom. Involve the customer's security contact early; it may trigger their breach process.
3. Root cause
- Indirect injection means the attacker's text arrived through data (a ticket, web page, PDF), not the user. OWASP lists prompt injection first in its 2025 LLM top 10 and says it is unclear whether any foolproof prevention exists, so design assuming failure.
- The real flaw is usually the combination: untrusted input + sensitive data access + an outbound channel. Remove any one leg.
4. Defences (layered)
- Least privilege: the agent only reads the current ticket's data, not the whole CRM; tools are scoped per task.
- Do privileged work in code, not in the model: recipient allow-lists, field-level redaction, send limits.
- Human approval for high-risk actions such as external email.
- Segregate and label untrusted content so it carries less authority; this reduces but does not eliminate risk.
- Input and output filtering for known patterns and for data that should never leave (PII, internal URLs, secrets).
- Monitoring: alert on unusual recipients or large payloads.
5. Verify
- Add the payload and variants (hidden text, other languages, encoded) to an adversarial test set, run it in CI, and red-team adjacent tools.
Tone with the customer: factual timeline, what was exposed, what is changed, what you will test.
- The customer asks if you can guarantee this can never happen again. What do you say?
- Which tools would you give human approval and which would you leave automatic?
- How do you test for injection in CI?
- LLM01:2025 Prompt Injection (OWASP GenAI) (docs, 2025) Defines indirect injection and lists mitigations: least privilege, human approval, segregating external content, filtering and adversarial testing.
- Senior AI interviews don't test what you know, they test what breaks at 2am (prep, unknown) Argues senior AI interviews probe production failure and recovery scenarios.