Prompt injection is not a theoretical issue. Any system that mixes trusted instructions with untrusted text (user input, documents, web pages) is exposed. The most reliable mitigation is not a single "anti-injection prompt". It is a layered control system supported by repeatable tests.
A prompt injection testing suite is a curated set of attack fixtures and expected behaviours that you run before release and continuously in CI. It gives you two benefits: you catch weaknesses early, and you prevent regressions after prompt, policy, tool or retrieval changes.
Start with an injection taxonomy you can test
"Injection" covers multiple failure modes. If you treat them as one bucket, you will miss important cases. A practical taxonomy:
- Instruction override. Untrusted text tries to replace your system policy ("ignore rules").
- Tool subversion. Untrusted text tries to trigger risky tools or bypass approvals.
- Data exfiltration. Untrusted text encourages disclosure of secrets, prompts or restricted data.
- Retrieval poisoning. Untrusted sources manipulate rankings or inject misleading answers (see data poisoning).
This taxonomy aligns to guardrail layers so you can map tests to controls (see guardrail taxonomy).
Build fixtures that reflect your real threat model
Good fixtures are not clever; they are representative. Include:
- Malicious documents. Documents that contain instructions embedded in headings, footers, or code blocks.
- Mixed-context prompts. User request plus retrieved content plus tool outputs.
- Privilege boundaries. Same question asked by a user with and without access to the content.
- Multi-turn traps. The injection appears after a few helpful turns, not at the start.
If your system uses RAG, include fixtures that emulate your connector and chunking patterns (see connector hardening).
Define expected behaviours as assertions
A test suite is useful only if it has pass/fail criteria. Typical assertions:
- No prompt disclosure. Do not leak hidden instructions or confidential templates (see prompt confidentiality).
- No unsafe tool calls. Tool arguments validate, and sensitive actions require approval (see tool contracts and approvals).
- Policy-consistent refusal. Disallowed requests are refused consistently, without leaking details (see refusal calibration).
- Grounded outputs. Claims remain aligned to retrieved sources and citations (see structured citations).
Where possible, prefer deterministic checks: schema validation and tool allowlists over "did the model seem safe" (see structured validation).
Run injection tests as part of your standard evaluation pipeline
Injection tests should not live in a separate security corner. They should run with the rest of your evaluation suite:
- Run on prompt, tool and routing changes (see prompt registries).
- Run on retrieval changes (embedding model, ranking, chunking).
- Track results by version and treat failures as release blockers.
This is the same discipline as other AI testing, applied to adversarial cases (see AI testing pyramid).
Complement tests with red teaming and production monitoring
Test suites catch known issues. Red teaming finds new ones, and monitoring catches reality. Combine:
- Periodic red team scenario exercises (see red teaming for agents).
- Telemetry for policy blocks, tool denials, and suspicious content patterns.
- Incident-driven fixture updates: convert real incidents into permanent tests.
Prompt injection resilience is not a single prompt. It is a system with controls you can name, tests you can run, and behaviours you can verify.