Prevent Prompt Injection Attacks: Practical Defences for AI Systems
Input sanitisation fails against modern language models; you must separate system instructions from user data using strict architectural boundaries.
Isolate user input from system prompts using delimiters. Implement output validation to catch deviations. Apply least-privilege access to external tools. Avoid relying solely on prompt engineering. Use input filtering to remove malicious patterns. Combine these layers to reduce attack surface significantly.
The Illusion of Prompt Engineering
Relying on clever wording to stop attackers is a fragile strategy. Many developers believe that adding phrases like "ignore previous instructions" or "you must not execute commands" creates a secure boundary. This approach fails because large language models process text as a sequence of tokens without inherent understanding of intent or hierarchy.
An attacker can craft a prompt that overrides your safety instructions by exploiting the model's tendency to prioritise recent or specific directives. This phenomenon is known as prompt injection. It occurs when malicious input alters the model's behaviour, causing it to execute unintended actions or reveal sensitive information.
The model does not distinguish between your system instructions and the user's input in the way a traditional application distinguishes code from data. It sees everything as text. Therefore, textual barriers are permeable. You must treat the model as an untrusted component that can be manipulated. The goal is not to convince the model to be safe, but to architect a system where the model cannot act maliciously even if it is tricked.
Structural Separation of Concerns
The most effective defence is to keep system instructions and user data in separate contexts. If your application architecture allows this, do not concatenate user input directly into the system prompt. Instead, pass user data through a dedicated input channel that the model treats as distinct from its operational guidelines. This is similar to how databases use prepared statements to prevent SQL injection. The database engine understands the difference between the query structure and the data values. You need a similar semantic separation in your AI integration.
If your API does not support separate fields for system and user messages, you must use robust delimiters. Wrap user input in clear markers that the model is instructed to treat as inert content. For example, you might instruct the model to treat text between """ and """ as data only. However, this is not foolproof. A sophisticated attacker can include the delimiter characters in their input to break the structure. This is known as a delimiter collision. To mitigate this, use unique, complex delimiters that are unlikely to appear in normal user input, and validate that the delimiters are balanced before processing.
Input Filtering and Sanitisation
Filtering user input before it reaches the model adds a layer of defence. You can scan for known injection patterns, such as commands to ignore instructions or requests to output internal data. Regular expressions can catch simple attempts. More advanced systems use a secondary, smaller model to classify input as benign or malicious. This classifier acts as a gatekeeper, blocking suspicious requests before they reach the primary application model.
However, input filtering has limitations. Language models are highly adaptable. Attackers can obfuscate their prompts using synonyms, encoding, or indirect phrasing to bypass static filters. A filter that blocks "ignore instructions" may miss "disregard prior guidance". Therefore, input filtering should be one layer in a defence-in-depth strategy, not the sole line of defence. It reduces the volume of obvious attacks but does not eliminate the risk of novel or obfuscated injections. For deeper insights on adversarial inputs, see our guide on jailbreaking AI chatbots.
Output Validation and Guardrails
Checking the model's output is critical because it catches successful injections that passed through input filters. You must validate that the response matches the expected format and content. If your application expects a JSON object, verify that the output is valid JSON. If it expects a short summary, check the length and relevance. Any deviation should trigger an error or rejection.
You can also use a secondary model to audit the primary model's output. This auditor model checks for policy violations, such as revealing internal instructions or executing unauthorized commands. This adds latency and cost, but it significantly improves security. Output validation ensures that even if the model is tricked into generating malicious content, that content does not reach the end user or trigger downstream actions. This approach aligns with broader principles discussed in AI governance.
Least Privilege for AI Agents
If your application uses AI agents that can interact with external tools, such as databases or APIs, you must restrict their permissions. An agent should only have the minimum access required to perform its task. Do not give an agent admin rights to a database if it only needs to read user profiles. Use service accounts with scoped permissions. If an injection succeeds, the attacker can only execute actions within that narrow scope.
Consider the principle of separation of duties. The model should not directly execute commands. Instead, it should generate a request that a separate, trusted system validates and executes. This intermediary system checks the request against security policies before carrying it out. This architecture prevents the model from becoming a direct vector for command execution. For more on building secure autonomous systems, read about AI agents.
See also: Training Data Leakage Mistakes and How to Fix Them
What Does Not Work Alone
Many developers believe that using a "system prompt" that explicitly forbids certain actions is enough. It is not. The model can be coerced into ignoring these rules through complex prompting techniques. Similarly, simple keyword filtering is easily evaded. You must assume that the model will eventually be tricked. Your architecture must remain secure even when the model behaves incorrectly. This mindset shift is fundamental to securing AI applications.
Practical Prevention Hierarchy
| Measure | Effort | What it stops |
|---|---|---|
| Structural Separation | Medium | Direct instruction override |
| Output Validation | High | Successful injection execution |
| Least Privilege | Medium | Unauthorized tool usage |
| Input Filtering | Low | Obvious injection patterns |
| Delimiter Encoding | Low | Simple delimiter collisions |
Structural separation removes the most risk by preventing the model from conflating instructions with data. Output validation catches failures that slip through. Least privilege limits the blast radius. Input filtering provides a basic barrier. Delimiter encoding is a low-cost mitigation for simple attacks. Prioritise these measures based on your application's risk profile. For continuous assessment of these defences, consider AI red teaming.
Checklist for Today
- Separate system instructions from user input in your API calls.
- Implement output validation to check format and content.
- Restrict AI agent permissions to the minimum necessary.
Key takeaways
- Structural separation of instructions and data is more effective than textual warnings.
- Output validation detects successful injections that input filters missed.
- Least-privilege controls limit the damage if an injection succeeds.
Treat the model as untrusted code that can be manipulated. Architect your system to remain secure even if the model is tricked.
Frequently asked questions
Can I stop prompt injection with a single rule?
No, no single rule is sufficient. You need multiple layers of defence, including structural separation and output validation.
Does using a smaller model help?
Smaller models may be less susceptible to complex injections, but they are not immune. Security depends on architecture, not just model size.
What if my API doesn't support separate prompts?
Use robust delimiters and validate their structure. Consider wrapping the model in a proxy that enforces separation.
How often should I update my defences?
Continuously. Attack techniques evolve rapidly. Regularly review your inputs, outputs, and architecture for new vulnerabilities.
How this guide was produced: written by the TechSec Observer editorial team with AI assistance, checked against the public references listed below, and reviewed when the facts change. See our editorial policy or report an error.
