AI Agent Guardrails Explained (2026): Filters Fail, Permissions Hold

I have spent fourteen years building infrastructure where the controls that matter are the boring ones: permissions, network boundaries, change approval. Then agents arrived, and with them a product category that promises safety by reading text. So when people ask me what AI agent guardrails are in 2026, my answer is that the word covers two very different things. One is a classifier that guesses whether text is bad. The other is a hard limit on what an agent is allowed to do. This post explains both layers, compares the main tools (Llama Guard 4, Prompt Guard 2, NeMo Guardrails, Guardrails AI, provider moderation APIs), shows what the research says guardrails cannot stop, and sketches the layered design I would actually ship.

What are AI guardrails, exactly?

A guardrail is any check that sits between a model and the world and can say no. In an agent, there are five places to put one:

1. Input filters inspect the user message (and ideally every retrieved document or tool result) before the model sees it. Typical jobs: harmful content, jailbreak attempts, PII, off-topic requests.
2. Output filters inspect what the model wrote before a human or a downstream system sees it. Same categories, plus things like leaked secrets or hallucinated citations.
3. Tool permissioning decides which tools the agent may call, with which arguments, under whose identity. This is authorisation, not classification.
4. Human approval pauses the agent before an irreversible or high-impact action and asks a person.
5. Sandboxing limits the blast radius when everything above fails: the code the agent runs, the network it can reach, the credentials it holds.

The first two are probabilistic: a model or a regex makes a call, and is sometimes wrong. The last three are deterministic. A tool that is not on the allowlist cannot be called, however persuasive the prompt. That difference is the argument of this post, the same one I made about policy as code for agents: put the decisions you cannot afford to get wrong somewhere a model cannot talk its way past.

Content classifiers: Llama Guard 4, Prompt Guard 2 and moderation APIs

Most products sold as "guardrails" are content classifiers. The open-weight reference point is Meta's Llama Guard family. Llama Guard 4 is a 12B dense model pruned from Llama 4 Scout, natively multimodal, and it labels prompts and responses against 14 hazard categories (S1 to S14) based on the MLCommons taxonomy, with S14 covering code interpreter abuse. Its model card is honest about accuracy: on Meta's English output-filtering set it reports 69% recall at an 11% false positive rate, and on single-image prompts 41% recall. That is a useful filter, not a wall.

For smaller deployments, Llama Guard 3 ships in 1B and 8B sizes on Ollama, 1.6 GB and 4.9 GB respectively, and answers with safe or unsafe followed by category codes such as S2. Llama Guard is about harmful content, not attacks on your app. For that Meta ships Prompt Guard 2, a separate jailbreak and injection detector. The LlamaFirewall paper describes two sizes, an 86M mDeBERTa-base model and a 22M DeBERTa-xsmall model, at 92.4 ms and 19.3 ms per 512-token classification on an A100.

If you would rather not host anything, the hosted options are cheap or free. OpenAI's moderation endpoint with omni-moderation-latest is free, accepts text and images, and scores 13 categories. Amazon Bedrock Guardrails charges $0.15 per 1,000 text units for content filters and denied topics and $0.10 for sensitive information filters and contextual grounding, where a text unit is up to 1,000 characters; word filters and regex PII filters are free.

Guardrail	What it is	Size or version	Price	Catches	Source
Llama Guard 4	Open-weight safety classifier	12B, text and images, S1 to S14	Self-hosted	Harmful prompts and responses	Model card
Llama Guard 3	Open-weight safety classifier	1B (1.6 GB), 8B (4.9 GB)	Self-hosted	Harmful prompts and responses	Ollama
Prompt Guard 2	Jailbreak and injection classifier	86M and 22M	Self-hosted	Known attack phrasing	LlamaFirewall paper
OpenAI moderation	Hosted classifier	omni-moderation-latest, 13 categories	Free	Harmful text and images	OpenAI docs
Bedrock Guardrails	Managed policy service	Content, topics, PII, grounding	$0.10 to $0.15 per 1,000 text units	Harmful content, PII, ungrounded answers	AWS pricing
NeMo Guardrails	Orchestration framework	0.24.1, Apache-2.0	Free	Whatever rails you wire in, plus tool-call validation	Release
Guardrails AI	Validator framework	0.11.0, Apache-2.0	Free	Schema, regex, toxicity, competitor checks	PyPI
LlamaFirewall	Agent guardrail framework	1.0.3	Free	Injection, misaligned agent reasoning, insecure code	PyPI

NeMo Guardrails vs Guardrails AI vs LlamaFirewall

Frameworks decide when to call the classifiers and what to do with the answer. Three open-source ones cover most of what I see.

NeMo Guardrails, currently 0.24.1 (16 September 2026, Python 3.10 to 3.13), wraps your LLM calls with input, output, retrieval and dialog rails defined in YAML and a small DSL called Colang. It has built-in flows for Llama Guard, and its newer IORails engine adds tool-calling rails: every tool call the model emits is checked against the tools you declared and their JSON Schemas, with no extra LLM call, and the check fails closed. NVIDIA labels IORails opt-in and experimental, and a typo in a tool flow name silently falls back to the old engine with only a log warning. Read that sentence twice before you rely on it.

Guardrails AI, at 0.11.0, is closer to a validation library: you compose validators (regex, toxic language, competitor mentions, schema checks) into a Guard and run it on inputs or outputs. This year validators moved to ordinary PyPI packages, and the project discontinued hosted remote inferencing, with a planned cutoff of 25 August 2026 per the README. If your pipeline still calls hosted validators, check it.

LlamaFirewall from Meta is the most agent-specific of the three. It bundles Prompt Guard 2, AlignmentCheck (a chain-of-thought auditor that reads the agent's reasoning for signs it has been hijacked) and CodeShield (static analysis of generated code). The PyPI package has sat at 1.0.3 since May 2025, so treat it as a reference design more than a maintained product.

Can guardrails stop prompt injection?

Not reliably, and the best evidence comes from the people who build the defences.

The LlamaFirewall paper's own AgentDojo numbers look good at first glance. With no guardrail, the attack success rate was 17.6% at 47.7% task utility. Prompt Guard 2 alone cut it to 7.5%, AlignmentCheck to 2.9%, and the two combined to 1.75%, at a utility cost (42.7%). Those were static attacks: a fixed set of injections written before the defence existed.

In October 2025, researchers from OpenAI, Anthropic, Google DeepMind and several universities published The Attacker Moves Second. They took 12 published jailbreak and prompt injection defences, most of which reported near-zero attack success, and attacked them adaptively with gradient methods, reinforcement learning, search and human red-teamers. Most defences ended up above 90% attack success. Human red-teaming succeeded in every scenario they tried. The defences had never faced an attacker who could see them and iterate.

The counterpoint is Anthropic's Constitutional Classifiers. On 10,000 synthetic jailbreak prompts, the guarded model's jailbreak success fell from 86% to 4.4%, at 23.7% extra compute and a 0.38% rise in refusals. The next generation, published in January 2026, estimates roughly 1% compute overhead if applied to Claude Opus 4.0 traffic, and reports that no red-teamer had found a universal jailbreak across 198,000 attempts. Real progress, but it targets universal jailbreaks for a narrow set of catastrophic harms, maintained by the model provider. It is not something you bolt onto your agent to stop a poisoned web page from telling it to email your customer list.

Simon Willison put the practical version best in his June 2025 post on the lethal trifecta: an agent with access to private data, exposure to untrusted content, and a way to communicate externally can be made to leak that data, and a vendor that blocks 95% of attacks is offering a failing grade by web security standards. If your agent has all three, a classifier is not the fix. Removing one leg is.

Tool permissioning and human approval: the guardrails that hold

The controls that survive adaptive attackers are the ones that do not read the prompt at all.

Tool allowlists with schemas. Give the agent the smallest set of tools for the task, each with a strict JSON Schema, and reject anything else before it executes. NeMo's tool call validation does exactly this check, and you can do it in twenty lines of your own code. The model can be fully hijacked and still only call search_tickets(query: string).

Scoped, short-lived credentials. The tool runs with a token that can read one customer's tickets, not a service account that can read all of them. This is ordinary identity work, and it decides how much a hijacked agent can actually touch. I covered the mechanics in the post on non-human identity for agents.

Separate the planner from untrusted data. The CaMeL design from Debenedetti and colleagues has a privileged model write a plan from the trusted user request, while a quarantined model handles untrusted content and can only return values, never new instructions. It solved 77% of AgentDojo tasks with provable security against injection, against 84% for an undefended system. Seven points of utility is a price I would pay for anything that touches money or customer data.

Human approval for irreversible actions. Sending email, merging code, moving money, deleting data: the agent proposes, a person approves, and the approval shows the exact arguments. Reserve it for actions that cannot be rolled back, or people stop reading. The LangGraph human-in-the-loop guide walks through one implementation.

Here is the shape of a deterministic tool gate. It is deliberately boring.

# tool_gate.py: deterministic checks that run before any tool executes.
# Python 3.12, jsonschema 4.x. No model is consulted at this layer.
import json
from jsonschema import Draft202012Validator

ALLOWED = {
    # tool name -> (JSON Schema for args, needs human approval?)
    "search_tickets": ({"type": "object",
                        "properties": {"query": {"type": "string", "maxLength": 200}},
                        "required": ["query"], "additionalProperties": False}, False),
    "send_reply":     ({"type": "object",
                        "properties": {"ticket_id": {"type": "integer"},
                                       "body": {"type": "string", "maxLength": 4000}},
                        "required": ["ticket_id", "body"], "additionalProperties": False}, True),
}

def gate(tool_name: str, raw_args: str, approve) -> dict:
    if tool_name not in ALLOWED:                      # 1. allowlist
        raise PermissionError(f"tool {tool_name!r} is not allowed")
    schema, needs_human = ALLOWED[tool_name]
    args = json.loads(raw_args)
    errors = list(Draft202012Validator(schema).iter_errors(args))
    if errors:                                        # 2. schema, fail closed
        raise ValueError(f"bad args for {tool_name}: {errors[0].message}")
    if needs_human and not approve(tool_name, args):  # 3. human approval
        raise PermissionError(f"{tool_name} rejected by reviewer")
    return args                                       # 4. caller runs it with a scoped token

Where sandboxing fits

Once an agent can run code, an allowlist of one tool called execute_python is not much of a limit. That is where isolation comes in: a container, gVisor or a microVM per task, no ambient cloud credentials, default-deny egress with an explicit allowlist of hosts. I build PandaStack, a Firecracker microVM platform for running agent code, so I have an obvious bias here; the vendor-neutral comparison is in container vs microVM vs gVisor for agent workloads. The point for guardrail design is that egress control is what breaks the third leg of the lethal trifecta. A hijacked agent in a sandbox that can only reach your own API has nowhere to send the data.

How to design layered guardrails for an agent

Put together, the design I recommend looks like this, from cheapest to most important:

• Classify inputs and outputs with a small model (Llama Guard 3 1B, Prompt Guard 2 22M) or a free API. Log every verdict so you can measure false positives.
• Treat every tool result and retrieved document as untrusted input. Classify them too, but never let that verdict be the only barrier before a sensitive action.
• Gate tools deterministically: allowlist, strict schemas, scoped credentials, rate limits per session.
• Require approval for irreversible actions, with the full arguments shown.
• Sandbox code execution with default-deny egress.
• Red-team continuously, with adaptive attacks, not a fixed list. The OWASP Top 10 for LLM and agentic applications is a reasonable checklist of what to try.

Classifiers add latency: a guard call on each input and output means up to three model calls per turn. I walk through wiring Llama Guard into NeMo Guardrails, including a tool-call allowlist and a small red-team harness that reports block rate and latency, in the companion guide: How to add LLM guardrails with Llama Guard and NeMo Guardrails.

FAQ

What is the difference between guardrails and prompt injection defence?

Guardrails is the umbrella term for any check around a model: content filters, PII redaction, schema validation, tool permissions, approvals. Prompt injection defence is one goal among them, and it is the one classifiers handle worst, because attackers can iterate against them. The defences that hold against injection are architectural: limiting tools, separating trusted and untrusted data, and blocking exfiltration paths.

Is Llama Guard enough to secure an AI agent?

No. Llama Guard classifies harmful content against a hazard taxonomy; its own model card reports 69% recall on English output filtering. It does not know whether a tool call is authorised, and it is not designed to detect injected instructions. Use it as one layer next to Prompt Guard 2 or a similar detector, and behind deterministic tool controls.

Should I use NeMo Guardrails or Guardrails AI?

NeMo Guardrails suits apps where you want the framework to sit in the request path and orchestrate several rails, including Llama Guard and tool-call validation. Guardrails AI suits apps where you want a library of validators you call yourself, especially for structured output and simple content checks. A thin wrapper around a classifier plus a tool gate is also fine.

Do guardrails add latency?

Yes. Each model-based rail is another inference call, and running input and output rails sequentially adds both to time to first token. Small classifiers help: Prompt Guard 2 22M runs in about 19 ms per 512 tokens on an A100 according to Meta. Deterministic checks such as schema validation cost microseconds.

This quarter, if I were running an agent in production, I would spend a day listing every tool it can call and every credential behind those tools, then cut both lists in half. After that I would put a small Llama Guard model on inputs and outputs, log every verdict, and run an adaptive red-team session against the result. The classifier is the part people buy; the permissions are the part that saves you.

Related: Policy as Code for AI Agents in 2026: Cedar at the Gateway, Rego in the Hook, and the 11.2% the Model Still Lets Through, How to Defend LLM Apps Against Prompt Injection: 8 Steps (2026) and OWASP Top 10 for LLMs 2026 and Agentic Apps: 20 Risks, Real Incidents.