Guardrails

Architecturally, guardrails are policy enforcement points in the request path of an LLM application, analogous to a WAF or DLP gateway. NVIDIA's NeMo Guardrails makes the stages explicit: input rails on the user message, retrieval rails on RAG chunks before they enter the context, dialog rails that steer conversation flow (defined in its Colang language), execution rails around tool calls, and output rails on the generated response. Each rail can reject, rewrite, redact, ask a clarifying question, or escalate to a human. Placing a check on retrieved content and tool results - not only on the user's message - is what makes guardrails relevant to indirect prompt injection.

Implementations fall into three classes. Deterministic checks: regular expressions and validators for card numbers (with a Luhn check), CPR numbers, API-key formats, URL and domain allow-lists, length limits, and JSON Schema validation of structured output. Classifier models: purpose-trained safety classifiers such as Meta's Llama Guard family (an LLM fine-tuned to label prompts and responses against a hazard taxonomy, from Llama Guard 3 aligned with the MLCommons taxonomy), prompt-injection detectors such as Azure AI Content Safety Prompt Shields, and PII detectors based on named-entity recognition. LLM-as-judge: a second model prompted with a policy that grades the output. Constrained decoding - grammar- or schema-guided generation - is a related technique that prevents malformed output at generation time instead of filtering it afterwards.

Every guardrail is a classifier with a false-positive and a false-negative rate, and both matter: over-blocking pushes users to unmanaged tools, under-blocking lets attacks through. Guardrails should be evaluated on labelled test sets, including multilingual and encoded variants, with thresholds tuned per use case. Known weaknesses include obfuscation (Base64, homoglyphs, splitting a payload across turns), low-resource languages the classifier was not trained on, the classifier itself being susceptible to prompt injection when it is an LLM, and streaming, where tokens reach the user before an output check has seen the full response; mitigations are chunked checking with the ability to retract, or buffering high-risk responses. Each extra model call also adds latency and cost.

Guardrails complement rather than replace alignment: alignment changes what the model tends to produce, guardrails constrain what the application accepts and emits, and they can be updated in hours without retraining. Neither replaces least privilege on tools, since a filter can be bypassed but a permission the agent does not hold cannot be abused. OWASP's LLM Top 10 recommends input and output filtering as a layer for LLM01 Prompt Injection, LLM02 Sensitive Information Disclosure and LLM05 Improper Output Handling, and NIST AI 600-1 treats content filtering as one of several risk-management actions for generative AI.