NeMo Guardrails

// Overview

NeMo Guardrails is an open-source toolkit from NVIDIA for adding programmable guardrails (or "rails") to LLM-based conversational applications. Rails are specific ways of controlling a model's behavior, such as keeping it off certain topics, following a predefined dialog path, or moderating its output before it reaches the user.

It sits between your application code and the LLM, so calling it is similar to calling the model directly. You load a guardrails configuration, create an LLMRails instance, and route your chat messages through its generate method. It works with multiple LLM providers and can also wrap LangChain chains.

It fits the guardrail-frameworks category by giving developers a structured way to enforce safety and dialog policies. Rails are grouped into input, dialog, retrieval, execution, and output types, letting you reject or alter content at each stage of a request.

// What it does

• Five rail types cover the full request flow: input, dialog, retrieval, execution, and output
• Colang-based dialog rails steer the model along predefined conversational paths and standard procedures
• Protection against common LLM vulnerabilities such as jailbreaks and prompt injections
• Works with multiple LLMs, including OpenAI GPT-3.5, GPT-4, LLaMa-2, Falcon, Vicuna, and Mosaic
• Optional LangChain integration to wrap a guardrails layer around existing chains
• Async-first design with both sync and async versions of public methods (generate and generate_async)

// Getting started

Install the package with pip, then load a guardrails configuration and route your chat messages through an LLMRails instance. Requires Python 3.10, 3.11, 3.12, or 3.13.

pip install nemoguardrails

from nemoguardrails import LLMRails, RailsConfig

# Load a guardrails configuration from the specified path.
config = RailsConfig.from_path("PATH/TO/CONFIG")
rails = LLMRails(config)

completion = rails.generate(
    messages=[{"role": "user", "content": "Hello world!"}]
)

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Question answering over documents (RAG) where you need fact-checking and output moderation
• Domain-specific chatbots that must stay on topic and follow designed conversational flows
• Adding a safety layer to a custom LLM endpoint for safer customer interaction
• Wrapping a guardrails layer around existing LangChain chains

// How NeMo Guardrails compares

NeMo Guardrails alongside other open-source guardrails & security tools AI/TLDR tracks, ranked by GitHub stars.