# NeMo Guardrails

> Add programmable rails to LLM chat apps to keep them safe and on-topic

- Category: Evaluation & Production Ops / Guardrails & Security
- Language: Python
- Repository: https://github.com/NVIDIA-NeMo/Guardrails
- Website: https://docs.nvidia.com/nemo/guardrails/latest/index.html
- Canonical page: https://ai-tldr.dev/tools/nemo-guardrails/

## In short

NeMo Guardrails is an open-source toolkit from NVIDIA for adding programmable guardrails (or "rails") to LLM-based conversational applications. Rails are specific ways of controlling a model's behavior, such as keeping it off certain topics, following a predefined dialog path, or moderating its output before it reaches the user. It sits in the Guardrails & Security category of the Evaluation & Production Ops tools and is open-source, written in Python.

## Overview

NeMo Guardrails is an open-source toolkit from NVIDIA for adding programmable guardrails (or "rails") to LLM-based conversational applications. Rails are specific ways of controlling a model's behavior, such as keeping it off certain topics, following a predefined dialog path, or moderating its output before it reaches the user.

It sits between your application code and the LLM, so calling it is similar to calling the model directly. You load a guardrails configuration, create an LLMRails instance, and route your chat messages through its generate method. It works with multiple LLM providers and can also wrap LangChain chains.

It fits the guardrail-frameworks category by giving developers a structured way to enforce safety and dialog policies. Rails are grouped into input, dialog, retrieval, execution, and output types, letting you reject or alter content at each stage of a request.

## What it does

- Five rail types cover the full request flow: input, dialog, retrieval, execution, and output
- Colang-based dialog rails steer the model along predefined conversational paths and standard procedures
- Protection against common LLM vulnerabilities such as jailbreaks and prompt injections
- Works with multiple LLMs, including OpenAI GPT-3.5, GPT-4, LLaMa-2, Falcon, Vicuna, and Mosaic
- Optional LangChain integration to wrap a guardrails layer around existing chains
- Async-first design with both sync and async versions of public methods (generate and generate_async)

## Getting started

Install the package with pip, then load a guardrails configuration and route your chat messages through an LLMRails instance. Requires Python 3.10, 3.11, 3.12, or 3.13.

1. Install with pip

   Install the nemoguardrails package from PyPI.

```bash
pip install nemoguardrails
```

2. Load a config and generate a response

   Load a guardrails configuration from a path, create an LLMRails instance, and call generate with chat-style messages.

```python
from nemoguardrails import LLMRails, RailsConfig

# Load a guardrails configuration from the specified path.
config = RailsConfig.from_path("PATH/TO/CONFIG")
rails = LLMRails(config)

completion = rails.generate(
    messages=[{"role": "user", "content": "Hello world!"}]
)
```

## When to use it

- Question answering over documents (RAG) where you need fact-checking and output moderation
- Domain-specific chatbots that must stay on topic and follow designed conversational flows
- Adding a safety layer to a custom LLM endpoint for safer customer interaction
- Wrapping a guardrails layer around existing LangChain chains