# garak

> Scan LLMs for prompt injection, jailbreaks, and data leakage from the command line

- Category: Evaluation & Production Ops / Evaluation & Red-Teaming
- Language: Python
- Repository: https://github.com/NVIDIA/garak
- Website: https://garak.ai
- Canonical page: https://ai-tldr.dev/tools/garak/

## In short

garak is a command-line tool from NVIDIA that checks whether a large language model can be pushed into behaving in ways you don't want. It sits in the Evaluation & Red-Teaming category of the Evaluation & Production Ops tools and is open-source, written in Python. To get started: install with pip.

## Overview

garak is a command-line tool from NVIDIA that checks whether a large language model can be pushed into behaving in ways you don't want. It runs a library of probes against a target model to test for issues like prompt injection, jailbreaks, data leakage, hallucination, misinformation, and toxic output, then reports which probes the model failed and how often.

If you've used network or security scanners like nmap or Metasploit, garak does a similar job but for LLMs and dialog systems. It combines static, dynamic, and adaptive probes, and works with many model backends including Hugging Face Hub, the OpenAI API, Replicate, AWS Bedrock, LiteLLM, local GGUF models via llama.cpp, and most REST-accessible endpoints.

As an evaluation framework, garak fits the red-teaming and safety-testing stage of building with LLMs. It's aimed at engineers and security teams who want repeatable, automated checks on a model's weaknesses rather than ad-hoc manual prompting.

## What it does

- 100+ probes covering prompt injection, jailbreaks (such as DAN), data leakage, toxicity, misinformation, and hallucination
- Works across many backends: Hugging Face Hub, OpenAI API, Replicate, AWS Bedrock, LiteLLM, GGUF/llama.cpp, and most REST endpoints
- Runs all known probes by default, or lets you target a specific probe family or single probe (e.g. promptinject, lmrc.SlurUsage)
- Detector-based scoring that marks undesirable responses as FAIL and reports the failure rate per probe
- Detailed logging to garak.log and a per-run .jsonl file, plus an analysis script to surface the prompts that caused the most hits
- Built-in probe listing with --list_probes to explore available attacks

## Getting started

garak is a command-line tool developed on Linux and macOS. Install it from PyPI, then point it at a model to start scanning.

1. Install with pip

   Grab the latest release from PyPI.

```bash
python -m pip install -U garak
```

2. List available probes

   See every probe garak can run before launching a scan.

```bash
garak --list_probes
```

3. Scan an OpenAI model

   Set your API key, then run the encoding probes against a target model. Use --target_type for the model family and --target_name for the exact model.

```bash
export OPENAI_API_KEY="sk-123XXXXXXXXXXXX"
python3 -m garak --target_type openai --target_name gpt-5-nano --probes encoding
```

4. Test a Hugging Face model for a jailbreak

   Load a model from the Hub and run a specific probe, such as the DAN 11.0 jailbreak.

```bash
python3 -m garak --target_type huggingface --target_name gpt2 --probes dan.Dan_11_0
```

## When to use it

- Red-teaming a model before deployment to find prompt-injection and jailbreak weaknesses
- Comparing how susceptible different model versions are to the same class of attack (e.g. encoding-based injection)
- Adding automated safety and vulnerability checks to an LLM project's testing pipeline
- Auditing a hosted or REST-accessible model for data leakage and toxic-output risks