NVIDIA · 2024-12-15 · major
Garak — LLM Vulnerability Scanner by NVIDIA
NVIDIA's open-source LLM vulnerability scanner. Probes for prompt injection, data leakage, hallucination, toxicity, and jailbreaks across 50+ attack families.
Automated red-teaming for LLMs — finds vulnerabilities before attackers do.
Key specs
| GitHub stars | 2.5k+ |
|---|---|
| Attack probes | 50+ |
What is it?
Garak is NVIDIA's open-source tool for probing LLM vulnerabilities. It tests models against prompt injection, jailbreaks, data leakage, and harmful content generation using 50+ attack families.
How does it work?
You point Garak at any LLM endpoint (OpenAI, local models, HuggingFace). It runs thousands of adversarial prompts organized by attack type, logs which ones succeed, and generates a vulnerability report.
Why does it matter?
Before deploying an LLM, you need to know its weaknesses. Garak automates what would take a red team weeks — systematic probing across known attack vectors.
Who is it for?
Security teams, ML engineers deploying LLMs to production.
Try it
pip install garak && garak --model_type openai --model_name gpt-4