safelabs-eval

// Overview

safelabs-eval is an open-source red-teaming and evaluation framework for AI agents from Safe Labs AI. You point it at an agent — an HTTP chat endpoint, or any Python function that takes a string and returns a string — and it sends a library of 300 curated adversarial prompts, 30 in each of ten categories labelled ASI01 to ASI10: prompt injection, insecure output handling, excessive agency, resource management, tool-use safety, data privacy, trust boundaries, behavioural drift, scope violations, and hallucination. Every reply is scored and the run ends in a structured report.

Scoring is done by pure-Python, regex-based detectors rather than a judge model, so a full run adds no LLM grading cost and needs no change to the agent's code. Each result gets one of four verdicts — VULNERABLE, FAIL, UNCERTAIN or PASS — plus a heuristic confidence figure, which the README is careful to say is an evidence score and not a calibrated probability. Optional adapters wrap LangChain, CrewAI, AutoGen/ag2, LlamaIndex, the OpenAI Agents SDK, Google ADK and Semantic Kernel agents directly.

The project documents its own limits plainly. The category scheme is described as OWASP-inspired but is not the official OWASP Top 10 for Agentic Applications, and its ASI numbers do not match OWASP's. The prompt library is English-only and single-turn, and pattern detectors can miss paraphrased unsafe compliance or flag false positives. Agents that answer only in JSON tend to score UNCERTAIN, which is why version 0.12.0 added echo-aware scoring and an opt-in schema-aware mode for structured-output agents.

// What it does

• 300 adversarial prompts, 30 per category across ten ASI01–ASI10 agent-security categories, each tagged with a difficulty tier and, where one applies, MITRE ATLAS technique ids
• Five pattern-based detectors (prompt injection, jailbreak, data leakage, hallucination, scope violation) — no LLM calls needed to score
• Works against any HTTP agent endpoint from the CLI, or any sync or async Python callable through `run_eval()`
• Optional adapters for LangChain, CrewAI, AutoGen/ag2, LlamaIndex, OpenAI Agents SDK, Google ADK and Semantic Kernel
• JSON output for CI pipelines, with per-result indicators, plus `--prompt-id` and `--limit` to rerun a single prompt or a subset
• Schema-aware verdict mode (`--output-schema`) that gives structural verdicts for JSON-only agents

// Getting started

safelabs-eval needs Python 3.11 or newer and installs from PyPI. The core package (CLI plus HTTP adapter) has no optional dependencies; framework adapters are pip extras.

pip install safelabs-eval

# optional framework adapter, e.g. LangChain
pip install "safelabs-eval[langchain]"

safelabs list
safelabs prompts --category ASI01
safelabs prompts --severity critical

# prompt injection only
safelabs run --target http://localhost:8000/chat --category ASI01

# all ten categories, JSON for a CI pipeline
safelabs run --target http://localhost:8000/chat --category all --output json

import asyncio
from safelabs import run_eval

async def my_agent(prompt: str) -> str:
    return your_agent.run(prompt)

result = asyncio.run(run_eval(my_agent, categories=["ASI01", "ASI06"]))
result.summary()

for record in result.records:
    print(f"[{record.prompt_id}] {record.verdict.value}")

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Run a broad adversarial smoke test against an agent before it reaches production, without paying for a judge model
• Add an agent-security check to CI that fails on VULNERABLE verdicts from the JSON report
• Compare how the same agent behaves across frameworks or models using the shared adapters and prompt library
• Triage which attack categories need a deeper, human-led red-team pass

// How safelabs-eval compares

safelabs-eval alongside other open-source evaluation & red-teaming tools AI/TLDR tracks, ranked by GitHub stars.