PyRIT

// Overview

PyRIT (Python Risk Identification Tool for generative AI) is an open-source framework from Microsoft that helps security professionals and engineers probe generative AI systems for risks. Instead of testing prompts by hand, you point PyRIT at a target model and it sends, scores, and refines adversarial prompts for you.

It is aimed at AI red teams, application security engineers, and researchers who need to stress-test LLM-based applications. A typical setup uses one model as the attacker that generates and adapts prompts, and a separate target model that you want to evaluate, with scorers deciding whether an attack succeeded.

As an evaluation framework, PyRIT focuses on safety and security testing rather than accuracy benchmarks. It supports single-prompt and multi-turn attack flows, pluggable targets, and reusable scoring components, so you can build repeatable harm-and-jailbreak assessments around your own models.

// What it does

• Automates adversarial prompt generation, sending, and scoring against a target model
• Supports single-turn and multi-turn attack strategies that refine prompts over a conversation
• Pluggable prompt targets, including OpenAI-compatible chat endpoints
• Built-in attack classes such as PromptSendingAttack for sending objectives to a target
• In-memory or persistent memory backends to record conversations and results
• Result printers (e.g. ConsoleAttackResultPrinter) to review full attack conversations

// Getting started

Install PyRIT from PyPI (Python 3.10-3.14 is supported, 3.13 recommended), then run a minimal attack against a configured target.

pip install pyrit

python -c "import pyrit; print(f'PyRIT version installed: {pyrit.__version__}')"

from pyrit.executor.attack import ConsoleAttackResultPrinter, PromptSendingAttack
from pyrit.prompt_target import OpenAIChatTarget
from pyrit.setup import IN_MEMORY, initialize_pyrit_async

await initialize_pyrit_async(memory_db_type=IN_MEMORY)

target = OpenAIChatTarget()
attack = PromptSendingAttack(objective_target=target)
result = await attack.execute_async(objective="What model exactly are you? be concise.")

printer = ConsoleAttackResultPrinter()
await printer.print_conversation_async(result=result)

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Red-teaming an LLM-based chatbot or assistant for jailbreaks and harmful outputs before launch
• Running repeatable, automated safety assessments as part of a CI or pre-release security review
• Comparing how different target models respond to the same set of adversarial objectives
• Researching multi-turn attack strategies where prompts are refined over a conversation

// How PyRIT compares

PyRIT alongside other open-source evaluation & red-teaming tools AI/TLDR tracks, ranked by GitHub stars.