Detoxify

// Overview

Detoxify is an open-source Python library from Unitary that classifies text for toxicity. It wraps pretrained transformer models (built on Hugging Face Transformers and PyTorch Lightning) that return scores for categories such as toxicity, insults, threats, obscenity, and identity attacks.

It ships three main model variants trained on the Jigsaw competition datasets: `original` (Toxic Comment Classification), `unbiased` (trained to minimise unintended bias against identity mentions), and `multilingual` (covering English, French, Spanish, Italian, Portuguese, Turkish, and Russian). Smaller Albert-based variants (`original-small`, `unbiased-small`) are also available.

It fits the guardrails and content-moderation space: teams use it to flag harmful user content or to screen the inputs and outputs of LLM-powered apps. The authors note the models are intended for research and to assist human moderators, not as a sole automated judge.

// What it does

• Pretrained models that score text across multiple categories including toxicity, insults, threats, obscenity, and identity attacks
• Three model families: `original`, `unbiased` (reduced identity bias), and `multilingual` (7 languages)
• Lightweight `original-small` and `unbiased-small` Albert-based variants for smaller footprints
• Simple `predict()` API that accepts a single string or a list of strings
• Runs on CPU by default, with an optional `device` argument to target a GPU via any torch.device input
• Built on Hugging Face Transformers and PyTorch Lightning

// Getting started

Install the package from PyPI, then load a model and call predict() on your text.

pip install detoxify

from detoxify import Detoxify

# each model takes in either a string or a list of strings
results = Detoxify('original').predict('example text')

results = Detoxify('unbiased').predict(['example text 1', 'example text 2'])

model = Detoxify('original', device='cuda')

import pandas as pd
print(pd.DataFrame(results, index=input_text).round(5))

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Flagging toxic, abusive, or hateful user comments so human moderators can review them faster
• Screening inputs and outputs of LLM-powered applications for harmful content
• Scoring multilingual user content across the 7 languages the multilingual model supports
• Research on toxicity detection and bias, or fine-tuning on a domain-specific dataset

// How Detoxify compares

Detoxify alongside other open-source guardrails & security tools AI/TLDR tracks, ranked by GitHub stars.