# Whisper

> General-purpose speech recognition that transcribes and translates audio in many languages

- Category: Multimodal Generation / Audio, Music & Voice
- Language: Python
- Repository: https://github.com/openai/whisper
- Website: https://openai.com/research/whisper
- Canonical page: https://ai-tldr.dev/tools/whisper/

## In short

Whisper is an open-source speech recognition model from OpenAI. It is trained on a large, diverse audio dataset and handles several tasks at once: multilingual speech recognition, speech translation, and spoken language identification. It sits in the Audio, Music & Voice category of the Multimodal Generation tools and is open-source, written in Python.

## Overview

Whisper is an open-source speech recognition model from OpenAI. It is trained on a large, diverse audio dataset and handles several tasks at once: multilingual speech recognition, speech translation, and spoken language identification. Under the hood it is a Transformer sequence-to-sequence model that treats these tasks as one sequence of tokens to predict, so a single model replaces many stages of a traditional speech pipeline.

It is aimed at developers and researchers who need to turn audio into text without calling a hosted API. You install it as a Python package and run it locally, choosing from six model sizes that trade off speed against accuracy and VRAM. There are English-only variants (.en) and multilingual variants, plus a turbo model optimized for faster transcription.

In the speech and audio category, Whisper covers two common needs in one tool: transcribing speech to text in its original language, and translating non-English speech into English. The multilingual models handle translation, while the default turbo model is tuned for fast English transcription.

## What it does

- Multitasking model: multilingual speech recognition, speech translation, and language identification in one model
- Six model sizes (tiny, base, small, medium, large, turbo) trading off speed, accuracy, and VRAM (~1 GB to ~10 GB)
- English-only .en variants that tend to perform better for the smaller tiny and base sizes
- Command-line tool that transcribes common audio formats like .flac, .mp3, and .wav
- Translate non-English speech into English using the multilingual models with --task translate
- Optional language hint via --language so you can target a known input language

## Getting started

Install the package with pip, make sure ffmpeg is available, then run the whisper command on an audio file.

1. Install Whisper

   Install the latest release from PyPI. The codebase targets Python 3.8-3.11 and recent PyTorch versions.

```bash
pip install -U openai-whisper
```

2. Install ffmpeg

   Whisper requires the ffmpeg command-line tool, available from most package managers.

```bash
# on Ubuntu or Debian
sudo apt update && sudo apt install ffmpeg

# on MacOS using Homebrew
brew install ffmpeg

# on Windows using Chocolatey
choco install ffmpeg
```

3. Transcribe an audio file

   Run whisper on one or more audio files. This example uses the turbo model, which works well for English transcription.

```bash
whisper audio.flac audio.mp3 audio.wav --model turbo
```

4. Translate non-English speech to English

   The turbo model is not trained for translation. Use a multilingual model with --task translate and specify the input language.

```bash
whisper japanese.wav --model medium --language Japanese --task translate
```

## When to use it

- Transcribe podcasts, interviews, or meeting recordings into text locally without a paid API
- Translate non-English audio into English text using the multilingual models
- Add captions or subtitles to videos by transcribing their audio tracks
- Identify the spoken language of an audio clip before further processing