WhisperLive

// Overview

WhisperLive is Collabora's near-live implementation of OpenAI's Whisper. Whisper itself transcribes a finished file; WhisperLive turns it into a service you can speak into, running a server that accepts streaming audio from a client and returns text as it goes. The same server handles pre-recorded files, so one deployment covers both live captioning and batch transcription.

Three backends sit behind the same server interface, which is the project's most practical feature: Faster-Whisper for a straightforward CPU or GPU setup, TensorRT for NVIDIA hardware where you have built a Whisper engine, and OpenVINO for efficient inference on Intel CPUs, integrated GPUs and discrete GPUs. Switching is a `--backend` flag rather than a different project, and the server takes `--max_clients` and `--max_connection_time` so a shared instance cannot be monopolised by one session.

The transcription features go past a plain text stream. It supports word-level timestamps with confidence scores, custom vocabulary and hotwords for names and jargon Whisper otherwise mangles, speaker diarization, batch inference, and raw PCM input for clients that already have audio buffers. There is an OpenAI-compatible REST interface, Chrome and Firefox browser extensions, and an iOS client. Python, MIT licensed, with published GHCR images for both CPU and GPU.

// What it does

• Streams transcription from a live microphone or a pre-recorded file through one websocket server
• Three interchangeable backends behind a single `--backend` flag: Faster-Whisper, TensorRT and OpenVINO
• Word-level timestamps with confidence scores, plus custom vocabulary and hotwords
• Speaker diarization, batch inference and raw PCM input
• OpenAI-compatible REST interface alongside the native client protocol
• `--max_clients` and `--max_connection_time` limits for a shared server
• Chrome and Firefox extensions and an iOS client; prebuilt CPU and GPU images on GHCR

// Getting started

Python 3.12 is the version the README sets up. PortAudio is a system dependency for microphone input via PyAudio; the bundled script installs it per platform.

bash scripts/setup.sh

python3.12 -m venv whisper_env
source whisper_env/bin/activate

pip install whisper-live

python3 run_server.py --port 9090 \
                      --backend faster_whisper \
                      --max_clients 4 \
                      --max_connection_time 600

python3 run_client.py --files <audio-file-name>

# GPU
docker run -it --gpus all -p 9090:9090 ghcr.io/collabora/whisperlive-gpu:latest

# CPU
docker run -it -p 9090:9090 ghcr.io/collabora/whisperlive-cpu:latest

python3 run_server.py --port 9090 --backend faster_whisper --max_clients 4 \
  --max_connection_time 600 --enable_rest \
  --cors-origins="http://localhost:8080,http://127.0.0.1:8080"

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Add live captions to a meeting, stream or call without sending audio to a hosted API
• Run one transcription service for several clients, with per-client connection limits
• Get word-level timestamps and speaker labels for editing or indexing recorded audio
• Serve Whisper on Intel iGPU/dGPU hardware through OpenVINO rather than buying NVIDIA GPUs

// How WhisperLive compares

WhisperLive alongside other open-source audio, music & voice tools AI/TLDR tracks, ranked by GitHub stars.