Pipecat

// Overview

Pipecat is an open-source Python framework for building real-time voice and multimodal conversational agents, maintained by Daily and the community under a BSD-2-Clause license. It orchestrates audio and video, AI services, transports and conversation pipelines, so the code you write is the part that makes your agent distinctive rather than the media plumbing underneath.

The unit of composition is a pipeline, and every pipeline is an agent. You can run a single voice agent or build a full multi-agent system where specialists hand off to each other, fan out in parallel, or run as sidecar workers over a shared bus — locally, or distributed across processes and machines. Latency is the design constraint throughout: transports include WebRTC (Daily, LiveKit, Vonage, or Pipecat's own SmallWebRTCTransport), WebSockets, WhatsApp, and telephony serializers for Twilio, Telnyx, Plivo, Exotel, Genesys and Vonage.

Services are pluggable across every layer — more than twenty speech-to-text providers, a similar range of LLMs, a long list of text-to-speech engines, and speech-to-speech models such as OpenAI Realtime, Gemini Multimodal Live, AWS Nova Sonic, Grok Voice Agent and Ultravox. Around the core sit Pipecat Flows for structured conversations, client SDKs for JavaScript, React, React Native, Swift, Kotlin, C++ and ESP32, the Voice UI Kit, the Whisker debugger and the Tail terminal dashboard.

// What it does

• Voice-first design that integrates speech recognition, text-to-speech and conversation handling out of the box.
• Composable pipelines: build complex behavior from modular processors, and treat each pipeline as an agent.
• Multi-agent ready — compose agents with handoff, parallel fan-out, sidecar workers, or distributed deployments over a shared bus.
• A wide service catalog across STT, LLM, TTS, speech-to-speech, video avatars, memory, vision and image generation, swappable without rewriting the pipeline.
• Real-time transports including WebRTC, WebSockets, WhatsApp and telephony serializers for major carriers.
• Audio processing utilities such as Silero VAD and Krisp Viva, Koala, ai-coustics and RNNoise noise filters.
• OpenTelemetry and Sentry integrations for tracing and metrics, plus the Whisker real-time pipeline debugger.
• Client SDKs for JavaScript, React, React Native, Swift, Kotlin, C++ and ESP32, so one agent serves web, mobile and hardware.

// Getting started

Pipecat is a Python package (3.11 minimum, 3.12+ recommended) installed with uv. The quickest path is the bundled CLI, which scaffolds a runnable phone or web/mobile bot; you can also add the module to an existing project and wire the pipeline yourself.

curl -LsSf https://astral.sh/uv/install.sh | sh

uv tool install "pipecat-ai[cli]"
pipecat init

uv init my-pipecat-app
cd my-pipecat-app
uv add pipecat-ai
uv add "pipecat-ai[option,...]"
cp env.example .env

git clone https://github.com/pipecat-ai/pipecat.git
cd pipecat
uv sync --group dev --all-extras --no-extra gstreamer --no-extra local
uv run pre-commit install
uv run pytest

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Build streaming voice assistants that hold a natural back-and-forth conversation over WebRTC or WebSockets.
• Put an agent on the phone through Twilio, Telnyx, Plivo or Vonage for intake, support lines and guided flows.
• Design structured, stateful dialogs with Pipecat Flows when a conversation needs predefined or dynamically generated paths.
• Compose a multi-agent system where a triage agent hands off to specialists, or fan work out in parallel across pipelines.
• Ship voice to hardware and mobile with the ESP32, Swift, Kotlin and React Native client SDKs against one server-side agent.

// How Pipecat compares

Pipecat alongside other open-source agent frameworks & builders tools AI/TLDR tracks, ranked by GitHub stars.