Haystack

// Overview

Haystack is an open-source AI orchestration framework from deepset for building LLM applications in Python. You compose modular pipelines and agent workflows from individual components, with explicit control over retrieval, routing, memory, and generation rather than a single black-box call.

It is aimed at Python developers building retrieval-augmented generation (RAG) systems, semantic search, question answering, and autonomous agents. The component-based design lets you swap models, retrievers, and data stores, and inspect what happens at each step of a pipeline.

As a general agent framework, Haystack sits alongside tools like LangChain and LlamaIndex. Its focus is a transparent, customizable pipeline architecture that you can experiment with locally and then deploy to production.

// What it does

• Modular pipeline architecture: build apps from composable components for retrieval, routing, memory, and generation
• Agent support: an Agent component that decides when to call tools (like web search) and returns a response without manual routing
• Built for RAG and semantic search, plus multimodal applications and question answering
• Connects to many model providers and data stores through its component ecosystem
• Installable via pip or conda, with Docker images and nightly pre-releases available
• Backed by documentation, a Discord community, and deepset's Haystack Enterprise platform

// Getting started

Install Haystack with pip, then run a minimal agent that can call a web-search tool. The agent example below uses an OpenAI chat model and requires API keys.

pip install haystack-ai

import os

from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.tools import ComponentTool
from haystack.components.websearch import SerperDevWebSearch

os.environ["OPENAI_API_KEY"] = "<YOUR OPENAI API KEY>"
os.environ["SERPERDEV_API_KEY"] = "<YOUR SERPERDEV API KEY>"

search_tool = ComponentTool(component=SerperDevWebSearch())

basic_agent = Agent(
    chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
    system_prompt="You are a helpful web agent.",
    tools=[search_tool],
)

result = basic_agent.run(messages=[ChatMessage.from_user("When was the first version of Haystack released?")])

print(result['last_message'].text)

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Build a RAG system that answers questions over your own documents
• Add semantic search to an application instead of keyword-only matching
• Create an agent that calls tools such as web search to gather information
• Prototype an LLM pipeline locally, then deploy it to production

// How Haystack compares

Haystack alongside other open-source app frameworks tools AI/TLDR tracks, ranked by GitHub stars.