Overview
Prompt Flow is a suite of development tools from Microsoft for building LLM-based applications across the whole lifecycle, from first idea and prototype through testing, evaluation, and production deployment. It treats an LLM app as an executable flow that links large language models, prompts, Python code, and other tools together.
The goal is to make prompt engineering easier while keeping quality high. You can trace every interaction with an LLM while debugging, run your flow against larger datasets to measure quality and performance, wire that testing into a CI/CD pipeline, and then deploy the finished flow to the serving platform you choose. It ships with a command-line interface and a VS Code extension for visual flow design.
What it does
- Define LLM apps as executable flows that connect models, prompts, Python code, and tools in one graph (flow.dag.yaml)
- Trace and debug LLM interactions step by step while you iterate on a flow
- Evaluate flow quality and performance against larger datasets, with metrics for batch runs
- Integrate testing and evaluation into your CI/CD system to guard quality on every change
- Deploy flows to your chosen serving platform or embed them directly in your app's codebase
- Visual flow designer through the official VS Code extension, plus a pf command-line interface
Getting started
Prompt Flow runs in a local Python environment (python 3.9 to 3.11 is recommended). Install the packages, scaffold a chat flow, connect your API key, then chat with the flow from the terminal.
Install the packages
Install the core library and the bundled tools from PyPI.
pip install promptflow promptflow-toolsScaffold a chatbot flow
Initialize a flow from the chat template. This creates a folder named my_chatbot with the required files inside.
pf flow init --flow ./my_chatbot --type chatCreate a connection for your API key
Set up a connection so the flow can reach your model. For an OpenAI key, use the openai.yaml file in the my_chatbot folder.
pf connection create --file ./my_chatbot/openai.yaml --set api_key=<your_api_key> --name open_ai_connectionChat with your flow
Run the flow in interactive mode to talk to your chatbot. Press Ctrl + C to end the session.
pf flow test --flow ./my_chatbot --interactiveCommands and code are distilled from the project's own documentation — always check the official repo for the latest.
When to use it
- Prototype a chatbot or LLM app, then iterate quickly while tracing each model call to find and fix issues
- Run batch evaluations of a flow against a dataset to measure answer quality and performance before release
- Add LLM flow testing and evaluation to a CI/CD pipeline so quality is checked on every code change
- Package a finished flow and deploy it to a serving platform or embed it inside an existing application
How Prompt Flow compares
Prompt Flow alongside other open-source app frameworks tools AI/TLDR tracks, ranked by GitHub stars.
| Tool | Stars | What it does |
|---|---|---|
| LangChain | ★ 140k | A widely used Python and JavaScript framework for building LLM applications by composing models, prompts, tools, retrievers, and memory into chains. |
| LlamaIndex | ★ 50.2k | A data framework for connecting language models to your own documents and data sources, with built-in agent and retrieval (RAG) tooling. |
| Haystack | ★ 25.6k | An orchestration framework from deepset for building modular LLM pipelines and agents for search, RAG, and question answering. |
| Jina | ★ 21.9k | Jina-serve is a Python framework for building, scaling, and deploying AI services and multi-step pipelines that communicate over gRPC, HTTP, and WebSockets. |
| Prompt Flow | ★ 11.2k | Build, test, and ship production-quality LLM apps with linked flows |