# Prefect

> Turn Python scripts into scheduled, observable data and ML pipelines

- Category: Models & Inference / GPU & Compute Clouds
- Language: Python
- Repository: https://github.com/PrefectHQ/prefect
- Website: https://www.prefect.io
- Canonical page: https://ai-tldr.dev/tools/prefect/

## In short

Prefect is a workflow orchestration framework for building data pipelines in Python. You add `@flow` and `@task` decorators to ordinary functions, and Prefect handles scheduling, retries, caching, and event-based automations so a plain script can run as a production workflow. It sits in the GPU & Compute Clouds category of the Models & Inference tools and is open-source, written in Python.

## Overview

Prefect is a workflow orchestration framework for building data pipelines in Python. You add `@flow` and `@task` decorators to ordinary functions, and Prefect handles scheduling, retries, caching, and event-based automations so a plain script can run as a production workflow.

It is aimed at data and ML teams who already write Python and want their pipelines to recover from failures, run on a schedule, and be observable without rewriting everything in a separate DSL. Workflow runs are tracked in a UI that you can host yourself with a self-hosted Prefect server or use through the managed Prefect Cloud dashboard.

Within the compute orchestration space, Prefect sits alongside tools like Airflow and Dagster but keeps the authoring experience close to regular Python code, using decorators instead of static DAG definitions.

## What it does

- Define workflows by decorating Python functions with `@flow` and `@task` - no separate DAG syntax
- Built-in scheduling, caching, retries, and event-based automations
- Self-hosted Prefect server with a UI at http://localhost:4200, or managed Prefect Cloud
- Turn any flow into a scheduled deployment with `.serve()` and a cron expression
- Run deployments manually from the UI or CLI, or trigger them in response to events
- Lightweight `prefect-client` package for ephemeral environments talking to Cloud or a remote server

## Getting started

Prefect requires Python 3.10+. Install it, write a flow, then start a server to watch runs in the UI.

1. Install Prefect

   Install the latest version with pip (or use uv).

```bash
pip install -U prefect
```

2. Write and run a flow

   Decorate your functions with `@flow` and `@task`, then run the file like any Python script.

```python
from prefect import flow, task
import httpx

@task(log_prints=True)
def get_stars(repo: str):
    url = f"https://api.github.com/repos/{repo}"
    count = httpx.get(url).json()["stargazers_count"]
    print(f"{repo} has {count} stars!")

@flow(name="GitHub Stars")
def github_stars(repos: list[str]):
    for repo in repos:
        get_stars(repo)

if __name__ == "__main__":
    github_stars(["PrefectHQ/prefect"])
```

3. Start the server and open the UI

   Fire up a Prefect server and open the UI at http://localhost:4200 to see what happened.

```bash
prefect server start
```

4. Schedule it as a deployment

   Replace the last line of your script to serve the flow on a cron schedule - here, every minute.

```python
if __name__ == "__main__":
    github_stars.serve(
        name="first-deployment",
        cron="* * * * *",
        parameters={"repos": ["PrefectHQ/prefect"]}
    )
```

## When to use it

- Schedule recurring ETL or data ingestion jobs to run every minute, hour, or day
- Add retries and error handling to flaky pipeline steps that hit external APIs
- Orchestrate multi-step ML pipelines and monitor their runs in a single UI
- Trigger workflows in response to events instead of fixed schedules