# Kedro

> Software-engineering discipline for data science pipelines — templates, a data catalog and a pipeline abstraction

- Category: Models & Inference / GPU & Compute Clouds
- Language: Python
- Licence: Apache-2.0
- Repository: https://github.com/kedro-org/kedro
- Website: https://kedro.org
- Canonical page: https://ai-tldr.dev/tools/kedro/

## In short

Kedro is a toolbox for production-ready data engineering and data science pipelines. It sits in the GPU & Compute Clouds category of the Models & Inference tools and is open-source, written in Python and licensed Apache-2.0. To get started: install from PyPI.

## Overview

Kedro is a toolbox for production-ready data engineering and data science pipelines. It applies software engineering practice — modularity, separation of concerns, testing, linting, documentation — to analytics code, so that pipelines come out reproducible, maintainable and modular instead of accumulating as notebooks, one-off scripts and glue code. It is an open-source Python framework hosted by the LF AI & Data Foundation.

The framework is built around a few pieces that fit together. A project template based on Cookiecutter Data Science gives every project the same shape. The Data Catalog is a set of lightweight connectors for saving and loading data across many file formats and file systems — local and network filesystems, cloud object stores and HDFS — and it includes data and model versioning for file-based systems. The pipeline abstraction resolves dependencies between pure Python functions automatically, and the companion Kedro-Viz project renders the resulting graph.

Kedro is deliberately deployment-agnostic. The same project can run on a single machine or be distributed, with additional support for deploying on Argo, Prefect, Kubeflow, AWS Batch and Databricks. The team frames the motivation as fixing the shortcomings of notebooks and glue code, improving collaboration between team members with different levels of software-engineering exposure, and making analytics code reusable.

## What it does

- Project template based on Cookiecutter Data Science, so every Kedro project has the same structure
- Data Catalog: lightweight connectors for many file formats and file systems, including cloud object stores and HDFS, with data and model versioning for file-based systems
- Pipeline abstraction that resolves dependencies between pure Python functions automatically
- Pipeline visualisation through Kedro-Viz
- Coding-standards support out of the box — pytest, Sphinx docs, ruff linting and the standard Python logging library
- Flexible deployment: single machine or distributed, with support for Argo, Prefect, Kubeflow, AWS Batch and Databricks

## Getting started

Kedro is a Python package on PyPI and conda-forge. The commands below are the ones given in the project README; the official Get Started guide covers virtual-environment setup in full.

1. Install from PyPI

   The README uses uv's pip interface.

```bash
uv pip install kedro
```

2. Or install with conda

   Kedro is published on conda-forge.

```bash
conda install -c conda-forge kedro
```

3. Install from source for the unreleased version

   To get changes ahead of an official release, install from the `main` branch.

```bash
uv pip install git+https://github.com/kedro-org/kedro@main
```

4. Work through the spaceflights tutorial

   The docs introduce the key Kedro concepts and then build a full project hands-on. Start at the tutorial: https://docs.kedro.org/en/stable/tutorials/spaceflights_tutorial/

## When to use it

- Reach for it when notebooks and one-off scripts have stopped scaling and the pipeline needs to be maintainable
- Reach for it when a team mixes data scientists and engineers and needs one agreed project structure
- Reach for it when the same pipeline has to run locally and on Argo, Prefect, Kubeflow, AWS Batch or Databricks
- Reach for it when reproducibility matters — versioned data and models, and a dependency graph you can visualise