█

AI/TLDR

Buy me a coffee
Buy me a coffee

Sacred

Configure, organize, log, and reproduce your machine learning experiments

Experiment TrackingOpen source
Language
Python
$pip install sacred
IN SHORT

What is Sacred?

Sacred is a Python library that handles the repetitive work around a machine learning experiment so you can focus on the research itself. It helps you keep track of every parameter, run the same experiment under different settings, save each run's configuration, and reproduce results later. It sits in the Experiment Tracking category of the Training & Fine-Tuning tools and is open-source, written in Python.

Overview

Sacred is a Python library that handles the repetitive work around a machine learning experiment so you can focus on the research itself. It helps you keep track of every parameter, run the same experiment under different settings, save each run's configuration, and reproduce results later.

It is aimed at researchers and ML engineers who run many experiments and need a reliable record of what was run and how. You wrap your training script in a Sacred Experiment, define parameters in a config function, and Sacred injects those values into your code and gives you a command-line interface to override them.

As an experiment-tracking tool, Sacred focuses on capturing runs and their metadata. Observers can write run details such as configuration, source dependencies, host information, and results to a backend like MongoDB, and separate frontends (for example, Omniboard) let you browse those stored runs.

What it does

  • Config Scopes let you define experiment parameters as plain local variables inside a function decorated with @ex.config
  • Config injection passes configuration values into any function automatically by argument name
  • A command-line interface is generated for each experiment so you can change parameters and run variants without editing code
  • Observers log parameters, source dependencies, host details, and results, and can save them to a MongoDB database
  • Automatic seeding controls randomness so experiment results stay reproducible
  • Works with external frontends such as Omniboard to browse stored runs

Getting started

Install Sacred with pip, then wrap your script in an Experiment that reads its parameters from a config function.

Install Sacred

Install the package from PyPI. The numpy and pymongo packages are optional but enable extra features such as saving runs to MongoDB.

bashbash
pip install sacred

Write an experiment

Create an Experiment, declare parameters in a function decorated with @ex.config, and mark your entry point with @ex.automain so the values are injected by name.

pythonpython
from numpy.random import permutation
from sklearn import svm, datasets
from sacred import Experiment

ex = Experiment('iris_rbf_svm')

@ex.config
def cfg():
    C = 1.0
    gamma = 0.7

@ex.automain
def run(C, gamma):
    iris = datasets.load_iris()
    per = permutation(iris.target.size)
    iris.data = iris.data[per]
    iris.target = iris.target[per]
    clf = svm.SVC(C=C, kernel='rbf', gamma=gamma)
    clf.fit(iris.data[:90], iris.target[:90])
    return clf.score(iris.data[90:], iris.target[90:])

Run and override parameters

Run the script directly, or use the generated command-line interface to override a config value for a single run.

bashbash
python experiment.py with C=0.5 gamma=0.1

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

When to use it

  • Tracking the configuration and result of every training run so you can compare and reproduce experiments later
  • Sweeping over hyperparameters by overriding config values from the command line instead of editing the script
  • Storing run metadata in MongoDB and browsing it with a frontend like Omniboard
  • Keeping randomness reproducible across runs through Sacred's automatic seeding

How Sacred compares

Sacred alongside other open-source experiment tracking tools AI/TLDR tracks, ranked by GitHub stars.

ToolStarsWhat it does
MLflow★ 28.9kAn open-source ML and GenAI platform that, alongside experiment tracking, provides evaluation and monitoring tools for scoring and comparing LLM and agent outputs.
DVC★ 16kA Git-like version control tool for datasets, models, and experiments that makes ML projects reproducible.
Weights & Biases (wandb)★ 11.3kAn open-source client library for logging experiments, running hyperparameter sweeps, and versioning models and datasets.
ClearML★ 6.9kAn open-source MLOps suite whose experiment manager auto-logs runs, metrics, and artifacts and ties them into data management and orchestration.
Aim★ 6.3kA self-hosted, open-source experiment tracker that logs and compares thousands of training runs through a fast web UI.
Sacred★ 4.4kConfigure, organize, log, and reproduce your machine learning experiments
Polyaxon★ 3.7kAn open-source platform for tracking, scheduling, and managing large-scale deep learning experiments on Kubernetes.
Trackio★ 1.8kA lightweight open-source experiment tracker from Hugging Face that stores runs locally in SQLite as a drop-in alternative to wandb.