# Sacred

> Configure, organize, log, and reproduce your machine learning experiments

- Category: Training & Fine-Tuning / Experiment Tracking
- Language: Python
- Repository: https://github.com/IDSIA/sacred
- Website: https://sacred.readthedocs.io
- Canonical page: https://ai-tldr.dev/tools/sacred/

## In short

Sacred is a Python library that handles the repetitive work around a machine learning experiment so you can focus on the research itself. It helps you keep track of every parameter, run the same experiment under different settings, save each run's configuration, and reproduce results later. It sits in the Experiment Tracking category of the Training & Fine-Tuning tools and is open-source, written in Python.

## Overview

Sacred is a Python library that handles the repetitive work around a machine learning experiment so you can focus on the research itself. It helps you keep track of every parameter, run the same experiment under different settings, save each run's configuration, and reproduce results later.

It is aimed at researchers and ML engineers who run many experiments and need a reliable record of what was run and how. You wrap your training script in a Sacred Experiment, define parameters in a config function, and Sacred injects those values into your code and gives you a command-line interface to override them.

As an experiment-tracking tool, Sacred focuses on capturing runs and their metadata. Observers can write run details such as configuration, source dependencies, host information, and results to a backend like MongoDB, and separate frontends (for example, Omniboard) let you browse those stored runs.

## What it does

- Config Scopes let you define experiment parameters as plain local variables inside a function decorated with @ex.config
- Config injection passes configuration values into any function automatically by argument name
- A command-line interface is generated for each experiment so you can change parameters and run variants without editing code
- Observers log parameters, source dependencies, host details, and results, and can save them to a MongoDB database
- Automatic seeding controls randomness so experiment results stay reproducible
- Works with external frontends such as Omniboard to browse stored runs

## Getting started

Install Sacred with pip, then wrap your script in an Experiment that reads its parameters from a config function.

1. Install Sacred

   Install the package from PyPI. The numpy and pymongo packages are optional but enable extra features such as saving runs to MongoDB.

```bash
pip install sacred
```

2. Write an experiment

   Create an Experiment, declare parameters in a function decorated with @ex.config, and mark your entry point with @ex.automain so the values are injected by name.

```python
from numpy.random import permutation
from sklearn import svm, datasets
from sacred import Experiment

ex = Experiment('iris_rbf_svm')

@ex.config
def cfg():
    C = 1.0
    gamma = 0.7

@ex.automain
def run(C, gamma):
    iris = datasets.load_iris()
    per = permutation(iris.target.size)
    iris.data = iris.data[per]
    iris.target = iris.target[per]
    clf = svm.SVC(C=C, kernel='rbf', gamma=gamma)
    clf.fit(iris.data[:90], iris.target[:90])
    return clf.score(iris.data[90:], iris.target[90:])
```

3. Run and override parameters

   Run the script directly, or use the generated command-line interface to override a config value for a single run.

```bash
python experiment.py with C=0.5 gamma=0.1
```

## When to use it

- Tracking the configuration and result of every training run so you can compare and reproduce experiments later
- Sweeping over hyperparameters by overriding config values from the command line instead of editing the script
- Storing run metadata in MongoDB and browsing it with a frontend like Omniboard
- Keeping randomness reproducible across runs through Sacred's automatic seeding
