Overview
MedSAM is an adaptation of Meta AI's Segment Anything Model (SAM) for medical image segmentation. It takes a bounding box prompt around a region of interest and returns a segmentation mask, working across medical imaging types rather than a single organ or task.
The model is fine-tuned from SAM and uses a ViT-B image encoder. It supports imaging modalities including CT and MR, and the repository documents training on abdominal CT scans covering 13 organ labels.
MedSAM is published in Nature Communications (2024). You can run it from the command line, from a Jupyter notebook, or through a graphical interface where you draw the bounding box by hand.
What it does
- Promptable segmentation driven by a bounding box drawn around the target region
- Fine-tuned from Meta AI's Segment Anything Model (SAM) using a ViT-B image encoder
- Works across medical imaging modalities including CT and MR
- Command-line inference through the MedSAM_Inference.py script
- Interactive GUI (python gui.py, requires PyQt5) for drawing bounding boxes on images
- Jupyter notebook and Colab quickstart tutorials for running inference
Getting started
Create a conda environment, clone the repository, install it, download the model checkpoint, then run inference with a bounding box prompt.
Create the environment and install
Create a conda environment with Python 3.10, clone the repository, and install it in editable mode. Install PyTorch 2.0 from pytorch.org as well.
conda create -n medsam python=3.10 -y
conda activate medsam
git clone https://github.com/bowang-lab/MedSAM
cd MedSAM
pip install -e .Download the model checkpoint
Download the MedSAM checkpoint from the Google Drive folder linked in the repository and place it at work_dir/MedSAM/medsam_vit_b.
# Place the downloaded checkpoint at:
# work_dir/MedSAM/medsam_vit_bRun command-line inference
Run the inference script, passing the input image, the output path, and the bounding box coordinates for the region to segment.
python MedSAM_Inference.py -i input_img -o output_path --box bounding_boxUse the interactive GUI
Install PyQt5 and launch the GUI, then draw a bounding box to specify the segmentation target.
pip install PyQt5
python gui.pyCommands and code are distilled from the project's own documentation — always check the official repo for the latest.
When to use it
- Segment anatomical structures in CT or MR images by placing a bounding box around each target
- Produce segmentation masks across multiple imaging modalities without training a separate model per task
- Interactively annotate medical images through the GUI to build or correct labeled datasets
- Prototype segmentation workflows from a notebook using the provided quickstart tutorials
How MedSAM compares
MedSAM alongside other open-source medical imaging tools AI/TLDR tracks, ranked by GitHub stars.
| Tool | Stars | What it does |
|---|---|---|
| nnU-Net | ★ 8.7k | Self-configuring deep-learning framework that automatically adapts U-Net pipelines to any biomedical image segmentation dataset. |
| MONAI | ★ 8.4k | PyTorch-based open-source framework for deep learning in healthcare imaging, from domain-specific data loaders to pretrained model zoos. |
| MedSAM | ★ 4.3k | Segment Anything adapted for medical images |
| TotalSegmentator | ★ 2.9k | Tool for robust automatic segmentation of over 100 anatomical structures in CT and MR images. |
| NeuroVFM | ★ 46 | Neuroimaging visual foundation model trained on 5.24M clinical MRI/CT volumes via self-supervised Vol-JEPA, covering 156 diagnostic tasks across both modalities. |