Grounding DINO

// Overview

Grounding DINO is an open-set object detector: instead of choosing from a fixed list of classes, you give it an image and a text prompt like "chair . person . dog ." and it returns boxes for the things you named. This means you can detect new categories without retraining the model on labeled data for them.

It is built for computer-vision engineers and researchers who need flexible detection without collecting and annotating a custom dataset first. The repo is a PyTorch implementation with pretrained Swin-T and Swin-B checkpoints, a Python inference API, and a command-line demo script. It runs on GPU and also supports a CPU-only mode.

Within the multimodal vision space, Grounding DINO sits between text and images: it pairs the DINO detection architecture with grounded language pre-training so prompts drive what gets detected. It is often used as a front end for downstream tools, for example feeding boxes into Segment Anything for masks or into auto-labeling pipelines.

// What it does

• Open-set detection: find objects described by free-form text rather than a fixed class list
• Zero-shot capable, reported at 52.5 AP on COCO without training on COCO data, and 63.0 AP after fine-tuning
• Pretrained checkpoints for Swin-T and Swin-B backbones, downloadable from GitHub releases or Hugging Face
• Simple Python API (load_model, load_image, predict, annotate) plus a CLI demo script
• CPU-only mode so it can run on machines without a GPU
• Works as a building block for Grounded SAM, image editing, and automated dataset annotation

// Getting started

Clone the repo, install it as an editable package, download a checkpoint, then run inference from Python. A CUDA GPU is recommended but a CPU-only mode is supported.

git clone https://github.com/IDEA-Research/GroundingDINO.git
cd GroundingDINO/
pip install -e .

mkdir weights
cd weights
wget -q https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth
cd ..

from groundingdino.util.inference import load_model, load_image, predict, annotate
import cv2

model = load_model("groundingdino/config/GroundingDINO_SwinT_OGC.py", "weights/groundingdino_swint_ogc.pth")
IMAGE_PATH = "weights/dog-3.jpeg"
TEXT_PROMPT = "chair . person . dog ."
BOX_TRESHOLD = 0.35
TEXT_TRESHOLD = 0.25

image_source, image = load_image(IMAGE_PATH)
boxes, logits, phrases = predict(model=model, image=image, caption=TEXT_PROMPT, box_threshold=BOX_TRESHOLD, text_threshold=TEXT_TRESHOLD)
annotated_frame = annotate(image_source=image_source, boxes=boxes, logits=logits, phrases=phrases)
cv2.imwrite("annotated_image.jpg", annotated_frame)

CUDA_VISIBLE_DEVICES={GPU ID} python demo/inference_on_a_image.py \
-c groundingdino/config/GroundingDINO_SwinT_OGC.py \
-p weights/groundingdino_swint_ogc.pth \
-i image_you_want_to_detect.jpg \
-o "dir you want to save the output" \
-t "chair"

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Detect novel or rare object categories in images without collecting and labeling a training set
• Auto-label datasets by generating boxes from text prompts to bootstrap training for other detectors
• Feed detected boxes into Segment Anything (Grounded SAM) to get masks for prompted objects
• Prototype prompt-driven detection for tasks like image editing or visual search

// How Grounding DINO compares

Grounding DINO alongside other open-source vision & understanding tools AI/TLDR tracks, ranked by GitHub stars.