RF-DETR

// Overview

RF-DETR is a real-time object detection and instance segmentation model from Roboflow. It uses a transformer (DETR-style) architecture built on a DINOv2 vision transformer backbone, and exposes both detection and segmentation through a single, consistent Python API.

It is aimed at computer-vision engineers who need a fast detector that runs in real time. The project reports accuracy and latency results on the COCO and RF100-VL benchmarks, and ships in several sizes (Nano, Small, Medium, Large) so you can trade speed for accuracy depending on your hardware.

Within the computer-vision space, RF-DETR is an alternative to YOLO-family detectors. The core rfdetr package and its models are released under Apache 2.0, while larger Plus models (RF-DETR-XL and 2XL) use a separate PML 1.0 license.

// What it does

• Single, consistent API covering both object detection and instance segmentation
• DINOv2 vision transformer backbone for the detection and segmentation heads
• Multiple model sizes (Nano, Small, Medium, Large) to balance latency and accuracy
• Benchmarked on Microsoft COCO and the RF100-VL dataset
• Works with the supervision library for annotating and drawing detections
• Open-source rfdetr package under Apache 2.0; larger Plus models available under PML 1.0

// Getting started

Install the rfdetr package in a Python 3.10+ environment, then load a model and run inference on an image.

pip install rfdetr

import supervision as sv
from rfdetr import RFDETRMedium
from rfdetr.assets.coco_classes import COCO_CLASSES

model = RFDETRMedium()
detections = model.predict("https://media.roboflow.com/dog.jpg", threshold=0.5)
labels = [f"{COCO_CLASSES[class_id]}" for class_id in detections.class_id]
annotated_image = sv.BoxAnnotator().annotate(detections.metadata["source_image"], detections)
annotated_image = sv.LabelAnnotator().annotate(annotated_image, detections, labels)

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Add real-time object detection to a camera or video pipeline
• Run instance segmentation to get per-object masks instead of just boxes
• Pick a smaller model (Nano/Small) when you need low latency on limited hardware
• Evaluate a transformer detector as an alternative to YOLO models on COCO or custom datasets

// How RF-DETR compares

RF-DETR alongside other open-source vision & understanding tools AI/TLDR tracks, ranked by GitHub stars.