# Roboflow Inference

> Run computer vision models like YOLO, SAM, and CLIP locally or at the edge

- Category: Multimodal Generation / Vision & Understanding
- Language: Python
- Repository: https://github.com/roboflow/inference
- Website: https://inference.roboflow.com
- Canonical page: https://ai-tldr.dev/tools/roboflow-inference/

## In short

Roboflow Inference is an open-source server that turns any computer or edge device into a place to run computer vision models. You can self-host your own fine-tuned models or use foundation models such as Florence-2, CLIP, and SAM2, and call them over an HTTP API. It sits in the Vision & Understanding category of the Multimodal Generation tools and is open-source, written in Python.

## Overview

Roboflow Inference is an open-source server that turns any computer or edge device into a place to run computer vision models. You can self-host your own fine-tuned models or use foundation models such as Florence-2, CLIP, and SAM2, and call them over an HTTP API.

It is aimed at developers who want to deploy vision models without sending images to a third-party cloud. The server pulls the right image for your machine, runs locally (by default on localhost:9001), and supports GPU acceleration when you have a CUDA-enabled GPU.

Within the computer vision space, it fits as the runtime layer. On top of model inference it adds Workflows, composable blocks that let you chain models, mix ML with classic CV (OCR, barcode and QR reading, template matching), and process video streams.

## What it does

- Runs your own fine-tuned models plus foundation models like Florence-2, CLIP, and SAM2
- Exposes models over a local HTTP API (default localhost:9001) with a Python SDK
- Workflows: composable blocks to track, count, time, measure, and visualize detections
- Combines ML with traditional CV such as OCR, barcode, QR, and template matching
- Manages cameras and video streams for real-time processing
- GPU acceleration via the NVIDIA Container Toolkit on CUDA-enabled hardware

## Getting started

Inference runs as a Docker container that you start with its CLI. Install Docker first (plus the NVIDIA Container Toolkit if you want GPU acceleration), then start the server and connect to it from Python.

1. Install the CLI and start the server

   Install the inference CLI with pip, then start the server in development mode. This pulls the right image for your machine and runs it.

```bash
pip install inference-cli && inference server start --dev
```

2. Open the quickstart notebook

   In development mode a Jupyter notebook server with a quickstart guide runs at http://localhost:9001/notebook/start, where you can tour the server's functionality.

3. Install the Python SDK

   To talk to your running server from Python, install the SDK. By default the server listens on localhost:9001.

```bash
pip install inference-sdk
```

## When to use it

- Self-host a fine-tuned object detection model and call it over a local API instead of a cloud service
- Build a Workflow that detects, tracks, and counts objects on a live video stream
- Read license plates or barcodes by combining a detection model with OCR in one Workflow
- Run foundation models like CLIP or SAM2 on an edge device for on-site image analysis
