PixelRAG

// Overview

PixelRAG is the official codebase for the paper *PixelRAG: Web Screenshots Beat Text for Retrieval-Augmented Generation*, from Berkeley SkyLab, BAIR and Berkeley NLP. Its premise is that the parsing step in conventional RAG is where the information goes missing: turning a page into text chunks discards tables, charts, infographics and layout, so a reader model cannot answer questions that depend on them. PixelRAG renders documents — web pages, PDFs, images — as screenshots and retrieves over those images directly, leaving the visual structure intact for the reader to look at.

Two pieces make it work. The first is rendering rather than parsing: a page becomes a set of screenshot tiles. The second is a `Qwen3-VL-Embedding` model LoRA-fine-tuned on screenshot data, which embeds page images into a space where visual content is actually retrievable. Retrieval then returns the right tile and the reader reads the number straight off the image instead of hunting for it in a flattened text chunk.

The project is usable without building anything: a hosted endpoint at `api.pixelrag.ai` serves a pre-built index of 8.28 million Wikipedia pages with no setup and no API key, and it accepts an image as the query as well as text. For local work the pipeline is split into stages behind the `pixelrag` umbrella with separate extras, so you install only what you need — `pixelshot` for capture, `chunk`/`embed`/`build-index` for turning tiles into a FAISS index, `index` to orchestrate the whole thing, and `serve` for a FastAPI search API on CPU or GPU. Indexing works on Linux with CUDA and on Apple Silicon via MPS, with `device: auto` picking the backend. The training code is a separate uv project with its own pinned environment.

The renderer also ships as a coding-agent plugin. The `pixelbrowse` skill lets Claude Code screenshot a page with `pixelshot` and read the image rather than fetching raw HTML — no MCP server and no backend, since the skill just shells out to the CLI on your machine. opencode users get the same `screenshot` tool through the `@startrail/pixelbrowse` npm package. PixelRAG is Apache-2.0 licensed.

// What it does

• Renders web pages, PDFs and images to screenshot tiles instead of parsing them to text, preserving tables, charts, layout and infographics
• A Qwen3-VL-Embedding model LoRA-fine-tuned on screenshot data, so page images embed into a retrievable space
• A hosted endpoint serving a pre-built index of 8.28M Wikipedia pages — no setup, no API key
• Visual search: an image can be the query, not just text
• Staged installation — pixelshot for capture, plus [embed], [index] and [serve] extras for the rest of the pipeline
• FAISS-backed search API served by FastAPI on CPU or GPU
• Pre-built FAISS indexes published on Hugging Face (base and LoRA Wikipedia pixel, Wikipedia text, news pixel)
• pixelbrowse plugin that gives Claude Code and opencode a screenshot tool backed by the same renderer
• Runs on Linux with CUDA and on Apple Silicon via MPS, with device: auto choosing the backend

// Getting started

The two core operations are rendering a page to screenshot tiles and searching a visual index. You can do the second against the hosted index before installing anything locally.

pip install pixelrag

pixelshot https://en.wikipedia.org/wiki/Python --output ./tiles

curl -X POST https://api.pixelrag.ai/search \
  -H "Content-Type: application/json" \
  -d '{"queries": [{"text": "What is the capital of France?"}], "n_docs": 5}'

pip install 'pixelrag[serve]'

huggingface-cli download StarTrail-org/pixelrag-faiss-indexes \
  --repo-type dataset --include "search_index_normed_v2/*" --local-dir ./index

pixelrag serve --index-dir ./index/search_index_normed_v2 --port 30001

pip install 'pixelrag[index]'

cat > pixelrag.yaml << 'EOF'
source:
  type: local
  path: ./my_docs

embed:
  model: Qwen/Qwen3-VL-Embedding-2B
  device: auto
EOF

pixelrag index

uv tool install pixelrag
claude plugin marketplace add StarTrail-org/PixelRAG
claude plugin install pixelbrowse@pixelrag-plugins

claude -p "screenshot https://news.ycombinator.com and summarize the top stories"

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Question answering over documents whose answers live in tables, charts or infographics that text extraction flattens away
• Retrieval across PDFs and scanned material where layout carries meaning
• Visual search — finding pages that look like a supplied image rather than matching its text
• Giving a coding agent the ability to read a rendered page instead of raw HTML
• Comparing visual and text-based retrieval on the same corpus using the published Wikipedia pixel and text indexes

// How PixelRAG compares

PixelRAG alongside other open-source rag frameworks & platforms tools AI/TLDR tracks, ranked by GitHub stars.