LMMs-Lab · 2026-05-25 · notable
LLaVA-OneVision-2 — Open 8B Vision-Language Model Reads Video as a Codec Bit-Cost Stream, Hits 74.9 JumpScore mAP vs Qwen3-VL-8B's 30.1
An 8B open vision-language model that tokenizes video as a codec bit-cost stream instead of uniform frames, scoring 74.9 JumpScore mAP against Qwen3-VL-8B's 30.1. Data, code, encoders, and checkpoints ship under Apache 2.0.
An open 8B vision-language model that reads video as a compression stream, not a stack of sampled frames.
Key specs
| License | Apache 2.0 |
|---|---|
| Parameters | 8B |
| GitHub stars | 963 |
| Jump score m ap | 74.9 |
What is it?
LLaVA-OneVision-2 is the new version of the open LLaVA-OneVision multimodal family from LMMs-Lab. The 8B model handles images, long-form video, and spatial understanding in one architecture, and ships its data, training code, encoders, and checkpoints openly under Apache 2.0.
How does it work?
Instead of uniformly sampling frames, it uses codec-stream tokenization: compressed video is treated as a continuous bit-cost stream, where bit-cost dynamics drive temporal grouping and motion-residual cues select salient regions into compact visual canvases. Images, uniform frames, and codec-aligned tokens all flow through one shared OneVision-Encoder with no per-modality routing.
Why does it matter?
Concentrating tokens on event-bearing content gives steadier long-video understanding on a fixed token budget. On matched budgets it adds 9.7 points on temporal grounding over frame sampling, and the 8B reports 74.9 JumpScore mAP against Qwen3-VL-8B's 30.1, with the full pipeline open for reproduction.
Who is it for?
multimodal and video-understanding researchers
Try it
github.com/EvolvingLMMs-Lab/LLaVA-OneVision-2