Is Mage-VL open source?

Yes. Microsoft released Mage-VL under Apache-2.0 with weights on Hugging Face at microsoft/Mage-VL and inference code at github.com/microsoft/Mage. The paired Mage-ViT visual encoder ships under MIT so it can be reused separately.

How does Mage-VL compare to Qwen3-VL-4B and Phi-4-MM?

Microsoft's technical report shows Mage-VL matches Qwen3-VL-4B and Phi-4-MM on static image benchmarks and posts strong gains on video understanding and 2D/3D spatial reasoning, while running up to 3.5x faster on video thanks to its codec-aligned token compression.

What does 'codec-native' actually mean for Mage-VL?

Mage-VL's tokenizer mirrors how modern video codecs allocate bits: it keeps every anchor (I) frame patch and only retains predicted-frame (P) patches where the codec itself spends bits. That codec-aligned sparsity drops visual-token count by more than 75% while preserving spatio-temporal context.

What can Mage-VL do in streaming mode?

Mage-VL couples the vision-language model with a frozen-backbone cognition gate that emits commentary only when an event fires. Microsoft reports state-of-the-art TimVal, F1, ROC-AUC and PR-AUC on SoccerNet streaming and shows generalization to real 2026 World Cup broadcasts.

Microsoft · 2026-07-27 · major

Microsoft Mage-VL — 4B codec-native multimodal cuts video tokens by 75%

Mage-VL pairs a from-scratch codec-native visual encoder with a Qwen3-4B backbone. It keeps anchor-frame patches, drops 75% of predicted-frame tokens, and runs video inference up to 3.5x faster. Apache-2.0 weights on Hugging Face.

Mage-VL model card banner on Hugging Face

Mage-VL treats a video like a codec — keep anchor frames, drop most predicted-frame patches, and cut visual tokens by 75%.

Key specs

Parameters	4B
Visual token cut	>75%
Video speedup	up to 3.5x

Quick facts

Maker	Microsoft
License	Apache-2.0 (Mage-ViT encoder: MIT)
Parameters	4B (Mage-ViT + Qwen3-4B-Instruct)
Modalities	Image, video (H.264/HEVC + DCVC-RT), streaming commentary
Visual token cut	>75% vs baselines
Video inference	up to 3.5x faster wall-clock
Availability	Open weights on Hugging Face + code on GitHub

What is it?

Mage-VL is Microsoft's new 4B multimodal foundation model that pairs a from-scratch codec-native visual encoder called Mage-ViT with a Qwen3-4B-Instruct language backbone. A single Apache-2.0 checkpoint handles static images, offline video reasoning, traditional codec video (H.264/HEVC), neural codec video (DCVC-RT), and event-gated streaming commentary.

How does it work?

The Mage-ViT encoder is trained from scratch and mirrors the structure of modern video codecs. Every anchor (I) frame patch is kept, but only the predicted-frame (P) patches where the codec itself allocated bits are retained. On top, a frozen-backbone cognition gate decides when a new event happens and lets Mage-VL emit low-latency streaming commentary rather than talking continuously.

Why does it matter?

Cutting visual tokens by more than 75% means Mage-VL can process long or live video without ballooning context length or GPU cost. Microsoft reports up to 3.5x wall-clock speedups on video versus baselines while matching Qwen3-VL-4B and Phi-4-MM on image tasks, which puts real-time video perception within reach of modest hardware.

Who is it for?

multimodal researchers, video-AI and robotics engineers

Frequently asked questions

Is Mage-VL open source?: Yes. Microsoft released Mage-VL under Apache-2.0 with weights on Hugging Face at microsoft/Mage-VL and inference code at github.com/microsoft/Mage. The paired Mage-ViT visual encoder ships under MIT so it can be reused separately.
How does Mage-VL compare to Qwen3-VL-4B and Phi-4-MM?: Microsoft's technical report shows Mage-VL matches Qwen3-VL-4B and Phi-4-MM on static image benchmarks and posts strong gains on video understanding and 2D/3D spatial reasoning, while running up to 3.5x faster on video thanks to its codec-aligned token compression.
What does 'codec-native' actually mean for Mage-VL?: Mage-VL's tokenizer mirrors how modern video codecs allocate bits: it keeps every anchor (I) frame patch and only retains predicted-frame (P) patches where the codec itself spends bits. That codec-aligned sparsity drops visual-token count by more than 75% while preserving spatio-temporal context.
What can Mage-VL do in streaming mode?: Mage-VL couples the vision-language model with a frozen-backbone cognition gate that emits commentary only when an event fires. Microsoft reports state-of-the-art TimVal, F1, ROC-AUC and PR-AUC on SoccerNet streaming and shows generalization to real 2026 World Cup broadcasts.

Try it

huggingface.co/microsoft/Mage-VL