# Microsoft Mage-VL — 4B codec-native multimodal cuts video tokens by 75%

> Mage-VL treats a video like a codec — keep anchor frames, drop most predicted-frame patches, and cut visual tokens by 75%.

- Org: Microsoft
- Date: 2026-07-27
- Category: model, paper, algorithm
- Source: https://microsoft.github.io/Mage/vl/
- Canonical page: https://ai-tldr.dev/releases/microsoft-mage-vl/

## In short

Mage-VL pairs a from-scratch codec-native visual encoder with a Qwen3-4B backbone. It keeps anchor-frame patches, drops 75% of predicted-frame tokens, and runs video inference up to 3.5x faster. Apache-2.0 weights on Hugging Face.

## What it is

Mage-VL is Microsoft's new 4B multimodal foundation model that pairs a from-scratch codec-native visual encoder called Mage-ViT with a Qwen3-4B-Instruct language backbone. A single Apache-2.0 checkpoint handles static images, offline video reasoning, traditional codec video (H.264/HEVC), neural codec video (DCVC-RT), and event-gated streaming commentary.

## How it works

The Mage-ViT encoder is trained from scratch and mirrors the structure of modern video codecs. Every anchor (I) frame patch is kept, but only the predicted-frame (P) patches where the codec itself allocated bits are retained. On top, a frozen-backbone cognition gate decides when a new event happens and lets Mage-VL emit low-latency streaming commentary rather than talking continuously.

## Why it matters

Cutting visual tokens by more than 75% means Mage-VL can process long or live video without ballooning context length or GPU cost. Microsoft reports up to 3.5x wall-clock speedups on video versus baselines while matching Qwen3-VL-4B and Phi-4-MM on image tasks, which puts real-time video perception within reach of modest hardware.

## Who it's for

multimodal researchers, video-AI and robotics engineers

## Try it

huggingface.co/microsoft/Mage-VL

## Quick facts

- Maker: Microsoft
- License: Apache-2.0 (Mage-ViT encoder: MIT)
- Parameters: 4B (Mage-ViT + Qwen3-4B-Instruct)
- Modalities: Image, video (H.264/HEVC + DCVC-RT), streaming commentary
- Visual token cut: >75% vs baselines
- Video inference: up to 3.5x faster wall-clock
- Availability: Open weights on Hugging Face + code on GitHub

## FAQ

### Is Mage-VL open source?

Yes. Microsoft released Mage-VL under Apache-2.0 with weights on Hugging Face at microsoft/Mage-VL and inference code at github.com/microsoft/Mage. The paired Mage-ViT visual encoder ships under MIT so it can be reused separately.

### How does Mage-VL compare to Qwen3-VL-4B and Phi-4-MM?

Microsoft's technical report shows Mage-VL matches Qwen3-VL-4B and Phi-4-MM on static image benchmarks and posts strong gains on video understanding and 2D/3D spatial reasoning, while running up to 3.5x faster on video thanks to its codec-aligned token compression.

### What does 'codec-native' actually mean for Mage-VL?

Mage-VL's tokenizer mirrors how modern video codecs allocate bits: it keeps every anchor (I) frame patch and only retains predicted-frame (P) patches where the codec itself spends bits. That codec-aligned sparsity drops visual-token count by more than 75% while preserving spatio-temporal context.

### What can Mage-VL do in streaming mode?

Mage-VL couples the vision-language model with a frozen-backbone cognition gate that emits commentary only when an event fires. Microsoft reports state-of-the-art TimVal, F1, ROC-AUC and PR-AUC on SoccerNet streaming and shows generalization to real 2026 World Cup broadcasts.

## Links

- [Project Page](https://microsoft.github.io/Mage/vl/)
- [Model Weights](https://huggingface.co/microsoft/Mage-VL)
- [Paper (arXiv)](https://arxiv.org/abs/2607.24904)
- [GitHub Repo](https://github.com/microsoft/Mage)