SenseTime · 2026-05-12 · notable
SenseNova-U1 — 8B Dense and 30B-A3B MoE Native Unified Multimodal Models With NEO-Unify Pixel-Space Backbone, Apache 2.0
SenseTime's open-source unified multimodal family drops the vision encoder + VAE split, jointly trains understanding and generation on raw pixels, and hits 0.91 on GenEval. Code and weights on GitHub and HF.
An open native-unified multimodal model that ditches the vision encoder and VAE: one transformer reads and draws pixels end-to-end.
Key specs
| GitHub stars | 1,593 |
|---|---|
| Dense params | 8B |
| Moe total params | 30B |
| Moe active params | 3B |
| Geneval | 0.91 |
| Hf papers trending | #1 |
What is it?
SenseNova-U1 is SenseTime's open multimodal model family with two variants: an 8B dense (SenseNova-U1-8B-MoT) and a 30B-total / 3B-active Mixture-of-Experts (SenseNova-U1-A3B-MoT). Unlike most VLMs, it has no pretrained vision encoder and no VAE — it operates directly on pixels and text using a Mixture-of-Transformers (MoT) where understanding and generation streams share routing but keep decoupled parameters.
How does it work?
The NEO-unify backbone uses near-lossless 32×32 patch encoding/decoding to move pixels in and out without a learned tokenizer. Understanding is autoregressive over text; generation is pixel-space flow matching. The two objectives are co-trained in a single transformer so the model can perceive, reason, and generate within one pass — useful for interleaved tasks like image editing, text-rich infographics, and vision-language-action.
Why does it matter?
Unified models have been the long-promised replacement for the vision-encoder + LLM + diffusion-decoder Frankenstein stack. SenseNova-U1 is one of the first openly licensed (Apache 2.0) takes that posts competitive numbers across MMMU, MMBench, and GenEval (0.91), with code, weights, GGUF builds, and a hosted demo. The paper is the #1 trending paper on Hugging Face today.
Who is it for?
ML researchers and infra teams evaluating unified multimodal architectures.
Try it
https://huggingface.co/collections/sensenova/sensenova-u1