ShengShu · 2026-05-28 · notable
ShengShu Open-Sources minWM — Full-Stack Pipeline Turns Wan2.1 and HunyuanVideo 1.5 Into Real-Time Camera-Controllable World Models With 224× First-Frame Latency Speedup
Tsinghua-affiliated ShengShu releases minWM, an end-to-end pipeline that converts bidirectional T2V/TI2V diffusion models into camera-controllable few-step autoregressive world models. 224× first-frame latency cut on HunyuanVideo 1.5, Apache 2.0.
An end-to-end recipe that distills heavyweight video diffusion models into few-step, camera-controllable world models that respond fast enough for live interaction.
Key specs
| License | Apache-2.0 |
|---|---|
| GitHub stars | 285 |
| First frame latency (hy1.5) | 3.4s |
| Speedup (hy1.5) | 224x |
| Speedup (wan2.1) | 237x |
| Paper upvotes (hf) | 185 |
What is it?
minWM is a full-stack open-source framework for building real-time interactive video world models. It is not a single trained model but a reproducible pipeline that takes off-the-shelf text-to-video or text-and-image-to-video diffusion backbones (Wan2.1-T2V-1.3B and HunyuanVideo 1.5-TI2V-8B) and converts them into camera-controllable, few-step autoregressive generators that can be steered by a user.
How does it work?
The pipeline has two phases. Phase one fine-tunes a bidirectional diffusion backbone on camera-annotated video using PRoPE to inject camera intrinsics and poses into self-attention. Phase two applies the lab's Causal Forcing / Causal Forcing++ recipe: teacher-forced AR diffusion training, causal ODE or causal consistency distillation, then an asymmetric DMD post-training stage that aligns the few-step student model with the bidirectional teacher. The result is a 4-step autoregressive generator that produces 480×832 video at 77 frames.
Why does it matter?
Existing world-model techniques are scattered across separate papers. minWM bundles data construction, controllability training, AR distillation, and inference into one reproducible repo with intermediate checkpoints for each stage, so graduate students and small labs can fine-tune or adapt existing world models like HY-WorldPlay without rebuilding the stack. Reported first-frame latency drops from 771s to 3.4s on HunyuanVideo 1.5 (224× speedup), and from 269s to 1.1s on Wan2.1 (237×).
Who is it for?
world-model researchers, video-diffusion labs, robotics teams
Try it
git clone https://github.com/shengshu-ai/minWM