ByteDance Seed · 2026-08-25 · notable
DiffusionOPSD — reward training for diffusion models at 40% fewer GPU-hours
DiffusionOPSD is a diffusion post-training method from ByteDance Seed that turns image reward scores into per-step training targets. It uses 40% fewer GPU-hours than DiffusionNFT on Stable Diffusion 3.5-Medium.
DiffusionOPSD turns image reward scores into step-by-step training targets, cutting diffusion post-training GPU-hours by 40 to 63%.
Key specs
| GitHub stars | 73 |
|---|---|
| Gpu hours saved (sd3.5 m) | 40% |
| Gpu hours saved (z image turbo) | 63% |
What is it?
DiffusionOPSD replaces endpoint-only reward training for diffusion models with explicit targets for the clean-image prediction at each sampled step. Reinforcement learning can align an image model with a human preference score, but one score at the end of sampling never says how an intermediate prediction should change. ByteDance Seed published the paper on arXiv on August 25, 2026 with Apache-2.0 code and three LoRA adapters.
How does it work?
Each outer iteration starts from a frozen behavior policy that generates trajectories and supplies query states and anchors. Reward gradients then build bounded positive and negative targets around each anchor, and the trainable policy fits those targets as detached supervision under a fixed budget. An exponential moving average update refreshes the behavior policy, which keeps DiffusionOPSD's targets on-policy as the model changes.
Why does it matter?
Tuning an image model to a preference score is expensive, and the paper reports 40% fewer GPU-hours than DiffusionNFT on Stable Diffusion 3.5-Medium and 63% fewer on Z-Image-Turbo. That lets a smaller cluster do the same reward-guided post-training run. The three published LoRA adapters also let people try the trained results without repeating the training.
Who is it for?
diffusion model researchers
Try it
git clone https://github.com/worldbench/DiffusionOPSD && pip install -e ".[rewards]"