# Nemotron Labs Sigma (3B / 8B)

> Released 12 November 2026: NVIDIA's open 3B and 8B continuous diffusion language models, which denoise whole blocks of text in a 16-dimensional embedding space instead of predicting one token at a time.

- Maker: NVIDIA
- Line: Nemotron Labs (research models)
- Canonical page: https://ai-tldr.dev/models/nemotron-labs-sigma/
- Updated: 2026-11-12

## In short

Nemotron Labs Sigma (3B / 8B) is an open-weight model from NVIDIA. NVIDIA released it on 12 November 2026. It has a 8K-token context window. You can use it through Hugging Face (weights, 8B instruct), Hugging Face (weights, 8B base) and Hugging Face (weights, 8B distilled, 8 steps).

## Facts

| Fact | Value | Source |
| --- | --- | --- |
| Context window | 8K | [1] |
| Licence | NVIDIA Nemotron Open Model License · open weights | [1] |
| Released | 12 November 2026 | [2] |
| Parameters | 3B and 8B | [1] |

## Sources

1. [Nemotron-Labs-Sigma-8B-instruct model card (Hugging Face)](https://huggingface.co/nvidia/Nemotron-Labs-Sigma-8B-instruct)
2. [Nemotron-Labs-Sigma collection (Hugging Face)](https://huggingface.co/collections/nvidia/nemotron-labs-sigma-6aed16ae59cfa91daa36c53b)

## Overview

Nemotron Labs Sigma is a family of continuous diffusion language models that NVIDIA published on Hugging Face on 12 November 2026, alongside version 2 of the paper Large Language Continuous Diffusion Models (arXiv 2610.02665). Instead of generating text one token at a time, Sigma starts each block of tokens as Gaussian noise in a 16-dimensional continuous token-embedding space and refines the whole block over a number of denoising steps. The denoiser and the embeddings are learned together by likelihood optimization, and previously committed tokens are reused through an ordinary KV cache.

The models were warm-started from autoregressive weights: the model cards describe about 1 trillion tokens of autoregressive initialization followed by about 0.3 trillion tokens of continuous-diffusion training, on the same English corpus as NVIDIA's Nemotron-Labs-Diffusion family. The backbone is a Ministral-style decoder-only Transformer. The release covers 3B and 8B sizes, each as a base model and a supervised fine-tuned instruct model. The instruct cards give a maximum of 8,192 tokens for prompt plus output; the base and distilled cards give 4,096.

Sigma's pitch is speed through fewer steps. The base and instruct models run DDPM or DDIM samplers with optional classifier-free guidance, and NVIDIA's cards note that quality drops noticeably below about 16 steps. For fewer steps NVIDIA ships distilled checkpoints, trained with Phased Distribution Matching Distillation for a fixed budget of 2, 4, 8 or 16 denoising steps per 32-token block. Separate distill-spec checkpoints use the diffusion head to draft a 32-token block that the model's own autoregressive head verifies, so the output is exactly the greedy autoregressive output and only the number of tokens committed per round changes.

In the paper's 8B table, Sigma-8B in diffusion mode with 256 DDPM steps per block averages 71.16 across GSM8K, Minerva Math, HumanEval, HumanEval+, MBPP and MBPP+, against 70.22 for Nemotron-Labs-Diffusion-8B, 58.74 for Dream-7B and 41.78 for LLaDA-8B in diffusion mode, and 68.91 for Qwen3-8B as an autoregressive model. It scores lower than Qwen3-8B and Nemotron-Labs-Diffusion-8B on MBPP. All weights are released under the NVIDIA Nemotron Open Model License, need an NVIDIA GPU in bfloat16 (the cards say CPU inference is not supported) and load through Hugging Face Transformers with trust_remote_code.

## Benchmarks

- GSM8K (Sigma-8B, DDPM 256 steps/block): 88.57% ([source](https://arxiv.org/abs/2610.02665))
- MBPP+ (Sigma-8B, DDPM 256 steps/block): 80.19% ([source](https://arxiv.org/abs/2610.02665))
- Minerva Math (Sigma-8B, DDPM 256 steps/block): 66.72% ([source](https://arxiv.org/abs/2610.02665))
- MBPP (Sigma-8B, DDPM 256 steps/block): 65.74% ([source](https://arxiv.org/abs/2610.02665))
- HumanEval (Sigma-8B, DDPM 256 steps/block): 65.12% ([source](https://arxiv.org/abs/2610.02665))
- HumanEval+ (Sigma-8B, DDPM 256 steps/block): 60.61% ([source](https://arxiv.org/abs/2610.02665))
- GSM8K CoT (distill-8B-nfe8, 8 steps): 88.44% ([source](https://huggingface.co/nvidia/Nemotron-Labs-Sigma-distill-8B-nfe8))
- MATH Minerva (distill-8B-nfe8, 8 steps): 64.4% ([source](https://huggingface.co/nvidia/Nemotron-Labs-Sigma-distill-8B-nfe8))

## Strengths

- Generates text in parallel blocks by denoising continuous embeddings, rather than one token per forward pass
- Distilled checkpoints run at a fixed 2, 4, 8 or 16 denoising steps per 32-token block with no guidance or temperature tuning
- Sigma-distill-8B-nfe8 scores 88.44 on GSM8K (CoT) and 79.37 on MBPP+ at 8 denoising steps, per its model card
- Self-speculative checkpoints draft with the diffusion head and verify with their own autoregressive head, returning exactly the greedy output
- Plain PyTorch inference through Transformers with no custom kernels or flash-attention, per the model cards
- Open weights for both sizes as base, instruct, distilled and self-speculative variants

## Best for

- Research into diffusion language models and few-step text generation from released, trained checkpoints
- Latency-sensitive generation experiments that trade denoising steps for speed
- Studying speculative and parallel decoding with a model that drafts and verifies itself
- Math and code generation tests against autoregressive models of the same size

## How to access

- [Hugging Face (weights, 8B instruct)](https://huggingface.co/nvidia/Nemotron-Labs-Sigma-8B-instruct) — model id `nvidia/Nemotron-Labs-Sigma-8B-instruct`
- [Hugging Face (weights, 8B base)](https://huggingface.co/nvidia/Nemotron-Labs-Sigma-8B) — model id `nvidia/Nemotron-Labs-Sigma-8B`
- [Hugging Face (weights, 8B distilled, 8 steps)](https://huggingface.co/nvidia/Nemotron-Labs-Sigma-distill-8B-nfe8) — model id `nvidia/Nemotron-Labs-Sigma-distill-8B-nfe8`

## Nemotron Labs (research models) versions

- Nemotron Labs Sigma (3B / 8B) (12 November 2026): First Nemotron Labs continuous diffusion language models: 3B and 8B base, instruct, few-step distilled and self-speculative checkpoints. — this page
- [Nemotron Labs Dory 1.0 4B](https://ai-tldr.dev/models/nemotron-labs-dory-1-0-4b/) (16 October 2026): First Nemotron Labs looped model: 4B parameters, 1–4 loops, 256K context.

## FAQ

### What is a continuous diffusion language model?

It generates text by denoising rather than by predicting the next token. Sigma maps tokens to points in a 16-dimensional continuous embedding space, starts each block of tokens as Gaussian noise there and refines the whole block over several denoising steps before reading the tokens out. Previously finished blocks act as a KV-cached prefix for the next one.

### Which Sigma checkpoint should I use?

The instruct models are for chat, reasoning and code with the bundled chat template. The base models suit few-shot and benchmark-style prompting. For few denoising steps NVIDIA recommends the distill variants, each trained for a fixed 2, 4, 8 or 16 steps per block, and the distill-spec variants are for self-speculative decoding.

### How does Sigma-8B compare with Qwen3-8B?

In the paper's Table 2, Sigma-8B with 256 DDPM steps per block averages 71.16 across six math and coding tasks against 68.91 for Qwen3-8B. Sigma-8B is ahead on GSM8K (88.57 vs 86.73), Minerva Math (66.72 vs 52.94) and HumanEval (65.12 vs 64.63), but behind on MBPP (65.74 vs 69.4) and MBPP+ (80.19 vs 83.07).

### What hardware does it need?

A CUDA-capable NVIDIA GPU running in bfloat16, with PyTorch 2.4 or later and Transformers 4.55 or later. The model cards list Ampere, Hopper, Lovelace and Blackwell GPUs and say CPU inference is not supported because the diffusion sampler degrades on CPU.

### What license is it under?

The NVIDIA Nemotron Open Model License. The model cards state the models are ready for commercial or non-commercial use.

## Links

- [Nemotron-Labs-Sigma-8B-instruct model card (Hugging Face)](https://huggingface.co/nvidia/Nemotron-Labs-Sigma-8B-instruct)
- [Nemotron-Labs-Sigma-8B base model card (Hugging Face)](https://huggingface.co/nvidia/Nemotron-Labs-Sigma-8B)
- [Nemotron-Labs-Sigma-distill-8B-nfe8 model card (Hugging Face)](https://huggingface.co/nvidia/Nemotron-Labs-Sigma-distill-8B-nfe8)
- [Nemotron-Labs-Sigma-distill-spec-8B model card (Hugging Face)](https://huggingface.co/nvidia/Nemotron-Labs-Sigma-distill-spec-8B)
- [Nemotron-Labs-Sigma collection (Hugging Face)](https://huggingface.co/collections/nvidia/nemotron-labs-sigma-6aed16ae59cfa91daa36c53b)
- [Large Language Continuous Diffusion Models (arXiv 2610.02665)](https://arxiv.org/abs/2610.02665)
- [NVIDIA Nemotron Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-nemotron-open-model-license/)

Source: AI/TLDR LLM registry, https://ai-tldr.dev/models/nemotron-labs-sigma/
