# Sam Witteveen — 'NVIDIA's Secret 2 Bit Model' on Nemotron 3.5 Bolt 74B-A9B

> A walkthrough of why NVIDIA's 2-bit Nemotron 3.5 Bolt works when most sub-4-bit models break.

- Org: Sam Witteveen
- Date: 2026-11-09
- Category: video, model
- Source: https://www.youtube.com/watch?v=WJpeW9lCSUM
- Canonical page: https://ai-tldr.dev/releases/sam-witteveen-nemotron-bolt-2-bit-nov9/

## In short

Sam Witteveen explains how NVIDIA's Nemotron 3.5 Bolt 74B-A9B squeezes into about 24GB with 2-bit experts, covering ParetoQ, pruning and quantization-aware training, then tests it locally on agentic tool calls.

## What it is

Sam Witteveen's video breaks down Nemotron 3.5 Bolt 74B-A9B, an NVIDIA model with 74B total and 9B active parameters whose experts are stored at about 2 bits, packing the whole model into roughly 24GB. He uses it to explain why the recipe matters more than the model itself.

## How it works

The video covers how standard quantization works and why it tends to break below 4 bits, the ParetoQ paper's case for 2-bit models, and mixed precision. It then walks through NVIDIA's three-stage pipeline of pruning, post-training and quantization-aware training, plus NVIDIA's QAD twist.

## Why it matters

A 74B-class model that fits in 24GB changes what runs on a single workstation GPU. Witteveen demos Bolt on an HP ZGX Fury with an SVG test, agentic tool-calling and concurrency runs, and discusses which hardware it suits.

## Who it's for

developers running models locally

## Try it

https://www.youtube.com/watch?v=WJpeW9lCSUM

## Links

- [Watch on YouTube](https://www.youtube.com/watch?v=WJpeW9lCSUM)
- [Model Card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB)
- [NVIDIA Research Report](https://research.nvidia.com/labs/nemotron/Nemotron-3.5-Bolt-24GB/)
- [ParetoQ Paper](https://arxiv.org/pdf/2502.02631)