# DeepSeek V4-Flash on one MI300X — 168 tok/s decode, no quantization

> Docker Compose stack that runs the full 304B DeepSeek V4-Flash unquantized on a single AMD MI300X, hitting 168 tok/s decode.

- Org: ryanzhou
- Date: 2026-08-04
- Category: repo, showcase, tool
- Source: https://github.com/ryanzhou/deepseek-v4-flash-mi300x
- Canonical page: https://ai-tldr.dev/releases/ryanzhou-deepseek-v4-flash-mi300x/

## In short

ryanzhou publishes a production stack that runs DeepSeek V4-Flash unquantized on a single AMD MI300X GPU. It hits 168 tok/s single-stream decode, 830 tok/s across 64 streams, and 256K validated context.

## What it is

deepseek-v4-flash-mi300x is an Apache-2.0 vLLM overlay from developer ryanzhou that runs the full 304B DeepSeek V4-Flash model on one AMD MI300X GPU without weight quantization. The whole 156.67 GiB of weights fits in HBM3.

## How it works

The stack fixes AMD's FNUZ FP8 cache-writer for the MI300X, patches MXFP4 routing corruption that hurt tool-calling accuracy, ships 21 GEMM shapes tuned for gfx942, and pairs a 20 GB GPU KV cache with a 96 GiB CPU KV tier. DSpark speculative decoding adds probabilistic drafting with causal verification.

## Why it matters

The MI300X has been an option on paper for months, but vanilla vLLM misbehaved on FP8 and MoE routing. This Apache-2.0 stack drops a ready-to-run Docker Compose setup that rents for about $1.99/hr on AMD Developer Cloud, giving self-hosted teams a cheaper, un-quantized alternative to NVIDIA for the flagship DeepSeek model.

## Who it's for

AMD GPU users, self-hosted inference teams

## Try it

git clone https://github.com/ryanzhou/deepseek-v4-flash-mi300x

## Links

- [GitHub Repo](https://github.com/ryanzhou/deepseek-v4-flash-mi300x)
- [HN Discussion](https://news.ycombinator.com/item?id=49166386)
- [DeepSeek V4-Flash Model](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731)