# Zyphra ZAYA1-8B — AMD-Trained MoE Reasoning Model With <1B Active Parameters

> Zyphra trained an 8.4B sparse MoE with under a billion active params on AMD MI300X — and it matches open models 10–100x larger on math and code.

- Org: Zyphra
- Date: 2026-05-06
- Category: model
- Source: https://www.zyphra.com/post/zyphra-releases-zaya1-8b-a-reasoning-model-trained-on-amd-and-optimized-for-maximum-intelligence-density-per-parameter
- Canonical page: https://ai-tldr.dev/releases/zyphra-zaya1-8b/

## In short

Open-weight 8.4B mixture-of-experts with only 760M active parameters, trained end-to-end on 1,024 AMD MI300X GPUs. Hits 89.1 on AIME26 and 71.6 on HMMT, matching open models 10–100x larger. Apache 2.0.

## What it is

ZAYA1-8B is a small mixture-of-experts language model from Zyphra: 8.4B total parameters but only ~760M active per token. It's a reasoning-first open-weight model targeting math, code, and long-form analysis, trained end-to-end on AMD hardware rather than NVIDIA — a first at this scale for any well-known frontier-style release.

## How it works

Zyphra trained on 1,024 AMD Instinct MI300X GPUs with Pensando Pollara networking on IBM Cloud. The architecture combines a Compressed Convolutional Attention (CCA) variant, an MLP-based expert router that improves routing stability, and learned residual scaling. Post-training adds a reasoning RL cascade and a new test-time compute method called Markovian RSA, which chunks parallel reasoning traces to keep memory constant during long deliberation.

## Why it matters

Two things are notable. First, it shows AMD's MI300X stack is now production-ready for end-to-end frontier-style training, not just inference. Second, sub-1B active params getting 89 on AIME26 and 71 on HMMT keeps Zyphra's claim that you can win on intelligence-per-parameter without scaling totals to hundreds of billions — meaningful for on-device and edge deployment.

## Who it's for

open-weight researchers, anyone optimizing reasoning per FLOP, AMD-stack teams

## Try it

huggingface.co/Zyphra/ZAYA1-8B or serverless via cloud.zyphra.com

## Links

- [Announcement](https://www.zyphra.com/post/zyphra-releases-zaya1-8b-a-reasoning-model-trained-on-amd-and-optimized-for-maximum-intelligence-density-per-parameter)
- [Technical Post](https://www.zyphra.com/post/zaya1-8b)
- [Model Weights](https://huggingface.co/Zyphra/ZAYA1-8B)
- [MarkTechPost](https://www.marktechpost.com/2026/05/06/zyphra-releases-zaya1-8b-a-reasoning-moe-trained-on-amd-hardware-that-punches-far-above-its-weight-class/)
- [PR Newswire](https://www.prnewswire.com/news-releases/zyphra-releases-zaya1-8b-a-reasoning-model-trained-on-amd-and-optimized-for-maximum-intelligence-density-per-parameter-302764700.html)