# MACaQ — Huawei open-sources a 4-bit quantizer for GGUF, vLLM and Diffusers

> Huawei's MACaQ shrinks text, image, video and audio models to 4 bits with one layer-by-layer recipe.

- Org: Huawei Computing Systems Lab
- Date: 2026-11-10
- Category: repo, tool, paper, algorithm
- Source: https://github.com/huawei-csl/MACaQ
- Canonical page: https://ai-tldr.dev/releases/huawei-csl-macaq/

## In short

MACaQ is Huawei CSL's Apache-2.0 quantizer that calibrates 4-bit weights and activations one layer at a time. Its Qwen3.6-35B-A3B GGUF is closer to FP16 than Unsloth's file of the same size.

## What it is

MACaQ (Model Agnostic Calibrated Quantization) is an open-source toolkit from Huawei Computing Systems Lab that compresses models to 4-bit weights and activations (MXFP4 W4A4) or to 2–4-bit GGUF files. One method covers LLMs, MoE models, FLUX.2 [klein] image, Wan2.2 video and Qwen3-TTS speech models, with backends for llama.cpp, vLLM, Diffusers and PyTorch.

## How it works

Each linear layer is handled on its own: a per-channel scale and a block-Hadamard rotation are applied to the activations at inference time, and the inverse transforms are folded into the weights, so nothing crosses a layer boundary. A Hessian-aware step called Flip-Refinement then adjusts individual FP4 codes to beat round-to-nearest. The paper reports a 122B model quantized in 64 minutes with 32.5 GB of memory.

## Why it matters

Most strong quantization methods need model-specific engineering for every new architecture. A layer-local recipe works on new models as they ship. For local users, the README shows a refined Qwen3.6-35B-A3B Q4_K_XL GGUF at KL 0.013367 versus 0.015731 for Unsloth's UD-Q4_K_XL at the same 22.36 GB, and W4A4 Qwen3.8-27B in vLLM decoding 2.58x faster than BF16.

## Who it's for

people running open models locally and inference engineers

## Try it

https://github.com/huawei-csl/MACaQ

## Links

- [GitHub Repo](https://github.com/huawei-csl/MACaQ)
- [Huawei CSL Blog](https://huawei-csl.com/blog/macaq/)
- [arXiv Paper](https://arxiv.org/abs/2611.10649)
- [HF Papers](https://huggingface.co/papers/2611.10649)
- [Hacker News Thread](https://news.ycombinator.com/item?id=50458166)