Huawei Computing Systems Lab · 2026-11-10 · notable
MACaQ — Huawei open-sources a 4-bit quantizer for GGUF, vLLM and Diffusers
In short
MACaQ is Huawei CSL's Apache-2.0 quantizer that calibrates 4-bit weights and activations one layer at a time. Its Qwen3.6-35B-A3B GGUF is closer to FP16 than Unsloth's file of the same size.
Huawei Computing Systems Lab · 2026-11-10 · 5 sources checked
Huawei's MACaQ shrinks text, image, video and audio models to 4 bits with one layer-by-layer recipe.
What is it?
MACaQ (Model Agnostic Calibrated Quantization) is an open-source toolkit from Huawei Computing Systems Lab that compresses models to 4-bit weights and activations (MXFP4 W4A4) or to 2–4-bit GGUF files. One method covers LLMs, MoE models, FLUX.2 [klein] image, Wan2.2 video and Qwen3-TTS speech models, with backends for llama.cpp, vLLM, Diffusers and PyTorch.
How does it work?
Each linear layer is handled on its own: a per-channel scale and a block-Hadamard rotation are applied to the activations at inference time, and the inverse transforms are folded into the weights, so nothing crosses a layer boundary. A Hessian-aware step called Flip-Refinement then adjusts individual FP4 codes to beat round-to-nearest. The paper reports a 122B model quantized in 64 minutes with 32.5 GB of memory.
Why does it matter?
Most strong quantization methods need model-specific engineering for every new architecture. A layer-local recipe works on new models as they ship. For local users, the README shows a refined Qwen3.6-35B-A3B Q4_K_XL GGUF at KL 0.013367 versus 0.015731 for Unsloth's UD-Q4_K_XL at the same 22.36 GB, and W4A4 Qwen3.8-27B in vLLM decoding 2.58x faster than BF16.
Who is it for?
people running open models locally and inference engineers
Try it
https://github.com/huawei-csl/MACaQ