ShinMegamiBoson · 2026-11-21 · notable
LookAtMyQuant — Inkling 2 Mini squeezed into 5.4 GB for Apple Silicon
In short
LookAtMyQuant, an MIT-licensed quantization compiler, shrinks Inkling 2 Mini's 18.89 GB of text weights to 5.38 GB at 1.63 bits per weight. The author reports 91.2% of the original's GSM8K and HumanEval score, on Apple Silicon.
ShinMegamiBoson · 2026-11-21 · 4 sources checked

A quantization compiler packs Thinking Machines' new open model into 5.4 GB so it runs on a Mac.
Key specs
| Parameters | 5.38 GB |
|---|---|
| Bits per weight | 1.63 |
What is it?
LookAtMyQuant now ships a 5.38 GB build of Inkling 2 Mini, the open model Thinking Machines released on November 20, down from 18.89 GB of original text weights. A second 8.87 GB build (2.69 bits per weight) is also published. Both run on Apple Silicon through the project's MLX runtime, with an optional small drafter model for faster decoding.
How does it work?
The LookAtMyQuant compiler first measures how much each piece of a model hurts the output when compressed, as KL divergence from the original. It then picks one compression code per piece so the whole file fits a bits-per-weight budget with the least total damage. For Inkling 2 Mini it uses trellis codes on blocks and rows, and keeps a higher-precision floor on the tail of the output layer.
Why does it matter?
In the README's own table, the 5.4 GB file solves 134 of 200 GSM8K problems and 60 of 164 HumanEval problems, ahead of Unsloth's larger 9.14 GB UD-Q2_K_XL file. Its KL divergence is still higher than Unsloth's 7.76 GB UD-IQ1_S, and Hacker News commenters questioned whether the benchmark gains are beyond noise. For now the files run only on Apple Silicon, not in llama.cpp.
Who is it for?
Mac users running local models, quantization researchers
Try it
pip install git+https://github.com/ShinMegamiBoson/lookatmyquant.git && lamq generate shinmegamiboson/Inkling-2-Mini-LookAtMyQuant-5.4GB -p "What is 17 times 23?"