# LookAtMyQuant — Inkling 2 Mini squeezed into 5.4 GB for Apple Silicon

> A quantization compiler packs Thinking Machines' new open model into 5.4 GB so it runs on a Mac.

- Org: ShinMegamiBoson
- Date: 2026-11-21
- Category: repo, tool, model
- Source: https://github.com/ShinMegamiBoson/lookatmyquant
- Canonical page: https://ai-tldr.dev/releases/shinmegamiboson-lookatmyquant-inkling-2-mini/

## In short

LookAtMyQuant, an MIT-licensed quantization compiler, shrinks Inkling 2 Mini's 18.89 GB of text weights to 5.38 GB at 1.63 bits per weight. The author reports 91.2% of the original's GSM8K and HumanEval score, on Apple Silicon.

## What it is

LookAtMyQuant now ships a 5.38 GB build of Inkling 2 Mini, the open model Thinking Machines released on November 20, down from 18.89 GB of original text weights. A second 8.87 GB build (2.69 bits per weight) is also published. Both run on Apple Silicon through the project's MLX runtime, with an optional small drafter model for faster decoding.

## How it works

The LookAtMyQuant compiler first measures how much each piece of a model hurts the output when compressed, as KL divergence from the original. It then picks one compression code per piece so the whole file fits a bits-per-weight budget with the least total damage. For Inkling 2 Mini it uses trellis codes on blocks and rows, and keeps a higher-precision floor on the tail of the output layer.

## Why it matters

In the README's own table, the 5.4 GB file solves 134 of 200 GSM8K problems and 60 of 164 HumanEval problems, ahead of Unsloth's larger 9.14 GB UD-Q2_K_XL file. Its KL divergence is still higher than Unsloth's 7.76 GB UD-IQ1_S, and Hacker News commenters questioned whether the benchmark gains are beyond noise. For now the files run only on Apple Silicon, not in llama.cpp.

## Who it's for

Mac users running local models, quantization researchers

## Try it

pip install git+https://github.com/ShinMegamiBoson/lookatmyquant.git && lamq generate shinmegamiboson/Inkling-2-Mini-LookAtMyQuant-5.4GB -p "What is 17 times 23?"

## Links

- [GitHub Repo](https://github.com/ShinMegamiBoson/lookatmyquant)
- [5.4 GB Model](https://huggingface.co/shinmegamiboson/Inkling-2-Mini-LookAtMyQuant-5.4GB)
- [Method Write-up](https://lookatmyquant.pages.dev/)
- [Base Model](https://huggingface.co/thinkingmachines/Inkling-2-Mini)
- [HN Discussion](https://news.ycombinator.com/item?id=50591853)
