Simon Willison · 2026-04-16 · notable
Simon Willison: Qwen3.6-35B-A3B on my laptop drew me a better pelican than Claude Opus 4.7
Simon Willison runs the 21GB Qwen3.6-35B-A3B quant locally on a MacBook Pro M5 and finds it beats Anthropic's Opus 4.7 at the pelican-riding-a-bicycle SVG benchmark — and at flamingo-on-a-unicycle too.
A laptop-sized open-weight model drew a better pelican than Anthropic's frontier closed model — Simon Willison's classic SVG benchmark finally breaks the correlation.
What is it?
A blog post from Simon Willison, one of the AI community's most-read commentators, comparing Alibaba's new Qwen3.6-35B-A3B open-weight model with Anthropic's freshly-shipped Claude Opus 4.7. The comparison uses Willison's long-running 'pelican riding a bicycle' SVG benchmark — a quick-and-dirty visual test he has been running against every new model for over a year.
How does it work?
Willison ran a 20.9 GB Q4_K_S quantization of Qwen3.6-35B-A3B (a 35B-parameter Mixture-of-Experts with 3B active) locally on a MacBook Pro M5 via LM Studio and the llm-lmstudio plugin. He hit Opus 4.7 through the Anthropic API, including its max-thinking mode. The Qwen SVG renders a recognizable bicycle and a pelican with a pouch; Opus 4.7 botches the bicycle frame in both standard and max-thinking attempts. A secondary 'flamingo on a unicycle' test gives the same result, which he argues rules out the 'they trained on the pelican benchmark' explanation.
Why does it matter?
The pelican benchmark has historically tracked overall model quality, so a 21 GB local model beating a fresh frontier closed model is news for people tracking where the open-weight frontier actually is. It is also an argument against the benchmark as a serious eval — Willison explicitly flags this — while still being the kind of grounded, fetched-and-verified spot-check this community listens to.
Who is it for?
ML practitioners, local-LLM tinkerers, and anyone tracking the open vs closed capability gap.
Try it
llm install llm-lmstudio && llm -m qwen3.6-35b-a3b 'Generate an SVG of a pelican riding a bicycle'