What can the esp32-ai model actually do?

esp32-ai writes short, coherent children's stories in the style of the TinyStories dataset it was trained on. The 28.9M-parameter model produces about 9.5 tokens per second to a small screen wired to the chip. It cannot answer questions, follow instructions, write code, or recall facts — those tasks need a much larger reasoning core than an ESP32-S3 can hold.

How does esp32-ai fit a 28.9M model on a chip with only 512 KB of SRAM?

esp32-ai stores most of the model — a 25M-parameter embedding table — in the chip's 16 MB of flash instead of RAM. The 512 KB SRAM only holds the compute core for each token, and the runtime reads roughly 450 bytes of embeddings from flash per token. The technique comes from Google's Gemma 3n/4 Per-Layer Embeddings idea, now applied to microcontrollers for the first time.

How does esp32-ai compare to previous on-device LLMs?

esp32-ai is roughly 100 times larger than the 260K-parameter models previously demonstrated on ESP32-class chips. Prior work (Karpathy's llama2.c line) topped out well under 1M parameters on this hardware. esp32-ai's per-layer-embedding trick unlocks models that would otherwise need a Raspberry Pi or Jetson to run.

Is esp32-ai open source and can I try it?

esp32-ai ships MIT-licensed at github.com/slvDev/esp32-ai, with training scripts, 4-bit quantization code, and flashing instructions. Anyone with an ESP32-S3 dev board and a screen can reproduce the demo. The repo grew past 1,200 stars in three days after landing on Hacker News on 2026-07-25.

slvDev · 2026-07-25 · major

esp32-ai — 28.9M-parameter LLM runs entirely on an $8 ESP32-S3

esp32-ai squeezes a 28.9M-parameter language model onto an $8 ESP32-S3 microcontroller — ~100 times larger than the best previous on-chip LLM. The model runs fully offline at 9.5 tokens per second and writes short stories to a wired screen.

GitHub OG card for slvDev/esp32-ai showing the ESP32-S3 LLM project

A hobbyist ports Gemma's Per-Layer Embeddings trick to a microcontroller and gets a 28.9M-param LLM running on $8 of silicon.

Quick facts

Maker	slvDev (independent)
License	MIT
Model size	28.9M params, 14.9 MB after 4-bit quantization
Hardware	ESP32-S3 microcontroller (~$8)
On-chip memory	512 KB SRAM + 8 MB PSRAM + 16 MB flash
Speed	9.5 tokens/sec end-to-end
Training data	TinyStories dataset
Key technique	Per-Layer Embeddings from Google Gemma 3n/4

What is it?

esp32-ai is an open-source project that runs a 28.9M-parameter language model entirely on an ESP32-S3 microcontroller — a chip that costs about $8 and has just 512 KB of SRAM. The model is trained on TinyStories, prints short stories to a wired screen at 9.5 tokens per second, and never touches the network.

How does it work?

The core trick is Per-Layer Embeddings, borrowed from Google's Gemma 3n/4 architecture. Most of the model's parameters (25M out of 28.9M) live in a big lookup table that gets read from, not multiplied against — so esp32-ai parks that table in the chip's 16 MB of slow flash memory and only pulls about 450 bytes into SRAM per token. The remaining compute core is 4-bit quantized to fit under 15 MB total, and each token cycle uses PSRAM as scratch space.

Why does it matter?

esp32-ai is roughly 100× larger than any language model previously demonstrated on an ESP32-class chip — earlier ports of Karpathy's llama2.c topped out around 260K parameters. That unlocks text generation on hardware small enough to sit inside a sensor, a toy, or a battery-powered node with no cloud dependency. The Per-Layer Embeddings recipe also generalizes: any model whose weights are dominated by an embedding table can now be flashed onto a microcontroller.

Who is it for?

embedded engineers, IoT tinkerers, edge-AI researchers

Frequently asked questions

What can the esp32-ai model actually do?: esp32-ai writes short, coherent children's stories in the style of the TinyStories dataset it was trained on. The 28.9M-parameter model produces about 9.5 tokens per second to a small screen wired to the chip. It cannot answer questions, follow instructions, write code, or recall facts — those tasks need a much larger reasoning core than an ESP32-S3 can hold.
How does esp32-ai fit a 28.9M model on a chip with only 512 KB of SRAM?: esp32-ai stores most of the model — a 25M-parameter embedding table — in the chip's 16 MB of flash instead of RAM. The 512 KB SRAM only holds the compute core for each token, and the runtime reads roughly 450 bytes of embeddings from flash per token. The technique comes from Google's Gemma 3n/4 Per-Layer Embeddings idea, now applied to microcontrollers for the first time.
How does esp32-ai compare to previous on-device LLMs?: esp32-ai is roughly 100 times larger than the 260K-parameter models previously demonstrated on ESP32-class chips. Prior work (Karpathy's llama2.c line) topped out well under 1M parameters on this hardware. esp32-ai's per-layer-embedding trick unlocks models that would otherwise need a Raspberry Pi or Jetson to run.
Is esp32-ai open source and can I try it?: esp32-ai ships MIT-licensed at github.com/slvDev/esp32-ai, with training scripts, 4-bit quantization code, and flashing instructions. Anyone with an ESP32-S3 dev board and a screen can reproduce the demo. The repo grew past 1,200 stars in three days after landing on Hacker News on 2026-07-25.

Try it

git clone https://github.com/slvDev/esp32-ai