# esp32-ai — 28.9M-parameter LLM runs entirely on an $8 ESP32-S3

> A hobbyist ports Gemma's Per-Layer Embeddings trick to a microcontroller and gets a 28.9M-param LLM running on $8 of silicon.

- Org: slvDev
- Date: 2026-07-25
- Category: repo, showcase
- Source: https://github.com/slvDev/esp32-ai
- Canonical page: https://ai-tldr.dev/releases/slvdev-esp32-ai/

## In short

esp32-ai squeezes a 28.9M-parameter language model onto an $8 ESP32-S3 microcontroller — ~100 times larger than the best previous on-chip LLM. The model runs fully offline at 9.5 tokens per second and writes short stories to a wired screen.

## What it is

esp32-ai is an open-source project that runs a 28.9M-parameter language model entirely on an ESP32-S3 microcontroller — a chip that costs about $8 and has just 512 KB of SRAM. The model is trained on TinyStories, prints short stories to a wired screen at 9.5 tokens per second, and never touches the network.

## How it works

The core trick is Per-Layer Embeddings, borrowed from Google's Gemma 3n/4 architecture. Most of the model's parameters (25M out of 28.9M) live in a big lookup table that gets read from, not multiplied against — so esp32-ai parks that table in the chip's 16 MB of slow flash memory and only pulls about 450 bytes into SRAM per token. The remaining compute core is 4-bit quantized to fit under 15 MB total, and each token cycle uses PSRAM as scratch space.

## Why it matters

esp32-ai is roughly 100× larger than any language model previously demonstrated on an ESP32-class chip — earlier ports of Karpathy's llama2.c topped out around 260K parameters. That unlocks text generation on hardware small enough to sit inside a sensor, a toy, or a battery-powered node with no cloud dependency. The Per-Layer Embeddings recipe also generalizes: any model whose weights are dominated by an embedding table can now be flashed onto a microcontroller.

## Who it's for

embedded engineers, IoT tinkerers, edge-AI researchers

## Try it

git clone https://github.com/slvDev/esp32-ai

## Quick facts

- Maker: slvDev (independent)
- License: MIT
- Model size: 28.9M params, 14.9 MB after 4-bit quantization
- Hardware: ESP32-S3 microcontroller (~$8)
- On-chip memory: 512 KB SRAM + 8 MB PSRAM + 16 MB flash
- Speed: 9.5 tokens/sec end-to-end
- Training data: TinyStories dataset
- Key technique: Per-Layer Embeddings from Google Gemma 3n/4

## FAQ

### What can the esp32-ai model actually do?

esp32-ai writes short, coherent children's stories in the style of the TinyStories dataset it was trained on. The 28.9M-parameter model produces about 9.5 tokens per second to a small screen wired to the chip. It cannot answer questions, follow instructions, write code, or recall facts — those tasks need a much larger reasoning core than an ESP32-S3 can hold.

### How does esp32-ai fit a 28.9M model on a chip with only 512 KB of SRAM?

esp32-ai stores most of the model — a 25M-parameter embedding table — in the chip's 16 MB of flash instead of RAM. The 512 KB SRAM only holds the compute core for each token, and the runtime reads roughly 450 bytes of embeddings from flash per token. The technique comes from Google's Gemma 3n/4 Per-Layer Embeddings idea, now applied to microcontrollers for the first time.

### How does esp32-ai compare to previous on-device LLMs?

esp32-ai is roughly 100 times larger than the 260K-parameter models previously demonstrated on ESP32-class chips. Prior work (Karpathy's llama2.c line) topped out well under 1M parameters on this hardware. esp32-ai's per-layer-embedding trick unlocks models that would otherwise need a Raspberry Pi or Jetson to run.

### Is esp32-ai open source and can I try it?

esp32-ai ships MIT-licensed at github.com/slvDev/esp32-ai, with training scripts, 4-bit quantization code, and flashing instructions. Anyone with an ESP32-S3 dev board and a screen can reproduce the demo. The repo grew past 1,200 stars in three days after landing on Hacker News on 2026-07-25.

## Links

- [GitHub repo](https://github.com/slvDev/esp32-ai)
- [HN discussion](https://news.ycombinator.com/item?id=49050512)
- [Author's X post](https://x.com/slvDev/status/2080322138410524737)
- [Source tree](https://github.com/slvDev/esp32-ai/tree/main/src)