TOON

// Overview

Token-Oriented Object Notation (TOON) is a compact, human-readable encoding of the JSON data model, designed for what you put into an LLM prompt. It borrows YAML's indentation for nested objects and CSV's tabular layout for uniform data: a header such as `forecast[3]{day,temp{min,max},condition,rainChance}:` declares the row count and the field list once, then each element is a single comma-separated row. The project frames it as a translation layer — keep using JSON in your code, encode to TOON at the boundary where the data becomes prompt text.

The format has four forms, picked automatically from the shape of the data: inline form for primitive arrays, tabular form for uniform object arrays, keyed tabular for maps of uniform objects (config maps, feature flags, records by ID), and list form as the fallback for mixed or non-uniform data. Round-trips are deterministic and lossless, and the declared lengths and field lists double as guardrails — truncated or malformed model output does not silently pass.

The repository's own benchmarks measure 244 retrieval questions across four models. On the mixed-structure track TOON scores 72.2% accuracy on 2,474 tokens against JSON's 71.4% on 4,308 — the same comprehension for 42.6% fewer tokens — and it ranks first on accuracy per 1,000 tokens. The README is equally clear about where TOON loses: deeply nested or non-uniform data, where compact JSON often wins outright; purely tabular data, where CSV is smaller; and latency-bound deployments, where compact JSON can be processed faster. The reference implementation is TypeScript on npm as `@toon-format/toon` with a separate CLI, MIT licensed, with a versioned spec and community ports in other languages.

// What it does

• Lossless, deterministic round-trip with the JSON data model — objects, arrays and primitives
• Four automatic forms: inline, tabular, keyed tabular and list, with nested field groups folded into headers
• Declared `[N]` row counts and `{fields}` lists that catch truncated or malformed model output
• Streaming encode and decode (`encodeLines`, `decodeFromLines`, `decodeStream`) for large datasets
• A CLI that converts either direction, reads stdin, and prints token savings with `--stats`
• A published spec with a shared conformance test suite, plus editor support and community ports

// Getting started

Try it on your own data with the CLI before installing anything, then encode at the point where data becomes prompt text.

cat data.json | npx @toon-format/cli --stats

npm install @toon-format/toon
npm install -g @toon-format/cli

import { encode } from '@toon-format/toon'

const data = {
  users: [
    { id: 1, name: 'Ada', role: 'admin' },
    { id: 2, name: 'Bob', role: 'user' }
  ]
}

console.log(encode(data))
// users[2]{id,name,role}:
//   1,Ada,admin
//   2,Bob,user

import { encodeLines } from '@toon-format/toon'

for (const line of encodeLines(largeData)) {
  process.stdout.write(`${line}\n`)
}

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Cut prompt cost when you feed a model tables of uniform records — orders, logs, rows from a database
• Give a model structural guardrails: a declared row count and field list it can check its reading against
• Shrink MCP or tool responses before they land in an agent's context window
• Compare formats honestly — the repo's benchmarks show where JSON or CSV is still the better choice

// How TOON compares

TOON alongside other open-source context compression & token budgets tools AI/TLDR tracks, ranked by GitHub stars.