Can You Stream JSON? Combining Streaming with Structured Outputs

// In plain English

Streaming lets a model send its answer to you word by word, the instant each piece is ready, instead of making you wait for the whole reply. Structured output is the opposite habit: you ask the model to return strict JSON that matches a fixed schema, so your code can JSON.parse it without guessing.

These two pull in opposite directions. Streaming wants to hand you partial text early. JSON is only valid once it is complete — a half-sent object like {"name": "Ja is not parseable. You cannot call a JSON parser on it; it just throws an error. So "can you stream JSON?" has an awkward answer: yes, the bytes arrive as a stream, but you can't treat each fragment as a finished object.

Think of receiving a long fax of a filled-out form. With plain prose, you can read each line as it prints — it makes sense on its own. With a strict form, half a page is useless: the closing brackets, the last field, the comma that separates two entries are all still in the machine. You either wait for the full page, or you teach yourself to read a form that is missing its bottom half — and fill in the blanks carefully. That second skill is what this article is about.

// Why it matters

The whole point of streaming is perceived speed. A user watching text appear feels a response is fast, even if the total time is the same. The moment you switch the model to strict JSON mode, that benefit is at risk — because the natural way to consume JSON is to wait for the last byte, parse once, and render. At that point you have re-introduced the very lag streaming was supposed to hide.

This bites hardest in three common situations a builder runs into:

• A UI that renders structured data live. You want a product card, a table, or a form to fill in field-by-field as the model writes it — not pop into existence three seconds later. That needs partial JSON you can read mid-stream.
• Tool calls / function calling. When a model decides to call a tool, its arguments are themselves a JSON object, and they stream as fragments. If you want to show "the assistant is searching for: flights to Tokyo…" while the argument is still being written, you are parsing partial JSON whether you meant to or not.
• Long structured outputs. A big JSON array of 50 items takes real time to generate. Blocking the entire thing means a long blank wait; streaming it means you can start processing item 1 while item 50 is still being written.

The tension is real and there is no magic setting that removes it. What exists instead is a small set of patterns — partial parsing, splitting prose from JSON, or simply deciding not to stream the structured part — and the skill is picking the right one for the job. Knowing them stops you from either shipping a laggy UI or fighting a parser that throws on every chunk.

// How it works

To stream structured output well, you first need to see what actually comes down the wire. A streaming API does not send you finished objects — it sends a series of small events, and the text (or JSON) arrives inside delta events, each carrying a tiny fragment that you append to a growing buffer.

What a stream actually delivers

A typical message stream emits a start event, then many delta events (each a piece of text), then stop events. For ordinary prose the deltas are text_delta fragments. For a tool call, the model's JSON arguments arrive as a different delta type — an input_json_delta — where each event carries a partial JSON string, not a parsed object. You concatenate those partial strings yourself; only when the block finishes do you have a complete, parseable argument object.

Notice that after the second delta your buffer holds {"city":"Tok — syntactically broken. A strict parser rejects it. Only at block stop is the buffer guaranteed to be a complete, valid object. That is the core problem in one picture: validity arrives only at the end, but you wanted to use the data along the way.

The accumulate-and-parse loop

The base pattern, regardless of language or provider, is: keep a string buffer, append every delta's fragment, and attempt to parse — but expect failure until the end. Here it is with a tool-call stream, where the streamed fragments are the tool's input JSON:

let jsonBuffer = "";

for await (const event of stream) {
  if (
    event.type === "content_block_delta" &&
    event.delta.type === "input_json_delta"
  ) {
    // Each delta is a PARTIAL JSON string, not an object.
    jsonBuffer += event.delta.partial_json;
    // jsonBuffer is usually invalid here — don't JSON.parse blindly.
  }
}

// Only safe to parse once the block is complete.
const args = JSON.parse(jsonBuffer);

This works, but it only gives you the final object — no live updates. To render during the stream, you need a parser that tolerates an unfinished buffer. That is the next section.

// Partial and relaxed JSON parsing

To show a half-built object as it streams, you need a partial JSON parser (sometimes called a relaxed, streaming, or forgiving parser). Instead of throwing on incomplete input, it does its best: it closes any open strings, objects, and arrays in its head, and returns whatever is unambiguous so far.

Given the broken buffer {"city": "Tok, a partial parser returns { city: "Tok" } — it understands you are mid-way through a string value and hands you the prefix. A strict parser returns an error. As more deltas arrive, the partial parse keeps improving until it matches the final object exactly.

What a partial parser can and cannot know

The key insight: a partial parser only surfaces fields it is sure about. A number that is still being typed ("age": 3, which could become 30 or 300) is usually withheld until the next token confirms it. This means your live UI should treat every field as provisional — it may appear, then refine — until the stream ends. Render optimistically, but don't act irreversibly on a value that isn't final.

Several small libraries do this (search for "partial json parser" or "streaming json" in your language's package registry), and some SDKs expose a partial-parse helper directly on the stream object so you get a progressively-completed object on each event without wiring up your own parser. Check your SDK's streaming docs before pulling in a dependency.

// Choosing an approach

There are three sane ways to handle structured output under streaming. Pick by how much you actually need live partial data versus how simple you want your code.

The "stream the prose, block the JSON" split

A very practical middle path: have the model stream a human-readable explanation as normal text (which the user watches appear live), and return the machine-readable structured part separately — for example as a tool call whose arguments you only read once complete, or as a final JSON block at the end. The user gets the streaming feel from the prose; your code gets clean, complete JSON to parse. You avoid partial-JSON handling entirely while keeping the perceived speed.

When to just not stream

If the structured output is the only thing you need and nothing renders it live — a classification label, an extraction feeding a database, a back-end pipeline step — streaming adds complexity for zero user benefit. Make a normal blocking request, parse the JSON once, done. Streaming earns its keep only when a human is watching output appear, or when the output is long enough that processing-as-you-go is a real win. (One unrelated reason to still stream: very large outputs can hit request timeouts on a blocking call, so streaming is sometimes a transport necessity even when you don't show partials.)

// Common pitfalls

• Calling a strict parser on every chunk. It throws on all but the last fragment. Either accumulate and parse once at the end, or use a partial parser on purpose — don't rely on catching exceptions in a hot loop.
• Assuming each delta is a whole token or a whole field. Deltas are arbitrary byte fragments. A single field's value can span several deltas, and one delta can straddle two fields. Always concatenate; never assume boundaries.
• Acting on provisional values. A field a partial parser shows mid-stream can still change as more text arrives. Don't fire a side effect (charge a card, send an email) off a value until the stream completes.
• Forgetting the stream can end early. If the model hits its max_tokens cap, the JSON may be truncated and never becomes valid. Check the finish/stop reason — a max_tokens stop means your buffer is incomplete on purpose, not a bug to retry blindly.
• Mixing up the delta types. Plain text streams as text deltas; tool-call arguments stream as input_json_delta fragments. They are different event shapes — branch on the delta type, don't treat all deltas as text.
• Ignoring schema constraints under streaming. Structured-output / JSON-mode guarantees the final object matches your schema, but says nothing about intermediate states — your partial-render code must tolerate missing and half-typed fields.

// Going deeper

Schema-constrained generation and streaming. Strict structured-output modes constrain the model so the finished response is guaranteed valid against your JSON schema. This is great for correctness, but remember the guarantee is about the end state. While streaming, you still pass through every invalid intermediate buffer on the way there. Constrained decoding makes the destination safe; it does not make the journey parseable.

Tool calls are the most common place this shows up. Most developers meet streamed partial JSON not through a JSON-mode response but through function calling: a model's tool-call arguments are a JSON object that arrives as input_json_delta fragments. If you build an agent UI that shows "calling search with query: …" as the argument is typed, you are doing partial JSON parsing of tool inputs. The same accumulate-and-partial-parse loop applies. See structured outputs explained for how schemas and tool arguments relate.

Validation belongs at block stop, not during. A clean architecture streams partial JSON only for display, then runs full schema validation once on the complete object before doing anything consequential with it. Treat the live partial view as a preview and the final parsed object as the source of truth. If they ever disagree, trust the final one.

Robustness details that bite in production. Network chunking is independent of the model's token boundaries, so a single SSE event can split a multi-byte UTF-8 character across two deltas — decode the byte stream carefully, or work at the string level your SDK already provides. Also handle the stream simply stopping: a dropped connection or a refusal mid-generation leaves you with a permanently-incomplete buffer, so always have a timeout and a "never completed" path, not just a success path.

Where to go next. Solidify the two halves separately: read how streaming works for the event model in depth, and the JSON-mode vs structured-outputs comparison for how the schema guarantee is enforced. Combine them with the patterns here and you can stream structured data without fighting your parser.