SeedRealtime

// Overview

SeedRealtime is a native audio-visual full-duplex large language model announced by ByteDance's Seed team on 5 August 2026. Seed describes it as 'a key step toward omni-modal interaction': it uses a unified architecture to natively fuse audio, video and text, enabling real-time interaction over continuous multimodal streams and what Seed calls a 'watch, listen, and speak' experience.

The design point is that everything happens inside one end-to-end model rather than a chain of parts. Seed contrasts it with two existing paradigms: cascaded systems that string together separate ASR, VLM and TTS modules and lose information and time at every handoff, and end-to-end models that are more fluent but still lean on an external voice-activity detector to decide turns, which keeps them 'essentially a half-duplex, one-question-one-answer interaction'. SeedRealtime instead unifies sound, vision, timing and expression in a single model, letting perception, understanding, decision-making and expression run in parallel over the incoming streams.

Seed groups the result into three claimed breakthroughs. Joint audio-visual understanding lets the model resolve homophone ambiguity from visual context and interpret temporal references in what it sees. Proactive interaction means it can speak up unprompted when the scene changes — for example flagging a target object the moment it appears — and weave tool calls into its replies. Natural conversational timing means turn-taking is decided continuously from multimodal signals rather than by external VAD rules, so it can stay quiet through bystander chatter and background noise instead of being falsely triggered.

For results, Seed reports end-to-end human evaluation rather than a public benchmark table: compared with cascaded models, SeedRealtime 'reduces audio-visual conversational pacing issues by half', with fewer mid-sentence cut-offs, fewer sluggish replies after a pause and fewer false triggers, and a significantly higher chance of completing a conversation smoothly. ByteDance has not published a technical report, a parameter count, a context window, weights or a public API endpoint for the model; the launch post states only that SeedRealtime 'has been fully rolled out, pioneering large-scale deployment of audio-visual full-duplex technology in the industry'.

// Strengths

• Single end-to-end model for audio, video and text — no ASR → VLM → TTS handoffs and no inter-stage information loss
• Turn-taking decided from multimodal context instead of an external voice-activity detector, so it holds up in noisy, multi-speaker scenes
• Proactive: can raise a reminder on its own when a watched object appears, and can weave tool calls into a spoken reply
• Uses the visual scene to disambiguate speech, including homophones and vague references like 'how do I do this'
• Human evaluation reports half as many audio-visual conversational pacing problems as cascaded systems

// Best for

• Real-time voice-and-camera assistants that need to watch a live scene and answer about what is on screen
• Hands-busy guidance — walking a user through a physical task and correcting them as the visual state changes
• Multi-speaker settings where the assistant must track who is talking and avoid replying to background chatter
• Watch-for-me tasks: holding an instruction in context and speaking up the moment the target appears