# DrivingBench — GPT-6 Astra is the only model to finish a real cone course > Four frontier models took turns steering a real car around cones; only GPT-6 Astra made it to the end. - Org: DrivingBench - Date: 2026-09-22 - Category: benchmark - Source: https://drivingbench.com/ - Canonical page: https://ai-tldr.dev/releases/drivingbench-llms-drive-a-corolla/ ## In short DrivingBench gives frontier models control of a real Toyota Corolla on a cone course, one command at a time. GPT-6 Astra finished it on its second try in 5:22. Claude Fable 5.1 got 45% of the way; Grok 4.6 and GPT-5.6 Sol barely started. ## What it is DrivingBench is an independent test of whether general-purpose AI models can drive a real car, not a simulator. Three researchers put GPT-6 Astra, Claude Fable 5.1, Grok 4.6 and GPT-5.6 Sol in charge of a Toyota Corolla's steering, accelerator and brakes on a fixed course of traffic cones. Each model got up to three attempts, and the site scores how far along the course it got. ## How it works The open harness exposes three MCP tools: observe() returns a road camera image plus speed and steering angle, set_motion() asks for a steering direction, a speed between 0.5 and 3.5 m/s and a duration, and stop_now() brakes. Commands go through an HTTP gateway to a comma device running a modified openpilot, whose own 100 Hz control loop turns the request into wheel angles and enforces safety limits. A licensed driver sat in the driver's seat, foot over the brake, for every run. ## Why it matters Most model rankings measure text, code or screen clicks. This one checks whether a model can turn camera frames into safe physical actions with real latency, and the gap is large: one model finished, one got under halfway, two failed near the start. Co-author Aditya Ramabadran told The Register that using a frontier model out of the box for real driving 'is definitely not practical', so read it as a test of perception and planning, not a path to self-driving cars. ## Who it's for embodied-AI and agent researchers ## Try it https://github.com/aditya-ramabadran/drivingbench_harness_v1 ## Quick facts - Built by: Aditya Ramabadran, Simon Mahns, Tobias Gessler - Car: Toyota Corolla with a comma device running openpilot - Course: About 130 m of traffic cones in an empty lot - Speed cap: 3.5 m/s - Attempts: Up to 3, in one continuous chat session - Harness license: MIT ## FAQ ### How long did GPT-6 Astra take to finish the DrivingBench course? GPT-6 Astra finished the DrivingBench cone course on its second attempt in 5 minutes 22 seconds. The Register reports it covered 134.7 meters at an average of 0.94 mph. The slow pace comes from the setup: the model drives one command at a time from inside a chat session, and speed is capped at 3.5 meters per second. ### How did Claude Fable 5.1 and Grok 4.6 do on DrivingBench? Claude Fable 5.1 was the closest rival on DrivingBench, reaching 45% of the course on its best attempt without finishing. Grok 4.6 reached 11% and GPT-5.6 Sol only 6%. The Register says the failed attempts mostly broke down at the first corner because of perception problems, meaning the models misread where the cones were. ### Can I run the DrivingBench harness myself? The DrivingBench harness is open source under the MIT license on GitHub. It needs a comma four device running openpilot v0.11.2, a supported Toyota with longitudinal control (it was tested on a Corolla), a macOS laptop with Python 3.12 and uv, and an empty private lot with cones. The README says a licensed, attentive safety driver must stay in the driver's seat at all times. ### How much did a DrivingBench run cost? The Register reports the DrivingBench runs used about 6.6 million tokens for $7.74 in API fees at discounted cache rates, plus $999 for the comma hardware. Counting both, the researchers put the cost at roughly $92.47 per mile driven, against about $0.18 per mile for fuel, which shows how far this is from a practical way to drive. ## Links - [DrivingBench Results](https://drivingbench.com/) - [Harness Code](https://github.com/aditya-ramabadran/drivingbench_harness_v1) - [The Register](https://www.theregister.com/ai-and-ml/2026/09/24/openais-astra-model-went-for-a-drive-and-no-one-died/5298715) - [HN Discussion](https://news.ycombinator.com/item?id=49817404)