Google / Kaggle · 2026-06-04 · notable
Kaggle Benchmarks Goes Local — Google Lets Coding Agents Author Eval Tasks From VS Code, Cursor, and Antigravity Through a Kaggle CLI, an SDK, and a 'write-kaggle-benchmarks' Skill
Google opens Kaggle Benchmarks task authoring to local dev environments. A new Kaggle CLI benchmarks subcommand, the kaggle-benchmarks Python SDK, and an agent skill let developers write, push, and run evaluations from VS Code, Cursor, or Antigravity instead of the web editor.
Google opens Kaggle Benchmarks authoring to any local IDE so coding agents can build new eval tasks themselves.
Key specs
| Community tasks | 10,000+ |
|---|---|
| Kaggle cli stars | 7,364 |
What is it?
Kaggle Benchmarks is the community leaderboard system Google launched late last year for measuring model capabilities on user-submitted evaluation tasks. Until now you could only build tasks in Kaggle's web editor; this update opens that to any local dev environment.
How does it work?
The Kaggle CLI gains a benchmarks subcommand tree: `auth` and `init` fetch Model Proxy credentials, and `tasks push / run / list / status / download / publish` manage tasks end-to-end. Tasks are defined in Python against the kaggle-benchmarks SDK. A 'write-kaggle-benchmarks' skill ships in Kaggle/kaggle-skills to teach coding agents like Antigravity and Cursor how to call the SDK and CLI through natural-language instructions.
Why does it matter?
Authoring evaluations is the slow part of a benchmark race. Closing the loop between Cursor or Antigravity and Kaggle Benchmarks means anyone can spin up a task in their IDE, push it to the public leaderboard, and have it scored against the catalogue of frontier models without leaving their editor.
Who is it for?
ML researchers, eval engineers, and Antigravity / Cursor / VS Code users who want their agent to author benchmark tasks.
Try it
kaggle benchmarks init