Show Lab, NUS · 2026-09-09 · notable
Show-Harness — one semantic interface lets a VLM drive a robot arm
Show-Harness is an open control layer that exposes a robot as discrete semantic action units a vision-language model can reason over. Show Lab at NUS shipped the Apache-2.0 code, six LoRA adapters and the demonstration data with the paper.
A small vocabulary of semantic action units lets an off-the-shelf vision-language model control a robot arm with no robot-specific pretraining.
What is it?
Show-Harness exposes a robot as a compact set of discrete semantic action units — the kind of step a language model can reason about — instead of raw joint targets. Embodiment-specific interpreters then ground each unit into motion deterministically, so the model stays responsible for every physical decision. Show Lab at the National University of Singapore released the paper, the Apache-2.0 code, six LoRA adapters and the training corpus on September 9.
How does it work?
Two modes share the one interface. A closed frontier VLM can run the loop zero-shot through an API key, or a small open model fine-tuned on demonstrations emits one action token per step with no planner in front of it. Swapping robots changes only the interpreter and a primitives config file: Franka, AgileX Piper, ManiSkill and Isaac Lab all keep the same model-facing vocabulary and prompts. Nine optional plugins mount on the perception, reasoning and action stages of the loop.
Why does it matter?
Collecting robot demonstrations normally needs teleoperation hardware. GUMI, the browser tool shipped with Show-Harness, maps every action unit to a key, so a person — or a GUI-driving agent — can demonstrate a task by playing the robot, and each step is recorded as a ready training pair. The paper reports that a small open VLM becomes a capable policy with less than a few H200 GPU-hours of fine-tuning, which puts a working policy inside a small lab's budget.
Who is it for?
robotics researchers and labs without teleoperation rigs
Try it
bash scripts/setup.sh base && .venv/bin/python gumi/collect_rollouts_web.py data/rollouts_demo --sim