agent-device

// Overview

agent-device is an open-source tool from Callstack that closes the loop between a coding agent and the app it is editing. Instead of writing a change and hoping it works, the agent can open the app on a simulator, emulator or physical device, inspect what is actually on screen, tap and type, and capture screenshots, video and logs as evidence. It covers iOS, Android and HarmonyOS, plus tvOS, Android TV, Amazon Vega OS, web, macOS and Linux.

The design choice that makes it work is the accessibility snapshot. Rather than asking a model to reason over raw screenshots, agent-device hands it a token-efficient tree of roles, labels and refs — `@e2 [button] "Add"` — and the agent acts through those refs and selectors. Commands that change the UI can wait for it to settle and return a diff of what changed, so the agent tracks state without re-screenshotting every step.

The same runtime is exposed three ways: a CLI, a built-in stdio MCP server (`agent-device mcp`), and a typed Node.js client for code that orchestrates its own agents. Sessions are scoped to the caller's git worktree and host-local device claims stop parallel agents from stealing each other's simulators. Working runs can be saved as `.ad` replay scripts for CI, or exported as strict Maestro YAML.

// What it does

• Accessibility-tree snapshots with refs and selectors, so agents act on structured UI state rather than screenshots alone
• One runtime behind three entry points: a CLI, an official stdio MCP server, and a typed Node.js API
• Targets iOS, Android and HarmonyOS simulators, emulators and physical devices, plus tvOS, Android TV, Vega OS, web, macOS and Linux
• Evidence capture for debugging: screenshots, video, logs, traces, network data, performance samples, crash details and React profiles
• Replay: save a working run as an `.ad` script to re-run locally or in CI, or export strict Maestro YAML
• Worktree-scoped sessions and device claims that keep parallel agents from taking over each other's devices
• Remote proxy and device-cloud routing for BrowserStack, AWS Device Farm and Limrun

// Getting started

agent-device needs Node.js 22.12 or newer (web automation needs Node.js 24+). Install the CLI and run `doctor` yourself before handing it to an agent — it checks that your platform toolchains are set up.

npm install -g agent-device@latest
agent-device doctor
agent-device help workflow

# Start a session
agent-device open Contacts --platform ios

# Inspect the screen (refs vary)
agent-device snapshot -i
# @e2 [button] "Add"

# Use the ref and wait for the UI to settle
agent-device press @e2 --settle
agent-device fill @e7 "Ada" --settle

# Capture evidence and close
agent-device screenshot ./contact-form.png
agent-device close

{
  "mcpServers": {
    "agent-device": {
      "command": "agent-device",
      "args": ["mcp"]
    }
  }
}

import { createAgentDeviceClient } from 'agent-device';

const client = createAgentDeviceClient({ session: 'qa-run' });
try {
  await client.apps.open({ app: 'com.apple.Preferences', platform: 'ios' });
  const snapshot = await client.capture.snapshot({ interactiveOnly: true });
  const button = snapshot.nodes.find((node) => node.role === 'button');
  if (button) await client.interactions.press({ ref: button.ref });
} finally {
  await client.sessions.close();
}

agent-device capabilities --platform android

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Letting a coding agent implement a screen, run it on the iOS simulator and Android emulator, and attach screenshots to the pull request
• Reproducing a crash on a device and capturing the logs and traces that lead up to it
• Checking whether a change causes unnecessary React Native re-renders, using the built-in React profiler capture
• Recording an exploratory run as an `.ad` replay script and re-running it in CI with screenshots and logs kept as artifacts

// How agent-device compares

agent-device alongside other open-source computer & browser use tools AI/TLDR tracks, ranked by GitHub stars.