agent-device

@e2edev/agent-device drives iOS simulators and Android emulators through agent-device, against the same engine contract the browser engine implements. agent, app, screen, and expect work as they do everywhere; this page covers what is specific to a device target. Import test from the package to have device typed, and declare requires: ['device'] in a suite that also runs on the web. Choose Mobile (iOS/Android) in e2e init to add the package with a Settings example. It defaults to iOS on macOS and Android elsewhere, uses one worker, and needs no APP_URL. iOS needs Xcode and a simulator; Android needs the Android SDK and an emulator. Bring your own agent-device: the package declares agent-device as a peer dependency (0.21.x) and does not install it. A project that already drives devices with the agent-device CLI keeps its version and one copy in node_modules; e2e init adds agent-device only to a project that has none. agent-device is 0.x and its minors break, so the range pins the minor the engine was built and tested against and moves with each engine release. A version outside the range may be rejected by your package manager as an unmet peer (npm’s ERESOLVE).

Engine

e2e.config.ts

import type { E2EConfig } from '@e2edev/e2e';
import { createAgent } from '@e2edev/e2e/agent';
import { agentDevice } from '@e2edev/agent-device';
import { agentDeviceTools } from '@e2edev/agent-device/tools';
import { gateway } from 'ai';

const iphone = agentDevice({ platform: 'ios', app: 'Settings' });
const pixel = agentDevice({ platform: 'android', app: 'com.android.settings' });

export default {
  targets: [
    { name: 'iphone', engine: iphone },
    { name: 'pixel', engine: pixel },
  ],
  workers: 1,
  agents: {
    default: createAgent({
      model: gateway('openai/gpt-5.6-luna'),
      tools: agentDeviceTools(iphone, pixel),
    }),
  },
} satisfies E2EConfig;

One engine per device family; a config declares as many as it needs. A test using the shared device APIs can run on iOS and Android. Platform-specific labels or operations need platforms: ['ios'] or platforms: ['android']. Device attempts open their configured app; app.open() requires a URL and is unavailable on these targets.

Option	Type	Default	Meaning
`platform`	`'ios' \| 'android'`	required	Which device family the target boots.
`app`	`string`	none	App opened fresh at the start of every attempt: a bundle id, a package name, or a display name. Declares `app.restart()` and `app.clearState()`.
`appPath`	`string`	none	An iOS `.app` bundle or Android `.apk`, resolved against the project root, installed once per worker after boot and before the first attempt. Without `app`, the installed bundle id or package is the app opened per attempt.
`device`	`string \| readonly string[]`	every booted device	Simulator or emulator name or UDID. A list is a pool: the engine declares one worker per entry and worker slot `n` drives the `n`th; an empty list is `INVALID_CONFIG`. Omitted, `prepare` discovers the booted devices of the platform and drives as many as the run has slots; with none booted, agent-device boots one.
`session`	`string`	`e2e-<target name>`	agent-device session name, before the worker slot: slot `n` drives its device under `<session>-<n>`. One run per session at a time.
`snapshot`	`'full' \| 'interactive'`	`'full'`	Whether observations include static text.
`settle`	`number \| false`	`150`	For agent actions: milliseconds the UI must hold still after a tap, fill, or back before the agent observes again; `false` skips the wait. A test’s own steps never settle; `expect` verifies their outcome.
`transition`	`number`	`500`	For a test’s steps: how long a control that appeared or moved with the last action gets to finish arriving before it is acted on. Accessibility frames report final positions from a transition’s first frame, so this budget is what keeps a test from tapping where a sliding control has not arrived. Controls already in place before the action are acted on at once.
`identity`	`string`	`app`, else `appPath`	Logical app identity for cache and session keying; see the app under test.
`environment`	`'test' \| 'staging' \| 'production'`	`'test'`	A label for the report and the identity digest.

Every optional value also accepts undefined, so a config passes device: process.env.E2E_DEVICE or appPath: process.env.E2E_APP_PATH straight through, with no conditional spread. The devices boot in prepare, once per run and outside every launch budget: each worker slot the run will use boots its device and opens the pinned app once, so its automation runner is up before any worker starts, one slot after another. init then finds a booted device. On a CI runner, where a cold boot and a runner launch take a minute or more, that time is spent before the run’s clock starts instead of inside launchTimeout. agent-device commands take no abort signal, so a cancelled or timed-out command is abandoned by its caller but keeps running on the device; the next attempt waits for every abandoned command to settle before it opens anything, within the launch timeout. Each attempt relaunches app when one is pinned; without it the surface observes whatever is in the foreground and the model opens apps with the open_app tool. The engine declares one worker per configured device: a single device is one worker and a pool is one per entry. With no device, prepare uses the booted devices it finds, capped by the run’s slots. When none are booted, it boots one and reports a one-worker cap. The scheduler respects this cap, so two workers never drive the same simulator. To split a target’s test files across several devices, pass a pool and let workers be at least its size, or omit device to discover booted devices. Four booted simulators can run four files at a time when workers is at least four. The discovered pool reaches workers through the run’s environment, E2E_AGENT_DEVICE_POOL_<TARGET>_<digest>, so each slot resumes the device prepare booted for it.

import type { E2EConfig } from '@e2edev/e2e';
import { agentDevice } from '@e2edev/agent-device';

const iphones = agentDevice({
  platform: 'ios',
  app: 'com.example.app',
  device: ['iPhone 17', 'iPhone 17 Pro'],
});

export default {
  targets: [{ name: 'ios', engine: iphones }],
  workers: 2,
} satisfies E2EConfig;

Capabilities

Contract member	On a device
`observe`	The accessibility tree with rects and a viewport. iOS: `Button` is `button`, `TextField` and `SearchField` are `textbox`, `SecureTextField` is a secure `textbox`, `Cell` is `listitem`, `StaticText` is `text`, `Switch` is `switch` with `checked`. Android: `TextView` is `text`, `EditText` is `textbox`, `Switch` and `CheckBox` keep their roles (no `checked` state: Android exposes none), layouts are `group`, `RecyclerView` is `list`; a label echoed as the value is dropped. Other types keep their kebab-cased name. Element identifiers surface under the configured `testIdAttribute`. Pixels on request, with every secure node’s bounds painted black; a secure node without bounds withholds the image.
`perform`	`tap`, `doubleTap`, `longPress`, `fill`, `clear`, `hover`, `check`, `uncheck`, `press` (`Enter` or one character), `swipe` within a node, `dragTo`, and `focus` on editable fields (a touch surface focuses by tapping, and a tap on a control would activate it). A tap, check, or uncheck on a `switch` or `checkbox` lands on its innermost control, because UIKit reports a settings row as a labelled switch spanning the row with the real toggle as an unlabelled child; `check` and `uncheck` need the tree to expose the checked state, which Android switches do not. `selectOption`, `setInputFiles`, `scrollIntoView`, focus on a control, and other keys are `UNSUPPORTED_CAPABILITY`.
`locate`	Every `screen` query. Role queries skip nodes the platform reports as not visible; `visible: true` applies the same rule to the other query kinds. Text queries answer with the innermost match, so a host view echoing its child’s label (iOS `Text` over `StaticText`) counts once.
`swipe`	Viewport scroll in four directions.
`app`	`back`; `restart` and `clearState` when `app` or `appPath` is pinned. No `navigate`: a device has no app URL.
`artifacts`	Redacted screenshots under `screenshots/NNN-<label>.png`: every secure field’s bounds are painted black first, and a screenshot that cannot be redacted fails instead of being written. Video with `--video`, as `video/video.mp4` with taps shown. No traces.
`url`	`app://device/<bundle id>/<screen title>`, the trace cache’s anchor; the cache compares pathnames, so the app identity is in the path. The title is the navigation bar’s on iOS and the collapsing toolbar’s on Android.
`state`	Not declared. `test.setup` and `session:` are unavailable.

Trace cache on a device

The cache anchors a recorded agent.act step on the path the surface reports when the step begins and ends. This engine reports app://device/<bundle id>/<title>, with the title read off the navigation bar (its label, its inner text, or its identifier). A step recorded on /com.apple.preferences/General replays only when Settings is on General again. Recorded node actions replay through perform; viewport scrolling uses swipe. A mutating project-tool call such as open_app ends a replay, as does a target that no longer relocates. The read-only screenshot tool creates no replay gap. Pin app so attempts start on the same screen, and keep in-step app switching to the device fixture between steps.

Device

export interface Device {
  locator(selector: string): Locator;
  setNetwork(state: 'online' | 'offline'): Promise<void>;
  setAirplaneMode(enabled: boolean): Promise<void>;
  setPermission(permission: DevicePermission, state: 'grant' | 'deny' | 'reset'): Promise<void>;
  setLocation(coordinates: { latitude: number; longitude: number }): Promise<void>;
  clearLocation(): Promise<void>;
  setAppearance(mode: 'light' | 'dark'): Promise<void>;
  setOrientation(orientation: DeviceOrientation): Promise<void>;
  setBiometrics(sensor: 'faceid' | 'touchid' | 'fingerprint', result: 'match' | 'nonmatch'): Promise<void>;
  enrollBiometrics(sensor: 'faceid' | 'touchid', enrolled: boolean): Promise<void>;
  installApp(appPath: string, options?: { app?: string; reinstall?: boolean }): Promise<{ app: string; bundleId?: string }>;
  openApp(app: string, options?: { relaunch?: boolean }): Promise<void>;
  closeApp(): Promise<void>;
  foregroundApp(): Promise<{ name: string; bundleId?: string }>;
  home(): Promise<void>;
  back(): Promise<void>;
  alert(action: 'accept' | 'dismiss'): Promise<void>;
  dismissKeyboard(): Promise<void>;
  clipboard(): Promise<string>;
  setClipboard(text: string): Promise<void>;
}

export type DevicePermission =
  | 'camera' | 'microphone' | 'photos' | 'contacts' | 'notifications' | 'calendar'
  | 'location' | 'reminders' | 'motion' | 'siri' | 'media-library';
export type DeviceOrientation = 'portrait' | 'portrait-upside-down' | 'landscape-left' | 'landscape-right';

Every async method is a recorded step named device.<method>, bounded by config.actionTimeout and the attempt signal, and never involves a model. A lost session is APP_NOT_OPEN, a command that timed out is ACTION_FAILED, a command the platform cannot run is UNSUPPORTED_CAPABILITY, and anything else is ENGINE_FAILURE carrying agent-device’s message prefixed with the operation. installApp puts a build on the device from inside a test, for upgrade and fresh-install paths the appPath option cannot express. A plain install replaces the binary and keeps its data; reinstall: true removes the app named by app (default: the pinned app) first, and is APP_NOT_OPEN when neither names one. It resolves to the bundle id or package to openApp the build by.

const { app } = await device.installApp('./build/MyApp.app', { reinstall: true });
await device.openApp(app);

locator is a synchronous accessor: it mints a core Locator from an agent-device selector, for nodes the closed screen vocabulary cannot name.

await expect(device.locator('role=NavigationBar id=About')).toBeVisible();
await expect(device.locator('role=StaticText label="iOS Version"')).toHaveText('iOS Version');
await device.locator('id=SW_VERSION_SPECIFIER role=Button').tap();

Selector terms are agent-device’s: id, role, text, label, value, appname, windowtitle, and the flags visible, hidden, editable, selected, focused, enabled, hittable. Text terms compare exactly after whitespace collapsing; role accepts either the platform type (StaticText) or the contract role (text). Polling and strictness are the runner’s, as for any locator.

Agent tools

agentDeviceTools(...engines) returns a defineTool pack for createAgent. Pass every device engine the config declares: tool names are fixed, so two packs cannot be merged; the pack is scoped to the platforms of those engines and dispatches each call to the engine whose attempt is running (a worker runs one attempt at a time):

Tool	Replay	Purpose
`open_app`	gap	Bring an app to the foreground, optionally relaunched.
`swipe`	gap	Free-form swipe between two points in logical pixels.
`type_text`	gap	Type into the focused field, optionally pressing Return; for editors that hide the focused field from the tree.
`alert`	gap	Accept or dismiss a system alert.

The grammar verbs (tap, type, scroll) come from the engine manifest and need no tool. screenshot and tap_at are the agent’s own pixel tools, offered while no secret has been filled; the engine declares tapAt, so a tap_at whose point lands on nothing the tree lists taps the bare point in logical pixels. type_secret is not offered: secret fills require an origin the runner can check, and app:// has none. After a secret fill the agent offers neither screenshot nor tap_at and captures no pixels, including in later members of the same serial group. Pixels whose masking the engine cannot prove are withheld as MASKING_UNPROVEN.

Test API

Project

Web

Mobile

Engine contract

agent-device

Engine

Capabilities

Trace cache on a device

Device

Agent tools

Project tools

Writing an engine

app

Config

​Engine

​Capabilities

​Trace cache on a device

​Device

​Agent tools

Project tools

Writing an engine

app

Config

Engine

Capabilities

Trace cache on a device

Device

Agent tools