e2e/engine

@e2edev/e2e/engine is the authoring import for an engine. defineEngine validates the body synchronously at config load, freezes it, computes its capability set, and returns the branded handle a target accepts.

import { defineEngine, type EngineHandle } from '@e2edev/e2e/engine';

export function defineEngine(spec: Engine): EngineHandle;
export function isEngineHandle(value: unknown): value is EngineHandle;

Validation fails with INVALID_CONFIG, never a silent demotion: a missing or empty version, a spiVersion other than 1, an unknown top-level key on an object literal (a class instance’s own fields are its state and are not checked), perform, locate, swipe, or tapAt without observe, a state, artifacts, or app manifest that is not an object, carries an unknown key, or has a non-function member (state needs both capture and restore; artifacts needs screenshot, and startTrace/stopTrace come together, as do startVideo/stopVideo), or a fixture name that is not a lower-camel identifier or shadows a universal fixture (agent, app, screen, platform, session). Every function on the handle is bound to the spec it came from, so a class instance is as valid a body as an object literal.

Engine

export interface Engine {
  readonly name: string;
  readonly version: string;
  readonly spiVersion: 1;
  readonly platform?: Platform;
  readonly workers?: number;

  observe?(context: OperationContext, options?: EngineObserveOptions): Promise<EngineSnapshot>;
  perform?(ref: NodeRef, action: LocatorAction, context: OperationContext): Promise<void>;
  locate?(expression: LocatorExpression, context: OperationContext): Promise<readonly SemanticNode[]>;
  swipe?(direction: ScrollDirection, momentum: Momentum | undefined, context: OperationContext): Promise<void>;
  tapAt?(point: ViewportPoint, context: OperationContext): Promise<void>;
  readonly fixtures?: Readonly<Record<string, EngineFixtureFactory>>;
  readonly state?: EngineStateCapability;
  readonly artifacts?: EngineArtifacts;
  readonly app?: EngineApp;
  url?(context: OperationContext): Promise<string>;

  prepare?(info: EnginePrepareInfo): Promise<void | EnginePrepareResult>;
  init?(info: EngineInitInfo): Promise<void>;
  startAttempt?(context: EngineAttemptContext): Promise<void>;
  endAttempt?(context: EngineCleanupContext): Promise<void>;
  dispose?(context: EngineCleanupContext): Promise<void>;
}

version is required and non-empty. It is recorded as provenance in the report and is part of the trace cache identity: an engine that resolves nodes differently must never replay another version’s traces. workers bounds how many workers the engine serves per target at once: one per surface it can drive concurrently, such as the size of a device pool. The scheduler never starts more for that target, whatever config.workers allows; an engine without such a bound omits it.

Member	Capability	Requires	Unlocks
`observe`	`observation`	-	judgment tier, prompt snapshots, trace replay
`perform`	`actions`	`observe`	the agent’s grammar verbs (`tap`, `type`, `press`, `select`, `type_secret`, node `scroll`, and `tap_at` when the point lands on a listed node) and the `Locator` action set with platform actionability checks
`locate`	`location`	`observe`	`screen`, `Locator`, `expect(locator)`
`swipe`	-	`observe`	the agent’s viewport `scroll` verb, `screen.swipe`, `scrollUntilVisible`
`tapAt`	`pointer`	`observe`	the agent’s `tap_at` verb when the point lands on nothing the tree lists: a tap at one viewport point in the CSS pixels of `SemanticNode.rect`, with no node behind it
`fixtures.<name>`	`<name>`	-	a contributed fixture; `requires: ['<name>']` gates at selection
`state`	`state`	-	`test.setup` sessions and the `session:` option
`artifacts`	`artifacts`	-	`app.screenshot()`, assertion evidence, `artifacts: ['trace']`, `artifacts: ['video']` and `--video`
`app`	-	-	`app.open()` and the agent’s `navigate` verb (`navigate`), `app.back()` (`back`), `app.restart()`, `app.clearState()`
`url`	-	-	trace start anchors, the secret-fill origin check

The agent offers the model only the verbs the engine declares: tap, type, press, select, and type_secret need perform; scroll needs swipe (a scroll aimed at a node is perform with { kind: 'swipe' }); navigate needs app.navigate; tap_at needs perform or tapAt. A tap_at whose point lands on nothing listed reaches tapAt; without it, the tool reports that this engine taps listed nodes only. An undeclared verb is absent from the toolset rather than a runtime surprise. engine.capabilities is the computed ReadonlySet of capability names and is what selection, the report, and UNSUPPORTED_CAPABILITY messages read.

Lifecycle

Hook	When	Budget
`prepare(info)`	once per run and target, in the runner process, after collection and before any worker starts: install a browser engine, fetch a toolchain, boot the devices of the `info.slots` worker slots the run will use. Progress reported through `info.log` streams as `notice` run events, which the list reporter prints above its live status block. `plan` is emitted, the report’s `startedAt` taken, and the app started only once every target is prepared, so nothing done here is on the run’s clock	none; only `info.signal` (the run’s interrupt) cuts it short, so a first-run download is never charged against `launchTimeout`; failure is infrastructure and ends the run before any test
`init(info)`	once per worker, before the first attempt; runs again after `dispose` when a config-held handle outlives an in-process worker, so it must work on a disposed engine as on a fresh one	`launchTimeout`; `info.signal` aborts on interrupt or timeout; failure is infrastructure
`startAttempt(context)`	before every attempt; `context.artifactsDir` is where artifacts go	`launchTimeout`; on failure or timeout `context.signal` aborts and `endAttempt` runs
`endAttempt(context)`	after every attempt, idempotent, and safe after a failed `startAttempt`	`cleanupTimeout`; `context.signal` aborts when the budget is spent; failure marks cleanup failed, the test keeps its status
`dispose(context)`	worker shutdown, whether or not `init` ran	`cleanupTimeout`; `context.signal` aborts when the budget is spent; failure is a run error

/** `workers` lowers the target's cap for this run to what prepare provisioned, `1` to `info.slots`. */
export interface EnginePrepareResult {
  readonly workers?: number;
}

export interface EnginePrepareInfo {
  readonly runId: string;
  readonly targetName: string;
  /** Worker slots the run starts for this target, `0` to `slots - 1`: the run cap, `workers`, and the units selected, whichever is smallest. */
  readonly slots: number;
  /** The run's environment, the one every worker starts with. */
  readonly env: NodeJS.ProcessEnv;
  readonly signal: AbortSignal;
  readonly log: (line: string) => void;
}

export interface EngineInitInfo {
  readonly runId: string;
  readonly targetName: string;
  readonly projectRoot: string;
  readonly app: { readonly baseUrl?: string; readonly allowedOrigins: readonly string[] };
  readonly testIdAttribute: string;
  readonly headed: boolean;
  /** 0-based slot among the target's workers; the lowest one free at spawn. */
  readonly workerSlot: number;
  readonly signal: AbortSignal;
}

export interface EngineAttemptContext {
  readonly attemptId: string;
  readonly artifactsDir: string;
  readonly signal: AbortSignal;
}

export interface EngineCleanupContext {
  readonly signal: AbortSignal;
  readonly timeoutMs: number;
}

app.baseUrl is absent when the project configured no app URL. A surface that needs one fails the call that needs it; it never navigates to a placeholder.

Observation

export interface EngineSnapshot {
  readonly url?: string;
  readonly nodes: readonly SemanticNode[];
  readonly viewport?: { width: number; height: number; scale: number };
  readonly pixels?: ObservationPixels;
  readonly maskedRegionCount?: number;
}

Return the surface as it is right now. The harness mints the revision, stamps it onto every ref, redacts registered secret values, bounds the byte size, and checks that maskedRegionCount covers every secure node when pixels are present. A pixels-only snapshot (nodes: []) is valid: that is how a screenshot-and-coordinates surface is observable. When location requires a snapshot, return url from that same capture. The harness reuses it for the executor’s path and trace-cache anchors. url() remains a fresh probe for replay polling after mutations; engines without snapshot location metadata still use that hook.

Actions

export type LocatorAction =
  | { kind: 'tap' | 'doubleTap' | 'check' | 'uncheck' | 'clear' | 'focus' | 'hover' | 'scrollIntoView' }
  | { kind: 'longPress'; durationMs?: number }
  | { kind: 'fill'; value: string; sensitive: boolean }
  | { kind: 'press'; key: string }
  | { kind: 'selectOption'; value: SelectOption }
  | { kind: 'setInputFiles'; paths: readonly string[] }
  | { kind: 'dragTo'; target: NodeRef }
  | { kind: 'swipe'; direction: ScrollDirection; momentum?: Momentum };

perform(ref, action, context) performs exactly one action on a ref this engine minted, from the newest observation or from locate (both live in one id space), with the platform’s actionability checks. The agent’s grammar verbs and the screen tier’s actions both bottom out here, so a surface implements each action once. Throw a retryable NODE_STALE when the ref no longer binds and ACTION_MAY_HAVE_COMMITTED when input may have reached the app; the harness never blindly repeats an uncertain mutation. An action kind the surface cannot express fails with UNSUPPORTED_CAPABILITY. tapAt(point, context) is the one action addressed by a point rather than a node: a ViewportPoint in the CSS pixels of SemanticNode.rect, already clamped to the viewport by the harness, which the agent’s tap_at reads off the observation’s pixels (divided by ObservationPixels.scale) when the tree lists nothing at the target. Dispatch the pointer at the point as given; there is no element to wait on. The grammar is closed. Anything beyond it is an agent-side defineTool.

App

The app manifest is the app under test: what the engine declares about it, and the hooks behind the universal app fixture.

export interface EngineAppDeclaration {
  readonly url?: string;
  readonly allowedOrigins?: readonly string[];
  readonly environment?: 'test' | 'staging' | 'production';
  readonly identity?: string;
  readonly command?: CommandConfig;
  readonly readyUrl?: string;
  readonly services?: readonly ServiceConfig[];
}

export interface EngineApp extends EngineAppDeclaration {
  navigate?(url: string, context: OperationContext): Promise<void>;
  back?(context: OperationContext): Promise<void>;
  restart?(context: OperationContext): Promise<void>;
  clearState?(context: OperationContext): Promise<void>;
}

The declaration is the engine’s to make and the runner’s to enforce: a browser engine passes its url and command options through, a device engine declares the bundle id it pins as identity. The runner resolves it once per target at config load (invalid values fail naming the target) and owns everything built on it: navigation and origin policy (allowedOrigins defaults to the URL’s origin), cache and session identity (identity defaults to the URL’s origin and path; a surface without a URL declares one or keys entries on the target alone), the report’s target record, and the app process (command, polled at readyUrl, default url; targets declaring the same command share one process). Every field is documented under the app under test; the resolved URL and origins come back to the engine in EngineInitInfo.app. navigate receives a URL the harness already resolved against the base URL and the origin policy, and is absent on a surface without addressable locations. Node actions never live here; they are perform.

Errors

export class EngineError extends Error {
  constructor(code: EngineErrorCode, message: string, options: { retryable: boolean; cause?: unknown });
  readonly code: EngineErrorCode;
  readonly retryable: boolean;
}

export const ENGINE_ERROR_CODES: readonly EngineErrorCode[];
export const RETRYABLE_ENGINE_ERROR_CODES: ReadonlySet<EngineErrorCode>; // NODE_STALE, FRAME_NOT_FOUND

export type EngineErrorCode =
  | 'NODE_STALE' | 'FRAME_NOT_FOUND' | 'FRAME_AMBIGUOUS' | 'NOT_ACTIONABLE'
  | 'ACTION_MAY_HAVE_COMMITTED' | 'OPERATION_TIMEOUT' | 'CANCELLED'
  | 'UNSUPPORTED_CAPABILITY' | 'INVALID_STATE' | 'ENGINE_FAILURE';

Code	Runner mapping	Retryable
`NODE_STALE`	re-resolve, then `LOCATOR_NOT_FOUND`	yes
`FRAME_NOT_FOUND`	re-resolve, then `LOCATOR_NOT_FOUND`	yes
`FRAME_AMBIGUOUS`	`LOCATOR_AMBIGUOUS`	no
`NOT_ACTIONABLE`, `ACTION_MAY_HAVE_COMMITTED`, `OPERATION_TIMEOUT`	`ACTION_FAILED`	no
`CANCELLED`	infrastructure `CANCELLED`	no
`UNSUPPORTED_CAPABILITY`	configuration `UNSUPPORTED_CAPABILITY`	no
`INVALID_STATE`	`APP_NOT_OPEN`	no
`ENGINE_FAILURE`, any non-`EngineError`	infrastructure `ENGINE_FAILURE`	no

Retryability is closed to RETRYABLE_ENGINE_ERROR_CODES, the two repeatable-read codes; a retryable claim on any other code is coerced to a non-retryable ENGINE_FAILURE. A contributed fixture method fails a step with the runner’s own taxonomy, exported from the same entry so an engine never carries a copy:

export class TestError extends Error {}            // a test failure: ASSERTION_FAILED, INVALID_LOCATOR, ACTION_FAILED, ...
export class ConfigurationError extends Error {}   // POLICY_DENIED, APP_URL_REQUIRED, UNSUPPORTED_CAPABILITY, ...
export class InfrastructureError extends Error {}  // the surface itself failed: BROWSER_INSTALL_FAILED, ...

Shared semantics

What the spec requires every engine and fixture to reproduce exactly is exported rather than re-implemented per engine:

Export	Spec	Use
`matchesText(actual, pattern)`, `toTextPattern(match, { exact? })`, `describePattern(pattern)`	04-locators	`TextPattern` matching for `locate` and text matchers
`urlMatches(current, expected, baseHref)`	03-assertions	`toHaveURL`-style comparison: relative strings resolve against the base, regexps test the serialized URL
`pollCondition({ deadline, signal, negated, evaluate, onTimeout })`, `Deadline`	03-assertions	Assertion polling cadence and the negation grace window
`validateJsonValue(value, label)`	04-locators	Rejects non-JSON data (`undefined`, `NaN`, cycles) a fixture would return

Beyond these and the four utilities described under Contributed fixtures (obj, raceAbort, withTimeout, withinCleanupBudget) there is no shared entry: anything else an engine needs is its own to write.

State

export interface EngineState {
  readonly format: string;
  readonly version: number;
  readonly data: unknown;
  readonly expiresAt?: string;
}

export interface EngineStateCapability {
  capture(context: OperationContext): Promise<EngineState>;
  restore(state: EngineState, context: OperationContext): Promise<void>;
}

A snapshot is a credential. The harness encrypts it at rest with a per-run AES-256-GCM key, binds it to run, target, engine, and app identity, deletes it at run end, and never logs, reports, digests, or sends it to a model. An engine must not persist, cache, log, or echo a snapshot, and restore replaces the surface’s persisted state rather than merging into it.

Artifacts

export interface EngineArtifacts {
  screenshot(label: string | undefined, context: OperationContext): Promise<string>;
  startTrace?(context: OperationContext): Promise<void>;
  stopTrace?(context: OperationContext): Promise<string | readonly string[]>;
  startVideo?(context: OperationContext): Promise<void>;
  stopVideo?(context: OperationContext): Promise<readonly VideoSegment[]>;
}

export interface VideoSegment {
  /** Relative to the attempt artifact directory. */
  readonly path: string;
  /** When the segment started recording, as an ISO timestamp; its first frame is at or just after it. */
  readonly startedAt: string;
}

Every returned path is relative to the artifactsDir the engine received in startAttempt. screenshot masks secure fields at the source. startTrace and stopTrace are declared together or not at all, and so are startVideo and stopVideo. The harness starts the video, then the trace, right after startAttempt (and after a session restore), under the launch budget, and stops them before endAttempt, under the cleanup budget. The video comes first because a surface that records both through one screencast sizes it for whichever client came first, and the recording is the one a person watches: the browser engine opens the attempt’s page when the video starts, so the trace that follows records its frames at the recording’s size. stopVideo returns every segment written, in order, each with the instant it started recording, so a consumer can place the report’s step timestamps on it. A surface whose recording is bound to a page returns one segment per page the attempt showed (a restart or a state reset opens a new one), and an attempt that never showed anything returns none. stopTrace returns the trace’s path, or every archive written in order when the trace had to be cut: a browser trace is bound to one context, so replacing the context closes the running trace as a segment and a new one records on from there (in the Playwright engine a state reset always replaces the context; a restart does so only while a recording is armed). The harness registers each returned archive once it is safe to keep: when a secret was filled in the attempt, every text entry of each archive is rewritten through the secret redactor first, and if that fails the archives are deleted and the attempt records TRACE_WITHHELD instead (see authentication). Recordings are the one artifact the harness cannot vouch for: it records them with redaction: 'incomplete', since a screencast masks nothing.

Contributed fixtures

export type EngineFixtureFactory = (context: EngineFixtureContext) => object;

export interface EngineFixtureContext {
  readonly targetName: string;
  fixture<T extends object>(name: string, surface: T, operations: FixtureOperations<T>): T;
  readonly app: { baseUrl?: string; allowedOrigins: readonly string[]; resolveUrl(url: string): string };
  readonly timeouts: { test: number; action: number; assertion: number };
  readonly signal: AbortSignal;
  operation(timeoutMs?: number): OperationContext;
  attachArtifact(kind: 'screenshot' | 'trace' | 'video' | 'download' | 'log', relativePath: string): void;
  attachViewport(viewport: { width: number; height: number; scale: number }): void;
  locator(expression: LocatorExpression): Locator;
  screen(scope: (expression: LocatorExpression) => LocatorExpression): Screen;
  expectable<T extends object, E extends object>(target: T, factory: () => E): T & Expectable<E>;
}

Use context.fixture(name, surface, operations) to declare which methods are recorded. Each operation specifies kind: 'resource' | 'assertion', an optional safe label(...args), an optional timeout (milliseconds, a function of the arguments, or false when the method owns its deadline), and optional verifies. The default timeout is actionTimeout; assertions verify by default. Passing verification steps confirm earlier staged action traces. Nested operation maps describe namespaces. Undeclared synchronous accessors retain their identity. Recording decorates the supplied fixture object in place, preserving mutable fields, private-field receivers, and namespace accessors. Create a fresh fixture surface in each factory invocation; declared methods and namespace properties must allow replacement. The harness opens the step before calling a declared method, so synchronous attachments, failures, and asynchronous work have the same owner. Events and artifacts remain attached to that operation across nested and overlapping calls. Work arriving after its step has closed cannot attach to a later step. A factory must return the surface it declared through context.fixture; a plain, undeclared surface is rejected with INVALID_CONFIG the first time a test reaches for the fixture, because its methods would run engine code outside any recorded step. expectable attaches an expectation surface so expect(fixture) returns it; use context.fixture('expect', matchers, operations) in its factory to declare matcher recording. Type the fixture with test.extend<{ name: Type }>(). app.resolveUrl throws APP_URL_REQUIRED when no app URL is configured and POLICY_DENIED for a disallowed origin or scheme. The engine entrypoint also exports obj(value), which drops undefined-valued keys and types them as absent optional properties, so a manifest or fixture result built from optional inputs satisfies exact optional property types without a conditional spread per key. It also exports raceAbort(work, signal, label), withTimeout(promise, timeoutMs, onTimeout), and withinCleanupBudget(promise, { signal, timeoutMs }). Pass a thunk to raceAbort to check cancellation before dispatch. Cleanup waiting absorbs failures and releases its listeners at cancellation or timeout; abandoning a wait does not stop an underlying command.

Vocabulary

OperationContext, TextPattern, SemanticQuery, LocatorExpression, NodeRef, SemanticNode, ObservationPixels, LocatorAction, ENGINE_SPI_VERSION, ENGINE_ERROR_CODES, RETRYABLE_ENGINE_ERROR_CODES, OBSERVED_NAME_LIMIT, OBSERVED_TEXT_LIMIT, and the shared semantics above are exported from @e2edev/e2e/engine and are the platform-neutral words every engine speaks. The selector expression kind carries a platform-native selector string (CSS or XPath on a document platform); frame scopes a query into a nested document. A SemanticQuery with visible: true must match only nodes whose states.hidden is false, judged by the same predicate the engine reports on SemanticNode, so getByText('x', { visible: true }) and toBeVisible() never disagree about a node. The harness drops hidden nodes from a top-level query as a backstop; only the engine can apply the predicate under a scope, filter, or index, so evaluate it wherever the query sits in the expression.

Test API

Project

Web

Mobile

Engine contract

e2e/engine

Engine

Lifecycle

Observation

Actions

App

Errors

Shared semantics

State

Artifacts

Contributed fixtures

Vocabulary

​Engine

​Lifecycle

​Observation

​Actions

​App

​Errors

​Shared semantics

​State

​Artifacts

​Contributed fixtures

​Vocabulary

Engine

Lifecycle

Observation

Actions

App

Errors

Shared semantics

State

Artifacts

Contributed fixtures

Vocabulary