Project tools

createAgent builds the built-in agent extended with your tools: AI SDK tools with declared semantics. A test API that seeds data, an internal endpoint, a device toolkit. The model can call them mid-flow, and the runner still budgets, serializes, and records every call.

e2e.config.ts

import type { E2EConfig } from '@e2edev/e2e';
import { createAgent, defineTool } from '@e2edev/e2e/agent';
import { playwright } from '@e2edev/playwright';
import { gateway, tool } from 'ai';
import { z } from 'zod';

const seedCart = defineTool(
  tool({
    description: 'Seed the cart with a SKU via the store test API',
    inputSchema: z.object({ sku: z.string() }),
    execute: async ({ sku }) => {
      const baseUrl = new URL(process.env.APP_URL ?? 'http://localhost:3000');
      baseUrl.pathname = `${baseUrl.pathname.replace(/\/+$/, '')}/`;
      const url = new URL('api/test/cart', baseUrl);
      url.searchParams.set('sku', sku);
      const response = await fetch(url);
      if (!response.ok) throw new Error(`Cart seeding failed: HTTP ${response.status}`);
      return response.text();
    },
  }),
  { mutates: true },
);

export default {
  targets: [{ engine: playwright({ url: process.env.APP_URL ?? 'http://localhost:3000' }) }],
  agents: { default: createAgent({ model: gateway('openai/gpt-5.6-luna'), tools: { seedCart } }) },
} satisfies E2EConfig;

Now agent.act('seed three items and check out') can call your test API. createAgent() with no tools is exactly what runs by default; system appends guidance and model sets the model. Tool names are the model’s vocabulary, and the agent’s own names (observe, tap, type, type_secret, press, select, scroll, navigate, screenshot, tap_at, complete_step) are reserved: createAgent rejects a project tool that reuses one, since it would be shadowed on one engine and live on another.

Declare what a tool does

defineTool takes the AI SDK tool and its semantics:

Annotation	Meaning
`mutates: true`	The tool changes state. The runner reserves an action slot before running it and queues it with the grammar actions. A mutating call marks a replay gap in the trace cache.
`mutates: false`	Read-only. Runs without an action slot and may observe the screen.
`platforms: ['ios', 'android']`	Offered only on targets of those platforms. A swipe tool never reaches a web target, and the model never sees a verb the surface cannot honor.

A failed call still consumes its reserved slot and is recorded as a failure. Parallel tool calls cannot exceed the budget by checking it at the same time.

Observe from a tool

A read-only tool can look at the screen through the same redacted view the executor uses. This tool saves the available pixels as report evidence:

import { randomUUID } from 'node:crypto';
import { defineTool, getToolContext } from '@e2edev/e2e/agent';
import { tool } from 'ai';
import { z } from 'zod';

const captureEvidence = defineTool(
  tool({
    description: 'Save a screenshot as evidence in the report',
    inputSchema: z.object({}),
    execute: async (_input, executionOptions) => {
      const context = getToolContext(executionOptions);
      const observation = await context.observe({ pixels: true });
      const pixels = observation.pixels;
      if (pixels === undefined) {
        return { withheld: observation.pixelsWithheld ?? 'UNSUPPORTED_CAPABILITY' };
      }
      const id = await context.attachScreenshot(pixels, `evidence-${randomUUID()}`);
      return { evidence: id };
    },
  }),
  { mutates: false },
);

observe() returns the redacted tree and, with pixels: true, the masked screenshot, under the same pixel-withholding policy as the executor. A mutating tool must request observations in a separate read-only call; its body already occupies the action queue. attachScreenshot(pixels, label) files the image as a screenshot artifact of the step. It lands in the attempt’s artifact directory, in the report’s artifact records, and in a configured artifact store, and the call resolves with the artifact’s id. Use a unique label for each capture so later images do not overwrite earlier ones.

Device toolkits

@e2edev/agent-device ships a ready tool pack for its engine:

import type { E2EConfig } from '@e2edev/e2e';
import { createAgent } from '@e2edev/e2e/agent';
import { agentDevice } from '@e2edev/agent-device';
import { agentDeviceTools } from '@e2edev/agent-device/tools';
import { gateway } from 'ai';

const iphone = agentDevice({ platform: 'ios', app: 'Settings' });

export default {
  targets: [{ engine: iphone }],
  agents: {
    default: createAgent({
      model: gateway('openai/gpt-5.6-luna'),
      tools: agentDeviceTools(iphone),
    }),
  },
} satisfies E2EConfig;

It adds open_app, swipe, type_text, alert, and screenshot, scoped to the device platforms. See the agent-device reference.

Custom executors

When tools are not enough: your own prompt or your own loop.

How agent steps work

What the model sees and what it may do.

Get started

Web

Mobile

Guides

Extend

Project tools

Declare what a tool does

Observe from a tool

Device toolkits

Custom executors

How agent steps work

​Declare what a tool does

​Observe from a tool

​Device toolkits

Custom executors

How agent steps work

Declare what a tool does

Observe from a tool

Device toolkits