Custom executors

The agent surface has two layers under createAgent:

createToolLoopExecutor: the same loop as the built-in agent, with your prompt and your tool vocabulary. You keep the verdict tool, budget hard stops, loop guards, wind-down near the clock, model accounting, and --debug transcripts.
StepExecutor: any object with runStep(ctx). No AI SDK, no model, any decision process you like.

Whichever you pick, the runner still bounds the step, redacts what you observe, polices what you do, and records the verdict.

Keep the loop, change the vocabulary

This example gives a browser agent two tools: read the screen and tap a button by its accessible name. The same loop can wrap a device SDK.

e2e.config.ts

import type { E2EConfig } from '@e2edev/e2e';
import { createToolLoopExecutor } from '@e2edev/e2e/agent';
import { playwright, surfaceOf } from '@e2edev/playwright';
import { gateway, tool } from 'ai';
import { z } from 'zod';

const engine = playwright({ url: 'http://localhost:3000' });

export default {
  targets: [{ engine }],
  agents: {
    default: createToolLoopExecutor({
      name: 'button-agent',
      model: gateway('openai/gpt-5.6-luna'),
      system: 'Read the screen before acting. Address buttons by their visible label.',
      buildPrompt: (ctx) => ctx.step.instruction,
      tools: (ctx, { guard }) => ({
        observe: tool({
          description: 'Read the screen as text',
          inputSchema: z.object({}),
          execute: () =>
            guard(() => ctx.budgets.runTool({ name: 'observe', mutates: false }, async () => {
              const observation = await ctx.observe();
              return observation.text;
            })),
        }),
        tap: tool({
          description: 'Tap a button by its visible label',
          inputSchema: z.object({ label: z.string() }),
          execute: ({ label }) =>
            guard(() => ctx.budgets.runTool({ name: 'tap', mutates: true }, async () => {
              await surfaceOf(engine)!.page().getByRole('button', { name: label, exact: true }).click({
                timeout: Math.max(1, ctx.budgets.remainingMs()),
              });
              return `Tapped ${label}.`;
            })),
        }),
      }),
    }),
  },
} satisfies E2EConfig;

buildPrompt returns a string, or a message history to carry a conversation across steps. prepareMessages runs between turns for compaction and may return { messages, stop } when your own history shows the step is not progressing. guard() stops a tool from running once the step is concluding and turns budget and timeout stops into a clean end of the loop. runTool reserves the action budget before the body runs, serializes mutations, and records their outcomes. A mutating body must call its engine directly; queuing another grammar operation from inside it would wait on itself. Keep tool bodies short and interruptible: the step deadline settles the verdict either way, but a device operation that ignores cancellation finishes on its own time.

Replace the executor

This model-free executor supports one assertion, agent.assert('the screen says Ready'). Other instructions return a blocked verdict so the executor cannot claim to have checked something it does not understand.

e2e.config.ts

import type { E2EConfig, StepExecutor } from '@e2edev/e2e';
import { playwright } from '@e2edev/playwright';

const scripted: StepExecutor = {
  name: 'scripted',
  version: '1',
  cache: 'off',
  async runStep(ctx) {
    if (ctx.step.kind !== 'assert' || ctx.step.instruction !== 'the screen says Ready') {
      return {
        status: 'blocked',
        errorCode: 'AUTOMATION_UNSUPPORTED',
        summary: 'This executor only checks whether the screen says Ready.',
      };
    }
    const screen = await ctx.observe();
    const ready = screen.text.includes('Ready');
    return {
      status: ready ? 'passed' : 'failed',
      summary: ready ? 'The screen says Ready.' : 'Ready is absent from the screen.',
    };
  },
};

export default {
  targets: [{ engine: playwright({ url: 'http://localhost:3000' }) }],
  agents: { default: scripted },
} satisfies E2EConfig;

The context tells the executor where it is and what it may do:

Member	What it is
`ctx.step`	The instruction, params, and `index` in the attempt’s timeline
`ctx.observe({ tree, pixels })`	The only view of the app: redacted text, optionally the node tree and masked pixels
`ctx.actions`	`tap`, `type`, `typeSecret`, `press`, `select`, `scroll`, `navigate`, `tapAt`, each policed and recorded. `tapAt(point)` routes a viewport point onto the tree: a listed control under it is tapped by id with a replayable descriptor, a point on nothing listed goes to the engine’s `tapAt` as a bare point
`ctx.target.verbs`	The subset of those verbs the target’s engine declared. Offer a model exactly that vocabulary.
`ctx.pixelsTainted`	True once a secret was filled in the attempt: `observe({ pixels: true })` withholds pixels from then on, so an executor can leave screenshot verbs out of its vocabulary
`ctx.ledger`	The runner’s serialization of every completed step in the test
`ctx.attempt`	Test id, attempt id, retry index, a signal that fires when the attempt ends, and `memory`, a per-attempt map the harness never reports
`ctx.budgets`	`runTool` for actions of your own, and model-call accounting

Secrets in params arrive as name-only placeholders. actions.typeSecret(target, name) performs the authorized fill without the plaintext passing through your code. A custom executor can run both agent.act and agent.assert without a model. Its agent.assert dispatches to runStep under the act budgets and config.timeout; vision and screenshot are UNSUPPORTED_CAPABILITY there. Without a configured executor, the runner checks for a model when agent is first acquired, after the engine attempt starts. A missing model raises MODEL_UNAVAILABLE then. A custom executor with no model passes that check. Its agent.waitFor and agent.extract calls still use the built-in judgment tier and raise MODEL_UNAVAILABLE when they need a model and none is configured.

Driving the page yourself

An executor that brings its own browser tooling needs the page e2e opened, not a second one. @e2edev/playwright exports surfaceOf(handle): the live Page and BrowserContext of the current attempt behind the handle you passed to the target. Both accessors throw INVALID_STATE before an attempt is running.

import type { E2EConfig, StepExecutor } from '@e2edev/e2e';
import { playwright, surfaceOf } from '@e2edev/playwright';

const engine = playwright({ url: 'http://localhost:3000' });
const own: StepExecutor = {
  name: 'own-tools',
  version: '1',
  cache: 'off',
  async runStep(ctx) {
    const page = surfaceOf(engine)!.page();   // the page app.open() navigated
    await ctx.budgets.runTool({ name: 'click', mutates: true }, () =>
      page.getByRole('button', { name: 'Save' }).click(),
    );
    return { status: 'passed', summary: 'saved' };
  },
};

export default {
  targets: [{ engine }],
  agents: { default: own },
} satisfies E2EConfig;

What you do there is out of band: the harness still bounds the step and records the verdict, but it witnesses no actions, records no trace for replay, and attaches no artifacts. Report tool calls and model calls through ctx.budgets so the step keeps its metrics.

The verdict

Whatever the executor, runStep resolves with passed, failed, or blocked and a summary. A blocked verdict carries the error code that names the cause; How agent steps work lists them and the exit each one produces.

agent reference

Every method, option, and budget.

Writing an engine

Plug in a whole new surface below the executor.

Get started

Web

Mobile

Guides

Extend

Custom executors

Keep the loop, change the vocabulary

Replace the executor

Driving the page yourself

The verdict

agent reference

Writing an engine

​Keep the loop, change the vocabulary

​Replace the executor

​Driving the page yourself

​The verdict

agent reference

Writing an engine

Keep the loop, change the vocabulary

Replace the executor

Driving the page yourself

The verdict