Middleware

Middleware lets you hook into an agent run without touching the model, the tools, or the agent class. A middleware can observe what happens, transform the configuration and streamed chunks, or react to the run’s terminal outcome. You describe one as a class implementing IMiddleware and register it with decorator.middleware().

Defining middleware

The only required member is getName; every hook is optional. Implement just the hooks you need:

import { decorator } from "@talosjs/ai";
import type { IMiddleware } from "@talosjs/ai";

@decorator.middleware()
class AuditMiddleware implements IMiddleware<{ userId: string }> {
  public getName = (): string => "audit";

  public onStart = (ctx) =>
    ctx.defer(audit.write({ userId: ctx.context.userId, requestId: ctx.requestId }));

  public onFinish = (ctx, info) =>
    ctx.defer(audit.write({ event: "finish", tokens: info.usage }));
}

The type parameter on IMiddleware<TContext> types the runtime context read via ctx.context. Add it to an agent’s getMiddlewares, or pass it per request:

public getMiddlewares = (): AiMiddlewareClassType[] => [AuditMiddleware];

// or, for a single call:
await chat.run<string>({ prompt: "Hi", middlewares: [AuditMiddleware] });

Lifecycle hooks

Hooks fire across the run’s lifecycle. All are optional.

Hook	When it fires
`setup`	First, before any `onConfig`; provisions capabilities for later middleware.
`onConfig`	At init and once per agent iteration, to observe or transform the run config.
`onBeforeModel`	Immediately before each model call, to adjust per-iteration knobs.
`onStructuredOutputConfig`	Before the final structured-output call, to transform its config and JSON Schema.
`onStart`	Once, when the run starts.
`onIteration`	At the start of each agent loop iteration.
`onChunk`	For every streamed chunk: pass through, replace, expand, or drop it.
`onToolPhaseComplete`	After all tool calls in an iteration are processed.
`onUsage`	Once per iteration that reports token usage.
`onFinish`	Terminal: the run completed normally.
`onAbort`	Terminal: the run was aborted.
`onError`	Terminal: an unhandled error occurred.

The three terminal hooks (onFinish, onAbort, onError) are mutually exclusive: exactly one fires per run.

Composition

Middleware composes in the order it appears in the array: the agent’s own middleware first, then any passed per request. How the hooks combine depends on their role. The piped hooks (onConfig, onStructuredOutputConfig, onChunk) each receive the previous middleware’s output, so transformations chain. onBeforeToolCall is first-win, stopping at the first middleware that returns a decision. Everything else runs sequentially, one after another.

Transforming the request

Config hooks return a partial config that is shallow-merged into the run. Use onConfig to adjust the whole run, or reach for onBeforeModel when you need a per-iteration tweak like raising the temperature on a retry:

@decorator.middleware()
class RetryHeatMiddleware implements IMiddleware {
  public getName = (): string => "retry-heat";

  public onBeforeModel = (ctx, config) => {
    if (ctx.iteration > 0) {
      return { temperature: Math.min((config.temperature ?? 0.7) + 0.1, 2) };
    }
  };
}

Return void (or nothing) to pass the config through unchanged.

Filtering chunks

onChunk sees every streamed chunk and decides its fate. Return a chunk to replace it, an array to expand it, null to drop it, or void to pass it through:

@decorator.middleware()
class RedactMiddleware implements IMiddleware {
  public getName = (): string => "redact";

  public onChunk = (_ctx, chunk) => {
    if (chunk.type === "TEXT_MESSAGE_CONTENT") {
      return { ...chunk, delta: chunk.delta.replace(/\d{16}/g, "[redacted]") };
    }
  };
}

Dropped chunks are not seen by later middleware.

Use ctx.defer(promise) inside a hook to run side effects (logging, audit writes) without blocking the run — they are awaited as part of the run’s completion.

Getting Started

AI

Framework

Command Line

Defining middleware

Lifecycle hooks

Composition

Transforming the request

Filtering chunks

​Defining middleware

​Lifecycle hooks

​Composition

​Transforming the request

​Filtering chunks

Defining middleware

Lifecycle hooks

Composition

Transforming the request

Filtering chunks