Batchable Tools

Tools that set batchable: true can implement an executeBatch() method. When multiple calls to the same batchable tool appear in a single round, the runtime routes them through executeBatch() instead of executing each call in parallel — enabling a single API request, database query, or I/O operation to handle all calls at once.

Declaring a batchable tool

export const lookupTool: ToolDefinition<{ id: string }, { name: string }> = {
  name: 'lookup_user',
  description: 'Look up a user by ID',
  batchable: true,
  parameters: z.object({ id: z.string() }),

  // Called when only one invocation exists in a round
  execute: async ({ id }) => {
    return await db.users.findOne(id)
  },

  // Called when multiple invocations exist in a round
  executeBatch: async (params) => {
    const ids = params.map(p => p.id)
    const users = await db.users.findMany(ids)
    return params.map(p => users.find(u => u.id === p.id) ?? null)
  },
}

Routing logic

The runtime calls executeBatch() only when all tool calls in a round target the same batchable tool. Mixed rounds (some batchable, some not) fall through to standard parallel execution.

Return shape

executeBatch() must return an array with the same length and order as the input params array. Index 0 of the result maps to params[0], etc.