Swarm

createSwarm executes a fixed directed acyclic graph of agent tasks and reducers. The scheduler persists a task's dispatch before starting its work and commits task results and control events atomically. Execution continues without an event subscriber.

Run a DAG in memory

This complete example runs two independent native agents and then reduces their accepted outputs. It is deterministic and needs no API key.

import { createAgent } from '@deuz-sdk/core/agent';
import { createMockModel } from '@deuz-sdk/core/testing';
import { createSwarm, createInMemorySwarmStore } from '@deuz-sdk/core/swarm';

const worker = createAgent({
  model: createMockModel({ responses: [{ text: 'checked' }] }),
  maxSteps: 4,
});

const swarm = createSwarm({
  agents: { worker: { agent: worker, version: 'worker-v1' } },
  reducers: {
    summary: {
      version: 'summary-v1',
      execute(results) {
        return Object.fromEntries(
          Object.entries(results).map(([id, result]) => [id, result.output]),
        );
      },
    },
  },
  store: createInMemorySwarmStore(),
  concurrency: 2,
  definitionVersion: 'review-v1',
});

const handle = await swarm.run({
  scope: 'tenant-a',
  runId: 'review-1',
  tasks: [
    { id: 'facts', agent: 'worker', prompt: 'Check the facts.' },
    { id: 'math', agent: 'worker', prompt: 'Check the calculations.' },
    { id: 'report', reducer: 'summary', dependsOn: ['facts', 'math'] },
  ],
});

const outcome = await handle.result;
console.log(outcome.run.status);
console.log(outcome.tasks.find(task => task.task.id === 'report')?.result?.output);

for await (const event of handle.events({ afterSequence: 0 })) {
  console.log(event.sequence, event.type, event.taskId);
}

concurrency limits live tasks, including reducers; it defaults to five. Logical task count and live concurrency are separate. Cycles, duplicate task IDs, missing dependencies and unknown bindings are rejected before the first run write. A dependency is usable only after its task completes. Failed branches block their descendants while independent branches can finish.

Agent dependency prompts receive predecessor outputs as labelled data. They do not inherit the predecessors' conversation histories. Reducers receive a result map keyed by their direct dependencies and a context containing scope, runId, taskId, signal and a shared child execution context. Return a serializable value; plain JSON objects, arrays and Uint8Array are supported.

Dynamic tasks

A swarm created with dynamic limits (2.2) lets finished tasks spawn more tasks at runtime. A reducer queues requests through context.spawn; an agent binding maps its accepted output through spawn(output, context). Each request names a key and either a reducer or an agent with a prompt. The spawned task's ID is parentId/key, and its dependsOn lists full IDs of existing tasks or of tasks spawned in the same request.

import { createInMemorySwarmStore, createSwarm } from '@deuz-sdk/core/swarm';

const swarm = createSwarm({
  agents: {},
  store: createInMemorySwarmStore(),
  dynamic: { maxTasks: 50, maxSpawnDepth: 2 },
  reducers: {
    plan: {
      execute(_results, context) {
        const shards = ['north', 'south'];
        context.spawn([
          ...shards.map(key => ({ key, reducer: 'count' })),
          {
            key: 'total',
            reducer: 'sum',
            dependsOn: shards.map(key => `${context.taskId}/${key}`),
          },
        ]);
        return shards.length;
      },
    },
    count: { execute: (_results, context) => context.taskId.length },
    sum: {
      execute: results =>
        Object.values(results).reduce((total, result) => total + Number(result.output), 0),
    },
  },
});

const handle = await swarm.run({
  scope: 'tenant-a',
  tasks: [{ id: 'plan', reducer: 'plan' }],
});
const outcome = await handle.result;
console.log(outcome.tasks.map(task => task.task.id)); // plan, plan/north, plan/south, plan/total

Spawned tasks are written in the same commit as their parent's terminal state. A crash before that commit replays the parent under the usual rules; a crash after it resumes the children without running the parent again. Every request is checked before anything commits: an unknown binding, a malformed key, a missing dependency, a cycle or an exceeded limit fails the spawning task and creates nothing. maxTasks (at most 10 000) bounds the whole run, maxSpawnDepth the generations below a root task, and maxSpawnPerTask one spawning task. The limits are stored with the run; a resumed run can only tighten them.

onFailure on an agent or reducer binding returns compensation tasks that commit with the failure. A hook that throws or requests invalid tasks is recorded on the failed task's error instead. Compensation tasks cannot depend on the failed task.

timeoutMs on any task (2.2) bounds each attempt. When it expires, the attempt's signal aborts and the task fails with SwarmTaskTimeout. A reducer's model calls also receive the budget as an execution deadline. An agent task relies on the signal alone, because its native run saves its policy and a changed deadline would stop it from resuming.

A dynamic run has version 2 and needs a store whose capabilities include 'spawn'; the memory, SQLite, Postgres and Redis stores do. A store without it is refused because it would drop spawned tasks from its commits. A run created without dynamic cannot spawn, and 2.1 cannot resume a version 2 run. The journal records a task.spawned event per child, with the parent ID in detail.

Groups and the blackboard

Agents working on the same problem can share notes (2.2). Give a task a group and its agent binding a blackboard: the agent then gets a blackboard_post tool that writes to its group's channel and a blackboard_read tool over its group's channel (read: 'group') or a listed set of channels. Tasks without a group share the main channel. Reducers page any channel with context.readChannel(channel, afterSequence?, limit?).

import { createAgent } from '@deuz-sdk/core/agent';
import { createMockModel } from '@deuz-sdk/core/testing';
import { createInMemorySwarmStore, createSwarm } from '@deuz-sdk/core/swarm';

const scout = createAgent({
  model: createMockModel({ responses: [
    { toolCalls: [{ toolName: 'blackboard_post', args: { text: 'A lead worth checking.' } }] },
    { text: 'Shared.' },
  ] }),
});

const swarm = createSwarm({
  store: createInMemorySwarmStore(),
  agents: { scout: { agent: scout, blackboard: { read: 'group', post: true } } },
});
const handle = await swarm.run({
  scope: 'tenant-a',
  tasks: [{ id: 'first', agent: 'scout', prompt: 'Share what you find.', group: 'euler' }],
});
await handle.result;

Notes are durable, ordered per channel, and journaled as channel.posted events. A post's optional data is JSON text, which the tool parses and stores as a value; strict providers reject an untyped parameter. Both tools are idempotent: a post is keyed by a hash of task, model step and tool call, so resuming an interrupted post never adds a second note. At the task level an interrupted agent task still follows the usual rule and waits in needs_reconciliation unless it is declared replay: 'safe' or retried. A blackboard binding needs a store with the 'channels' capability; the memory, SQLite, Postgres and Redis stores have it.

Soft dependencies

dependsOn requires a task to complete: a failed dependency blocks its dependents. after (2.2) only requires the listed tasks to settle, in any terminal state. The task then receives the results of those that completed, and a reducer sees every listed task's final status in context.settled. Use it for a join that must run even when some of the work it gathers fails.

Rounds

createRounds (2.2) builds the pattern of large research runs from these pieces: rounds of agents working in blackboard groups, with a consolidator between rounds. The consolidator receives the round's completed results by group, can read the boards and make its own model calls through execution, and decides the next round: which groups continue, with how many agents and what prompt. Returning stop, returning no groups, or reaching maxRounds ends the chain.

import { createAgent } from '@deuz-sdk/core/agent';
import { createMockModel } from '@deuz-sdk/core/testing';
import { createInMemorySwarmStore, createRounds, createSwarm } from '@deuz-sdk/core/swarm';

const solver = createAgent({
  model: createMockModel({ responses: [
    { text: 'The simpler case breaks down at t=1.' },
    { text: 'Attempt A.' },
    { text: 'Attempt B.' },
  ] }),
});

const rounds = createRounds({
  id: 'search',
  maxRounds: 3,
  initial: { warmup: { agent: 'solver', count: 1, prompt: 'Solve the simpler case.' } },
  consolidate({ round, groups }) {
    if (round === 1) {
      const lead = String(groups.warmup?.[0]?.output);
      return { groups: { main: { agent: 'solver', count: 2, prompt: `Build on: ${lead}` } } };
    }
    return { stop: true, summary: groups.main?.map(result => result.output) };
  },
});

const swarm = createSwarm({
  store: createInMemorySwarmStore(),
  dynamic: { maxTasks: 100, maxSpawnDepth: 4 },
  agents: { solver },
  reducers: rounds.reducers,
});
const outcome = await (await swarm.run({ scope: 'lab', tasks: rounds.tasks })).result;
console.log(outcome.tasks.at(-1)?.result?.output);

Each consolidator waits for its round with after, so a failed explorer does not block it. A decision may only schedule the agent bindings named in agents, which defaults to the ones used in initial; this guards bindings chosen by a model. The swarm needs dynamic limits with maxSpawnDepth of at least maxRounds. Round tasks take the replay and timeoutMs options of createRounds.

SQLite persistence and event cursors

Import SQLite through @deuz-sdk/core/swarm/sqlite, a Node-only entry. It uses node:sqlite or an injected synchronous compatible database connection. Its schema version is independent of the existing store pack's PRAGMA user_version.

2.2 uses swarm schema 2. Opening a file written by 2.1 upgrades it once, inside one transaction: runs gain indexed status and updated_at columns and a channel table is added. Existing runs, tasks and events are kept and resume as before. After the upgrade, 2.1 refuses the file, so back it up before upgrading if you may need to roll back.

import { createAgent } from '@deuz-sdk/core/agent';
import { createMockModel } from '@deuz-sdk/core/testing';
import { createSwarm } from '@deuz-sdk/core/swarm';
import { createSqliteSwarmStore } from '@deuz-sdk/core/swarm/sqlite';

const store = createSqliteSwarmStore({ path: './swarm.sqlite' });
const worker = createAgent({
  model: createMockModel({ responses: [{ text: 'stored answer' }] }),
});
const swarm = createSwarm({
  store,
  agents: { worker: { agent: worker, version: 'worker-v1' } },
  concurrency: 2,
  definitionVersion: 'durable-demo-v1',
});
const key = { scope: 'tenant-a', runId: 'durable-demo' };

try {
  const existing = await swarm.get(key);
  const handle = existing
    ? await swarm.resume(key)
    : await swarm.run({ ...key, tasks: [
        { id: 'answer', agent: 'worker', prompt: 'Give the answer.' },
      ] });

  const readEvents = (async () => {
    for await (const event of handle.events({ afterSequence: 0 })) {
      console.log(event.sequence, event.type);
      // Persist this sequence after your consumer processes the event.
    }
  })();
  const result = await handle.result;
  await readEvents;
  console.log(result.tasks[0]?.result?.output);
} finally {
  await store.close();
}

swarm.get(key) returns the persisted snapshot. swarm.events(key, { afterSequence }) and handle.events(...) read the same durable, ordered journal. A cursor is exclusive: passing 25 starts after event 25. Consumers read bounded pages rather than accumulating an unbounded producer queue. An event iterator ends when the current execution settles, including suspension; open another iterator when the run resumes. If the executor fails, handle.events() throws the error that handle.result rejects with, once the executor has stopped. Breaking iteration or aborting its signal does not cancel execution.

The journal stores task/run control events, not every model token. Retrieve outputs from the snapshot. A disconnected consumer can replay control events from its last stored sequence.

Between pages, an event reader checks whether the run is still running. A store can implement the optional head(key) (2.2) to return the run record alone; the memory, SQLite, Postgres and Redis stores do. Without it the reader falls back to load, which reads every task, so a custom store for large runs should provide head.

A settled run stays in the store until you prune it (2.3). store.prune({ before, scope?, limit? }) deletes up to limit (default 100) completed, partial or cancelled runs whose updatedAt is before before, oldest first, with their tasks, events and blackboard notes, and returns how many it deleted. Running and suspended runs stay. A pruned run can no longer be resumed, read or replayed. The memory, SQLite, Postgres and Redis stores have it; see Retention.

Redis persistence

@deuz-sdk/core/swarm/redis (2.3), a Node-only entry, persists runs on Redis 7 or later: createRedisSwarmStore({ client | url, prefix? }) takes a client that runs Lua scripts, such as a connected node-redis client, or ioredis through wrapIoredisClient. It has every capability above ('spawn', 'channels', 'list'), head and prune. A commit is one script that writes only while the run is still at the revision its validation read, so a concurrent writer gets SwarmConflictError and nothing of its commit is written. Its keys share the hash tag {<prefix>:swarm}, so a Redis Cluster keeps the store in one slot. The server must keep them: an evicted key loses a run, or hides it from listRuns and recover(). Use maxmemory-policy noeviction or a volatile-* policy, and AOF; see Redis in Operations.

import { createClient } from 'redis';
import { createSwarm } from '@deuz-sdk/core/swarm';
import { createRedisSwarmStore } from '@deuz-sdk/core/swarm/redis';

const redis = createClient({ url: process.env.REDIS_URL });
await redis.connect();
const swarm = createSwarm({
  agents: {},
  reducers: {
    one: { execute: () => 1 },
    sum: {
      execute: (results) =>
        Object.values(results).reduce((total, result) => total + Number(result.output), 0),
    },
  },
  store: createRedisSwarmStore({ client: redis, prefix: 'jobs' }),
});
const handle = await swarm.run({
  scope: 'tenant-a',
  tasks: [
    { id: 'a', reducer: 'one' },
    { id: 'b', reducer: 'one' },
    { id: 'total', reducer: 'sum', dependsOn: ['a', 'b'] },
  ],
});
const outcome = await handle.result;
console.log(outcome.run.status); // 'completed'
console.log(outcome.tasks.find((task) => task.task.id === 'total')?.result?.output); // 2
await redis.quit();

See Operations for leases, budgets and retention on the same server.

Suspension, reconciliation and cancellation

handle.result settles when the run completes, becomes partial, is cancelled, or has only suspended/reconciliation work remaining. Inspect each task's status; partial can include useful completed results. Approval requests live in a suspended task's result.agentResult.

With the swarm created above, an authenticated application's approval handler can resume selected tasks as follows:

const snapshot = await swarm.get({ scope: 'tenant-a', runId: 'durable-demo' });
const task = snapshot?.tasks.find(item => item.task.id === 'answer');
const native = task?.result?.agentResult;

if (native?.status === 'suspended') {
  const handle = await swarm.resume({
    scope: 'tenant-a',
    runId: 'durable-demo',
    approvals: {
      answer: native.pendingApprovals.map(request => ({
        approvalId: request.approvalId,
        approved: false,
        reason: 'The user declined this operation.',
        token: request.token,
      })),
    },
  });
  console.log((await handle.result).run.status);
}

Approval verdicts are scoped by task ID. Resuming a run without verdicts leaves approval tasks suspended. Task/run scopes separate persisted identities; the application must still authenticate callers and enforce tenant access. For client-executed tools, pass clientToolResults keyed by task ID; each value is an array of { toolCallId, output, isError? } results for that task's pending calls. These results pass through native approval and validation handling.

Completed tasks are reused on recovery. If an agent's terminal envelope committed before its task's final journal write, recovery can finish the task from that envelope without another model call. Otherwise an interrupted task defaults to needs_reconciliation. After checking its external effects, pass its ID in retryTaskIds to authorize another attempt. A task declared replay: 'safe' allows automatic task replay; use that declaration only for work your application knows is safe to repeat.

Native tool receipts add a separate effect check. An uncertain executing tool receipt still requires reconciliation. Pass its call ID under retryToolCallIds: { taskId: ['call-id'] } together with that task in retryTaskIds after establishing that repeating the call is appropriate. Task replay permission alone does not silently authorize an uncertain tool effect.

Call await handle.cancel() to persist cancellation and stop new dispatches. Pass a signal to run or resume for the same behavior. Active work receives an abort signal; cancellation waits for active effects to settle and preserves already-completed task results. Tools and reducers must cooperate with that signal; JavaScript cannot forcibly undo an external effect.

Policy, budgets and versioning

Set policy and budget on createSwarm for the parent constraints. Agent and reducer bindings may supply narrower policies and budgets. Every child shares the run ledger, including descendants and auxiliary model calls that use the native execution context. The scheduler persists reservations before admission and retains unknown amounts across recovery. Restoring with weaker defaults cannot relax the saved parent limits. Admission uses estimates; it does not promise a hard ceiling on a provider's eventual bill.

admission on createSwarm (2.2) also admits every task's model calls against persistent scopes in a shared BudgetStore, such as a per-user or per-org budget that outlives the run. The run records the scopes, so resuming it needs the same admission store; see Persistent budgets.

The run ledger grows with live work, not with history (2.2). When a task becomes completed, failed, blocked or cancelled it never reserves again, so the scheduler folds its reservations into executionState.ledger.aggregates: one entry per ancestor chain that keeps counting spent, held and unknown amounts against the parent limits. Global totals are unchanged, and a later cap still fails closed for folded usage that had no estimate. Per-request records of finished tasks are gone; read a task's own cost from its agent result's accounting. Each agent task checkpoints only its slice of the shared ledger. A run whose ledger has folded entries uses ledger snapshot version 2, which 2.1 cannot resume.

Use swarm/binding policy and budget fields for agent templates; a template with a pre-bound live execution context is rejected because the swarm cannot adopt its external persistence and accounting ownership. A template's ordinary budget also narrows its task's budget.

Agent bindings also accept native tools, toolsContext, output, verify, maxOutputAttempts and maxVerifyAttempts. Binding tools replace the template's tool set when supplied. An inconclusive or exhausted rejected verification does not complete a task, so its dependents cannot consume an uncertified output. Templates with legacy completion hooks, chat, memory or fallbackModels are unsupported by the native runner; configure the binding's native verifier instead.

Keep definitionVersion stable for a DAG implementation and each binding's version stable for its semantics. Their defaults are '1'; set explicit values for durable applications. Stored bindingVersion values are checked on resume. Bump versions when instructions, tools, output schemas, reducers or verifier semantics change incompatibly; an old run will then fail resume instead of silently adopting new behavior.

Operational boundary

One executor drives a run at a time. The scheduler rejects overlapping executors using the same store object, and stores enforce revision checks, so a stale writer's commit is rejected rather than applied.

Several processes can share runs with the lease option (2.2). The executor holding a run's lease drives it and renews the lease on a heartbeat; a second executor's run or resume fails with SwarmLeaseError code 'held', and an executor that loses its lease stops before its next write. handle.drain(), swarm.recover() and swarm.requestCancel(key) drain a run for a deploy, take over runs whose executor died, and cancel a run from any process. See Operations for leases, the SQLite, Postgres and Redis backends (@deuz-sdk/core/ops/sqlite, @deuz-sdk/core/swarm/postgres, @deuz-sdk/core/ops/redis, @deuz-sdk/core/swarm/redis) and their clock requirements. Without a lease provider, do not drive one run from several processes.

The in-memory store is volatile. SQLite, Postgres and Redis atomically commit task changes, run state and journal events, and fail closed on write errors or unsupported schema versions. These transactions do not include external APIs. Exactly-once external effects still require the service's idempotency mechanism or application reconciliation: a task interrupted by a takeover follows the same replay rules as one interrupted by a crash. Runtime task spawning requires dynamic limits.