Video Generation

Video generation is asynchronous by nature: a clip takes tens of seconds to minutes, so there is no synchronous counterpart to generateImage. The shape is submit → poll → download, the same state machine the Midjourney proxy runs — you create a job, poll it through the injected clock until it reaches a terminal status, and then either use the result URL or pull the bytes.

The wire is the OpenAI Videos shape, which the relays that actually serve Sora / Veo / Kling / Hailuo mirror:

POST {baseURL}/videos              → create a job   → { id, status }
GET  {baseURL}/videos/{id}         → poll the job   → { id, status, progress, url, … }
GET  {baseURL}/videos/{id}/content → download bytes → video/mp4

Everything is pure and edge-safe: HTTP goes through deps.fetch, the poll delay through deps.clock.setTimeout (no ambient timers), and the API key is resolved from the factory, createClient, or a deps.keyProvider — never from the environment.

Video models are a separate kind from chat LanguageModel (surface: 'video'), like ImageModel. They cannot be passed to streamChat or generateText.

When to use it

Reach for this when a clip is a deliverable, not part of a conversation: a marketing asset, a product shot, a background loop, a storyboard frame brought to life. The whole design assumes minutes, not seconds — you submit, you go away, you come back.

Do not reach for it when:

generateVideo

generateVideo is the one-call entry point: it submits the job, polls it to a terminal status, and — with download: true — fetches the finished clip's bytes.

import { generateVideo } from '@deuz-sdk/core/video';
import { createYunwu } from '@deuz-sdk/core/yunwu';

const yunwu = createYunwu({ apiKey: process.env.YUNWU_API_KEY! });

const { task } = await generateVideo({
  model: yunwu.video('sora-2'),
  prompt: 'a tidy robot watering a fern on a windowsill, slow dolly in',
  size: '1280x720',
  seconds: 8,
  pollIntervalMs: 5000,  // default 5000
  timeoutMs: 600_000,    // default 600_000 (10 min)
  onProgress: (t) => console.log(t.status, t.progress), // e.g. "in_progress 50"
});

console.log(task.status); // 'completed'
console.log(task.url);    // the finished clip's URL

Ask for the bytes when you want to persist the clip yourself instead of handing a relay URL to the browser:

const { task, video, mediaType } = await generateVideo({
  model: yunwu.video('veo3.1'),
  prompt: 'a neon city skyline at dusk, aerial',
  download: true,
});

if (task.status === 'completed') {
  console.log(mediaType);        // 'video/mp4'
  console.log(video!.byteLength); // Uint8Array
}

video/mediaType are present only when download: true and the job completed — a failed job never triggers a download.

A failed job is returned, not thrown

A job the model refuses or botches resolves with status: 'failed' and a failReason. It is not an exception: the failure is the model's, not the transport's, and failReason is the useful part. (This is the same contract waitForTask has for Midjourney.) Only transport failures, a timeout, or an abort reject.

const { task } = await generateVideo({ model, prompt: '…' });

if (task.status === 'failed') {
  console.error('rejected:', task.failReason); // e.g. 'content policy'
}

Options

generateVideo(options) takes one object: the submitVideo fields, the poll fields, and download.

The pieces

Use the individual functions when you want to own the job's lifetime — submit in a request handler, store the id, and poll from a worker instead of holding a connection open for ten minutes.

import { submitVideo, fetchVideoTask, downloadVideo } from '@deuz-sdk/core/video';
import { createYunwu } from '@deuz-sdk/core/yunwu';

const yunwu = createYunwu({ apiKey: process.env.YUNWU_API_KEY! });
const model = yunwu.video('kling-2.6');

// 1. In the request handler — return immediately.
const job = await submitVideo({ model, prompt: 'a paper boat in a storm drain' });
await db.save(job.id);

// 2. Later, from a worker — the model descriptor addresses the same relay.
const task = await fetchVideoTask(job.id, { model });
if (task?.status === 'completed') {
  const { video, mediaType } = await downloadVideo(task.id, { model });
  await storage.put(`${task.id}.mp4`, video, mediaType);
}

fetchVideoTask returns null — not an error — when the relay says the job does not exist (a 404, or an envelope with neither an id nor a status). Every other failing status maps to a typed error.

waitForVideo stops at completed or failed, otherwise polls at pollIntervalMs until timeoutMs elapses (throwing a TimeoutError). An aborted signal rejects with an AbortError.

Four things the poll loop will do that may surprise you

• An unrecognized status is not terminal. The alias table covers the spellings the aggregators use; anything else passes through verbatim and keeps polling. A relay that finishes with, say, finished — a word not in the table — polls to the timeout even though the job is done. onProgress is where you would notice; task.raw is where you would confirm it.
• timeoutMs is a floor, not a ceiling. The elapsed check runs after each poll, so the real deadline is timeoutMs plus one poll interval plus one request. With the defaults that is up to ~10:05, not 10:00.
• onProgress fires per poll, not per change. It gets a snapshot every pollIntervalMs whether or not anything moved, and it is skipped entirely when the poll came back null. Deduplicate on task.progress yourself if you are pushing to a UI.
• A five-second cadence is a choice, not a law. pollIntervalMs costs you one HTTP request per tick against a relay that may rate-limit. Ten minutes at the default is ~120 requests per job; raise the interval for long models and lower it only for jobs you know are quick.

The production shape: submit here, poll there

Holding a connection open for ten minutes is the failure mode this API is designed to let you avoid. The state that matters is one string — the job id.

import { submitVideo, fetchVideoTask, downloadVideo } from '@deuz-sdk/core/video';
import { createYunwuVideo } from '@deuz-sdk/core/yunwu';

const video = createYunwuVideo({ apiKey: process.env.YUNWU_API_KEY! });

/** HTTP handler: returns in one round-trip. */
export async function enqueue(prompt: string): Promise<string> {
  const job = await submitVideo({ model: video('sora-2'), prompt, size: '1280x720' });
  await jobs.insert({ id: job.id, status: job.status, prompt });
  return job.id; // the client polls YOUR endpoint, not the relay
}

/** Cron/queue worker: one tick per job, no long-lived poll loop at all. */
export async function tick(jobId: string): Promise<void> {
  const task = await fetchVideoTask(jobId, { model: video('sora-2') });
  if (!task) return; // unknown yet — the relay may still be registering it

  await jobs.update(jobId, { status: task.status, progress: task.progress });

  if (task.status === 'completed') {
    const { video: bytes, mediaType } = await downloadVideo(task.id, { model: video('sora-2') });
    await storage.put(`${task.id}.mp4`, bytes, mediaType);
  }
  if (task.status === 'failed') {
    await jobs.update(jobId, { error: task.failReason ?? 'unknown' });
  }
}

Two details that make this work: the model descriptor is just settings (key, baseURL, headers on a private symbol), so re-minting it in the worker addresses the same relay without any shared state; and fetchVideoTask is the one function here that emits no observation event, so a tight polling worker does not flood your observer — only video.submit, video.wait, and video.download do.

Give the worker its own idea of "too old". waitForVideo's timeout only exists inside one process; a job row that has been in_progress for an hour is your database's problem to notice.

VideoTask

type VideoTaskStatus = 'queued' | 'in_progress' | 'completed' | 'failed' | (string & {});

interface VideoTask {
  id: string;
  status: VideoTaskStatus;
  progress?: number;   // 0-100
  model?: string;
  seconds?: string;    // clip length as the relay reports it
  size?: string;       // e.g. '1280x720'
  url?: string;        // result URL on a completed job
  failReason?: string; // why a failed job failed
  raw: unknown;        // the untouched relay JSON
}

Normalization — the tolerant half

Relays disagree about spelling, so the task response is normalized before it reaches you. raw always holds the untouched JSON if you need a field this table drops.

cancelled maps to failed deliberately — it is terminal, and leaving it unmapped would make waitForVideo poll a dead job until timeoutMs. An unrecognized status is not rewritten (that is what the open (string & {}) arm of VideoTaskStatus is for), but it is also not terminal, so a relay with a status this table misses will poll to its timeout.

Three more fields are normalized the same way:

• progress — a number stays a number; a '42%' (or '42') string becomes 42; anything unparseable is dropped.
• url — read from url, then video_url, then output.url, then data[0].url.
• failReason — read from error.message (or a bare error string), then fail_reason.

Image-to-video

Pass inputReference to seed the clip with a still or an earlier clip. The request switches from JSON to multipart/form-data and the reference is sent as the input_reference part. A bare Blob/File is sent as-is; raw bytes need a mediaType so the relay can tell a PNG from an MP4.

import { generateVideo } from '@deuz-sdk/core/video';

const { task } = await generateVideo({
  model: yunwu.video('sora-2'),
  prompt: 'the photo comes alive, leaves rustling in the wind',
  seconds: 4,
  inputReference: {
    data: pngBytes,           // Uint8Array | ArrayBuffer | Blob
    mediaType: 'image/png',
    filename: 'still.png',    // optional; derived from mediaType otherwise
  },
});

The SDK never sets a content-type header on the multipart request — fetch writes the boundary itself, and overriding it produces a body the relay cannot parse.

providerOptions

Video relays expose vendor-specific knobs (seed, aspect_ratio, resolution, camera_fixed, …) that no cross-provider option could name honestly. providerOptions is merged into the request body verbatim, and the canonical fields (model, prompt, size, seconds) always win — so a stray key can never silently retarget the model or the prompt.

await generateVideo({
  model: yunwu.video('seedance-1-5-pro-250928'),
  prompt: 'a lighthouse in fog',
  providerOptions: { seed: 42, aspect_ratio: '21:9', camera_fixed: true },
});

Creating video model descriptors

createVideoProvider(settings) returns a VideoProvider — a (modelId: string) => VideoModel function. The factory settings ride on a private symbol, so the public VideoModel shape stays { provider, modelId, surface } and the key never leaks via Object.keys/JSON.stringify.

The relay path is a setting, not a constant. /videos under a versioned root is the OpenAI Videos convention and what the aggregators mirror today, but a given host may mount its video jobs elsewhere or version them differently — point baseURL at whatever root makes {baseURL}/videos correct for your provider:

import { createVideoProvider } from '@deuz-sdk/core/video';

const relay = createVideoProvider({
  provider: 'my-relay',
  apiKey: process.env.RELAY_KEY!,
  baseURL: 'https://relay.example.com/api/v2', // → POST https://relay.example.com/api/v2/videos
});

const model = relay('some-video-model');

For Yunwu, prefer createYunwuVideo or the unified client's .video(modelId) — both pin provider: 'yunwu' and derive /v1/videos from the relay root. See the Yunwu provider page.

import { generateVideo } from '@deuz-sdk/core/video';
import { createYunwuVideo } from '@deuz-sdk/core/yunwu';

const yunwuVideo = createYunwuVideo({ apiKey: process.env.YUNWU_API_KEY! });

const { task } = await generateVideo({
  model: yunwuVideo('MiniMax-Hailuo-2.3'),
  prompt: 'a hot air balloon over terraced fields at sunrise',
});

Addressing a job without the model

Every function accepts the shared VideoConfig — model, apiKey, baseURL, provider, fetch, headers, signal, deps. Passing model is the easy path (it carries the factory settings), but a worker that only stored a job id can address the relay directly:

const task = await fetchVideoTask(jobId, {
  apiKey: process.env.YUNWU_API_KEY!,
  baseURL: 'https://yunwu.ai/v1',
});

What it costs, and how long it takes

The honest answer to both is: the SDK does not know, and does not pretend to.

• No usage, no cost. VideoTask has no usage field, deps.priceProvider is never consulted, PRICES_2026 has no video rows, and no cost event reaches the observation stream. What you do get is task.seconds and task.size as the relay reports them — the two inputs every vendor prices on — plus raw if the relay put a cost in its envelope. Meter on your side, per job.
• Duration is unbounded in practice. The defaults (pollIntervalMs: 5000, timeoutMs: 600_000) encode the assumption that a clip takes minutes. A long or busy model can exceed ten minutes; that is a TimeoutError, not a failure of the job, and the job may well finish afterwards — which is another reason to store the id rather than rely on the poll.
• Every knob that costs money is provider-specific. size and seconds are the only canonical ones, and both are pass-through strings the SDK never validates. Resolution tiers, frame rates, and quality presets live in providerOptions, spelled the way your relay spells them.

A failed job may still have cost you — the relay decides that, not this SDK. Log task.id and task.failReason on every failure; they are what a support ticket needs.

What generateVideo does not do

• No streaming and no partial output. There is no progressive preview; progress is a number the relay reports, nothing more.
• No retries, no breaker. Like every media module this is a plain fetch per step. A 429 on submit is an immediate RateLimitError carrying retryAfterMs; retrying is yours. (The poll naturally retries, because polling is what it does — but a poll that throws, throws.)
• No cancellation of the remote job. Aborting the signal stops your polling and rejects with AbortError. There is no DELETE /videos/{id} here, so the relay may keep generating — and billing.
• No resume across processes. waitForVideo holds no state; it is a loop over fetchVideoTask. That is a feature (any process holding the id can take over), but it means nothing recovers a lost id.
• No capability matrix, no model catalog. VideoModel is not a LanguageModel, so getModelCapabilities does not apply and there is no per-slug registry. YUNWU_VIDEO_MODELS is a pinned list of what that one relay served when it was checked, not a validated enum — any string reaches the wire.
• No input validation. size, seconds, and every providerOptions key go to the relay untouched. A bad size is a 4xx from the host, not a local error.

Observation

Video calls emit operation.* events on the 'video' subsystem (they are auxiliary operations, not runs — same as image and embedding), with one operation per step: video.submit, video.wait (the whole poll, not one event per tick), and video.download. See Observability.

Errors

HTTP failures map to the canonical error classes: 401/403 → AuthenticationError, 404 → ModelNotFoundError, 429 → RateLimitError (with Retry-After), 529 → OverloadedError, other 4xx → InvalidRequestError, 5xx → a retryable APICallError. When no API key can be resolved, the call throws an AuthenticationError before any network request is made, and a relay that accepts a job but names no id throws an APICallError.

fetchVideoTask is the one exception: a 404 there means "unknown job" and yields null.

Related

• Image generation — the synchronous generateImage and the async Midjourney proxy.
• Yunwu provider — one config for chat, image, video, embeddings, and Midjourney.
• Dependencies — the fetch / clock / keyProvider injection seam.
• Errors — the canonical error hierarchy.