Transcription (STT)

transcribe is the other half of generateSpeech: audio goes in, text comes out. Like every media module it is one request and one answer — no streaming, no loop.

Two wires ship in 2.0, and they are far less alike than the two TTS wires:

Transcription models are a separate kind from chat LanguageModel (surface: 'openai-transcription' | 'deepgram-transcription') — they cannot be passed to streamChat or generateText. The module is pure and edge-safe: HTTP goes through deps.fetch and the API key is resolved from the factory, createClient, or a deps.keyProvider, never from the environment.

Choosing a wire

The two providers are not interchangeable, and the differences show up in the first hour rather than the second month:

And when not to use this module at all: if you want a chat model to reason about how something was said (tone, background, overlapping speakers), a transcript throws that away — send the audio to a model whose registry row sets audio: true as a filePart instead. If you want live captions, see what this does not do: there is no streaming path here.

transcribe

Import transcribe from @deuz-sdk/core/transcription. Build a model descriptor with a factory, hand it the audio bytes, and pass the mediaType along with them.

import { transcribe, createOpenAITranscription } from '@deuz-sdk/core/transcription';
import { readFile } from 'node:fs/promises';

const openaiStt = createOpenAITranscription({
  apiKey: process.env.OPENAI_API_KEY!,
});

const { text, usage } = await transcribe({
  model: openaiStt('gpt-4o-transcribe'),
  audio: await readFile('meeting.mp3'),
  mediaType: 'audio/mpeg',
  language: 'tr',
});

console.log(text);              // 'Merhaba dünya.'
console.log(usage.inputTokens); // token-billed on this model

Options

transcribe(options) takes a single TranscribeOptions object.

Result shape

interface TranscribeResult {
  text: string;
  language?: string;        // detected or echoed, when the provider reports one
  durationSeconds?: number; // audio length, when the provider reports one
  segments?: TranscriptionSegment[]; // { start, end, text }
  words?: TranscriptionWord[];       // { word, start, end, confidence? }
  usage: TranscriptionUsage;
  raw: unknown;             // the untouched provider JSON
}

Every optional field is absent, not undefined, when the wire did not produce it — a plain json OpenAI response yields exactly { text, usage, raw }. raw always carries the provider's own JSON, so a field this module does not model is never lost.

Timestamps

timestamps: true asks for per-sentence segments and per-word words. What that costs — and whether it is even legal — depends entirely on the wire.

const { segments, words } = await transcribe({
  model: openaiStt('whisper-1'),
  audio: bytes,
  mediaType: 'audio/mpeg',
  timestamps: true,
});

segments?.[0]; // { start: 0, end: 4.2, text: ' Merhaba dünya.' }
words?.[0];    // { word: 'Merhaba', start: 0, end: 0.8 }

Media types and filenames

OpenAI does not read the multipart part's Content-Type — it dispatches on the file extension. So the upload filename is load-bearing, and this module derives it from mediaType:

Parameters are ignored (audio/wav; codecs=1 is still audio.wav), a Blob's own type fills in when mediaType is omitted, and filename overrides the derivation outright. On the Deepgram wire the same value is sent as the request Content-Type instead, falling back to application/octet-stream.

Deepgram

Deepgram is not OpenAI-shaped. Three differences are worth internalizing.

1. Authorization: Token, not Bearer. Deepgram uses its own scheme; a bearer token is a 401. The adapter writes it for you:

import { transcribe, createDeepgram } from '@deuz-sdk/core/transcription';

const deepgram = createDeepgram({ apiKey: process.env.DEEPGRAM_API_KEY! });

const { text, words, durationSeconds } = await transcribe({
  model: deepgram('nova-3'),
  audio: bytes,
  mediaType: 'audio/mpeg',
  language: 'en',
});
// POST https://api.deepgram.com/v1/listen?model=nova-3&language=en&smart_format=true
// Authorization: Token <key>          ← not Bearer
// Content-Type: audio/mpeg            ← the audio IS the body

2. The audio is the request body, and a URL works too. Bytes are POSTed raw — no multipart wrapper. Deepgram can also fetch the audio itself, which is the cheapest path for anything already sitting in object storage:

const { text } = await transcribe({
  model: deepgram('nova-3'),
  audio: { url: 'https://cdn.example/talk.mp3' },
});
// body → {"url":"https://cdn.example/talk.mp3"}, content-type: application/json

Passing { url } to an OpenAI model throws an InvalidRequestError before any request is made — that wire uploads bytes, full stop.

3. smart_format is on by default. It adds punctuation and casing, and it is what produces the paragraphs block that this module maps into segments. Turn it off through providerOptions if you want the raw token stream:

providerOptions: { deepgram: { smart_format: false } } // → the param is simply omitted

Word entries prefer Deepgram's punctuated_word over the raw word when smart formatting produced one, and carry Deepgram's per-word confidence. language comes from results.channels[0].detected_language, durationSeconds from metadata.duration.

A real pipeline: recording → notes

Transcription is rarely the deliverable; it is the step before one. The whole point of segments is that they survive into the summary as citations you can jump to.

import { generateText } from '@deuz-sdk/core';
import { createAnthropic } from '@deuz-sdk/core/anthropic';
import { transcribe, createDeepgram } from '@deuz-sdk/core/transcription';

const deepgram = createDeepgram({ apiKey: process.env.DEEPGRAM_API_KEY! });
const anthropic = createAnthropic({ apiKey: process.env.ANTHROPIC_API_KEY! });

export async function meetingNotes(recordingUrl: string) {
  // Deepgram pulls the file itself: a 90-minute recording never touches this
  // process, and `diarize` labels who spoke — a Deepgram query param, not a
  // canonical option, so it rides providerOptions.
  const { text, segments, durationSeconds, usage } = await transcribe({
    model: deepgram('nova-3'),
    audio: { url: recordingUrl },
    language: 'en',
    providerOptions: { deepgram: { diarize: true, utterances: true } },
  });

  // Timestamped lines, so the summary can cite a position in the audio.
  const timeline = (segments ?? [])
    .map((s) => `[${Math.floor(s.start)}s] ${s.text}`)
    .join('\n');

  const { text: notes } = await generateText({
    model: anthropic('claude-sonnet-4-6'),
    instructions:
      'Summarize this meeting transcript into Decisions, Action items (with owners), ' +
      'and Open questions. Cite the [Ns] marker for every claim.',
    prompt: timeline || text,
    maxOutputTokens: 2_000,
  });

  return { notes, durationSeconds, billedSeconds: usage.seconds };
}

Notes on the shape, all of them things that bite once:

• segments is optional and this code treats it as such. On Deepgram it exists only when smart_format produced paragraphs (it is on by default); on OpenAI it exists only for whisper-1 with timestamps: true. segments ?? [] with a fall back to plain text is not defensive noise — it is the actual contract.
• Diarization does not reach the canonical shape. TranscriptionSegment is { start, end, text } — no speaker field. Deepgram's speaker labels are in raw, which is exactly what raw is for: read result.raw when you need a field this module does not model.
• A long recording is a long wait. There is no streaming and no progress signal; the promise resolves when the provider is done. Put this behind a queue, not an HTTP request handler.
• The transcript is untrusted input. It came from a microphone someone else controlled. If it flows into a tool-calling agent rather than a summary, treat it the way you would a user message — see request validation and guardrails.

usage — two meters, one shape

Providers do not agree on what they charge for, so TranscriptionUsage carries every field optionally and fills in only what the response actually reported:

interface TranscriptionUsage {
  seconds?: number;      // audio duration billing
  inputTokens?: number;
  outputTokens?: number;
  audioTokens?: number;  // the audio-derived slice of inputTokens
  totalTokens?: number;
}

Read usage.seconds ?? usage.totalTokens if you need one number for a meter, and branch on which one is present if you are pricing the call.

Two honest caveats about that number. Nothing here converts it to money: deps.priceProvider is never consulted by transcribe, PRICES_2026 has no STT rows, and no cost event reaches the observation stream. And usage can be empty: every field is optional, so a relay that omits both meters yields {} rather than a zero — usage.seconds ?? usage.totalTokens ?? 0 is the safe read, and durationSeconds is a second source for the same fact when the provider reports it.

Practically: seconds scales with the recording, inputTokens scales with the recording and the model's tokenizer, and neither scales with the length of the transcript. A silent hour costs the same as a talkative one.

providerOptions

providerOptions is keyed by provider name and is a shallow escape hatch for wire fields the SDK does not model. Canonical fields always win — the hatch may add, never redefine:

await transcribe({
  model: openaiStt('whisper-1'),
  audio: bytes,
  mediaType: 'audio/mpeg',
  language: 'tr',
  providerOptions: {
    openai: { temperature: 0, chunking_strategy: 'auto', language: 'de' },
  },
});
// form fields → file, model, language=tr (NOT 'de'), response_format=json,
//               temperature=0, chunking_strategy=auto

Values are coerced with String(value); undefined / null entries are dropped. On the OpenAI wire the entries become extra multipart fields; on Deepgram they become extra query parameters (diarize, utterances, detect_language, keyterm, …), with smart_format special-cased as described above.

Creating transcription model descriptors

Both factories return a TranscriptionProvider — a (modelId: string) => TranscriptionModel function. Factory settings are carried on a private symbol, so the public TranscriptionModel shape stays clean and the key never leaks via enumeration.

Key resolution

The same G1 precedence chain as every other module: deps.keyProvider (highest) → factory apiKey → createClient's apiKeys[provider] (lowest). When none of them produce a key, the call throws an AuthenticationError before any network request is made.

import { createClient } from '@deuz-sdk/core';

const client = createClient({
  apiKeys: { openai: process.env.OPENAI_API_KEY!, deepgram: process.env.DEEPGRAM_API_KEY! },
});

baseUrls[provider] works the same way for the base URL, sitting between the factory setting and the per-wire default.

Observability

A transcription call emits the auxiliary operation.started / operation.completed / operation.failed events under the 'transcription' subsystem with operation: 'transcription.transcribe' — the same shape image, speech, and embedding use. Without an observer the fast path applies: no events are built at all. See Observability.

Errors

The canonical status→class table, shared with every media module: 401/403 → AuthenticationError, 404 → ModelNotFoundError, 429 → RateLimitError (with Retry-After parsed into retryAfterMs), 529 → OverloadedError, other 4xx → InvalidRequestError, 5xx → a retryable APICallError.

Deepgram wraps its failures in { err_code, err_msg } instead of OpenAI's { error: { message } }; the envelope is normalized before mapping, so err_msg lands on error.message and err_code on error.upstreamType.

Two failures happen before the request is ever sent: an unsupported audio shape, and { url } on the OpenAI wire — both InvalidRequestError. A missing API key is the third, from the G1 chain. Failed bodies are read once and parsed as JSON when they can be (a proxy's HTML 502 becomes the message instead of a second exception), and an x-request-id response header lands on error.requestId.

What transcribe does not do

• No streaming, no partial results. Deepgram and OpenAI both offer live/websocket transcription; this module speaks only the batch endpoints. Live captions are out of scope in 2.0.
• No chunking, and no size check. Providers cap how much audio one request may carry, and the SDK neither measures your bytes nor splits them — an over-long file comes back as an InvalidRequestError from the provider. Segment it yourself, or hand Deepgram a { url } so the bytes never pass through you.
• No retries, no breaker, no timeout. One plain fetch, like every media module. maxRetries does not exist here; retryAfterMs on the error is advice you act on. Pass signal for a deadline.
• No format conversion or resampling. Whatever bytes you pass are the bytes uploaded. The only transformation is the filename derivation, which exists because OpenAI dispatches on the extension.
• No speaker labels in the canonical shape. TranscriptionSegment has no speaker; Deepgram diarization output lives in raw.
• No capability matrix. TranscriptionModel is not a LanguageModel, so getModelCapabilities does not apply and an unknown slug produces no warning — the provider's 404 → ModelNotFoundError is the only signal.
• Only two wires. A third STT vendor needs a new TranscriptionAdapter; today's extension point is pointing baseURL at a relay that mimics one of these two.

Related

• Speech (TTS) — the text → audio direction.
• Image generation — the other synchronous media module.
• Dependencies — the fetch / clock / keyProvider injection seam.
• Errors — the canonical error hierarchy.
• Observability — the operation.* event protocol.