Streaming

Enable streaming

Pass stream: true on the request body, or use the dedicated SDK helpers:

typescriptfor await (const chunk of client.chat.completions.stream({
  model: "atlas-1",
  messages: [{ role: "user", content: "Write a haiku about reliability." }],
  metadata: { operation_id: "haiku_demo" },
})) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

bashcurl https://api.newmen.ai/v1/chat/completions \
  -N \
  -H "Authorization: Bearer $NEWMEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "atlas-1",
    "messages": [{ "role": "user", "content": "Write a haiku about reliability." }],
    "stream": true
  }'

SSE format

Each event has the shape data: {...}\n\n and the stream ends with data: [DONE]. Chunk bodies follow OpenAI’s delta shape — content is in choices[0].delta.content. The first chunk includes the call id; subsequent chunks omit it.

Cancellation

Streaming responses honour AbortController.abort(). On cancellation we still record the partial call — prompt tokens are billed, output tokens up to the abort are billed, and the call appears in the console with finish_reason: "cancelled".

typescriptconst ctl = new AbortController();
setTimeout(() => ctl.abort(), 250);

const stream = client.chat.completions.stream({
  model: "atlas-1",
  messages,
}, { signal: ctl.signal });

try {
  for await (const chunk of stream) {
    process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
  }
} catch (e) {
  if (e.name === "AbortError") {
    // Cleaned up. Tokens accumulated so far are still recorded.
  } else throw e;
}