Six stages from call to next version.

Define the operation

An operation is a named, observable unit of work — `summarize_ticket`, `extract_invoice`, `classify_intent`. You can route traffic, build datasets, and gate releases per operation, never on the model as a whole.

await client.operations.create({
  key: "summarize_ticket",
  name: "Support ticket summary",
  ship_gates: [{ evaluator_id: "ev_regex_pii", min_score: 1.0 }],
});

Tag traffic at the call site

Every production request carries `metadata.operation_id`. Atlas attaches that label to the recorded call. There is no separate trace pipeline to build, no payload to re-serialize.

await client.chat.completions.create({
  model: "atlas-1",
  messages,
  metadata: { operation_id: "summarize_ticket" },
});

Review and tag in the console

Engineers open calls in the console, rate them thumbs-up / thumbs-down, or write a corrected output. No CSV export. No re-upload. The correction lives next to the original call forever.

// or programmatically:
await client.feedback.create({
  call_id: "chatcmpl-…",
  rating: "thumbs_down",
  correction: "Summary should be one sentence, not three.",
});

Build a golden dataset

Filter tagged calls by operation, rating, and date range. Promote the selection into a draft dataset. Each dataset has a version number — every promotion is auditable.

const ds = await client.datasets.create({
  operation_id: "summarize_ticket",
  name: "Tickets v3",
});
await client.datasets.items.add(ds.id, items);

Run evaluators and ship gates

Bind evaluators — regex, JSON-schema, LLM-judge, embedding-match — to the operation. Configure ship gates with minimum scores. Eval runs report per-item and aggregate results.

const evalRun = await client.evaluations.create({
  dataset_id: ds.id,
  evaluator_ids: ["ev_regex_pii", "ev_judge_quality"],
});
// → summaryScores.overall = 0.94

Promote — or request training

Promote the dataset to golden when every gate passes; a failed gate returns `400 ship_gates_unmet` with the specific failures. Once golden, request training and a solutions engineer responds within one business day.

await client.datasets.promote(ds.id);
// → status: "golden"

await client.datasets.training(ds.id, {
  notes: "Quarterly retrain.",
});