Batch API

Overview

A normal call to /v1/chat/completions runs immediately and you pay the standard rate. A batch bundles many such requests into one submission; the underlying provider works through them over minutes to hours (up to 24h) instead of instantly, and in exchange charges roughly 50% less per token. Mizan bills you the same discounted rate — the savings pass straight through.

How it works

You submit an array of requests, each with your own custom_id so you can match results back up later. Mizan forwards the whole batch to the underlying provider's own batch endpoint under the hood, tracks it, and settles the cost once it actually completes — you never need to know which provider's batch API you're really talking to, the request/response shape is identical either way.

Submitting a batch

model picks which provider handles the whole batch (every request in one batch uses the same model). Each request object is otherwise shaped exactly like a normal /v1/chat/completions body, plus the required custom_id.

curl https://api.app-mizan.com/v1/batches \
  -H "Authorization: Bearer mizan_rt_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4.1-mini",
    "requests": [
      {
        "custom_id": "req-1",
        "messages": [{ "role": "user", "content": "Summarize this in one sentence: ..." }]
      },
      {
        "custom_id": "req-2",
        "messages": [{ "role": "user", "content": "Summarize this in one sentence: ..." }]
      }
    ]
  }'

{
  "id": "2b2d40a6-5855-4caa-a4f7-e1d144365b34",
  "provider": "openai",
  "model": "gpt-4.1-mini",
  "status": "processing",
  "request_count": 2,
  "completed_count": 0,
  "failed_count": 0,
  "cost_usd": null,
  "error_message": null,
  "created_at": "2026-09-15T22:10:08.145Z",
  "completed_at": null
}

Checking status

Poll for status whenever you like — there's no webhook yet. Status moves from processing to a terminal state: completed, failed, expired, or cancelled. Cost is only known — and only billed — once a batch actually completes.

curl https://api.app-mizan.com/v1/batches/2b2d40a6-5855-4caa-a4f7-e1d144365b34 \
  -H "Authorization: Bearer mizan_rt_..."

{
  "id": "2b2d40a6-5855-4caa-a4f7-e1d144365b34",
  "provider": "openai",
  "model": "gpt-4.1-mini",
  "status": "completed",
  "request_count": 2,
  "completed_count": 2,
  "failed_count": 0,
  "cost_usd": 0.000002,
  "error_message": null,
  "created_at": "2026-09-15T22:10:08.145Z",
  "completed_at": "2026-09-15T22:13:51.446Z"
}

Getting results

Once status is completed, fetch the results — one entry per request, matched back to your original custom_id. A single bad request never fails the whole batch: it just shows up with error set instead of response.

curl https://api.app-mizan.com/v1/batches/2b2d40a6-5855-4caa-a4f7-e1d144365b34/results \
  -H "Authorization: Bearer mizan_rt_..."

{
  "object": "list",
  "data": [
    {
      "custom_id": "req-1",
      "response": {
        "id": "chatcmpl-...",
        "model": "gpt-4.1-mini-2025-04-14",
        "object": "chat.completion",
        "choices": [{ "message": { "role": "assistant", "content": "..." }, "finish_reason": "stop" }],
        "usage": { "prompt_tokens": 14, "completion_tokens": 1, "total_tokens": 15 }
      },
      "error": null
    },
    {
      "custom_id": "req-2",
      "response": { "...": "..." },
      "error": null
    }
  ]
}

Supported providers & models

Batch is currently available for OpenAI and Anthropic models only — the two providers with a real batch API of their own. Any active model from either provider (see GET /v1/models) works. Other providers will be added if/when they ship their own batch endpoints.

Pricing

Batch requests are billed at exactly 50% of Mizan's normal per-token rate for that model, matching the discount both providers apply on their end. Billing is platform-only (your Mizan wallet) for now — BYOK batch billing isn't supported yet. Cost is debited once, when the batch settles, based on real token usage across every successful request in it.

Monitoring in the dashboard

Every batch you submit shows up on the Batch Jobs page — status, request counts, cost, and how much you saved versus the regular rate. There's no UI to create a batch from the dashboard; batches are meant for bulk, programmatic workloads, so submission always happens through the API.

Limitations

• OpenAI and Anthropic models only — no other provider yet.
• Platform-billed (wallet) only — no BYOK batch billing yet.
• No webhooks — poll the status endpoint.
• No scheduling/recurring batches — each batch is a one-off submission.
• Provider limits still apply underneath (e.g. request-count and size caps per batch).