Batch jobs

Queue up to 50,000 requests from one JSONL file and collect them later.

5 min read

The file is the body

This is not OpenAI's batch API

client.batches.create(input_file_id=…) cannot work here and will fail confusingly if you try it: it sends a small JSON object, this endpoint reads the body as a JSONL file, and you get a 400 listing parse errors for a body that was never JSONL. There is no /v1/files and no input_file_id. Cancel is DELETE /batches/{id}, not POST /batches/{id}/cancel. The response carries cost_ceiling_usd and results_url rather than output_file_id and expires_at.

Everything else on this gateway is OpenAI-shaped on purpose, so it is worth being blunt about the one surface that is not. Use curl --data-binary, or any HTTP client. There is no SDK method to reach for and pretending otherwise wastes an afternoon.

No multipart, no upload step, no file id to keep track of. POST the JSONL itself; curl --data-binary @requests.jsonl is the whole client.

requests.jsonl

{"custom_id": "ticket-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "openai/gpt-5-nano", "messages": [{"role": "user", "content": "…"}]}}
{"custom_id": "ticket-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "openai/gpt-5-nano", "messages": [{"role": "user", "content": "…"}]}}

bash

curl -X POST "https://api.multigrid.ai/v1/batches?name=nightly-triage&completion_window_hours=24" \
  -H "Authorization: Bearer $MULTIGRID_API_KEY" \
  -H "Content-Type: application/jsonl" \
  --data-binary @requests.jsonl

Each line needs custom_id, method, url and body. url is either /v1/chat/completions or /v1/embeddings, and every line in one file must use the same one. The body is exactly what you would have posted to that endpoint, including models and provider if you want them.

Validation happens up front and reports up to 25 problems at once, each with its line number and what to put instead. A file with one bad line is rejected whole rather than half-run. Those problems arrive in an issues array beside the usual error object (the one place in the API where the error envelope grows a field rather than only changing its contents) plus issues_omitted when there were more than 25.

Limits

Limit	Description
50,000 lines	Per file. The whole file is parsed in memory and every line becomes a row.
16 MB	Per file. Split larger jobs.
128 characters	Maximum `custom_id` length. It is how you match results back to inputs, so it must be yours and unique.
1 to 168 hours	The completion window, defaulting to 24. Passed as `?completion_window_hours=`.

Submitting and collecting

POST /batches submit. The response includes the parsed line count and a cost ceiling. ?name= labels the job so it is identifiable in the dashboard later, and X-Multigrid-Batch-Name does the same thing if a header suits your client better.
GET /batches every batch on the account, newest first. Capped at 100 with no cursor: an account with more than that cannot currently page through them all from the API, and the dashboard is the answer until it can.
GET /batches/{id} progress and counts for one.
DELETE /batches/{id} cancel. Stops dispatch; items already upstream are left to finish and are billed honestly, and everything that ran stays downloadable.
GET /batches/{id}/results the results, keyed by your custom_id, in submitted order so the output zips against the input without a lookup table. ?filter=failed narrows it to the items that did not run, which is the file you actually want when you are about to resubmit; ?filter=succeeded does the opposite. The response carries X-Multigrid-Batch-Status, so a partial file is explicable rather than alarming: results are downloadable while the job is still running, because a long batch is a long time to wait to find out whether the prompt is even right.

A batch needs an inference key, and its management calls count against that key’s ordinary per-minute rate limit. A polling loop that did not would be the cheapest possible way to ignore the limit. Every response carries X-RateLimit-Remaining, so a poll loop can pace itself instead of finding out with a 429.

What it costs

Batching is not cheaper here

Other gateways sell batch at a discount because they resell tokens at a margin and can give part of it back. Multigrid passes provider list price straight through, so there is no margin to discount and a batched token costs exactly what a live one costs. Batch is for throughput and for not writing your own queue, not for the bill.

The quote you get at submission is a ceiling, not a forecast: it assumes every prompt is as long as it looks and every completion runs to its full max_tokens. Real batches land well under it. It is deliberately the same pessimistic arithmetic the live gateway uses for its balance check, so a batch cannot be admitted that the gateway then refuses item by item.

Something here disagrees with what the API actually did? That is a bug in this page, and worth reporting.

Report it