How to use the OpenAI Batch API

By the end of this guide you’ll be able to call the OpenAI Batch API to run thousands of model requests as a single async job and pull back every result at a 50% discount. You’ll package your prompts into a JSONL file, submit one batch, poll until it finishes, and download the output, then test each step in Apidog before wiring it into production. If your work is more interactive, the synchronous path is a better fit, and you can test the ChatGPT API with Apidog instead.

What the Batch API is and when to use it

The Batch API is an asynchronous endpoint for large volumes of model calls that can tolerate a delay. Instead of one HTTP request per prompt, you package many requests into a single JSONL file, submit it as one job, and poll for completion. OpenAI runs the job off-peak and returns every result in an output file.

You get two concrete benefits. First, a flat 50% discount on both input and output tokens versus the synchronous API. Second, higher throughput, since batch jobs use a separate rate-limit pool and don’t compete with your live traffic. The trade-off is latency. OpenAI commits to finishing within 24 hours; many jobs land sooner, but you can’t count on that.

Reach for batch processing when the work is offline and bulk-shaped:

• Classifying or tagging a backlog of records
• Generating embeddings for a whole corpus
• Bulk content generation (product descriptions, summaries, translations)
• Running evaluation suites or model comparisons over a dataset

Skip it for anything a user is waiting on. Chat UIs, autocomplete, and live agents need the synchronous endpoints. If you’re generating many model or agent configurations at once, batch processing pairs well with that workload; see our walkthrough on generating 100+ agent configurations with batch processing.

What you need before you start

The whole flow spans two endpoints, /v1/files and /v1/batches, and four steps. Here’s the shape before we look at the calls.

To follow along you need an OpenAI API key exported as OPENAI_API_KEY, a JSONL file of requests, and a tool to fire and inspect the calls. Each step returns an object you can assert on, which makes the whole lifecycle straightforward to test.

Step 1: build and upload the JSONL file

Your input is a JSONL file where every line is one self-contained request. Each line needs four fields: a custom_id you choose (so you can match results back to inputs), the method (POST), the url (the target endpoint like /v1/chat/completions), and a body holding the actual request parameters.

{"custom_id": "req-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-4.1-mini", "messages": [{"role": "user", "content": "Classify the sentiment of: 'shipping was slow but the product is great'"}]}}
{"custom_id": "req-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-4.1-mini", "messages": [{"role": "user", "content": "Classify the sentiment of: 'returned it the same day'"}]}}

The custom_id has to be unique within the file. Results come back in no guaranteed order, so that ID is how you reconnect each response to its source row. A single batch can hold up to 50,000 requests and the file can be up to 200 MB.

Upload it to the Files API with the batch purpose:

curl https://api.openai.com/v1/files \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -F purpose="batch" \
  -F file="@requests.jsonl"

The response carries a file id (something like file-abc123). That’s your input_file_id for the next step.

Step 2: create the batch

Now create the job. You pass the input_file_id, the endpoint you’re targeting, and the completion_window. Today completion_window accepts a single value, "24h".

curl https://api.openai.com/v1/batches \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file-abc123",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h",
    "metadata": {"job": "sentiment-backfill"}
  }'

The endpoint field has to match the url used inside your JSONL lines. Supported targets include /v1/chat/completions, /v1/responses, /v1/embeddings, /v1/completions, and /v1/moderations, among others. The optional metadata object holds up to 16 key-value pairs, which is handy for tagging jobs you’ll want to filter later.

The call returns a batch object. The fields you’ll care about most:

{
  "id": "batch_abc123",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "input_file_id": "file-abc123",
  "completion_window": "24h",
  "status": "validating",
  "output_file_id": null,
  "error_file_id": null,
  "request_counts": { "total": 0, "completed": 0, "failed": 0 },
  "created_at": 1733452800,
  "metadata": { "job": "sentiment-backfill" }
}

Step 3: poll the batch status

A fresh batch starts at validating. From there it moves through a set of documented states. Poll GET /v1/batches/{batch_id} and read the status field.

The request_counts object (total, completed, failed) gives you live progress while the status sits at in_progress. There’s no webhook to wait on here, so polling on a sensible interval (every few minutes, not every second) is the right pattern. You can also cancel a running job with POST /v1/batches/{batch_id}/cancel if you submitted it by mistake.

Step 4: retrieve the output

When status reads completed, the batch object carries an output_file_id. Download that file’s contents through the Files API:

curl https://api.openai.com/v1/files/file-output456/content \
  -H "Authorization: Bearer $OPENAI_API_KEY" > results.jsonl

The output is JSONL again, one line per request. Each line echoes the custom_id you set, plus a response object containing the status code and body. Any requests that errored show up in the file referenced by error_file_id, so check both. Match results to inputs on custom_id, not on line order.

Mind the cost and window tradeoffs

The math is simple: you save 50% on tokens and you accept up to a 24-hour turnaround. For a one-time backfill or a nightly job, that’s an easy call. For a feature in your product’s critical path, it isn’t.

A few practical notes:

• The discount applies to both input and output tokens, across the supported models.
• The 24-hour window is a ceiling, not a target. Plan for the worst case; if a job hits expired, the requests that did finish are still billed and returned, and the rest are not.
• Batch jobs draw from a separate enqueued-token limit, so a large batch won’t eat into the rate limits your live traffic depends on. If you’re already bumping into caps, our guide to GPT API rate limits and how to test them covers the synchronous side.
• Half-price tokens still add up at volume. Track spend per job using the metadata tag so you can attribute cost later; here’s a cost-attribution playbook for OpenAI spend.

How to test it in Apidog

The Batch API is more error-prone than a single chat call, because the failure modes are spread across files, JSONL formatting, and a polling loop. A malformed line in a 50,000-request file fails the whole upload, and you won’t know until validation runs. Testing the lifecycle endpoints before you automate them saves you that round trip.

Apidog is an API platform where you can run each step as a request, chain them, and assert on the responses. It tests and mocks the endpoints; it isn’t an OpenAI SDK. A realistic setup looks like this:

• Validate the JSONL shape first. Before you upload anything, confirm each line carries custom_id, method, url, and body, and that body.model and the messages are present. Catching a missing field locally is cheaper than a failed batch.
• Run the multipart upload. Send POST /v1/files with purpose=batch and your file, then capture the returned id into an environment variable.
• Create the batch and assert on the object. Fire POST /v1/batches, then assert that status is validating and that endpoint matches what you sent. Apidog’s visual assertions let you check those fields without writing scripts.
• Poll and retrieve. Hit GET /v1/batches/{id} on a loop, assert when status becomes completed, then pull output_file_id and download the results.
• Exercise cancel and error paths. Submit a deliberately broken file and confirm you get a failed status, and test POST /v1/batches/{id}/cancel so your error handling is real, not assumed.

Because the output file arrives later, you can also stand up a mock API that returns a sample completed batch object and a canned results file. That lets you build and test your retrieval and parsing logic without waiting on a real 24-hour job, and without burning tokens during development. When your team works spec-first, you can also generate a test collection straight from an OpenAPI spec and keep the batch endpoints under regression coverage in CI.

Frequently asked questions

How long does a batch actually take?

OpenAI commits to completing a batch within 24 hours. In practice many jobs finish faster, but the guarantee is the 24-hour ceiling, so build your system around that. If the window closes with work unfinished, the batch moves to expired and only the completed requests are returned and billed.

What’s the real discount, and does it stack?

The Batch API gives a flat 50% discount versus the synchronous endpoints, on both input and output tokens. It’s the single largest cost lever OpenAI offers for offline workloads. If you’re trying to attribute that spend back to features or jobs, the cost-attribution playbook shows how to slice it.

Which endpoints can I run in a batch?

You set the target in both the JSONL url and the batch’s endpoint field, and they have to match. Supported targets include /v1/chat/completions, /v1/responses, /v1/embeddings, /v1/completions, and /v1/moderations, plus image and video endpoints. Check the current docs for the full list, since OpenAI adds to it over time.

Why are my results out of order?

They aren’t ordered, by design. The output JSONL doesn’t preserve input line order, which is exactly why every request needs a unique custom_id. Match results to inputs on that ID. If two lines share a custom_id, you can’t reliably tell their responses apart.

Conclusion

You now have the full flow: package your prompts into a JSONL file, upload it, create the batch, poll the status, and retrieve the output, all at half the token cost inside a 24-hour window. Each step returns an object you can verify, so once the JSONL shape and the polling loop are right, the rest is mechanical.

Before you automate it, run the lifecycle by hand. Download Apidog to validate your request file, exercise the upload, create, poll, and cancel endpoints, and assert on the batch object fields so a malformed line never costs you a 24-hour round trip.