OpenAI’s AI Batch Requests API Support Added to Aiomatic – 50% Discount For Your API Batch Requests!

What are AI Batch Requests?
The Batch API is a powerful feature of the OpenAI platform that allows you to send asynchronous groups of requests, offering significant cost savings, higher rate limits, and a clear 24-hour turnaround time. This service is ideal for processing jobs that don’t require immediate responses. You can use the Batch API for tasks such as:

– Running evaluations
– Classifying large datasets
– Embedding content repositories
Compared to using standard endpoints directly, the Batch API provides:

– Better cost efficiency: 50% cost discount compared to synchronous APIs
– Higher rate limits: Substantially more headroom compared to the synchronous APIs
– Fast completion times: Each batch completes within 24 hours (and often more quickly)
Check details about the Batch API, in OpenAI’s official documentation and FAQ.

How to Use the Batch API
Step 1: Prepare Your Batch File
You can use the ‘Manual Batch File Generator’ tab to create a .jsonl file which will be able to be uploaded directly to OpenAI and used as a AI Batch Request file.

You can also manually create a .jsonl file where each line contains the details of an individual request to the API. Each request must include a unique custom_id value. Here’s an example of an input file with 2 requests:

{“custom_id”: “request-1”, “method”: “POST”, “url”: “/v1/chat/completions”, “body”: {“model”: “gpt-3.5-turbo-0125”, “messages”: [{“role”: “system”, “content”: “You are a helpful assistant.”},{“role”: “user”, “content”: “Hello world!”}],”max_tokens”: 1000}}
{“custom_id”: “request-2”, “method”: “POST”, “url”: “/v1/chat/completions”, “body”: {“model”: “gpt-3.5-turbo-0125”, “messages”: [{“role”: “system”, “content”: “You are an unhelpful assistant.”},{“role”: “user”, “content”: “Hello world!”}],”max_tokens”: 1000}}
Step 2: Upload Your Batch Input File
Upload your .jsonl file using the ‘Manage AI Batch Request Files’ tab of this menu.

Step 3: Create the Batch
Use the input File object’s ID to create a batch, in the ‘Manage AI Batch Requests’ tab from this menu.

Step 4: Check the Status of a Batch & Retrieve the Results
You can check the status of a batch at any time. Once the batch is complete, download the output from the ‘Manage AI Batch Requests’ tab.

That’s it! You’ve successfully set up an AI-powered Batch API request, using the Aiomatic plugin.