Batch inference
Score whole portfolios or embed large lists of companies asynchronously, from the API or the dashboard.
Batch inference complements live predictions when you need to score or embed many records at once. You upload one or more files, create a batch that points a model version at them, wait while it runs, and download one Parquet file with the results.
| Use case | Recommended path |
|---|---|
| Nightly portfolio rescoring | Batch |
| Real-time onboarding | Live predictions |
| Marketing list enrichment | Batch |
| Underwriting one application at a time | Live predictions |
How a batch runs
A batch takes four steps: upload the input files through signed URLs, create the batch, wait for it to finish, and download the result through another signed URL. You can wait by polling the batch, or let a webhook tell you when it ends.
Batch statuses
A batch starts as pending, waits in a queue for resources as queued, processes records as running, and consolidates its results as concluding. It ends as completed, failed, or cancelled.
The diagram leaves out rarer paths. For example, a batch can also fail while it’s queued, and Avra can stop a batch at any status before it finishes, not only while it’s running.
| Status | What it means | Webhook |
|---|---|---|
pending |
Created, and not yet sent to run | batch-request is sent when you create the batch |
queued |
Waiting in the queue for resources | None |
running |
Processing records | batch-running |
concluding |
Processing has ended, and the results are being consolidated | None |
completed |
Finished, with a result to download | batch-ended |
failed |
Couldn’t finish | batch-ended |
cancelling |
A stop was requested | None |
cancelled |
Stopped before it finished | batch-ended |
A batch can’t be cancelled through the v2 API. Contact Avra support if you need to stop one: it moves to cancelling, then cancelled.
Prepare the input files
Each file holds one row per record, with a column for each input the version’s schema declares, named exactly as in the schema. Every input the schema doesn’t mark as optional must be there. Read the schema with GET /v1/api/models/{model_id}/versions/{alias_or_version_id}, as in Live predictions. In the dashboard, the batch form also offers an example CSV built from the version’s schema.
Prefer Parquet over CSV. Parquet is typed and compressed, so uploads and jobs are faster and values keep their precision. CSV works for simple schemas, and the dashboard rejects a CSV file whose header has a column the schema doesn’t declare. All the files in a batch must share the same format, such as all Parquet or all CSV.
In Parquet files, aim for at most 100k rows per row group and 5 GB per file. Split large inputs across files: a batch accepts many files at once, and several medium files beat one huge one. Very large jobs may need sizing beforehand, so reach out to your Avra team.
| Requirement | Value |
|---|---|
| Formats | Parquet (recommended) or CSV, the same format for every file in a batch |
| Columns | One per input in the version’s schema, named as in the schema |
| Parquet layout | At most 100k rows per row group and 5 GB per file, as guidance |
| Path you declare | Relative, such as portfolio/part-0001.parquet: no leading /, no .., ~, or \, and no empty segments |
| Checksum | The file’s MD5 digest, base64-encoded, such as KPRxZBtAn5UfexFEKfKr7Q== |
| Media type | Optional, as content_type. Left out, it’s inferred from the extension: application/vnd.apache.parquet for .parquet, text/csv for .csv |
| Files per upload request | Up to 1,000. Request URLs for more files in several calls |
To compute the checksum on the command line, run openssl dgst -md5 -binary portfolio.parquet | base64.
Run a batch from Python
This script runs the whole flow: it computes each file’s MD5, requests the upload URLs, uploads the files, creates the batch, polls until it finishes, and downloads the result. It needs the requests package. Set MODEL_ID and FILES before you run it.
"""Upload files, run a batch on them, wait for it, and download the result."""
import base64
import hashlib
import os
import sys
import time
from pathlib import Path
import requests
API = "https://api.avra.ai"
HEADERS = {"Authorization": f"Bearer {os.environ['AVRA_API_KEY']}"}
MODEL_ID = "<model_id>"
ALIAS = "production"
FILES = [Path("portfolio/part-0001.parquet"), Path("portfolio/part-0002.parquet")]
FINISHED = {"completed", "failed", "cancelled"}
def md5_base64(path: Path) -> str:
digest = hashlib.md5()
with path.open("rb") as f:
for chunk in iter(lambda: f.read(1024 * 1024), b""):
digest.update(chunk)
return base64.b64encode(digest.digest()).decode()
# 1. Request one signed upload URL per file. The URLs expire after one hour.
targets = [{"file": path.as_posix(), "md5": md5_base64(path)} for path in FILES]
response = requests.post(
f"{API}/v1/api/files/upload", headers=HEADERS, json={"targets": targets}, timeout=60
)
response.raise_for_status()
upload = response.json()
# 2. Upload each file with the Content-Type and Content-MD5 its URL was signed for.
for path, target in zip(FILES, upload["targets"]):
with path.open("rb") as f:
put = requests.put(
target["upload_url"],
data=f,
headers={"Content-Type": target["content_type"], "Content-MD5": target["md5"]},
timeout=600,
)
put.raise_for_status()
# 3. Create the batch from the files' avra:// paths.
response = requests.post(
f"{API}/v2/api/batches",
headers=HEADERS,
json={
"model_id": MODEL_ID,
"alias": ALIAS,
"input": {"files": [target["file"] for target in upload["targets"]]},
},
timeout=60,
)
response.raise_for_status()
batch = response.json()
print("created batch", batch["id"])
# 4. Poll until the batch finishes.
while batch["status"] not in FINISHED:
time.sleep(30)
response = requests.get(f"{API}/v2/api/batches/{batch['id']}", headers=HEADERS, timeout=30)
response.raise_for_status()
batch = response.json()
print(batch["status"])
print("statistics:", batch.get("statistics"))
if batch["status"] != "completed":
sys.exit(f"batch {batch['id']} ended as {batch['status']}")
# 5. Download the result. The signed URL is valid for 15 minutes.
response = requests.get(
f"{API}/v2/api/batches/{batch['id']}/result", headers=HEADERS, timeout=30
)
response.raise_for_status()
result = response.json()
out_path = Path(f"{batch['id']}-{result['name']}")
with requests.get(result["download_url"], stream=True, timeout=600) as download:
download.raise_for_status()
with out_path.open("wb") as out:
for chunk in download.iter_content(chunk_size=1024 * 1024):
out.write(chunk)
print("saved", out_path)
The upload response lists the targets in the order you sent them, which is what lets the script pair each file with its URL. To pin an exact version instead of following the alias, send model_version_id in place of alias. Add "reference_date": "2025-12-31" to the batch to get predictions as of a past date, for a backtest.
The same steps with cURL
The same flow for one file, with jq to read the responses:
Request an upload URL
FILE="portfolio.parquet"
MD5=$(openssl dgst -md5 -binary "$FILE" | base64)
jq -n --arg file "$FILE" --arg md5 "$MD5" '{targets: [{file: $file, md5: $md5}]}' |
curl -s -X POST https://api.avra.ai/v1/api/files/upload \
-H "Authorization: Bearer $AVRA_API_KEY" \
-H "Content-Type: application/json" \
-d @- > upload.jsonThe response has one target per file. file is the path to reference in the batch, and expiration is when the upload URLs stop working:
{
"targets": [
{
"file": "avra://uploads/1784669990715/portfolio.parquet",
"content_type": "application/vnd.apache.parquet",
"md5": "KPRxZBtAn5UfexFEKfKr7Q==",
"upload_url": "<signed upload URL>"
}
],
"expiration": "2026-11-09T15:00:00Z"
}Upload the file
Send the Content-Type and Content-MD5 the URL was signed for. Storage rejects an upload whose digest doesn’t match.
curl -X PUT "$(jq -r '.targets[0].upload_url' upload.json)" \
-H "Content-Type: $(jq -r '.targets[0].content_type' upload.json)" \
-H "Content-MD5: $MD5" \
--upload-file "$FILE"Create the batch
MODEL_ID="<model_id>"
jq -n --arg model "$MODEL_ID" --arg path "$(jq -r '.targets[0].file' upload.json)" \
'{model_id: $model, alias: "production", input: {files: [$path]}}' |
curl -s -X POST https://api.avra.ai/v2/api/batches \
-H "Authorization: Bearer $AVRA_API_KEY" \
-H "Content-Type: application/json" \
-d @- > batch.json
BATCH_ID=$(jq -r '.id' batch.json)Avra responds with 201 and the batch:
{
"id": "0d9c3f6e-4b1a-4f7e-9a2b-5c8d7e6f1a20",
"created_at": "2026-11-09T14:05:12Z",
"model_id": "8f14e45f-ceea-467f-a8f0-5c3f0c7a1b2d",
"model_version_id": "c9bf9e57-1685-4c89-bafb-ff5af830be8a",
"reference_date": null,
"input": {
"files": ["avra://uploads/1784669990715/portfolio.parquet"]
},
"status": "pending"
}Wait for it to finish
Check the status until it’s completed, failed, or cancelled:
curl -s "https://api.avra.ai/v2/api/batches/$BATCH_ID" \
-H "Authorization: Bearer $AVRA_API_KEY" | jq '{status, statistics}'Download the result
curl -s "https://api.avra.ai/v2/api/batches/$BATCH_ID/result" \
-H "Authorization: Bearer $AVRA_API_KEY" > result.json
curl -o result.parquet "$(jq -r '.download_url' result.json)"The result response names the file and says when its URL expires:
{
"name": "result.parquet",
"download_url": "<signed download URL>",
"expires_at": "2026-11-09T16:20:00Z"
}The batch object
POST /v2/api/batches, GET /v2/api/batches/{id}, and GET /v2/api/batches all return batches with these fields. The list returns the batches of the API key’s workspace.
| Field | Type | Description |
|---|---|---|
id |
string | The batch’s ID |
created_at |
string | When the batch was created, in RFC 3339 |
model_id |
string | The model it runs |
model_version_id |
string | The exact version it runs, resolved from the alias when you sent one |
reference_date |
string or null | The date it predicts as of, when you set one |
input.files |
array of strings | The avra:// paths of its input files |
status |
string | One of the batch statuses |
statistics |
object | Counts for the run, once the batch has them |
statistics has total, success, and failed record counts. When records fail, errors lists each error message with its count. For a model that returns a score, score describes their distribution: min, max, mean, std, median, and a histogram.
The batch keeps the version it resolved when you created it. Moving an alias afterward doesn’t change a batch that’s already running.
Download the result
Once the batch has completed or failed, GET /v2/api/batches/{id}/result returns a signed download_url that’s valid for 15 minutes, with the file’s name and the URL’s expires_at. The result is a single Parquet file, result.parquet. Asking earlier returns 409 Conflict with batch result is not ready.
When a URL expires, request the result again for a new one. Each successful request sends a batch-result-download webhook event.
Use webhooks instead of polling
Subscribe an endpoint to batch-running and batch-ended in Alerts → Webhooks, and let Avra tell you when a batch changes, instead of polling. batch-ended covers all three endings: completed, failed, and cancelled. A delivery doesn’t carry the batch’s status, so when one arrives, read the batches you’re waiting on with GET /v2/api/batches/{id} and act on the ones that finished.
Store each batch’s id when you create it. Then call a function like this one from your webhook endpoint, after it verifies the delivery:
import json
import os
from pathlib import Path
import requests
API = "https://api.avra.ai"
HEADERS = {"Authorization": f"Bearer {os.environ['AVRA_API_KEY']}"}
PENDING = Path("pending-batches.json") # the IDs you stored, kept in your database in production
FINISHED = {"completed", "failed", "cancelled"}
def on_batch_ended() -> None:
pending = json.loads(PENDING.read_text()) if PENDING.exists() else []
still_running = []
for batch_id in pending:
response = requests.get(f"{API}/v2/api/batches/{batch_id}", headers=HEADERS, timeout=30)
response.raise_for_status()
batch = response.json()
if batch["status"] not in FINISHED:
still_running.append(batch_id)
elif batch["status"] == "completed":
result = requests.get(
f"{API}/v2/api/batches/{batch_id}/result", headers=HEADERS, timeout=30
)
result.raise_for_status()
print(batch_id, "download from", result.json()["download_url"])
else:
print(batch_id, batch["status"], batch.get("statistics"))
PENDING.write_text(json.dumps(still_running))
Deliveries are at least once, so the same batch-ended can arrive twice. The function is safe to run again: a batch it already handled is no longer pending.
Run a batch from the dashboard
You can do the same from the dashboard under Batches. Pick the model and version, then add new Parquet or CSV files or reference files you already uploaded by their avra:// path. Optionally, choose a reference date up to two years back to get predictions as of that date, then review and submit. Each batch’s details show its progress and, once it has completed, let you download its results.
Creating batches takes the permission to create batches, and downloading results in the dashboard takes the permission to download them. See Roles and permissions.
Errors
Errors return JSON with a numeric code and a message. These are the ones specific to batches:
| Status | message |
What it means |
|---|---|---|
400 |
too many files in one request |
An upload request with more than 1,000 files |
400 |
md5 must be a base64-encoded 128-bit digest |
A checksum that isn’t a base64-encoded MD5 digest |
400 |
invalid file path |
A path that breaks the path rules, or that isn’t one of your workspace’s files |
400 |
batch input files must all have the same format |
Input files with different extensions |
400 |
invalid reference date |
A reference_date that isn’t a YYYY-MM-DD date |
400 |
model version does not support batch v2 |
A version that can’t run v2 batches. Ask your Avra team |
404 |
uploaded file not found |
A path with no uploaded file behind it, such as an upload that didn’t finish |
404 |
model alias not found |
An alias the model doesn’t have |
409 |
alias and model version do not match; choose only one reference |
Both alias and model_version_id, pointing at different versions |
409 |
batch result is not ready |
A result requested before the batch completed or failed |
429 |
too many requests |
Over the files endpoints’ rate limit. Wait for the seconds in the Retry-After header |
Limits and practices that help
Each upload request accepts up to 1,000 files, so request URLs for more files in several calls. The files endpoints have rate limits of 1,500 requests per minute per organization, and 300 per caller, where a caller is an API key or a member in the dashboard. Upload URLs expire after one hour, and result URLs after 15 minutes. Batch jobs don’t count against the prediction rate limit. They run in queues with resource quotas. See Limits and quotas.
Prefer Parquet, keep the files of one batch in one format, and split large inputs across several medium files. When a batch fails, read statistics.errors before you send the same files again.