---
title: Batch inference
description: Score whole portfolios or embed large lists of companies asynchronously, from the API or the dashboard.
---

Batch inference complements live predictions when you need to score or embed many records at once. You upload one or more files, create a batch that points a model version at them, wait while it runs, and download one Parquet file with the results.

| Use case | Recommended path |
| -------- | ---------------- |
| Nightly portfolio rescoring | Batch |
| Real-time onboarding | [Live predictions](/core/predictions) |
| Marketing list enrichment | Batch |
| Underwriting one application at a time | Live predictions |

:::info
Batch inference is enabled per organization. Ask your Avra team to turn it on.
:::

## How a batch runs

A batch takes four steps: upload the input files through signed URLs, create the batch, wait for it to finish, and download the result through another signed URL. You can wait by polling the batch, or let a [webhook](/core/webhooks) tell you when it ends.

```mermaid
sequenceDiagram
  autonumber
  participant C as Your code
  participant A as Avra API
  participant S as Storage
  C->>A: POST /v1/api/files/upload
  A-->>C: Upload URLs, avra:// paths
  C->>S: PUT each file
  C->>A: POST /v2/api/batches
  A-->>C: 201 with the batch id
  alt Poll
    loop Until completed, failed, or cancelled
      C->>A: GET /v2/api/batches/{id}
      A-->>C: status
    end
  else Webhooks
    A--)C: batch-ended delivery
    C->>A: GET /v2/api/batches/{id}
    A-->>C: status
  end
  C->>A: GET /v2/api/batches/{id}/result
  A-->>C: download_url, for 15 minutes
  C->>S: GET download_url
  S-->>C: result.parquet
```

## Batch statuses

A batch starts as `pending`, waits in a queue for resources as `queued`, processes records as `running`, and consolidates its results as `concluding`. It ends as `completed`, `failed`, or `cancelled`.

```mermaid
stateDiagram-v2
  [*] --> pending
  pending --> queued
  queued --> running
  running --> concluding
  concluding --> completed
  running --> failed
  concluding --> failed
  running --> cancelling: stop requested
  cancelling --> cancelled
  completed --> [*]
  failed --> [*]
  cancelled --> [*]
```

The diagram leaves out rarer paths. For example, a batch can also fail while it's queued, and Avra can stop a batch at any status before it finishes, not only while it's running.

| Status | What it means | Webhook |
|---|---|---|
| `pending` | Created, and not yet sent to run | `batch-request` is sent when you create the batch |
| `queued` | Waiting in the queue for resources | None |
| `running` | Processing records | `batch-running` |
| `concluding` | Processing has ended, and the results are being consolidated | None |
| `completed` | Finished, with a result to download | `batch-ended` |
| `failed` | Couldn't finish | `batch-ended` |
| `cancelling` | A stop was requested | None |
| `cancelled` | Stopped before it finished | `batch-ended` |

A batch can't be cancelled through the v2 API. Contact Avra support if you need to stop one: it moves to `cancelling`, then `cancelled`.

## Prepare the input files

Each file holds one row per record, with a column for each input the version's schema declares, named exactly as in the schema. Every input the schema doesn't mark as optional must be there. Read the schema with `GET /v1/api/models/{model_id}/versions/{alias_or_version_id}`, as in [Live predictions](/core/predictions#2-read-the-versions-schema). In the dashboard, the batch form also offers an example CSV built from the version's schema.

Prefer Parquet over CSV. Parquet is typed and compressed, so uploads and jobs are faster and values keep their precision. CSV works for simple schemas, and the dashboard rejects a CSV file whose header has a column the schema doesn't declare. All the files in a batch must share the same format, such as all Parquet or all CSV.

In Parquet files, aim for at most 100k rows per row group and 5 GB per file. Split large inputs across files: a batch accepts many files at once, and several medium files beat one huge one. Very large jobs may need sizing beforehand, so reach out to your Avra team.

| Requirement | Value |
|---|---|
| **Formats** | Parquet (recommended) or CSV, the same format for every file in a batch |
| **Columns** | One per input in the version's schema, named as in the schema |
| **Parquet layout** | At most 100k rows per row group and 5 GB per file, as guidance |
| **Path you declare** | Relative, such as `portfolio/part-0001.parquet`: no leading `/`, no `..`, `~`, or `\`, and no empty segments |
| **Checksum** | The file's MD5 digest, base64-encoded, such as `KPRxZBtAn5UfexFEKfKr7Q==` |
| **Media type** | Optional, as `content_type`. Left out, it's inferred from the extension: `application/vnd.apache.parquet` for `.parquet`, `text/csv` for `.csv` |
| **Files per upload request** | Up to 1,000. Request URLs for more files in several calls |

To compute the checksum on the command line, run `openssl dgst -md5 -binary portfolio.parquet | base64`.

## Run a batch from Python

This script runs the whole flow: it computes each file's MD5, requests the upload URLs, uploads the files, creates the batch, polls until it finishes, and downloads the result. It needs the `requests` package. Set `MODEL_ID` and `FILES` before you run it.

```python batch.py expandable
"""Upload files, run a batch on them, wait for it, and download the result."""
import base64
import hashlib
import os
import sys
import time
from pathlib import Path

import requests

API = "https://api.avra.ai"
HEADERS = {"Authorization": f"Bearer {os.environ['AVRA_API_KEY']}"}
MODEL_ID = "<model_id>"
ALIAS = "production"
FILES = [Path("portfolio/part-0001.parquet"), Path("portfolio/part-0002.parquet")]
FINISHED = {"completed", "failed", "cancelled"}


def md5_base64(path: Path) -> str:
    digest = hashlib.md5()
    with path.open("rb") as f:
        for chunk in iter(lambda: f.read(1024 * 1024), b""):
            digest.update(chunk)
    return base64.b64encode(digest.digest()).decode()


# 1. Request one signed upload URL per file. The URLs expire after one hour.
targets = [{"file": path.as_posix(), "md5": md5_base64(path)} for path in FILES]
response = requests.post(
    f"{API}/v1/api/files/upload", headers=HEADERS, json={"targets": targets}, timeout=60
)
response.raise_for_status()
upload = response.json()

# 2. Upload each file with the Content-Type and Content-MD5 its URL was signed for.
for path, target in zip(FILES, upload["targets"]):
    with path.open("rb") as f:
        put = requests.put(
            target["upload_url"],
            data=f,
            headers={"Content-Type": target["content_type"], "Content-MD5": target["md5"]},
            timeout=600,
        )
    put.raise_for_status()

# 3. Create the batch from the files' avra:// paths.
response = requests.post(
    f"{API}/v2/api/batches",
    headers=HEADERS,
    json={
        "model_id": MODEL_ID,
        "alias": ALIAS,
        "input": {"files": [target["file"] for target in upload["targets"]]},
    },
    timeout=60,
)
response.raise_for_status()
batch = response.json()
print("created batch", batch["id"])

# 4. Poll until the batch finishes.
while batch["status"] not in FINISHED:
    time.sleep(30)
    response = requests.get(f"{API}/v2/api/batches/{batch['id']}", headers=HEADERS, timeout=30)
    response.raise_for_status()
    batch = response.json()
    print(batch["status"])

print("statistics:", batch.get("statistics"))
if batch["status"] != "completed":
    sys.exit(f"batch {batch['id']} ended as {batch['status']}")

# 5. Download the result. The signed URL is valid for 15 minutes.
response = requests.get(
    f"{API}/v2/api/batches/{batch['id']}/result", headers=HEADERS, timeout=30
)
response.raise_for_status()
result = response.json()
out_path = Path(f"{batch['id']}-{result['name']}")
with requests.get(result["download_url"], stream=True, timeout=600) as download:
    download.raise_for_status()
    with out_path.open("wb") as out:
        for chunk in download.iter_content(chunk_size=1024 * 1024):
            out.write(chunk)
print("saved", out_path)
```

The upload response lists the targets in the order you sent them, which is what lets the script pair each file with its URL. To pin an exact version instead of following the alias, send `model_version_id` in place of `alias`. Add `"reference_date": "2025-12-31"` to the batch to get predictions as of a past date, for a backtest.

## The same steps with cURL

The same flow for one file, with `jq` to read the responses:

1. **Request an upload URL**

    ```bash
    FILE="portfolio.parquet"
    MD5=$(openssl dgst -md5 -binary "$FILE" | base64)

    jq -n --arg file "$FILE" --arg md5 "$MD5" '{targets: [{file: $file, md5: $md5}]}' |
      curl -s -X POST https://api.avra.ai/v1/api/files/upload \
        -H "Authorization: Bearer $AVRA_API_KEY" \
        -H "Content-Type: application/json" \
        -d @- > upload.json
    ```

    The response has one target per file. `file` is the path to reference in the batch, and `expiration` is when the upload URLs stop working:

    ```json
    {
      "targets": [
        {
          "file": "avra://uploads/1784669990715/portfolio.parquet",
          "content_type": "application/vnd.apache.parquet",
          "md5": "KPRxZBtAn5UfexFEKfKr7Q==",
          "upload_url": "<signed upload URL>"
        }
      ],
      "expiration": "2026-11-09T15:00:00Z"
    }
    ```

2. **Upload the file**

    Send the `Content-Type` and `Content-MD5` the URL was signed for. Storage rejects an upload whose digest doesn't match.

    ```bash
    curl -X PUT "$(jq -r '.targets[0].upload_url' upload.json)" \
      -H "Content-Type: $(jq -r '.targets[0].content_type' upload.json)" \
      -H "Content-MD5: $MD5" \
      --upload-file "$FILE"
    ```

3. **Create the batch**

    ```bash
    MODEL_ID="<model_id>"

    jq -n --arg model "$MODEL_ID" --arg path "$(jq -r '.targets[0].file' upload.json)" \
      '{model_id: $model, alias: "production", input: {files: [$path]}}' |
      curl -s -X POST https://api.avra.ai/v2/api/batches \
        -H "Authorization: Bearer $AVRA_API_KEY" \
        -H "Content-Type: application/json" \
        -d @- > batch.json

    BATCH_ID=$(jq -r '.id' batch.json)
    ```

    Avra responds with `201` and the batch:

    ```json
    {
      "id": "0d9c3f6e-4b1a-4f7e-9a2b-5c8d7e6f1a20",
      "created_at": "2026-11-09T14:05:12Z",
      "model_id": "8f14e45f-ceea-467f-a8f0-5c3f0c7a1b2d",
      "model_version_id": "c9bf9e57-1685-4c89-bafb-ff5af830be8a",
      "reference_date": null,
      "input": {
        "files": ["avra://uploads/1784669990715/portfolio.parquet"]
      },
      "status": "pending"
    }
    ```

4. **Wait for it to finish**

    Check the status until it's `completed`, `failed`, or `cancelled`:

    ```bash
    curl -s "https://api.avra.ai/v2/api/batches/$BATCH_ID" \
      -H "Authorization: Bearer $AVRA_API_KEY" | jq '{status, statistics}'
    ```

5. **Download the result**

    ```bash
    curl -s "https://api.avra.ai/v2/api/batches/$BATCH_ID/result" \
      -H "Authorization: Bearer $AVRA_API_KEY" > result.json

    curl -o result.parquet "$(jq -r '.download_url' result.json)"
    ```

    The result response names the file and says when its URL expires:

    ```json
    {
      "name": "result.parquet",
      "download_url": "<signed download URL>",
      "expires_at": "2026-11-09T16:20:00Z"
    }
    ```

## The batch object

`POST /v2/api/batches`, `GET /v2/api/batches/{id}`, and `GET /v2/api/batches` all return batches with these fields. The list returns the batches of the API key's workspace.

| Field | Type | Description |
|---|---|---|
| `id` | string | The batch's ID |
| `created_at` | string | When the batch was created, in RFC 3339 |
| `model_id` | string | The model it runs |
| `model_version_id` | string | The exact version it runs, resolved from the alias when you sent one |
| `reference_date` | string or null | The date it predicts as of, when you set one |
| `input.files` | array of strings | The `avra://` paths of its input files |
| `status` | string | One of the [batch statuses](#batch-statuses) |
| `statistics` | object | Counts for the run, once the batch has them |

`statistics` has `total`, `success`, and `failed` record counts. When records fail, `errors` lists each error `message` with its `count`. For a model that returns a score, `score` describes their distribution: `min`, `max`, `mean`, `std`, `median`, and a `histogram`.

The batch keeps the version it resolved when you created it. Moving an alias afterward doesn't change a batch that's already running.

## Download the result

Once the batch has completed or failed, `GET /v2/api/batches/{id}/result` returns a signed `download_url` that's valid for 15 minutes, with the file's `name` and the URL's `expires_at`. The result is a single Parquet file, `result.parquet`. Asking earlier returns `409 Conflict` with `batch result is not ready`.

When a URL expires, request the result again for a new one. Each successful request sends a `batch-result-download` [webhook](/core/webhooks) event.

## Use webhooks instead of polling

Subscribe an endpoint to `batch-running` and `batch-ended` in **Alerts → Webhooks**, and let Avra tell you when a batch changes, instead of polling. `batch-ended` covers all three endings: completed, failed, and cancelled. A delivery doesn't carry the batch's status, so when one arrives, read the batches you're waiting on with `GET /v2/api/batches/{id}` and act on the ones that finished.

Store each batch's `id` when you create it. Then call a function like this one from your webhook endpoint, after it [verifies the delivery](/core/webhooks#verify-deliveries):

```python
import json
import os
from pathlib import Path

import requests

API = "https://api.avra.ai"
HEADERS = {"Authorization": f"Bearer {os.environ['AVRA_API_KEY']}"}
PENDING = Path("pending-batches.json")  # the IDs you stored, kept in your database in production
FINISHED = {"completed", "failed", "cancelled"}


def on_batch_ended() -> None:
    pending = json.loads(PENDING.read_text()) if PENDING.exists() else []
    still_running = []
    for batch_id in pending:
        response = requests.get(f"{API}/v2/api/batches/{batch_id}", headers=HEADERS, timeout=30)
        response.raise_for_status()
        batch = response.json()
        if batch["status"] not in FINISHED:
            still_running.append(batch_id)
        elif batch["status"] == "completed":
            result = requests.get(
                f"{API}/v2/api/batches/{batch_id}/result", headers=HEADERS, timeout=30
            )
            result.raise_for_status()
            print(batch_id, "download from", result.json()["download_url"])
        else:
            print(batch_id, batch["status"], batch.get("statistics"))
    PENDING.write_text(json.dumps(still_running))
```

Deliveries are at least once, so the same `batch-ended` can arrive twice. The function is safe to run again: a batch it already handled is no longer pending.

## Run a batch from the dashboard

You can do the same from the dashboard under **Batches**. Pick the model and version, then add new Parquet or CSV files or reference files you already uploaded by their `avra://` path. Optionally, choose a reference date up to two years back to get predictions as of that date, then review and submit. Each batch's details show its progress and, once it has completed, let you download its results.

Creating batches takes the permission to create batches, and downloading results in the dashboard takes the permission to download them. See [Roles and permissions](/dashboard/roles-permissions).

## Errors

Errors return JSON with a numeric `code` and a `message`. These are the ones specific to batches:

| Status | `message` | What it means |
|---|---|---|
| `400` | `too many files in one request` | An upload request with more than 1,000 files |
| `400` | `md5 must be a base64-encoded 128-bit digest` | A checksum that isn't a base64-encoded MD5 digest |
| `400` | `invalid file path` | A path that breaks the [path rules](#prepare-the-input-files), or that isn't one of your workspace's files |
| `400` | `batch input files must all have the same format` | Input files with different extensions |
| `400` | `invalid reference date` | A `reference_date` that isn't a `YYYY-MM-DD` date |
| `400` | `model version does not support batch v2` | A version that can't run v2 batches. Ask your Avra team |
| `404` | `uploaded file not found` | A path with no uploaded file behind it, such as an upload that didn't finish |
| `404` | `model alias not found` | An alias the model doesn't have |
| `409` | `alias and model version do not match; choose only one reference` | Both `alias` and `model_version_id`, pointing at different versions |
| `409` | `batch result is not ready` | A result requested before the batch completed or failed |
| `429` | `too many requests` | Over the files endpoints' rate limit. Wait for the seconds in the `Retry-After` header |

## Limits and practices that help

Each upload request accepts up to 1,000 files, so request URLs for more files in several calls. The files endpoints have rate limits of 1,500 requests per minute per organization, and 300 per caller, where a caller is an API key or a member in the dashboard. Upload URLs expire after one hour, and result URLs after 15 minutes. Batch jobs don't count against the [prediction rate limit](/core/predictions#limits). They run in queues with resource quotas. See [Limits and quotas](/core/limits).

Prefer Parquet, keep the files of one batch in one format, and split large inputs across several medium files. When a batch fails, read `statistics.errors` before you send the same files again.

:::warning
The v1 batch endpoints (`/v1/api/batch-inputs/*` and `/v1/api/batches/*`) are **deprecated** and kept only for backward compatibility. Build new integrations on the v2 endpoints on this page. See [Migrating to Batch v2](/changelog/migrations/batch-v2).
:::

## Where to go next

**[Webhooks](/core/webhooks)**

Verify deliveries and handle retries.

**[Model versioning](/core/model-versioning)**

Choose the version a batch runs.

**[Limits and quotas](/core/limits)**

Every limit for files, batches, and URLs.

**[API reference](/api-reference)**

Every endpoint, field, and status code.
