Batch Inference
Process portfolios, refresh segments, or generate large embedding sets asynchronously.
Batch inference complements the low-latency API when you need to score or enrich large portfolios.
When to choose batch
| Use case | Recommended path |
|---|---|
| Nightly portfolio rescoring | Batch |
| Real-time onboarding | REST API |
| Marketing list enrichment | Batch |
| High-touch underwriting | REST API with on-demand enrichment |
Workflow
Upload data
Request signed upload URLs with POST /v2/api/files/upload, declaring each file’s path, content type, and base64-encoded MD5 digest.
Upload each file with a PUT to its upload_url, sending matching Content-Type and Content-MD5 headers.
Submit job
Request a batch prediction with POST /v2/api/batches, referencing the uploaded files, an available model, and an optional reference_date.
Avra responds with a batch id so you can track the job.
Processing
Jobs run in prioritized queues with resource usage quotas. Expect minutes for thousands of records and hours for millions.
Waiting
Poll GET /v2/api/batches/{id} checking job status or subscribe to the batch-lifecycle webhook to be notified on status transitions.
Download results
Request a download link at GET /v2/api/batches/{id}/result to obtain a signed short-lived download_url.
Limits and Best Practices
- No hard limits. The Files API does not impose size or row-count limits, but following good practices keeps job queues no longer than needed and jobs reliable.
- Split large inputs across many files. A batch accepts many files at once, prefer uploading and referencing several medium files over one huge one.
- Prefer Parquet over CSV. Parquet datasets have strong typing and compression, providing faster uploads and batch jobs while preserving data quality and precision.
- Size Parquet datasets sensibly. A Parquet dataset can be split into
row_groups(the minimal decompressed block in a read operation) andpartitions(the segmentation of the dataset across multiple files). Aim for at most 100k rows perrow_groupand 5 GB perpartition. - Very large jobs. These may require fine-tuning to avoid failures, reach out so we can size the run properly.
Integrations
- Use SFTP Batch Inference for scheduled large files.
- Configure webhooks triggered by many batch-related events. See Webhooks.
Monitoring
Track batch throughput and failures in the dashboard under Batches.