xeroctl batches

Create, inspect, cancel, and download results for batch inference jobs from the command line. An endpoint slug is always required.

Overview

The xeroctl batches command wraps the Batch API for use from the command line. Every operation requires the --endpoint <slug> flag.

The typical workflow is:

  1. Prepare a JSONL file where each line is a single inference request.
  2. Run xeroctl batches --endpoint <slug> --input requests.jsonl to upload the file and create the batch in one step.
  3. Poll status with --wait, or check manually using xeroctl batches <id> --endpoint <slug>.
  4. Download the output JSONL once the batch completes.

Back to xeroctl CLI overview or the Batch API reference.

JSONL Format

Each line in the input file is a self-contained JSON object representing one inference request. The format mirrors the OpenAI Batch API JSONL specification.

Required Fields

Field Type Description
custom_id string Unique identifier for this request. Echoed back in the output file to correlate results.
method string HTTP method. Must be "POST".
url string Target API path, e.g. "/v1/chat/completions" or "/v1/embeddings".
body object The full request body for the target endpoint. Must include model and the required body fields for that endpoint.

Example -- Chat Completions Batch

JSONL
{"custom_id": "req-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "my-model", "messages": [{"role": "user", "content": "What is the capital of France?"}], "max_tokens": 64}} {"custom_id": "req-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "my-model", "messages": [{"role": "user", "content": "Summarize the Pythagorean theorem."}], "max_tokens": 128}} {"custom_id": "req-3", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "my-model", "messages": [{"role": "user", "content": "Write a haiku about the ocean."}], "max_tokens": 64}}

Example -- Embeddings Batch

JSONL
{"custom_id": "doc-1", "method": "POST", "url": "/v1/embeddings", "body": {"model": "bge-large-en-v1.5", "input": "Retrieval-augmented generation improves factual accuracy."}} {"custom_id": "doc-2", "method": "POST", "url": "/v1/embeddings", "body": {"model": "bge-large-en-v1.5", "input": "Large language models can reason over long contexts."}} {"custom_id": "doc-3", "method": "POST", "url": "/v1/embeddings", "body": {"model": "bge-large-en-v1.5", "input": "Fine-tuning adapts a pre-trained model for a specific task."}}

Note: Each line must be a complete, valid JSON object. Blank lines are not permitted. The custom_id must be unique within the file.

List Batches

List all batch jobs for the specified endpoint. The table shows ID, Status, Total requests, Completed, Failed, and Created time.

bash
# List batches for an endpoint xeroctl batches --endpoint my-llm # Limit results xeroctl batches --endpoint my-llm --limit 20 # Paginate using a cursor (batch ID to start after) xeroctl batches --endpoint my-llm --after batch_abc123 # JSON output xeroctl batches --endpoint my-llm -o json

When more results exist beyond the current page, a hint is printed showing the last batch ID to use as the --after cursor.

Get Batch

Show full details for a specific batch job.

bash
xeroctl batches batch_abc123 --endpoint my-llm # JSON output xeroctl batches batch_abc123 --endpoint my-llm -o json

The detail view includes: ID, Status, Endpoint, Input File ID, Output File ID, Error File ID, Completion Window, Total / Completed / Failed request counts, Created, Started At, Expires, Completed At, Cancelled At, and any metadata.

Create Batch

Provide an --input path to upload the JSONL file and submit the batch in a single command. The CLI uploads the file first (printing the file ID), then creates the batch job.

bash
# Create and return immediately xeroctl batches --endpoint my-llm --input requests.jsonl # Create and wait for completion xeroctl batches --endpoint my-llm --input requests.jsonl --wait # Attach metadata key=value pairs xeroctl batches --endpoint my-llm \ --input requests.jsonl \ --metadata project=finance \ --metadata run_id=batch-2026-04-09 # Preview without uploading or creating xeroctl batches --endpoint my-llm --input requests.jsonl --dry-run

Create Options

Option Description
--input <file.jsonl> Path to the JSONL input file. Triggers batch creation.
--metadata <key=value> Attach metadata to the batch. Repeat the flag for multiple pairs.
--wait After creating, poll until the batch reaches a terminal state. Can also be combined with an existing batch ID.

Cancel Batch

Cancel an in-progress batch. The batch transitions to cancelling and then cancelled. Requests that were already processed will have results in the partial output file.

bash
# Cancel with confirmation xeroctl batches batch_abc123 --endpoint my-llm --cancel # Preview without cancelling xeroctl batches batch_abc123 --endpoint my-llm --cancel --dry-run

Download Output

Download the output JSONL file from a completed batch. The output file contains one result object per successful request, keyed by custom_id.

bash
# Download to default filename (<batch-id>-output.jsonl) xeroctl batches batch_abc123 --endpoint my-llm --download # Specify output path xeroctl batches batch_abc123 --endpoint my-llm --download -o results.jsonl

If no output file is available (the batch has not yet completed, or all requests failed), an error message is printed along with the current batch status.

Download Errors

Download the error JSONL file for a batch. The error file contains one entry per failed request, including the custom_id and error details.

bash
# Download error file to default filename (<batch-id>-errors.jsonl) xeroctl batches batch_abc123 --endpoint my-llm --errors # Specify output path xeroctl batches batch_abc123 --endpoint my-llm --errors -o batch-errors.jsonl

Wait for Completion

Poll a batch every 2 seconds until it reaches a terminal state: completed, failed, cancelled, or expired. An inline progress bar is printed when request counts are available.

bash
# Wait on an existing batch xeroctl batches batch_abc123 --endpoint my-llm --wait # Create and immediately wait xeroctl batches --endpoint my-llm --input requests.jsonl --wait

On completion, the terminal status, request counts, and file IDs are printed. Use -o json to receive a machine-readable summary on completion.

Options Reference

Full reference for all flags and options accepted by xeroctl batches.

Required

Option Description
--endpoint <slug> Endpoint slug. Required for all batch operations.

Action Flags (mutually exclusive)

Flag / Option Requires Batch ID Description
--input <file.jsonl> No Upload and create a new batch from the given JSONL file.
--cancel Yes Cancel a running batch.
--download Yes Download the output JSONL file.
--errors Yes Download the error JSONL file.
--wait Yes (or use with --input) Poll until the batch reaches a terminal state.

Other Options

Option Description
-o, --output <path> Destination file path for --download or --errors. Defaults to <batch-id>-output.jsonl or <batch-id>-errors.jsonl.
--metadata <key=value> Metadata to attach when creating a batch. Repeatable.
--limit <n> Maximum number of results when listing.
--after <id> Pagination cursor: batch ID to start results after.

Examples

End-to-End Batch Workflow

bash
#!/bin/bash ENDPOINT="my-llm" INPUT="requests.jsonl" # Create batch and wait for it to finish xeroctl batches --endpoint "$ENDPOINT" --input "$INPUT" --wait # Retrieve the batch ID from the list BATCH_ID=$(xeroctl batches --endpoint "$ENDPOINT" -o json \ | jq -r '.data[0].id') # Download output xeroctl batches "$BATCH_ID" --endpoint "$ENDPOINT" --download -o output.jsonl # Download any errors xeroctl batches "$BATCH_ID" --endpoint "$ENDPOINT" --errors -o errors.jsonl

Monitor an Existing Batch

bash
# Check current status xeroctl batches batch_abc123 --endpoint my-llm # Block until done xeroctl batches batch_abc123 --endpoint my-llm --wait

Create with Metadata Tags

bash
xeroctl batches \ --endpoint my-llm \ --input prompts.jsonl \ --metadata env=production \ --metadata team=data-science \ --metadata version=1.2.0

xeroctl CLI

Overview of all xeroctl commands, installation, and configuration.

Batch API

REST API reference for batch inference, JSONL format, and output schema.

xeroctl files

Manage uploaded files used as batch inputs and outputs.