Capabilities
Async Execution
Batch API and background mode for long-running or bulk inference
Read this when
- Processing a large queue of independent prompts
- Running deep-research / long agentic flows
Batch API
Async bulk inference. Submit a JSONL of requests, run within a window (typically 24h, often <1h), retrieve results at ~50% of synchronous cost.
| Provider | API | pi-go | Notes |
|---|---|---|---|
| Anthropic | ✅ Message Batches; up to 300k output tokens with output-300k-2026-03-24 beta | ❌ | not wired |
| OpenAI Chat | ✅ Batch API; 50% discount | ❌ | not wired |
| OpenAI Responses | ✅ | ❌ | not wired |
| Google Gemini | ✅ via Vertex AI batch predictions | ❌ | not wired |
| Claude CLI | ❌ | — |
Background Mode
Long-running async generation: submit, poll status, retrieve.
| Provider | API | pi-go | Notes |
|---|---|---|---|
| Anthropic | ❌ (use Batch instead) | — | |
| OpenAI Chat | ❌ | — | |
| OpenAI Responses | ✅ background: true (preview) | ❌ | not wired |
| Google Gemini | ❌ | — | Live API is realtime, not background |
Provider Documentation
- Anthropic — Message Batches
- OpenAI — Batch API
- OpenAI Responses — Background mode
- Google — Vertex AI batch predictions
pi-go Gaps
- No
BatchProvidercapability interface; no JSONL builder, submit/poll/retrieve helpers. - No
BackgroundProvidercapability interface. - Both belong as optional capability interfaces alongside
ImageProviderandObjectProvider. - Without these, callers must use synchronous calls even for embarrassingly parallel workloads — forfeiting the ~50% Batch discount.