Rate limits & retries
Treat 429, brief 5xx, and long model latency as normal on an aggregation gateway.
| Symptom | Likely cause | Action |
|---|---|---|
| 429 | Platform or upstream limit | Exponential backoff; lower concurrency |
| 5xx | Transient upstream/gateway issue | Retry a few times; escalate if persistent |
| Client timeout | Slow reasoning / network | Timeout ≥ 60s; higher for long jobs |
Retry only idempotent-safe cases; do not blindly retry 400 / 401 / 403. Split keys per app for isolation. Keep timestamps, model ids, and request ids for support.