Model High Availability
Keep your agent answering when an LLM provider throttles, runs out of credit, or has a blip. Foldspace automatically fails over to the next healthy model and key, usually without the end user noticing.
High availability is the uptime half of model routing. Token Quotas control how much your agents spend; high availability keeps them responding when an upstream provider fails.
How failover works
When an agent calls an LLM, Foldspace builds an ordered list of options — each option is a model + API key (and region, where it applies) — and sends the request to the first healthy one. If that option fails with a recoverable error, Foldspace moves to the next option in the list automatically.
"Unavailable" always applies to a specific model + key + region combination, never to an entire provider. A single throttled key doesn't take a provider offline; only that one rung is skipped, and it's retried again once it recovers.
Fallback order
Foldspace works down two dimensions in order: it tries the keys for a model first, then moves to a peer model.
Key order (for each model)
- Your configured BYOK — the API key set on the agent.
- Another of your keys for that provider, if one exists.
- The Foldspace platform key, when platform fallback is allowed.
Model order
Foldspace starts with the model configured on the agent. If every key for that model is unavailable, it may try one peer model in the same capability tier on the other provider (US / global regions).
In the EU, agents stay on Gemini and do not cross providers — this preserves data residency, so EU failover has fewer rungs by design.
Model pairs
Each tier has a matched peer on the other provider. When the primary model is exhausted, failover crosses to its peer at the same tier.
| Tier | Google Gemini | OpenAI peer |
|---|---|---|
| Lite | Gemini 3.1 Flash Lite | GPT-5.6 Luna |
| Standard | Gemini 3.5 Flash | GPT-5.6 Terra |
| Pro | Gemini 3.1 Pro Preview | GPT-5.6 Sol |
Typical full order
A request on a US agent with BYOK and platform fallback enabled walks the list like this:
| # | What we try | Meaning |
|---|---|---|
| 1 | Requested model + your BYOK | Primary path |
| 2 | Requested model + your other key | Same model, different key of yours |
| 3 | Requested model + platform key | Same model, Foldspace key |
| 4 | Peer model + your BYOK | Other provider, same tier |
| 5 | Peer model + platform key | Other provider on Foldspace key |
Foldspace always picks the first healthy option. If a request fails partway through, it may retry briefly on the same option (for flaky errors) or hop to the next. For streaming replies, failover only happens before the first tokens reach the user — once tokens start streaming, that option is committed.
Error types
Not every failure is treated the same. Some are transient and masked by failover; others are yours to fix and stop before any spend is redirected.
| Error type | What it usually means | What failover does | What you should do |
|---|---|---|---|
| Capacity | Provider shared capacity exhausted (common 429) | Retries briefly, then moves to the next option; parks the option after repeated failures | Usually nothing — failover masks it. It's typically transient. |
| Unavailable | Provider timeout or temporary outage (408 / 425 / 5xx) | Retries briefly, then moves to the next option | Nothing — a transient provider issue that failover handles. |
| Billing / spend cap | Your provider quota or billing is exhausted | Moves to another key or model; platform fallback may keep chat working | Check the billing and quota on your provider account. |
| Rate limited | Your key hit its RPM / TPM limits | Puts the key on a short cooldown (~2 minutes) and uses another key or model | Raise your provider limits, or reduce traffic. |
| Auth | Invalid or unauthorized API key (401 / 403) | No failover — does not fall back to the platform key | Fix or rotate your BYOK. Chat fails until the key is corrected. |
| Client error | Bad request or unsupported model (400 / 404 / 422) | No failover — this is a config or request issue, not capacity | Review the model configuration on the agent. |
:::note Auth failures are intentionally fail-loud. If your BYOK is wrong or revoked, Foldspace does not quietly spend platform budget in the background to cover it. Update the key to restore service. :::
What if the requested model is already unavailable?
If a model + key combination was recently marked unavailable, new requests skip it and go straight to the next healthy option:
- Your key is down, platform key is healthy → the same model continues on the platform key.
- All keys for the requested model are down → Foldspace tries the peer model, when allowed.
- Nothing healthy is left → the turn fails, and the user may see a brief technical-difficulty message.
:::tip If you run in the EU, remember there's no cross-provider peer — Gemini issues have fewer alternate rungs, so healthy BYOK and provider limits matter more. Keep your keys valid and your provider quotas ahead of demand. :::