Skip to main content

​
Governed external capacity

InferCrane can route to a registered external API when all primary targets are unhealthy or an explicit bounded queue policy is satisfied. This is emergency capacity, not a model marketplace: InferCrane does not resell access, select a model, or hide the external provider from the operator.
External fallback can transmit prompts and generated output outside infrastructure you control and can create a separate provider charge. Enabling it requires an explicit privacy acknowledgement and hard request and cost-reservation limits.
External capacity has two explicit scopes:
  • Stable endpoint binding: an authenticated external API participates in an immutable manual, primary-fallback, or weighted serving plan. This is the preferred one-endpoint composition.
  • Deployment overflow: emergency external capacity is selected when one concrete deployment’s primary targets are unhealthy or its bounded queue policy is satisfied.
Both scopes use reference-only credentials, consent, in-memory budget leases, and no request replay. They are configured separately so a deployment policy cannot silently change an endpoint plan. See Stable endpoints for first-class endpoint bindings.

​
Configure OpenRouter fallback

Inject the credential into the control-plane environment, then register only its reference:
Register the exact endpoint and model mapping. Replace SECRET_REFERENCE_ID with the returned ID:
The policy attaches the registered fallback target to qwen-prod; it is excluded from ordinary healthy-primary routing. Inspect the durable policy and reserved budget:
For queue overflow, opt in to hysteresis and cooldown explicitly:
Missing or stale queue metrics never imply an empty queue. Consecutive observations and cooldown prevent route oscillation. Every selection, recovery, denial, reason change, and hysteresis-counter transition is persisted with the exact policy and signal snapshot. Identical steady-state holds are coalesced rather than writing an unbounded row on every health interval.

​
Selection and budget behavior

The reconciler publishes one external route after every ordinary target is unhealthy or after the configured queue breach persists for the required intervals. Before each external transmission, the gateway atomically consumes a request reservation and the configured worst-case cost reservation. Exhaustion returns an error before sending bytes. Reservations are deliberately conservative. A request that reserves more than its eventual provider charge does not receive an automatic refund because InferCrane does not ingest an authoritative provider invoice. cost_limit_usd is therefore an authorization ceiling, not an estimated bill. InferCrane selects the provider before transmission. It never replays a request after a possible send, never duplicates streaming traffic, and does not silently shadow user requests. Selection, denial, budget counters, and health changes remain available through persisted events and request records.

​
Disable fallback

Re-run external configure without --enable using the same target, reference, acknowledgement, and limits. This replaces the policy with a disabled policy; it does not delete the target or secret reference.

​
Current limits

  • Weighted, semantic, shadow, and request-duplicating external routing are not implemented.
  • Environment references are currently the only secret resolver.
  • Provider prices are not fetched or fabricated.
  • Real OpenRouter billing qualification is deferred to the consolidated manual release gate.