Skip to main content

​
A slow provider operation should not own your terminal

Cloud deployment is submitted as one durable operation. The operation worker, not the CLI process - owns provider mutation, retries, and reconciliation.
The command reports the durable operation ID before waiting. You may close the terminal and resume from another session without sending another provider create request:
Idempotency, provider-resource identity, leased execution, and reconciliation protect the operation across CLI disconnects and control-plane worker restarts. They cannot make unavailable GPU capacity appear faster, but they prevent waiting from becoming an unsafe client-side workflow.

​
Explain what the backend actually exposed

Cold-start explanations label observed and unavailable timing boundaries independently. A provider that exposes only zero-worker state and gateway time to first response byte does not produce a fabricated artifact-download or model-load waterfall.

​
Compare measured serving plans

Use AIPerf once per real deployment, then compare compatible persisted results:
Inference Lab currently emits MEASURED candidates from benchmark history. It does not silently provision hardware, estimate missing performance, or mix incompatible workloads. Cost is shown only when a trustworthy source and observation timestamp were persisted.

​
Add governed external overflow

When every ordinary target is unhealthy - or a bounded queue policy remains breached - InferCrane can select an explicitly configured external target:
External fallback can transmit prompts and generated output outside infrastructure you control and can create a separate provider charge. InferCrane requires explicit acknowledgement and hard request and worst-case cost reservations. It never fabricates a provider price or silently retries a request after a possible send.
This is a safety valve, not automatic provider shopping. Selection, denial, recovery, budget counters, and the signal snapshot remain available as persisted evidence.