A slow provider operation should not own your terminal
Cloud deployment is submitted as one durable operation. The operation worker—not the CLI process—owns
provider mutation, retries, and reconciliation.
The command reports the durable operation ID before waiting. You may close the terminal and resume
from another session without sending another provider create request:
Idempotency, provider-resource identity, leased execution, and reconciliation protect the operation
across CLI disconnects and control-plane worker restarts. They cannot make unavailable GPU capacity
appear faster, but they prevent waiting from becoming an unsafe client-side workflow.
Explain what the backend actually exposed
Cold-start explanations label observed and unavailable timing boundaries independently. A provider
that exposes only zero-worker state and gateway time to first response byte does not produce a
fabricated artifact-download or model-load waterfall.
Compare measured serving plans
Use AIPerf once per real deployment, then compare compatible persisted results:
Inference Lab currently emits MEASURED candidates from benchmark history. It does not silently
provision hardware, estimate missing performance, or mix incompatible workloads. Cost is shown only
when a trustworthy source and observation timestamp were persisted.
Add governed external overflow
When every ordinary target is unhealthy—or a bounded queue policy remains breached—InferCrane can
select an explicitly configured external target:
External fallback can transmit prompts and generated output outside infrastructure you control and
can create a separate provider charge. InferCrane requires explicit acknowledgement and hard request
and worst-case cost reservations. It never fabricates a provider price or silently retries a request
after a possible send.