Skip to main content

​
Know what a deployment is waiting for

Provisioning is a durable operation. The CLI can disconnect while the control plane continues to reconcile one deterministic provider identity per replica intent. Reconnect with the operation ID printed by deploy:

​
Choose and check an adapter before provisioning

integrations distinguishes registration, local evidence, and real-system qualification. plan validates the requested intent without allocating capacity. When at least three successful, tenant-scoped observations match the exact provider adapter, runtime, region, and accelerator, the plan also shows an observed p50 from durable replica intent through runtime readiness. P95 is withheld until twenty successes. These values are historical evidence, never a stock reservation or readiness guarantee. Follow the exact adapter guide for credentials, networking, immutable images, runtime contract, preflight, cleanup, and current limits: InferCrane cannot safely choose a provider from an unspecified workload. Decide these inputs first: immutable model/artifact, runtime and version, compute mode, accelerator/topology, region/network, replica bounds, protocol needs, and required evidence. Then use this boundary: You do not need to read every provider guide. Select one only after the inputs above identify the operator-owned boundary: If these requirements do not select exactly one adapter, stop and collect workload, security, residency, capacity, and cost constraints; InferCrane will not silently choose infrastructure.
deploy and apply can create billable provider resources. Do not execute either until provider credentials and read-only doctor checks pass, the plan is reviewed, current provider inventory is recorded, and the exact combination’s qualification gap is accepted. Cost remains unknown when no trustworthy provider evidence exists.
status shows whether the deployment is serving and whether desired capacity is still converging. operation watch follows the blocking mutation. logs follows durable events and is the best view when you need the transition history rather than the latest snapshot.

​
Read the current stage

Not every provider exposes container download, artifact transfer, model load, and runtime initialization separately. InferCrane reports the narrowest boundary supported by fresh evidence. For example, provider_capacity_or_worker_initialization means exactly that the product cannot separate those phases from the available observation.

​
Diagnose a long wait

  1. Copy the blocking operation ID from status.
  2. Run operation watch and note the last changed stage, provider message, attempt, and retry time.
  3. Run logs --follow in a second terminal to distinguish a repeated observation from a real state transition.
  4. Run infercrane inspect DEPLOYMENT --output json for the persisted provider identity and non-secret infrastructure details.
  5. If the provider retains a failed resource - for example after an interrupted image pull - cancel the durable operation before replacing it. Do not start a second deploy with a different identity.
Known bootstrap failures such as interrupted image pulls or exhausted host storage remain visible and retryable while the provider retains the resource. InferCrane does not silently select a different accelerator or create a second billable resource.

​
Capacity evidence is advisory

Before the first create, InferCrane discovers the deterministic resource. An existing resource is adopted before mutable stock is consulted. If no resource exists, a provider adapter may report available, constrained, unavailable, or unknown:
  • unavailable defers creation with a retryable error;
  • constrained proceeds with an explicit warning because stock is not a reservation;
  • unknown permits the normal provider attempt but cannot support a capacity claim.
The first stock advisor queries RunPod secure-cloud availability. A region-qualified request is still labeled global when the provider response cannot prove stock in the requested region. Credentials are sent in headers and are not persisted in progress or events. Capacity history deduplicates repeated reconciliation polls by logical resource identity. A multi-minute startup therefore counts as one placement, not dozens of failed attempts, and its successful duration begins at the durable replica intent rather than the final health-check poll. Pending placements are disclosed separately and excluded from the success-rate denominator.

​
Reuse prewarmed runtime images

AWS and GCP worker bootstrap checks for the immutable runtime image locally before pulling it. A customer-maintained AMI or VM image may therefore prewarm the exact qualified digest and avoid retransferring those container layers. Startup logs expose identity_start, identity_ready, image_check, image_cache_hit, image_pull_start, image_pull_complete, runtime_start, and runtime_container_started timestamps without credentials. AWS observations parse only this closed marker grammar, discard all other console output, and persist it with the independently observed runtime_ready_at timestamp. infercrane inspect DEPLOYMENT renders the measured waterfall. AWS can additionally enforce INFERCRANE_AWS_IMAGE_CACHE_POLICY=required. A miss then fails before the registry pull instead of silently violating a fast-start pool’s contract. The default prefer policy retains the safe fallback to an immutable pull. This optimization concerns container layers only. It does not prove model weights are cached. Model-artifact locality remains a separate, provider-native observation through the artifact cache contract. InferCrane does not silently enable billed snapshot acceleration or accept a mutable image tag.

​
Idempotency and cleanup

The provider resource key is stored before any external create call. If the create succeeds but its response is lost, retry performs discovery and adopts the resource. Delete re-observes asynchronous provider removal and cannot mark the replica deleted while the resource remains visible.

​
Reconcile an interrupted create

  1. Do not create a new deployment, revision, or idempotency key.
  2. Reattach to the persisted operation and export the stored identities:
  3. In the provider’s read-only inventory, locate the exact provider_resource_id, deterministic external key, ownership tags/labels, revision, and replica ordinal from inspect. A same-name resource with mismatched ownership is not adoptable.
  4. If the identities match, let the existing operation resume. Reissuing the original deploy/apply request is safe only with its original idempotency key and identical intent. Replaying rollout provision DEPLOYMENT REVISION_ID --wait is safe for the same persisted revision; do not run rollout create again.
  5. If provider inventory is unavailable, incomplete, or mismatched, stop mutation and preserve PostgreSQL. Resolve ownership manually before create or delete.
Provider-specific inventory is intentionally outside one generic command: use the RunPod Pod/ endpoint inventory, AWS EC2 tag-scoped inventory, GCP label-scoped instance inventory, or Kubernetes namespace/UID/managed-fields inventory described by that adapter. An empty InferCrane orphans response cannot prove a provider credential can see every external resource. After a paid qualification run, use both views:
Then verify the provider account inventory directly. InferCrane inventory cannot prove the absence of resources hidden by stale, incomplete, or mis-scoped provider credentials.

​
Qualification boundary

Existing targets are implemented. Provider adapters have independent evidence states. SkyPilot elastic lifecycle logic passes local lost-response, replay, and cleanup fixtures, but real RunPod provider/region/GPU qualification remains external. Runtime inspection requires both a healthy endpoint and the expected served model. Development fake workers prove control flow only; they do not prove GPU performance, model compatibility, or provider reliability. See compatibility and qualification, cold-start intelligence, and the provider contract.