RunPod
RunPod is one registered infrastructure provider with separate executable adapters. Built-in vLLM elastic deployments use SkyPilot. Immutable custom OCI workloads use the nativerunpod-pods REST
adapter, which does not require SSH inside the runtime image. Serverless deployments use RunPod’s
native endpoint lifecycle. InferCrane’s base production stack does not require RunPod; enable it
explicitly with compose.production.runpod.yaml.
Native Pods and Serverless endpoints use RunPod’s
REST API v2 at https://api.runpod.io/v2.
InferCrane requests on-demand Secure Cloud Pods. RunPod no longer offers Spot (interruptible) Pods,
so there is no interruptible option or price to choose.
AMD Instinct MI300X and MI350X run through runpod-pods with the pinned ROCm vLLM image; SkyPilot and
Serverless stay NVIDIA-only. See AMD Instinct GPUs.
Before the first deployment
1
Store a scoped RunPod key
Give the key only the Pod and Serverless permissions you intend this control plane to use. Keep
it outside the repository and restrict the file:
2
Start the explicit RunPod production profile
Copy The base production profile is provider-neutral. The additional overlay mounts the key
read-only and enables native RunPod Pods and optional provider-native Serverless. SkyPilot is
not enabled implicitly by a RunPod credential.Large custom OCI workloads may need more than the default 100 GiB container disk. Set, for
example, For gated repositories, create a scoped RunPod secret and configure only its name:Native Pod requests contain RunPod’s secret reference syntax, never the Hugging Face token.
InferCrane fails closed on an invalid secret name and cannot verify the secret value without
launching a workload.To populate a new identity-named volume with a resumable transfer, first set the exact
model/commit, data center, size, current provider rates, retention window, and watchdog, then run
.env.production.example to a private path, replace its placeholder values, and add the
key-file path from .env.runpod.example. The Serverless template ID is optional unless you use
provider-native Serverless.INFERCRANE_RUNPOD_CONTAINER_DISK_GIB=500 in the same private environment file. Disk
size is validated before the control plane starts and may affect provider cost.Native Pods can instead reuse a model-specific RunPod network volume. Create the volume in the
intended data center and name it infercrane-artifact- followed by the first 20 hexadecimal
characters of SHA-256 over the exact model@commit identity. Then configure the exact mapping:make plan-runpod-artifact-cache. Review its JSON worst-case cost before setting
INFERCRANE_RUNPOD_MAX_COST_USD and using make build-runpod-artifact-cache. The build command is
billable; its hermetic safety test is make test-runpod-artifact-cache.Before creating a Pod, InferCrane performs a read-only volume lookup, verifies its ID, exact
identity-derived name, positive size, and data center, then mounts it at /workspace. Standard
Hugging Face downloads and filesystem-materialized model profiles use that persistent path.
The first deployment may still download the model; required proves the persistent volume was
attached, not that its bytes were already warm. Runtime readiness remains the proof that the
exact model became serveable. Deleting a Pod does not delete this operator-owned volume.3
Connect and run read-only checks
Use the same URL and API key configured in the private environment file:
doctor checks local dependencies and credentials without creating a Pod. Stop here if a
required check fails.4
Plan before paying
Deploy and follow durable state
- Elastic
- Custom OCI
- Serverless
Delete and verify cleanup
Deletion is also a durable operation. Preview it, confirm explicitly, then follow provider cleanup:orphans check proves InferCrane has no known unmanaged resource. For a credentialed
qualification, also verify the RunPod inventory itself: zero run-owned Pods and zero run-owned
Serverless endpoints. Never erase the PostgreSQL state while provider cleanup is unresolved.
Check for leftover resources
RunPod retired REST API v1 on 2026-11-15. A control plane built before InferCrane moved to REST v2 could not create, stop, or delete RunPod Pods or Serverless endpoints from that date. Each deletion it tried failed withprovider_delete_failed and, once its retries ran out, ended as
attempts_exhausted. The RunPod resource kept running and billing. Upgrade the control plane first,
then:
- In the RunPod console, open the Pods and Serverless pages. InferCrane names every Pod
infercrane-…and every endpointinfercrane-…-serverless. Note each one’s status and age. - Compare them with
infercrane deployments,infercrane status <name>, andinfercrane orphans. A Pod or endpoint whose deployment was deleted, or whose deletion failed, is a leftover. - Retry the deletion through InferCrane so its records stay correct, using a new idempotency key:
infercrane delete <name> --yes --wait --idempotency-key <new-key>. Delete in the RunPod console only a resource InferCrane no longer tracks.
infercrane doctor --serverless to confirm the configured
template still exists. Network volumes named infercrane-artifact-… are operator-owned caches that
deletion keeps on purpose.
See provider setup for the credential boundary, production operations
for self-hosting, serverless lifecycle for worker-zero behavior, and
compatibility and qualification for the current real-provider evidence boundary.