A model is enough to start

InferCrane turns a model reference and a serving plan into a durable deployment. The CLI submits the intent to the control plane and can disconnect safely while provider allocation, artifact transfer, runtime startup, readiness, and route publication continue.

deploy can create billable provider resources. Start with plan, complete the selected provider setup and qualification review. InferCrane does not fabricate a price when the provider has not supplied trustworthy cost evidence.

For the first real deployment, the selected adapter’s setup path must include credential mounting, production control-plane startup, read-only preflight, durable reconnection, teardown, and direct provider-inventory verification. RunPod is only one independently qualified adapter.

infercrane deploy Qwen/Qwen3-8B --name qwen-production

If you want a reviewed immutable starting point first:

infercrane models
infercrane models inspect mistral-7b-instruct
infercrane workload init ./support-model --recipe mistral-7b-instruct

Catalog entries make repository, commit, license, protocol, and configuration evidence visible. They are not performance or provider-availability claims. See Verified Models. Qwen3-8B is intentionally used as the release acceptance baseline: it is small enough for frequent single-GPU qualification and exercises the production lifecycle without making every test unusually expensive. It is not embedded in the InferCrane domain model.

Use the model your workload needs

The same model-to-endpoint path accepts other Hugging Face repository identities. These examples show portable input syntax; they do not claim that every model fits every GPU or that every adapter has been production-qualified for it.

infercrane plan openai/gpt-oss-20b

infercrane plan google/gemma-4-26B-A4B-it

infercrane plan deepseek-ai/DeepSeek-V4-Pro-0813

infercrane plan moonshotai/Kimi-K3

infercrane plan Qwen/Qwen3.8-2.4T-A95B

plan validates InferCrane’s serving intent without creating resources. Before deploy, confirm the model card license or access grant, total and active parameter memory, quantization, tensor-parallel topology, runtime version, multimodal requirements, and provider capacity. Frontier MoE models such as Kimi K3, Qwen3.8 A95B, and DeepSeek V4 require materially different topology from a compact dense model. Omitting a GPU flag is deliberate: use Inference Lab or an explicit qualified serving plan rather than presenting one accelerator as a sizing recommendation.

Example family	Why teams use it	Qualification concern
Kimi	Frontier long-context, reasoning, coding, and multimodal work	Custom license, very large MoE topology, and version-specific runtime support
DeepSeek	High-adoption reasoning, agents, and cost-sensitive API workloads	Exact Pro/Flash artifact, serving framework, and multi-node plan
Qwen	Frontier and compact variants across agents, coding, vision, and chat	Exact chat template, license, tool calling, quantization, and topology
GLM	Coding and long-horizon agent workloads	Artifact availability, security policy, and exact runtime qualification
gpt-oss	Widely adopted open-weight reasoning and agent workloads	20B/120B GPU fit, reasoning protocol, and runtime version
Google Gemma	Efficient multimodal open-weight family	Model terms, multimodal runtime support, and artifact access
Meta Llama	Broad ecosystem and open-weight deployment	Meta license/access and multi-GPU topology for larger variants
Your fine-tune	Private or domain-specific behavior	Immutable artifact commit, tokenizer compatibility, and registry access

Model popularity changes faster than InferCrane releases. The source of truth is the immutable model artifact and the evidence attached to its serving plan, not a marketing list of “supported models.”

The short path uses configured defaults. Make infrastructure choices explicit when you need them:

The second command below creates provider capacity and can incur cost. The first command is the required side-effect-free review. For a deliberately small first qualification, keep exactly one replica with --min 1 --max 1, confirm the provider’s own price/stock view, and record the returned operation ID before running deploy. A replica ceiling is not a monetary budget.

infercrane plan Qwen/Qwen3-8B \
  --name qwen-production \
  --cloud runpod \
  --gpu L40S \
  --min 1 \
  --max 1

infercrane deploy Qwen/Qwen3-8B \
  --name qwen-production \
  --cloud runpod \
  --gpu L40S \
  --min 1 \
  --max 1

plan is side-effect free. deploy returns a durable operation unless --wait is supplied. Closing the terminal stops only the local watch; use the returned operation ID to reconnect:

infercrane operation watch OPERATION_ID
infercrane status qwen-production
infercrane events qwen-production

Diagnose readiness before changing the plan

Do not submit a second deployment when readiness fails. Preserve the operation and provider identity, then collect:

infercrane status qwen-production
infercrane events qwen-production --output json
infercrane logs qwen-production --since 1h
infercrane inspect qwen-production --output json
infercrane doctor qwen-production --window 1h
infercrane operation inspect OPERATION_ID --output json

Evidence	Safe action
Waiting for provider capacity with one persisted identity	Reattach to the original operation. Do not redeploy or change its idempotency key.
Container or artifact still progressing	Inspect provider/runtime logs and fresh cache evidence; wait within the approved timeout and cost bound.
Runtime unhealthy or expected model absent	Verify immutable model commit, runtime version/arguments, memory/topology, and artifact access. Correct it through a new reviewed spec, not an in-container edit.
Candidate revision permanently unhealthy	Preserve operation and Guard evidence, then reject the exact candidate and confirm candidate-only cleanup.
Initial deployment has no healthy active revision	Cancel the original operation only when intended so its cleanup handler can run; then use `delete --plan` and explicit deletion after ownership is unambiguous.
Provider identity or inventory is missing/mismatched	Stop mutation. Resolve ownership manually; do not create, adopt, or delete by name alone.

Retry the same operation or desired state only with the original idempotency key. After a terminal failure, choose between an immutable corrected candidate and plan-first deletion. Never erase PostgreSQL to clear readiness state. Cleanup is complete only when InferCrane shows no deployment or orphan and direct provider inventory proves zero run-owned resources. An uncertain create or delete must keep the same persisted replica identity and operation; never submit a replacement by name.

Clean up the exact deployment

Deletion can terminate billable provider capacity. First render the semantic plan, verify the deployment name and every persisted provider identity, then submit one durable delete operation:

infercrane delete qwen-production --plan

infercrane delete qwen-production \
  --yes \
  --wait \
  --idempotency-key qwen-production-cleanup-01

infercrane deployments --output json
infercrane orphans --output json

Do not guess a resource ID, retry with a different idempotency key, or delete by provider display name. delete --plan is side-effect free; --yes authorizes termination. InferCrane inventory is necessary but not sufficient: use the selected provider guide to query its authoritative inventory read-only and prove that every run-owned pod, instance, endpoint, service, and attached billable resource reached zero. If provider deletion is eventually consistent, keep watching the same durable operation and inventory; do not create replacement capacity during cleanup. Unknown ownership is a stop condition: preserve durable state, retry only read-only observation, and never create, adopt, or delete until one authoritative identity or absence is proven. Capture canonical provider inventory before deployment and after deletion with the command for the selected adapter. In a shared account, require equality with the baseline rather than global zero; never delete another deployment’s resource:

RunPod
AWS EC2
GCP Compute
Kubernetes or KServe

runpod_key=$(tr -d '\r\n' < "$RUNPOD_KEY_FILE")
curl -fsS -H "Authorization: Bearer $runpod_key" \
  https://rest.runpod.io/v1/pods |
  jq -S '[.[] | select((.name // "") | startswith("infercrane-")) |
    {id,name,desiredStatus}] | sort_by(.id)'
curl -fsS -H "Authorization: Bearer $runpod_key" \
  'https://rest.runpod.io/v1/endpoints?includeWorkers=true' |
  jq -S '[.[] | select((.name // "") | startswith("infercrane-")) |
    {id,name,workersMin,workersMax}] | sort_by(.id)'
unset runpod_key

aws ec2 describe-instances --region "$INFERCRANE_AWS_REGION" \
  --filters Name=tag:infercrane:managed,Values=true \
    Name=instance-state-name,Values=pending,running,stopping,shutting-down \
  --query 'Reservations[].Instances[].{id:InstanceId,state:State.Name}' \
  --output json --no-cli-pager | jq -S 'sort_by(.id)'

Run this with the same assumed-role identity validated by doctor --aws; an ambient account identity is not equivalent.

gcloud compute instances list \
  --project "$INFERCRANE_GCP_PROJECT" \
  --filter='labels.infercrane-managed=true' \
  --format=json --quiet |
  jq -S '[.[] | {id,name,status,zone}] | sort_by(.id)'

kubectl --context "$INFERCRANE_KUBERNETES_CONTEXT" \
  --namespace "$INFERCRANE_KUBERNETES_NAMESPACE" \
  get deployment,service \
  -l app.kubernetes.io/managed-by=infercrane -o name

kubectl --context "$INFERCRANE_KUBERNETES_CONTEXT" \
  --namespace "$INFERCRANE_KUBERNETES_NAMESPACE" \
  get inferenceservices.serving.kserve.io \
  -l app.kubernetes.io/managed-by=infercrane -o name

Run the KServe query only when the CRD is installed; API absence is not an empty KServe inventory.

Any authentication, authorization, parsing, pagination, or API failure makes cleanup unproven. An empty successful result is distinct from a command that could not see inventory.

Before opening a shared endpoint

A ready GPU is not the complete production boundary. Use a non-bootstrap, tenant-scoped credential, persist endpoint admission, set the distributed tenant request ceiling, and qualify only the protocols your exact runtime/model pair claims:

If authentication, quota exhaustion, request correlation, or a required protocol cannot be proven in the target environment, keep the endpoint private and the candidate unpromoted. Local fixture success does not upgrade an unqualified runtime/provider combination.

Send the first real request

Wait until status reports the deployment ready before sending application traffic. The application uses the control-plane URL as its stable OpenAI-compatible base URL and the deployment name as model; provider worker addresses remain infrastructure details:

infercrane status qwen-production
infercrane request qwen-production \
  --message "Explain continuous batching in one sentence."

curl -i -fsS "$INFERCRANE_URL/v1/chat/completions" \
  -H "Authorization: Bearer $INFERCRANE_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "qwen-production",
    "messages": [{"role":"user","content":"Explain continuous batching in one sentence."}]
  }'

from openai import OpenAI

client = OpenAI(
    base_url="https://infercrane.example/v1",
    api_key="ISSUED_INFERCRANE_CREDENTIAL",
)
response = client.chat.completions.create(
    model="qwen-production",
    messages=[{"role": "user", "content": "Explain continuous batching."}],
)
print(response.choices[0].message.content)

Copy the X-Request-Id response header from cURL to prove which revision and target served it:

infercrane request inspect REQUEST_ID --output json

The real provider/runtime/model/GPU combination remains unqualified until its separate acceptance evidence passes; a successful single request is readiness evidence, not a production performance or reliability claim.

RunPod is the first real-infrastructure acceptance target, not the InferCrane domain model. Its elastic SkyPilot adapter remains experimental until the credentialed lifecycle and zero-leak gate is complete. AWS BYOC, Kubernetes, existing targets, and future providers implement the same provider contract with independent evidence. Check compatibility and qualification before selecting a backend.

Bring an immutable runtime when a model flag is not enough

For a qualified custom runtime, describe the OCI image, digest, startup contract, health contract, and protocol capabilities in a DeploymentSpec. InferCrane persists that configuration as an immutable revision and manages its lifecycle; it does not build images or execute arbitrary build scripts.

infercrane plan deployment.yaml
infercrane apply deployment.yaml --idempotency-key release-2026-08-12

See Custom OCI workloads for the exact schema and safety requirements.

What “build” means here

InferCrane owns	Existing tools keep owning
Serving intent and stable endpoint identity	Model implementation
Provider and runtime adapter selection	Container registry and image build
Durable create, update, rollback, scale, and delete	CUDA/runtime internals
Immutable revisions and artifact identity	Inference engine execution
Readiness, routing, telemetry, and policy evidence	Provider infrastructure primitives

This boundary keeps the product extensible: vLLM, SGLang, a custom OCI runtime, or another qualified runtime can change without changing the application-facing endpoint.

Separate development, staging, and production

Environments are first-class, tenant-scoped resources. Bind separate stable endpoints to the same logical model so application configuration and policy do not leak across stages:

infercrane environment create development
infercrane environment create staging
infercrane environment create production

infercrane logical-model create coder --description "Stable coding model"
infercrane endpoint create coder-development --model coder --environment development
infercrane endpoint create coder-staging --model coder --environment staging
infercrane endpoint create coder-production --model coder --environment production

Each endpoint has independent active and candidate serving plans. Request records preserve logical model, endpoint, and environment identity. Stage the exact serving plan from staging as a production candidate without changing production traffic:

infercrane environment promote coder-staging --to coder-production
infercrane environment promote coder-staging --to coder-production --yes

The first command is a side-effect-free plan. The second atomically clones destination-scoped bindings and stages an immutable candidate. Production Release Guard remains the authority for activation. See Environment promotion.

Make the next revision safely

Use the same declarative path for updates:

infercrane plan deployment-v2.yaml
infercrane apply deployment-v2.yaml --idempotency-key release-2026-08-13
infercrane rollout inspect qwen-production

InferCrane creates an immutable candidate and preserves the active revision until promotion policy is satisfied. Continue with the safe rollout showcase.

Start here

Deploy and connect

Operate

Evidence and decisions

Build your inference deployment

A model is enough to start

Use the model your workload needs

Diagnose readiness before changing the plan

Clean up the exact deployment

Before opening a shared endpoint

Send the first real request

Bring an immutable runtime when a model flag is not enough

What “build” means here

Separate development, staging, and production

Make the next revision safely

​A model is enough to start

​Use the model your workload needs

​Diagnose readiness before changing the plan

​Clean up the exact deployment

​Before opening a shared endpoint

​Send the first real request

​Bring an immutable runtime when a model flag is not enough

​What “build” means here

​Separate development, staging, and production

​Make the next revision safely

A model is enough to start

Use the model your workload needs

Diagnose readiness before changing the plan

Clean up the exact deployment

Before opening a shared endpoint

Send the first real request

Bring an immutable runtime when a model flag is not enough

What “build” means here

Separate development, staging, and production

Make the next revision safely