CLI reference

The public CLI uses Cobra for grouped help, typo suggestions, aliases, and shell completion. It talks to the authenticated control-plane API and never opens PostgreSQL directly. Run infercrane help for the current command tree and infercrane version for the build version.

Global conventions

Commands that return data accept --output human|json.
status --watch --output json emits one JSON document per state refresh.
Mutation commands accept --idempotency-key; generated keys are printed in human and JSON output and attached to uncertain-request errors. Reuse the same key after an uncertain result.
--wait polls persisted operation state and prints only changed progress. Closing the client does not cancel server-side work; the operation ID and exact infercrane operation watch ID resume command are printed before waiting begins.
--wait-timeout 20m bounds the local watcher without cancelling the durable operation. Resume with infercrane operation watch ID; cancel only with infercrane operation cancel ID.
JSON failures contain code, category, message, retryable, remediation, and provider HTTP status when available.
Command help is available before authentication: infercrane COMMAND --help never contacts the control plane.
infercrane --context NAME COMMAND selects a context for one invocation.
infercrane --no-color COMMAND and the standard NO_COLOR environment variable disable ANSI styling. Redirected output and JSON are never decorated.

Configure and diagnose

`init`

Validate and store a control-plane URL and an already-issued credential in a named private client context.

INFERCRANE_API_KEY='ISSUED_CREDENTIAL' \
  infercrane init --context production --url https://infercrane.example

Flag	Default	Meaning
`--url`	`http://127.0.0.1:8080`	Control-plane base URL
`--context`	`default`	Context name to create and select
`--api-key`	environment	Existing credential; prefer `INFERCRANE_API_KEY` to avoid shell history
`--output`	`human`	`human` or `json`

init verifies authentication through the read-only identity endpoint before writing configuration. Use --skip-check only when intentionally configuring an offline control plane. It does not create or enroll a principal.

Contexts and authentication

infercrane context list
infercrane context show
infercrane context use production
infercrane --context staging status qwen-prod
infercrane auth status
infercrane auth status --output json

JSON output uses stable lowercase fields (id, tenant_id, name, role, kind, and scopes) so authentication checks can be consumed directly by scripts without depending on Go field names. Legacy single-context configuration is migrated on the next init. Context listing and display never print stored credentials.

Shell completion

infercrane completion bash
infercrane completion zsh
infercrane completion fish
infercrane completion powershell

Homebrew installs generated completions automatically. Completion is read-only and may suggest deployment names from the configured control plane; an unavailable control plane produces no completion error or mutation.

`doctor`

Ask the control plane to check its own dependencies.

infercrane doctor
infercrane doctor --cloud
infercrane doctor --serverless --output json
infercrane doctor --aws
infercrane doctor --gcp
infercrane doctor --kubernetes
infercrane doctor coder-production --window 1h

--cloud adds SkyPilot credentials and RunPod advisory accelerator availability; --serverless adds RunPod Serverless credentials and template validation; --aws performs a read-only role assumption and identity probe for the configured AWS BYOC adapter; --gcp checks Application Default Credentials plus read-only Compute zone access; --kubernetes checks the explicit context, optional KServe CRD, and required namespaced permissions. Diagnostics are read-only and do not create provider resources. The capability table distinguishes supported, unsupported, and unknown. InferCrane never silently changes hardware, and never claims cache or fast-resume behavior that an adapter cannot observe.

Operations workspaces

`inbox`

infercrane inbox
infercrane inbox --limit 100 --output json

The inbox reads tenant-scoped endpoint and deployment summaries, ranks non-serving state and staged candidates deterministically, and prints the exact persisted field behind each item. It fails the whole read if either fleet API is unavailable, so a partial inventory cannot look healthy. It does not run Doctor implicitly and never reads prompt or response content.

`observe`

infercrane observe coder-production
infercrane observe qwen-prod --watch
infercrane observe coder-production --diagnose

The default snapshot is read-only. Endpoint snapshots combine logical identity, serving plans, Guard, admission, and alerts; deployment snapshots combine lifecycle, traffic, active operation, Guard, and events. --diagnose explicitly persists a fresh deterministic Doctor evaluation.

`artifact`

infercrane artifact inspect ARTIFACT_ID
infercrane artifact prefetch ARTIFACT_ID --provider aws --location cache://models
infercrane artifact observe ARTIFACT_ID --provider aws --location cache://models \
  --state present --source aws-ebs-adapter --ttl 10m

Prefetch records durable intent; only a fresh provider observation is cache-hit evidence.

`evaluation`

infercrane evaluation keygen --file quality-evidence.key
infercrane evaluation ingest DEPLOYMENT REVISION --result evaluator-result.json \
  --key quality-evidence.key --file evidence.json --attach
infercrane evaluation sign DEPLOYMENT REVISION --suite SUITE --suite-version VERSION \
  --evaluator NAME --evaluator-version VERSION --score 0.95 --passed=true \
  --samples 100 --artifact-digest sha256:DIGEST \
  --key quality-evidence.key --file evidence.json
infercrane evaluation verify evidence.json
infercrane evaluation attach DEPLOYMENT --file evidence.json
infercrane evaluation list DEPLOYMENT

ingest accepts the strict evaluator-result v1 interchange contract, rejects unknown content fields, binds it to one immutable revision, signs it, and optionally attaches it. Evidence is signature-verified and contains aggregate values only.

`mcp`

infercrane mcp

Starts a stdio MCP server with read-only deployment, endpoint, request, operation, and curated-recipe tools. No mutation tool is registered. See MCP for coding agents.

Admission and async inference

infercrane admission get coder-production
infercrane admission set coder-production --max-concurrency 32 --max-queue 64 \
  --queue-timeout-ms 5000 --max-output-tokens 8192 --priorities normal,high

infercrane async submit coder-production --file request.json \
  --idempotency-key document-42
infercrane async get JOB_ID
infercrane async cancel JOB_ID

async submit persists protocol-native request content only after explicit encrypted-storage consent, returns a durable job ID, and does not depend on the client remaining connected. See Admission and async inference for retry, retention and webhook rules.

Terminal

infercrane ui

The terminal workspace is reconnectable, supports state-valid guarded actions, and can be forced read-only with infercrane ui --read-only. It requires an interactive terminal and never uses tmux for persistence. See Terminal operations workspace.

Plan and deploy

`workload`

Create a repository-local inference project, validate it without allocating a GPU, and use the same DeploymentSpec for build, plan, and deployment:

infercrane workload init ./fraud-explainer \
  --model mistralai/Mistral-7B-Instruct-v0.3
infercrane workload validate ./fraud-explainer
infercrane workload plan ./fraud-explainer
infercrane workload deploy ./fraud-explainer --wait

workload init --recipe NAME pins a reviewed model commit. workload build --tag IMAGE performs a local Docker Buildx build for custom OCI projects; add --push to record a registry-confirmed immutable digest. workload dev binds a custom OCI workload only to loopback. See Inference projects.

`plan`

Preview semantic changes without provider or database mutation.

infercrane plan Qwen/Qwen3-8B \
  --name qwen-prod \
  --cloud runpod \
  --gpu L40S \
  --min 1 \
  --max 4

Provisioned and serverless plans include an explicit readiness-evidence boundary:

Readiness: unavailable — fresh provider capacity and artifact-cache observations are not available during this static plan; no startup time is fabricated
Artifact cache: unknown
Capacity:       unknown
Startup stages: capacity -> container -> artifact -> runtime -> readiness

This is intentionally not a time estimate. Once an operation starts, durable events replace unknown stages with provider and runtime observations where those boundaries are actually exposed.

`deploy` and `apply`

deploy creates a cloud or existing-target deployment. apply declaratively converges a deployment using the same input shape.

infercrane deploy Qwen/Qwen3-8B \
  --cloud runpod \
  --gpu L40S \
  --min 1 \
  --max 4 \
  --idempotency-key qwen-prod-initial \
  --wait

infercrane apply examples/infercrane.yaml \
  --idempotency-key qwen-prod-spec-01

Flag	Default	Applies to	Meaning
`--name`	derived from model	all	Logical deployment name
`--targets`	none	existing	Comma-separated registered targets
`--cloud`	`runpod` on primary path	elastic/serverless	Registered provider cloud; `runpod`, `aws`, and `kubernetes` have distinct qualification states
`--gpu`	`L40S` on primary path	elastic/serverless	Requested GPU
`--region`	provider-selected	provisioned	Provider region; required for AWS BYOC and ignored by the namespace-scoped Kubernetes adapter
`--compute`	`elastic`	provisioned	`elastic` or `serverless`
`--min`	`1`; serverless defaults to `0`	provisioned	Minimum replicas/workers
`--max`	`1`	provisioned	Maximum replicas/workers
`--wait`	false	mutations	Follow durable progress to a terminal state
`--wait-timeout`	unlimited	waited mutations	Stop the local wait after a Go duration such as `20m`; the operation continues
`--idempotency-key`	generated	mutations	Stable safe-retry identity
`--output`	`human`	all	`human` or `json`

A YAML path cannot be combined with deployment flags. See DeploymentSpec. SGLang and custom OCI use the YAML form because their runtime contract is part of the immutable revision. See SGLang and Custom OCI workloads. After submission, human output prints the exact commands to follow durable progress. A successful wait prints the command for the first inference request.

`integrations`

infercrane integrations
infercrane integrations --output json

Displays registered provider, runtime, and external composition capabilities, evidence state, and the exact qualified runtime/provider/compute-mode combinations. Registration alone never implies production support.

`sandbox`

Compose an externally operated sandbox with one stable inference endpoint:

infercrane sandbox connect \
  --provider e2b \
  --external-id sandbox-01JCGW \
  --external-revision template-v3 \
  --endpoint coder-production \
  --ttl 30m

infercrane sandbox list --output json
infercrane sandbox rotate SANDBOX_REFERENCE_ID --output json
infercrane sandbox revoke SANDBOX_REFERENCE_ID --yes

connect and rotate reveal a credential once. It expires within 24 hours, can invoke only the selected endpoint alias, and cannot use control-plane routes. revoke disables InferCrane access without mutating the external sandbox. See External agent sandboxes.

`training`

Verify a signed, content-free artifact handoff from an external training system:

infercrane training keygen --file training-handoff.key
infercrane training sign DEPLOYMENT REVISION_ID \
  --provider mlflow --run RUN_ID \
  --repository mlflow://registry/model/VERSION \
  --immutable-revision VERSION --digest sha256:DIGEST \
  --key training-handoff.key --file artifact.handoff.json
infercrane training verify artifact.handoff.json
infercrane training attach DEPLOYMENT artifact.handoff.json
infercrane training list DEPLOYMENT --output json

The private signing key remains local. Attachment binds immutable identity to one revision but does not promote it or claim that InferCrane executed training. See Training artifact handoffs.

Send a request

Use the logical deployment name without assembling an HTTP request by hand:

infercrane request qwen-prod --message "Explain continuous batching in one sentence."
infercrane request qwen-prod --message "Count to five." --stream
infercrane request search-production --protocol embeddings --message "Document to embed"
infercrane request coder-production --protocol responses --message "Review this function"

--output json returns the OpenAI-compatible choices and usage object for non-streaming requests. The command uses the configured endpoint and credential; applications can continue to use any OpenAI-compatible SDK directly. --protocol accepts chat, responses, embeddings, completions, or batch; the endpoint must explicitly qualify the selected surface. Inspect the persisted routing and timing evidence for a returned X-Request-Id without retrieving prompt or output content:

infercrane request inspect req_123
infercrane request inspect req_123 --output json

Stable endpoints

Separate the name applications call from the deployment that currently serves it:

infercrane logical-model create coder --description "Stable coding model"
infercrane endpoint create coder-production --model coder --environment production
infercrane endpoint bind coder-production --name primary --deployment qwen-prod
infercrane endpoint plan coder-production --policy manual --bindings primary
infercrane request coder-production --message "Review this function."

endpoint plan makes the first plan active and stages later plans as candidates. Inspect and promote explicitly:

infercrane endpoints
infercrane endpoint inspect coder-production
infercrane endpoint guard coder-production --evaluate
infercrane endpoint promote coder-production PLAN_ID

--bindings is ordered and accepts optional bounded weights, for example primary:80,overflow:20. See stable endpoints and serving plans. Authenticated external APIs add provider-neutral policy flags to endpoint bind:

infercrane endpoint bind coder-production \
  --name managed-fallback \
  --target managed-coder \
  --ownership traffic-managed \
  --external-adapter openai-compatible-external \
  --secret-reference SECRET_REFERENCE_ID \
  --request-limit 1000 \
  --cost-limit-usd 25.00 \
  --max-request-cost-usd 0.10 \
  --acknowledge-external-data \
  --enable-external

The control plane rejects raw credentials, missing consent, absent hard limits, cross-tenant secret references, and adapter/target mismatches before creating the binding. Stage one environment’s active immutable plan as another endpoint’s candidate:

infercrane environment promote coder-staging --to coder-production
infercrane environment promote coder-staging --to coder-production --yes

The first command is a non-mutating preview. Staging never switches production traffic; destination Release Guard and explicit endpoint promotion remain required.

Connect an existing workload

The simple path verifies the upstream model list through the control-plane API and starts in observe-only mode:

infercrane connect https://vllm.internal/v1 --as coder-production --type vllm

Use --model when the endpoint exposes more than one physical model. Use --manage-traffic only when the endpoint has been qualified and should immediately enter the InferCrane logical route. The advanced ownership-compatible command remains available:

infercrane adopt endpoint coder-production \
  --url https://vllm.internal/v1 \
  --model coder \
  --upstream-model Qwen/Qwen3-8B \
  --ownership observe-only
infercrane adopt promote coder-production --ownership traffic-managed

Use traffic-managed only when InferCrane should publish the healthy existing target. Neither mode transfers provider lifecycle ownership. See Adopt and diagnose.

Existing targets and routing

infercrane target add gpu-a \
  --url http://gpu-a:8000 \
  --runtime vllm \
  --upstream-model Qwen/Qwen3-8B

infercrane target list --output json
infercrane route qwen-prod --strategy cache-aware

Supported strategies are round-robin, consistent-hash, power-of-two, and cache-aware, delegated to the pinned vLLM Router.

Observe and explain

Command	Purpose
`inbox [--limit N]`	Rank persisted fleet state requiring operator attention without mutation
`observe ENDPOINT_OR_DEPLOYMENT`	Join one resource’s lifecycle, traffic, Guard, policy, operation, and event evidence
`deployments [--output json]`	List logical deployments
`status DEPLOYMENT [--watch]`	Show serving health separately from convergence, readiness, traffic, revisions, and the blocking operation
`events DEPLOYMENT`	Show ordered durable deployment events
`logs DEPLOYMENT [--follow]`	Stream a filtered durable operational timeline
`request ENDPOINT`	Send a capability-qualified OpenAI-compatible inference request
`inspect DEPLOYMENT`	Show raw targets, replicas, artifacts, request statistics, and infrastructure metadata
`operation ID`	Inspect durable progress and result once
`operation watch ID`	Resume following a durable operation after disconnect or from another terminal
`operation cancel ID`	Request cooperative cancellation
`orphans`	List unmanaged provisioned resources visible to the control plane
`integrations`	Inspect provider, runtime, and external composition capabilities and qualification state
`sandbox connect/list/rotate/revoke`	Manage only endpoint-scoped access for externally owned sandboxes
`training keygen/sign/verify/attach/list`	Verify and bind signed external training artifact lineage

infercrane status qwen-prod --watch
infercrane logs qwen-prod --follow
infercrane logs qwen-prod --type runtime --since 15m
infercrane events qwen-prod --output json
infercrane inspect qwen-prod
infercrane operation watch OPERATION_ID
infercrane integrations
infercrane integrations --output json

Long operations belong to the control plane, not the terminal. deploy, apply, rollout, scale, and deletion continue after the CLI disconnects. Pressing Ctrl-C stops only the local watcher. Run infercrane operation watch ID later to resume from persisted progress; add --wait-timeout 20m to bound only the new local watch. Explicit infercrane operation cancel ID is the separate action that requests cooperative server-side cancellation and provider cleanup. status deliberately separates two concerns. Serving answers whether the current route can accept inference traffic. Convergence answers whether desired capacity, a rollout, or deletion is still progressing. A deployment can therefore be serving · converging while a provider allocates another replica; this is not reported as an outage. JSON output exposes the same stable fields under lifecycle_status, including ready and desired replica counts, provisioning and draining counts, candidate state, and the blocking durable operation. Deterministic explanations use only persisted state and measurements:

infercrane explain qwen-prod
infercrane explain scaling qwen-prod
infercrane explain rollout qwen-prod
infercrane explain cold-start qwen-prod

Benchmark

Run AIPerf and persist the exact reproduction metadata.

infercrane benchmark qwen-prod \
  --requests 1000 \
  --concurrency 32 \
  --random-seed 17 \
  --revision active

--revision accepts active, candidate, or an explicit revision ID. Defaults are 100 requests, concurrency 10, and seed 17. Capture and search immutable recipes only after a matching benchmark exists:

infercrane recipe create qwen-prod --name qwen3-balanced --version 1.0.0
infercrane recipes qwen3 --limit 20

Compare persisted measured configurations without provisioning new capacity:

infercrane lab 'Qwen/Qwen3-8B@IMMUTABLE_COMMIT' --max-ttft-p95-ms 250

See Recipes and Inference Lab for the provenance and missing-evidence contract.

Replay and capacity

infercrane replay coder-production --window 24h
infercrane capacity --window 720h

Replay captures content-free production shape by default. Running an AIPerf approximation requires both --execute and --acknowledge-cost. See Replay and capacity intelligence.

Revisions and Release Guard

infercrane rollout inspect qwen-prod
infercrane rollout policy get qwen-prod
infercrane rollout policy set qwen-prod \
  --require-compatibility \
  --require-synthetic \
  --auto-rollback \
  --validation-max-requests 100 \
  --validation-max-concurrency 4

infercrane rollout create qwen-prod \
  --model Qwen/Qwen3-8B \
  --cloud runpod \
  --gpu H100 \
  --min 1 \
  --max 2 \
  --idempotency-key qwen-prod-rev-19

infercrane rollout provision qwen-prod REVISION_ID --wait
infercrane rollout validate qwen-prod --acknowledge-validation-cost --wait
infercrane rollout promote qwen-prod REVISION_ID --reason 'guard accepted'
infercrane rollout reject qwen-prod REVISION_ID --reason 'TTFT regression'
infercrane rollout rollback qwen-prod REVISION_ID --wait

Candidate creation also accepts --model-revision, --runtime, --runtime-version, --runtime-args, --routing, and --region. Promotion remains policy-gated; no LLM decides the outcome. rollout validate is explicit synthetic traffic, not shadowing. It runs the existing AIPerf adapter against active and candidate revisions using the same workload and persisted hard bounds. The acknowledgement is mandatory because both runs may incur provider cost.

Signed release evidence

infercrane passport keygen
infercrane passport issue qwen-prod --file qwen-prod.passport.json
infercrane passport verify qwen-prod.passport.json
infercrane passport list qwen-prod --output json

passport keygen and passport verify work without a configured control-plane context. Issue and list use the authenticated API. See Inference Passports.

Delete

Preview cleanup, then confirm it explicitly.

infercrane delete qwen-prod --plan
infercrane delete qwen-prod \
  --yes \
  --wait \
  --idempotency-key qwen-prod-delete-01
infercrane orphans

--plan is side-effect-free. --yes is required for mutation. After a paid test, also verify the provider inventory reaches zero.

Tenant administration

These commands require an admin/bootstrap credential.

infercrane tenant create team-a --name 'Team A'
infercrane principal create deploy-bot --role operator --scopes read,deploy
infercrane principal rotate PRINCIPAL_ID
infercrane principal revoke PRINCIPAL_ID

New and rotated tokens are returned once. Service-account scopes can only restrict the role ceiling; they can never grant an action unavailable to viewer, operator, or admin. Omitting --scopes uses every action allowed by the selected role for compatibility.

Secret references

Register metadata that points to an injected environment value. InferCrane never accepts the raw secret on the command line or through its API.

infercrane secret create openrouter --from-env OPENROUTER_API_KEY
infercrane secret list
infercrane secret list --output json
infercrane secret delete SECRET_ID --yes

Signed alerts

infercrane alert list coder-production
infercrane alert configure coder-production --webhook https://alerts.example.com/hook \
  --secret-reference SECRET_ID --minimum-severity warning
infercrane alert evaluate coder-production

Alerts contain deterministic Doctor findings and HMAC headers; delivery is idempotent and bounded.

Governed external fallback

External targets require manage_external. They are selected only when ordinary targets are unhealthy and require an explicit privacy acknowledgement plus hard budgets.

infercrane target add openrouter-qwen \
  --provider openrouter \
  --url https://openrouter.ai/api/v1 \
  --upstream-model qwen/qwen3-8b

infercrane external configure qwen-prod \
  --target openrouter-qwen \
  --secret-reference SECRET_REFERENCE_ID \
  --request-limit 100 \
  --cost-limit-usd 10 \
  --max-request-cost-usd 0.10 \
  --acknowledge-external-data \
  --enable

infercrane external inspect qwen-prod --output json

See Governed external capacity for transmission and reservation semantics.

SLO policy and recommendations

Define explicit fail-closed thresholds, then evaluate persisted benchmark evidence. Recommendations are advisory and never mutate the deployment.

infercrane slo set qwen-prod --ttft-p95 250 --error-rate 0.01
infercrane slo get qwen-prod --output json
infercrane recommend qwen-prod
infercrane recommend qwen-prod --history --output json
infercrane slo delete qwen-prod

recommended identifies a qualified candidate satisfying the policy. no_match means measured candidates violate it. unknown means required evidence is missing. See Inference decisions.

​CLI reference

​Global conventions

​Configure and diagnose

​init

​Contexts and authentication

​Shell completion

​doctor

​Operations workspaces

​inbox

​observe

​artifact

​evaluation

​mcp

​Admission and async inference

​Terminal

​Plan and deploy

​workload

​plan

​deploy and apply

​integrations

​sandbox

​training

​Send a request

​Stable endpoints

​Connect an existing workload

​Existing targets and routing

​Observe and explain

​Benchmark

​Replay and capacity

​Revisions and Release Guard

​Signed release evidence

​Delete

​Tenant administration

​Secret references

​Signed alerts

​Governed external fallback

​SLO policy and recommendations