CLI reference

The public CLI uses Cobra for grouped help, typo suggestions, aliases, and shell completion. It talks to the authenticated control-plane API and never opens PostgreSQL directly. Run infercrane help for the current command tree and infercrane version for the build version.

Global conventions

Commands that return data accept --output human|json.
status --watch --output json emits one JSON document per state refresh.
Mutation commands accept --idempotency-key; generated keys are printed in human and JSON output and attached to uncertain-request errors. Reuse the same key after an uncertain result.
--wait polls persisted operation state and prints only changed progress. Closing the client does not cancel server-side work; the operation ID and exact infercrane operation watch ID resume command are printed before waiting begins.
--wait-timeout 20m bounds the local watcher without cancelling the durable operation. Resume with infercrane operation watch ID; cancel only with infercrane operation cancel ID.
JSON failures contain code, category, message, retryable, remediation, and provider HTTP status when available.
Command help is available before authentication: infercrane COMMAND --help never contacts the control plane.
infercrane --context NAME COMMAND selects a context for one invocation.
infercrane --no-color COMMAND and the standard NO_COLOR environment variable disable ANSI styling. Redirected output and JSON are never decorated.

Configure and diagnose

`init`

Validate and store a control-plane URL and an already-issued credential in a named private client context.

INFERCRANE_API_KEY='ISSUED_CREDENTIAL' \
  infercrane init --context production --url https://infercrane.example

Flag	Default	Meaning
`--url`	`http://127.0.0.1:8080`	Control-plane base URL
`--context`	`default`	Context name to create and select
`--api-key`	environment	Existing credential; prefer `INFERCRANE_API_KEY` to avoid shell history
`--output`	`human`	`human` or `json`

init verifies authentication through the read-only identity endpoint before writing configuration. Use --skip-check only when intentionally configuring an offline control plane. It does not create or enroll a principal.

Contexts and authentication

infercrane context list
infercrane context show
infercrane context use production
infercrane --context staging status qwen-prod
infercrane auth status
infercrane auth status --output json

JSON output uses stable lowercase fields (id, tenant_id, name, role, kind, and scopes) so authentication checks can be consumed directly by scripts without depending on Go field names. Legacy single-context configuration is migrated on the next init. Context listing and display never print stored credentials.

Shell completion

infercrane completion bash
infercrane completion zsh
infercrane completion fish
infercrane completion powershell

Homebrew installs generated completions automatically. Completion is read-only and may suggest deployment names from the configured control plane; an unavailable control plane produces no completion error or mutation.

`doctor`

Ask the control plane to check its own dependencies.

infercrane doctor
infercrane doctor --cloud
infercrane doctor --serverless --output json
infercrane doctor --aws
infercrane doctor --gcp
infercrane doctor --kubernetes
infercrane doctor coder-production --window 1h

--cloud adds SkyPilot credentials and RunPod advisory accelerator availability; --serverless adds RunPod Serverless credentials and template validation; --aws performs a read-only role assumption and dependency probe for the configured AWS BYOC adapter; --gcp checks Application Default Credentials, every configured Compute dependency, immutable images, and Private Google Access; --kubernetes checks the explicit context, optional KServe CRD, and required namespaced permissions. Diagnostics are read-only and do not create provider resources. The capability table distinguishes supported, unsupported, and unknown. InferCrane never silently changes hardware, and never claims cache or fast-resume behavior that an adapter cannot observe.

Local accelerator discovery

Inspect NVIDIA GPU inventory without installing a runtime or changing the host:

infercrane discover local
infercrane discover local --output json

Discovery reports its evidence source and limitations. It does not infer runtime compatibility, model fit, or performance from a device name alone.

Run the control plane

infercrane serve

serve starts the control plane and inference gateway from validated environment configuration. Production deployments should use the container and service guidance in Production operations, including PostgreSQL, TLS, bootstrap authentication, stable instance identity, and graceful shutdown.

Operations workspaces

`inbox`

infercrane inbox
infercrane inbox --limit 100 --output json

The inbox reads tenant-scoped endpoint and deployment summaries, ranks non-serving state and staged candidates deterministically, and prints the exact persisted field behind each item. It fails the whole read if either fleet API is unavailable, so a partial inventory cannot look healthy. It does not run Doctor implicitly and never reads prompt or response content.

`observe`

infercrane observe coder-production
infercrane observe qwen-prod --watch
infercrane observe coder-production --diagnose

The default snapshot is read-only. Endpoint snapshots combine logical identity, serving plans, Guard, admission, and alerts; deployment snapshots combine lifecycle, traffic, active operation, Guard, and events. --diagnose explicitly persists a fresh deterministic Doctor evaluation.

`telemetry collect dcgm`

infercrane telemetry collect dcgm coder-production \
  --url http://127.0.0.1:9400/metrics \
  --selector deployment=coder-production \
  --replica replica-03 --ttl 2m

The command parses a bounded, documented subset of DCGM Prometheus metrics locally and sends normalized content-free evidence through the authenticated control API. Use --file SNAPSHOT for offline input. The snapshot is immutable, active-revision-bound, and becomes stale at its declared TTL; arbitrary Prometheus metrics are not accepted.

`finops collect opencost`

infercrane finops collect opencost coder-production \
  --url http://127.0.0.1:9003/allocation \
  --window 24h --aggregate controller \
  --allocation coder-production --currency USD

The command selects exact OpenCost allocation keys, records a currency-explicit average hourly rate against the active immutable revision, and creates a persisted FinOps report. Multiple exact keys may be comma-separated. --allocation and --currency are mandatory; cluster-wide attribution and currency conversion are never inferred. Use --file allocation.json for offline evidence.

`artifact`

infercrane artifact inspect ARTIFACT_ID
infercrane artifact prefetch ARTIFACT_ID --provider aws --location cache://models
infercrane artifact observe ARTIFACT_ID --provider aws --location cache://models \
  --state present --source aws-ebs-adapter --ttl 10m

Prefetch records durable intent; only a fresh provider observation is cache-hit evidence.

`evaluation`

infercrane evaluation keygen --file quality-evidence.key
infercrane evaluation ingest DEPLOYMENT REVISION --result evaluator-result.json \
  --key quality-evidence.key --file evidence.json --attach
infercrane evaluation sign DEPLOYMENT REVISION --suite SUITE --suite-version VERSION \
  --evaluator NAME --evaluator-version VERSION --score 0.95 --passed=true \
  --samples 100 --artifact-digest sha256:DIGEST \
  --key quality-evidence.key --file evidence.json
infercrane evaluation verify evidence.json
infercrane evaluation attach DEPLOYMENT --file evidence.json
infercrane evaluation list DEPLOYMENT

ingest accepts the strict evaluator-result v1 interchange contract, rejects unknown content fields, binds it to one immutable revision, signs it, and optionally attaches it. Evidence is signature-verified and contains aggregate values only.

`mcp`

infercrane mcp

Starts a stdio MCP server with read-only deployment, endpoint, request, operation, and curated-recipe tools. No mutation tool is registered. See MCP for coding agents.

Admission and async inference

infercrane admission get coder-production
infercrane admission set coder-production --max-concurrency 32 --max-queue 64 \
  --queue-timeout-ms 5000 --max-output-tokens 8192 --priorities normal,high

infercrane async submit coder-production --file request.json \
  --idempotency-key document-42
infercrane async get JOB_ID
infercrane async cancel JOB_ID

async submit persists protocol-native request content only after explicit encrypted-storage consent, returns a durable job ID, and does not depend on the client remaining connected. See Admission and async inference for retry, retention and webhook rules.

Terminal

infercrane ui

The terminal workspace is reconnectable, supports state-valid guarded actions, and can be forced read-only with infercrane ui --read-only. It requires an interactive terminal and never uses tmux for persistence. See Terminal operations workspace.

Plan and deploy

`optimize propose`

Generate inspectable candidate DeploymentSpecs without contacting a control plane or provider. Check the optional modeled-candidate source first:

infercrane optimize doctor

The default --source auto uses the pinned AIConfigurator adapter only when its exact supported Python tuple is available and visibly falls back to the reviewed catalog otherwise. Use --source catalog for a deterministic offline proposal, or --source aiconfigurator to require the estimator and fail rather than falling back.

infercrane optimize propose mistral-7b-instruct \
  --provider aws --region eu-central-1 --gpu L40S \
  --objective interactive --profile interactive \
  --source auto --target-concurrency 8 \
  --max-ttft-p95-ms 200 --max-tpot-p95-ms 25 \
  --write-dir .infercrane/candidates/mistral-interactive

Catalog output is unmeasured; AIConfigurator output is modeled. Neither is qualified. Use --output json to retain the input digest, source version, modeled-result digest, compatibility evidence, missing evidence, and limitations. --workload-fingerprint can bind a content-free observed workload identity to the proposal. Simulated compatibility is excluded unless --include-simulated is explicit. The command refuses to overwrite a non-empty output directory. The reviewed catalog spans instruction, reasoning, coding, embeddings, multimodal, and enterprise-text workloads. A profile for one model is never evidence for another model, precision, GPU, runtime, or topology. See Evidence-gated optimization.

`optimize create`

Persist the proposal and candidate identities without provider mutation, then grant bounded paid authority explicitly:

infercrane optimize create llama-3.1-8b-instruct \
  --provider aws --region eu-central-1 --gpu L40S \
  --objective interactive --max-candidates 3

# Safely evolve an existing production deployment instead.
infercrane optimize create llama-3.1-8b-instruct \
  --provider aws --region eu-central-1 --gpu L40S \
  --objective interactive --max-candidates 3 \
  --target-deployment llama-production
infercrane optimize inspect CAMPAIGN_ID
infercrane optimize approve CAMPAIGN_ID --max-cost-usd 20 --expires-in 1h
infercrane optimize results CAMPAIGN_ID
infercrane optimize activate CAMPAIGN_ID --candidate CANDIDATE_ID
infercrane operation watch ACTIVATION_OPERATION_ID
infercrane optimize cancel CAMPAIGN_ID

Approval retries do not extend the first persisted expiry. Creation and inspection allocate no infrastructure. The default new_endpoint intent stops at human activation after qualification. --target-deployment selects evolve_endpoint, which must stop at Release Guard and human promotion. optimize activate is the explicit boundary in both cases: it marks the qualified new deployment active, or delegates an evolution to the existing guarded rollout operation. Retrying the command adopts the same durable operation; InferCrane never promotes from the optimizer loop.

`artifact optimize`

Plan and attest an externally built optimized checkpoint, speculator, or TensorRT engine:

infercrane artifact optimize plan BASE_ARTIFACT_ID \
  --preset eagle3 --tool-version 0.6.0 \
  --builder-image-digest sha256:BUILDER_DIGEST \
  --license-spdx Apache-2.0
infercrane artifact optimize start OPTIMIZED_ARTIFACT_ID
infercrane artifact optimize attest OPTIMIZED_ARTIFACT_ID \
  --state ready --output-repository hf://ORG/MODEL \
  --output-revision IMMUTABLE_COMMIT --output-digest sha256:OUTPUT_DIGEST \
  --evidence-file build-evidence.json
infercrane artifact optimize qualify OPTIMIZED_ARTIFACT_ID \
  --candidate-run CANDIDATE_RUN_ID --quality-evidence QUALITY_EVIDENCE_ID

Presets are fp8, awq, gptq, nvfp4, eagle3, mtp, dflash, and tensorrt. Builders are never executed inside the InferCrane API process, and no optimized artifact qualifies without exact candidate, revision, digest, and semantic-quality evidence.

`workload`

Create a repository-local inference project, validate it without allocating a GPU, and use the same DeploymentSpec for build, plan, and deployment:

infercrane workload init ./fraud-explainer \
  --recipe mistral-7b-instruct \
  --profile vllm-interactive
infercrane workload validate ./fraud-explainer
infercrane workload plan ./fraud-explainer
infercrane workload deploy ./fraud-explainer --wait

workload init --recipe NAME --profile PROFILE pins a reviewed model commit and writes the selected candidate’s runtime version, immutable workload when present, GPU type and count, provider hint, replica bounds, and routing policy. The default profile is the recipe’s general starting point; profiles are not benchmark claims. workload build --tag IMAGE performs a local Docker Buildx build for custom OCI projects; add --push to record a registry-confirmed immutable digest. workload dev binds a custom OCI workload only to loopback. See Inference projects.

`plan`

Preview semantic changes without provider or database mutation.

infercrane plan Qwen/Qwen3-8B \
  --name qwen-prod \
  --cloud runpod \
  --gpu L40S \
  --min 1 \
  --max 4

Provisioned and serverless plans include an explicit readiness-evidence boundary:

Readiness: unavailable: fresh provider capacity and artifact-cache observations are not available during this static plan; no startup time is fabricated
Artifact cache: unknown
Capacity:       unknown
Startup stages: capacity -> container -> artifact -> runtime -> readiness

This is intentionally not a time estimate. If the tenant has at least three successful historical observations for the exact provider adapter, runtime, region, and GPU, plan instead shows an observed p50 from durable replica intent through runtime readiness. It shows p95 only after twenty successes and labels the sample count and boundary. That is historical evidence, not a capacity guarantee. For a DeploymentSpec pinned to an immutable model revision, plan can additionally show per-stage observed p50 after three successes and p95 after twenty only when model commit, runtime version and arguments, provider, region, GPU type/count, and compute mode all match. A stage with no provider marker or insufficient samples remains absent/unknown. Once an operation starts, durable events replace unknown stages with fresh provider and runtime observations where those boundaries are actually exposed.

`deploy` and `apply`

deploy creates a cloud or existing-target deployment. apply declaratively converges a deployment using the same input shape.

infercrane deploy Qwen/Qwen3-8B \
  --cloud runpod \
  --gpu L40S \
  --min 1 \
  --max 4 \
  --idempotency-key qwen-prod-initial \
  --wait

infercrane apply examples/infercrane.yaml \
  --idempotency-key qwen-prod-spec-01

Flag	Default	Applies to	Meaning
`--name`	derived from model	all	Logical deployment name
`--targets`	none	existing	Comma-separated registered targets
`--cloud`	`runpod` on primary path	elastic/serverless	Registered provider cloud; `runpod`, `aws`, and `kubernetes` have distinct qualification states
`--backend`	standard provider path	elastic	`dynamo` expands to the safe aggregated Dynamo baseline; advanced topology uses YAML
`--provider-adapter`	selected from cloud/runtime	provisioned	Exact adapter identity; advanced escape hatch, normally inferred by `--backend` or YAML
`--runtime`	`vllm`	provisioned	`vllm` or `sglang` where the selected adapter has compatible evidence
`--gpu`	`L40S` on primary path	elastic/serverless	Requested GPU
`--region`	provider-selected	provisioned	Provider region; required for AWS BYOC and ignored by the namespace-scoped Kubernetes adapter
`--compute`	`elastic`	provisioned	`elastic` or `serverless`
`--min`	`1`; serverless defaults to `0`	provisioned	Minimum replicas/workers
`--max`	`1`	provisioned	Maximum replicas/workers
`--wait`	false	mutations	Follow durable progress to a terminal state
`--wait-timeout`	unlimited	waited mutations	Stop the local wait after a Go duration such as `20m`; the operation continues
`--idempotency-key`	generated	mutations	Stable safe-retry identity
`--output`	`human`	all	`human` or `json`

A YAML path cannot be combined with deployment flags. See DeploymentSpec. Custom OCI uses the YAML form because its runtime contract is part of the immutable revision. Advanced Dynamo topology also uses YAML. See SGLang, NVIDIA Dynamo, and Custom OCI workloads. After submission, human output prints the exact commands to follow durable progress. A successful wait prints the command for the first inference request.

`integrations`

infercrane integrations
infercrane integrations --output json

Displays registered provider, runtime, and external composition capabilities, evidence state, and the exact qualified runtime/provider/compute-mode combinations. Registration alone never implies production support.

`sandbox`

Compose an externally operated sandbox with one stable inference endpoint:

infercrane sandbox connect \
  --provider e2b \
  --external-id sandbox-01JCGW \
  --external-revision template-v3 \
  --endpoint coder-production \
  --ttl 30m

infercrane sandbox list --output json
infercrane sandbox rotate SANDBOX_REFERENCE_ID --output json
infercrane sandbox revoke SANDBOX_REFERENCE_ID --yes

connect and rotate reveal a credential once. It expires within 24 hours, can invoke only the selected endpoint alias, and cannot use control-plane routes. revoke disables InferCrane access without mutating the external sandbox. See External agent sandboxes.

`training`

Verify a signed, content-free artifact handoff from an external training system:

infercrane training keygen --file training-handoff.key
infercrane training sign DEPLOYMENT REVISION_ID \
  --provider mlflow --run RUN_ID \
  --repository mlflow://registry/model/VERSION \
  --immutable-revision VERSION --digest sha256:DIGEST \
  --key training-handoff.key --file artifact.handoff.json
infercrane training verify artifact.handoff.json
infercrane training attach DEPLOYMENT artifact.handoff.json
infercrane training list DEPLOYMENT --output json

The private signing key remains local. Attachment binds immutable identity to one revision but does not promote it or claim that InferCrane executed training. See Training artifact handoffs.

Send a request

Use the logical deployment name without assembling an HTTP request by hand:

infercrane request qwen-prod --message "Explain continuous batching in one sentence."
infercrane request qwen-prod --message "Count to five." --stream
infercrane request search-production --protocol embeddings --message "Document to embed"
infercrane request coder-production --protocol responses --message "Review this function"

--output json returns the OpenAI-compatible choices and usage object for non-streaming requests. The command uses the configured endpoint and credential; applications can continue to use any OpenAI-compatible SDK directly. --protocol accepts chat, responses, embeddings, completions, or batch; the endpoint must explicitly qualify the selected surface. Inspect the persisted routing and timing evidence for a returned X-Request-Id without retrieving prompt or output content:

infercrane request inspect req_123
infercrane request inspect req_123 --output json

Stable endpoints

Separate the name applications call from the deployment that currently serves it:

infercrane logical-model create coder --description "Stable coding model"
infercrane endpoint create coder-production --model coder --environment production
infercrane endpoint bind coder-production --name primary --deployment qwen-prod
infercrane endpoint plan coder-production --policy manual --bindings primary
infercrane request coder-production --message "Review this function."

endpoint plan makes the first plan active and stages later plans as candidates. Inspect and promote explicitly:

infercrane endpoints
infercrane endpoint inspect coder-production
infercrane endpoint guard coder-production --evaluate
infercrane endpoint promote coder-production PLAN_ID

--bindings is ordered and accepts optional bounded weights, for example primary:80,overflow:20. See stable endpoints and serving plans. Authenticated external APIs add provider-neutral policy flags to endpoint bind:

infercrane provider connect managed-coder \
  --adapter openai-compatible-external \
  --url https://models.example.com/v1 \
  --model company/coder \
  --from-env MANAGED_MODEL_API_KEY

infercrane endpoint bind coder-production \
  --name managed-fallback \
  --connection managed-coder \
  --request-limit 1000 \
  --cost-limit-usd 25.00 \
  --max-request-cost-usd 0.10 \
  --acknowledge-external-data \
  --enable-external

The control plane rejects raw credentials, missing consent, absent hard limits, cross-tenant secret references, and adapter/target mismatches before creating the binding. infercrane provider list shows reusable connection metadata without resolving credentials. infercrane provider delete NAME removes only the reusable configuration; immutable endpoint bindings and their release history are deliberately unchanged. --from-env creates or reuses a secret reference by variable name, making a disconnected retry idempotent without accepting the raw key. Use --secret-reference ID instead when secret references are provisioned separately. The two flags are mutually exclusive. Stage one environment’s active immutable plan as another endpoint’s candidate:

infercrane environment promote coder-staging --to coder-production
infercrane environment promote coder-staging --to coder-production --yes

The first command is a non-mutating preview. Staging never switches production traffic; destination Release Guard and explicit endpoint promotion remain required.

Connect an existing workload

The simple path verifies the upstream model list through the control-plane API and starts in observe-only mode:

infercrane connect https://vllm.internal/v1 --as coder-production --type vllm

Use --model when the endpoint exposes more than one physical model. Use --manage-traffic only when the endpoint has been qualified and should immediately enter the InferCrane logical route. The advanced ownership-compatible command remains available:

infercrane adopt endpoint coder-production \
  --url https://vllm.internal/v1 \
  --model coder \
  --upstream-model Qwen/Qwen3-8B \
  --ownership observe-only
infercrane adopt promote coder-production --ownership traffic-managed

Use traffic-managed only when InferCrane should publish the healthy existing target. Neither mode transfers provider lifecycle ownership. See Adopt and diagnose.

Existing targets and routing

infercrane target add gpu-a \
  --url http://gpu-a:8000 \
  --runtime vllm \
  --upstream-model Qwen/Qwen3-8B

infercrane target list --output json
infercrane route qwen-prod --strategy cache-aware

Supported strategies are round-robin, consistent-hash, power-of-two, and cache-aware, delegated to the pinned vLLM Router.

Observe and explain

Command	Purpose
`inbox [--limit N]`	Rank persisted fleet state requiring operator attention without mutation
`observe ENDPOINT_OR_DEPLOYMENT`	Join one resource’s lifecycle, traffic, Guard, policy, operation, and event evidence
`deployments [--output json]`	List logical deployments
`system instances [--output json]`	Inspect live control-plane membership and mixed-version protocol evidence
`status DEPLOYMENT [--watch]`	Show serving health separately from convergence, readiness, traffic, revisions, and the blocking operation
`events DEPLOYMENT`	Show ordered durable deployment events
`logs DEPLOYMENT [--follow]`	Stream a filtered durable operational timeline
`request ENDPOINT`	Send a capability-qualified OpenAI-compatible inference request
`inspect DEPLOYMENT`	Show raw targets, replicas, artifacts, request statistics, and infrastructure metadata
`operation ID`	Inspect durable progress and result once
`operation watch ID`	Resume following a durable operation after disconnect or from another terminal
`operation cancel ID`	Request cooperative cancellation
`orphans`	List unmanaged provisioned resources visible to the control plane
`integrations`	Inspect provider, runtime, and external composition capabilities and qualification state
`sandbox connect/list/rotate/revoke`	Manage only endpoint-scoped access for externally owned sandboxes
`training keygen/sign/verify/attach/list`	Verify and bind signed external training artifact lineage

infercrane status qwen-prod --watch
infercrane logs qwen-prod --follow
infercrane logs qwen-prod --type runtime --since 15m
infercrane events qwen-prod --output json
infercrane inspect qwen-prod
infercrane operation watch OPERATION_ID
infercrane integrations
infercrane integrations --output json

Long operations belong to the control plane, not the terminal. deploy, apply, rollout, scale, and deletion continue after the CLI disconnects. Pressing Ctrl-C stops only the local watcher. Run infercrane operation watch ID later to resume from persisted progress; add --wait-timeout 20m to bound only the new local watch. Explicit infercrane operation cancel ID is the separate action that requests cooperative server-side cancellation and provider cleanup. status deliberately separates two concerns. Serving answers whether the current route can accept inference traffic. Convergence answers whether desired capacity, a rollout, or deletion is still progressing. A deployment can therefore be serving · converging while a provider allocates another replica; this is not reported as an outage. JSON output exposes the same stable fields under lifecycle_status, including ready and desired replica counts, provisioning and draining counts, candidate state, and the blocking durable operation. Deterministic explanations use only persisted state and measurements:

infercrane explain qwen-prod
infercrane explain scaling qwen-prod
infercrane explain rollout qwen-prod
infercrane explain cold-start qwen-prod

Models and recipes

Explore the built-in reviewed catalog without contacting a control plane:

infercrane models
infercrane models embeddings
infercrane models inspect qwen3-8b
infercrane models --output json

models inspect prints the immutable revision, license URL, protocol capabilities, serving profiles, accelerator topology, immutable runtime workload when present, and the exact qualification boundary. Create a project with infercrane workload init --recipe NAME. Catalog profiles are configuration-verified, not benchmark, price, capacity, or universal GPU-fit claims. See Verified Models.

Benchmark

Run AIPerf and persist the exact reproduction metadata.

infercrane benchmark mistral-prod \
  --profile interactive \
  --ttft-slo-ms 250 \
  --tpot-slo-ms 25 \
  --revision active

--profile accepts balanced, buffered, interactive, long-context, long-generation, overload, or throughput. Profiles are immutable workload shapes; explicit request, concurrency, token, and streaming flags override their defaults. --revision accepts active, candidate, or an explicit revision ID. Without a profile, defaults remain 100 requests, concurrency 10, and seed 17. Declared TTFT, TPOT, or total-latency SLOs produce measured goodput. Capture and search immutable recipes only after a matching benchmark exists:

infercrane recipe create mistral-prod --name mistral-balanced --version 1.0.0
infercrane recipes mistral --limit 20

Compare persisted measured configurations without provisioning new capacity:

infercrane lab 'mistralai/Mistral-7B-Instruct-v0.3@IMMUTABLE_COMMIT' \
  --objective interactive \
  --profile interactive \
  --max-ttft-p95-ms 200ms \
  --max-hourly-cost 3

The objective can be interactive, latency, throughput, or cost-efficiency. InferCrane recommends only across an exact common workload digest; mixed shapes remain UNRANKED. See Recipes and Inference Lab for the provenance and missing-evidence contract.

Replay and capacity

infercrane replay coder-production --window 24h
infercrane capacity --window 720h

Replay captures content-free production shape by default. Running an AIPerf approximation requires both --execute and --acknowledge-cost. See Replay and capacity intelligence. Create or inspect a best-effort logical session hint when an application benefits from affinity:

infercrane session create qwen-prod --ttl 1h
infercrane session inspect CONTEXT_PASSPORT_ID

The returned Context Passport is an opaque, expiring routing hint. It never promises durable KV state, and health always overrides a stale preferred target or binding.

Revisions and Release Guard

infercrane rollout inspect qwen-prod
infercrane rollout policy get qwen-prod
infercrane rollout policy set qwen-prod \
  --require-compatibility \
  --require-synthetic \
  --auto-rollback \
  --validation-max-requests 100 \
  --validation-max-concurrency 4

infercrane rollout create qwen-prod \
  --model Qwen/Qwen3-8B \
  --cloud runpod \
  --gpu H100 \
  --min 1 \
  --max 2 \
  --idempotency-key qwen-prod-rev-19

# Preserve a complete advanced serving topology without flattening it to flags.
infercrane rollout create llama-production \
  --file infercrane-dynamo-candidate.yaml \
  --idempotency-key llama-production-dynamo-r2

infercrane rollout provision qwen-prod REVISION_ID --wait
infercrane rollout validate qwen-prod --acknowledge-validation-cost --wait
infercrane rollout promote qwen-prod REVISION_ID --reason 'guard accepted'
infercrane rollout reject qwen-prod REVISION_ID --reason 'TTFT regression'
infercrane rollout rollback qwen-prod REVISION_ID --wait

Candidate creation also accepts --model-revision, --runtime, --runtime-version, --runtime-args, --routing, and --region. --file preserves a complete validated DeploymentSpec, including advanced serving topology, and cannot be mixed with candidate override flags. Promotion remains policy-gated; no LLM decides the outcome. rollout validate is explicit synthetic traffic, not shadowing. It runs the existing AIPerf adapter against active and candidate revisions using the same workload and persisted hard bounds. The acknowledgement is mandatory because both runs may incur provider cost.

Signed release evidence

infercrane passport keygen
infercrane passport issue qwen-prod --file qwen-prod.passport.json
infercrane passport verify qwen-prod.passport.json
infercrane passport list qwen-prod --output json

passport keygen and passport verify work without a configured control-plane context. Issue and list use the authenticated API. See Inference Passports.

Delete

Preview cleanup, then confirm it explicitly.

infercrane delete qwen-prod --plan
infercrane delete qwen-prod \
  --yes \
  --wait \
  --idempotency-key qwen-prod-delete-01
infercrane orphans

--plan is side-effect-free. --yes is required for mutation. After a paid test, also verify the provider inventory reaches zero.

Tenant administration

These commands require an admin/bootstrap credential.

infercrane tenant create team-a --name 'Team A'
infercrane principal create deploy-bot --role operator --scopes read,deploy
infercrane principal rotate PRINCIPAL_ID
infercrane principal revoke PRINCIPAL_ID

New and rotated tokens are returned once. Service-account scopes can only restrict the role ceiling; they can never grant an action unavailable to viewer, operator, or admin. Omitting --scopes uses every action allowed by the selected role for compatibility.

Secret references

Register metadata that points to an injected environment value. InferCrane never accepts the raw secret on the command line or through its API.

infercrane secret create openrouter --from-env OPENROUTER_API_KEY
infercrane secret list
infercrane secret list --output json
infercrane secret delete SECRET_ID --yes

Signed alerts

infercrane alert list coder-production
infercrane alert configure coder-production --webhook https://alerts.example.com/hook \
  --secret-reference SECRET_ID --minimum-severity warning
infercrane alert evaluate coder-production

Alerts contain deterministic Doctor findings and HMAC headers; delivery is idempotent and bounded.

Governed external fallback

External targets require manage_external. They are selected only when ordinary targets are unhealthy and require an explicit privacy acknowledgement plus hard budgets.

infercrane target add openrouter-qwen \
  --provider openrouter \
  --url https://openrouter.ai/api/v1 \
  --upstream-model qwen/qwen3-8b

infercrane external configure qwen-prod \
  --target openrouter-qwen \
  --secret-reference SECRET_REFERENCE_ID \
  --request-limit 100 \
  --cost-limit-usd 10 \
  --max-request-cost-usd 0.10 \
  --acknowledge-external-data \
  --enable

infercrane external inspect qwen-prod --output json

See Governed external capacity for transmission and reservation semantics. Evaluate a bounded overflow decision without allowing a second controller to mutate replicas:

infercrane burst qwen-prod \
  --queue-depth 8 \
  --breaches 3 \
  --external-healthy \
  --incremental-cost-microusd-hour 2500000 \
  --max-incremental-cost-microusd-hour 3000000

Burst Guard requires fresh health, repeated pressure, and sourced cost within the hard limit. Its result is a policy decision; the configured mutation owner remains responsible for execution.

SLO policy and recommendations

Define explicit fail-closed thresholds, then evaluate persisted benchmark evidence. Recommendations are advisory and never mutate the deployment.

infercrane slo set qwen-prod --ttft-p95 250 --error-rate 0.01
infercrane slo get qwen-prod --output json
infercrane recommend qwen-prod
infercrane recommend qwen-prod --history --output json
infercrane slo delete qwen-prod

recommended identifies a qualified candidate satisfying the policy. no_match means measured candidates violate it. unknown means required evidence is missing. See Inference decisions. Advisory Autopilot turns that evidence into a reviewable plan without mutating production:

infercrane autopilot plan qwen-prod --objective minimize_cost
infercrane autopilot inspect PLAN_ID
infercrane autopilot approve PLAN_ID

Approval records human intent. It does not bypass Release Guard, evidence requirements, hard budgets, or the single mutation-owner boundary.

​CLI reference

​Global conventions

​Configure and diagnose

​init

​Contexts and authentication

​Shell completion

​doctor

​Local accelerator discovery

​Run the control plane

​Operations workspaces

​inbox

​observe

​telemetry collect dcgm

​finops collect opencost

​artifact

​evaluation

​mcp

​Admission and async inference

​Terminal

​Plan and deploy

​optimize propose

​optimize create

​artifact optimize

​workload

​plan

​deploy and apply

​integrations

​sandbox

​training

​Send a request

​Stable endpoints

​Connect an existing workload

​Existing targets and routing

​Observe and explain

​Models and recipes

​Benchmark

​Replay and capacity

​Revisions and Release Guard

​Signed release evidence

​Delete

​Tenant administration

​Secret references

​Signed alerts

​Governed external fallback

​SLO policy and recommendations