CLI reference
The public CLI uses Cobra for grouped help, typo suggestions, aliases, and shell completion. It talks to the authenticated control-plane API and never opens PostgreSQL directly. Runinfercrane help for the current command tree and infercrane version for the build version. infercrane version --output json returns version plus a build object with the Go version, platform, and, when the binary embeds them, the source commit, commit_time, and modified flag. Build fields that were not embedded are omitted, never guessed.
Global conventions
- Commands that return data accept
--output human|json. It can follow the command’s arguments or come before the command:infercrane --output json status qwen-prod. - With
--output json, a successful command writes JSON to stdout and every failure, including a usage error, writes one JSON error object to stderr. The CLI never prints Go’s flag usage text. - A resource argument such as
DEPLOYMENTorIDcomes right after the command or action, before flags. If it is missing, or a flag stands in its place, the command fails with a usage error before contacting the control plane. An extra positional argument is also a usage error, so a later--outputis never silently ignored. status --watch --output jsonemits one JSON document per state refresh.logs --output jsonemits JSON Lines: one event object per line, including while--followstreams new events. Parse it line by line, not as one document.- Mutation commands accept
--idempotency-key; generated keys are printed in human and JSON output and attached to uncertain-request errors. Reuse the same key after an uncertain result. --waitpolls persisted operation state and prints only changed progress. In a terminal it redraws one progress line with a spinner and prints a line as each phase finishes; otherwise it prints one line per change on stderr. Closing the client does not cancel server-side work; the operation ID and exactinfercrane operation watch IDresume command are printed before waiting begins.--wait-timeout 20mbounds the local watcher without cancelling the durable operation. Resume withinfercrane operation watch ID; cancel only withinfercrane operation cancel ID.- JSON failures contain
code,category,message,retryable,remediation, and provider HTTP status when available. - Command help is available before authentication:
infercrane COMMAND --helpnever contacts the control plane. infercrane --context NAME COMMANDselects a context for one invocation.--no-color, before or after the command, and the standardNO_COLORenvironment variable disable ANSI styling. Redirected output and JSON are never decorated. See Terminal output.- When a dashboard is configured, human output from
deploy,apply,endpoint create,rollout,optimize create|approve|report, andauth statusends withView in dashboard → URL. See Sign in the CLI.
Keep the original idempotency key after a timeout or disconnect. Inspect the durable operation and its events before retrying; a new key represents a new mutation intent.
Words
Help and human-readable output use the dashboard’s words. Command names, flag names, and--output json keep the control API’s names, so scripts keep working:
A deployment’s staged revision is still the candidate revision (
--revision candidate,
candidate_revision_id), and an endpoint’s staged serving plan is its candidate plan.
Control API error messages use the same words, so the CLI and the dashboard read alike:
optimization run was not found, setup was not found in this run. Error codes are identifiers
that clients match on and keep the API’s names, for example campaign_state_conflict and
candidate_not_decided.
Where the CLI turns a state into words, a run or setup the API calls promoted reads
rolled out (live in optimize report), and the current setup reads baseline where the API
says rejected because another setup was chosen. That applies to the styled optimize list and
inspect, to optimize watch, results, and report, and to shell completion. Plain list and
inspect output and JSON print the API’s own state.
Terminal output
In a terminal, human output is styled like the dashboard: state badges such as● SERVING,
before → after changes in green when better and red when worse, aligned tables with relative
times, and cards for optimize report. Every color repeats a word that is printed next to it, and
unknown values stay unknown. Layouts fit an 80-column terminal and use the extra width of a
wider one.
Output stays plain, with no ANSI escape codes and no prompts, when:
- stdout is redirected or piped, as it is for scripts, CI, and coding agents;
NO_COLORor--no-coloris set;TERM=dumb;--output jsonis used.
Optimization run ID · STATE and SETUPS.
Colors follow the terminal’s background when it sets COLORFGBG. Otherwise the CLI uses mid-tone
colors that read on dark and light backgrounds. It never queries the terminal, so a command never
waits on one.
For example, infercrane optimize list in an 80-column terminal:
infercrane deploy MODEL --wait while it runs:
delete DEPLOYMENTwithout--yesshows the deletion plan and asks you to type the deployment name. Without a terminal, or with--output json, it still requires--yes.optimize approveshows the spend cap, when the approval expires, and every setup the run starts, then asksApprove spending up to $25.00? [y/N].--yesapproves without asking. Without a terminal it approves as before, without a prompt.
Configure and diagnose
login
Sign the CLI in through the dashboard. The terminal and the browser show the same one-time code;
approving it issues a key with your role and scopes, stored in the OS keychain when available and
otherwise in the 0600 config file. The key is never printed.
auth login is the same command. Details, storage, and revocation are in
Sign in the CLI.
logout
Revoke this CLI’s key on the control plane and remove it from the keychain and config file. The
context keeps its URLs. A key pasted with init is removed locally but stays active on the control
plane; revoke it with principal revoke. auth logout is the same command.
open
Open a dashboard page for a resource, or print its URL with --no-browser.
init
Validate and store a control-plane URL and an already-issued credential in a named private client
context. Prefer login on a workstation; use init or INFERCRANE_API_KEY for service accounts
and CI.
init verifies authentication through the read-only identity endpoint before writing configuration. Use --skip-check only when intentionally configuring an offline control plane. It does not create or enroll a principal.
Contexts and authentication
id, tenant_id, name, role, kind, and scopes)
under principal, plus context, control_url, console_url, and credential_source (where the
key was found: the keychain, the config file, or INFERCRANE_API_KEY), so authentication checks can
be consumed directly by scripts without depending on Go field names. whoami is the same as
auth status.
Legacy single-context configuration is migrated on the next init. Context listing and display
never print stored credentials.
Shell completion
zsh and fish show the description next to each value. Completion only reads, gives up after 1.5
seconds, and stays silent when the control plane is unreachable or the CLI is not signed in: it
never prints an error and never mutates anything. A GPU or region suggestion says where it is
offered, not that capacity is available.
doctor
Ask the control plane to check its own dependencies.
--cloud adds SkyPilot credentials and RunPod advisory accelerator availability; --serverless
adds RunPod Serverless credentials and template validation; --aws performs a read-only role
assumption and dependency probe for the configured AWS BYOC adapter; --gcp checks Application
Default Credentials, every configured Compute dependency, immutable images, and Private Google
Access; --kubernetes checks the explicit context, optional KServe CRD, and required namespaced
permissions. Diagnostics are read-only and do not create provider resources.
The capability table distinguishes supported, unsupported, and unknown. InferCrane never
silently changes hardware, and never claims cache or fast-resume behavior that an adapter cannot
observe.
Local accelerator discovery
Inspect NVIDIA GPU inventory without installing a runtime or changing the host:infercrane agent detect also reports Apple
Silicon and AMD GPUs, with their unknown fields named, and the model servers on this machine.
Run the control plane
serve starts the control plane and inference gateway from validated environment configuration.
Production deployments should use the container and service guidance in
Production operations, including PostgreSQL, TLS, bootstrap authentication,
stable instance identity, and graceful shutdown.
Operations workspaces
inbox
observing endpoint is observe-only
by design and is not listed), and prints the exact persisted field behind each item. It fails the
whole read if any of those APIs is unavailable, so a partial inventory cannot look healthy. It does
not run Doctor implicitly and never reads prompt or response content.
Optimization runs appear with the same rules as run notifications:
a run waiting for spend approval, a result waiting for your decision (critical once its approval is
about to end), a running run whose approval is about to end, setups that have waited more than five
minutes for your signed evaluation or for the eval set the run names, and a run that failed in the
last 24 hours. Each item names the next command, such as infercrane optimize approve or
infercrane evaluation set upload. The inbox reads the newest 100 runs.
observe
--diagnose explicitly persists a fresh deterministic Doctor evaluation.
A field the control plane has not recorded prints as none (for example, no staged candidate plan)
or unknown. JSON keeps the raw values and reports an unstaged candidate as "candidate_plan": null.
telemetry collect dcgm|amd
dcgm reads NVIDIA’s dcgm-exporter and amd reads the AMD Device Metrics Exporter (default port
5000; --metric-prefix if it sets one). The command parses a bounded, documented subset of the
exporter’s Prometheus metrics locally and sends normalized content-free evidence through the
authenticated control API. Use --file SNAPSHOT for
offline input. The snapshot is immutable, active-revision-bound, and becomes stale at its declared
TTL; arbitrary Prometheus metrics are not accepted.
--interval 15s keeps collecting until stopped; each snapshot stays fresh for three intervals.
profile
/start_profile and /stop_profile, analyzes it locally, prints
the verdict and the numbers behind it, and records the capture against the deployment’s active
revision. It states that the replica slows while the profiler runs and asks before starting; --yes
skips the question. --dcgm-url samples dcgm-exporter during the capture. --trace analyzes existing
Kineto JSON, rocprofv3 CSV, or Nsight Systems CSV files; with --no-upload it needs no sign-in.
--open opens the capture in the dashboard. --output json prints the stable
infercrane.profile-result/v1 document. See GPU profiling.
finops collect opencost
--allocation and --currency are mandatory; cluster-wide attribution and
currency conversion are never inferred. Use --file allocation.json for offline evidence.
artifact
evaluation
ingest accepts the strict evaluator-result v1 interchange contract, rejects unknown content fields,
binds it to one immutable revision, signs it, and optionally attaches it. Evidence is
signature-verified and contains aggregate values only. The evaluator name infercrane-builtin is
reserved for InferCrane’s own answer check and is refused.
evaluation set manages the prompt sets that replace the built-in prompts of an optimization run’s
answer check. default prints the built-in suite to copy and adapt, digest validates a file
without uploading it, and upload prints the digest a run names with --eval-set. See
Semantic quality evidence.
mcp
Admission and async inference
async submit persists protocol-native request content only after explicit encrypted-storage
consent, returns a durable job ID, and does not depend on the client remaining connected. See
Admission and async inference for retry, retention and webhook rules.
Terminal
infercrane ui --read-only. It requires an interactive terminal and never uses tmux
for persistence. See Terminal operations workspace.
Plan and deploy
optimize propose
Preview the setups a run would measure, as inspectable DeploymentSpecs, without contacting a
control plane or provider. Check the optional estimator first:
--source auto uses the pinned AIConfigurator adapter only when its exact supported
Python tuple is available and visibly falls back to the reviewed catalog otherwise. Use --source catalog for a deterministic offline proposal, or --source aiconfigurator to require the estimator
and fail rather than falling back.
unmeasured; AIConfigurator output is modeled. Neither is qualified. Use
--output json to retain the input digest, source version, modeled-result digest, compatibility
evidence, missing evidence, and limitations. --workload-fingerprint can bind a content-free
observed workload identity to the proposal. Simulated compatibility is excluded unless
--include-simulated is explicit. The command refuses to overwrite a non-empty output directory.
The reviewed catalog spans instruction, reasoning, coding, embeddings, multimodal, and
enterprise-text workloads. A profile for one model is never evidence for another model, precision,
GPU, runtime, or topology. See Evidence-gated optimization.
optimize create
Create a run: persist the proposal and its setups without provider mutation, then grant bounded
paid authority explicitly:
optimize approve first shows the spend cap, when the approval expires, and every
setup the run starts, and asks before approving. --yes skips the question; scripts, pipes, and
--output json never see it. Approval retries do not extend the first persisted expiry. Creation and inspection allocate no
infrastructure. The default new_endpoint intent stops after qualification and waits for you to
roll out the winner. --target-deployment selects evolve_endpoint, which must also pass Release
Guard before it waits for you. optimize activate rolls out the winning setup in both cases: it
marks the qualified new deployment active, or hands an evolution to the existing guarded rollout
operation. Retrying the command adopts the same durable operation; InferCrane never rolls out a
setup from the optimizer loop.
optimize watch follows a run live and optimize results prints the same summary once: each setup
by name, the current setup first, which setup and measured turn the run is on, time elapsed, the
control plane’s estimate of time left, spend so far against the cap (estimated from the quoted
rate), the measured comparison, the answer checks, and the decision with the exact activate
command. inspect is the full record, with every setup’s name next to its ID. See
Follow a run live.
Every setup’s answers are checked before ranking. By default InferCrane runs its built-in answer
check against your current setup. --eval-set sha256:DIGEST replaces the built-in prompts with an
uploaded eval set, and --quality-check customer_evidence makes every setup wait for evidence you
sign and attach instead. inspect prints each setup’s answer-check result and, while a setup
waits on you, the exact command that unblocks it. --output json returns the control API’s
quality object for every setup, under candidates.
inspect, results, approve, activate, and cancel print a Proposal warnings block when
the run’s proposal recorded warnings, such as skipped speculative-decoding setups.
--output json keeps warnings as the control API returned it: an empty list means none were
recorded, and an absent field means the stored proposal could not be read. Warnings are not
performance evidence.
artifact optimize
Plan and attest an externally built optimized checkpoint, speculator, or TensorRT engine:
fp8, awq, gptq, nvfp4, eagle3, mtp, and dflash. There is no TensorRT engine
preset: TensorRT-LLM removed its engine backend, and its PyTorch backend loads checkpoints directly.
An engine built by an earlier TensorRT-LLM release can still be recorded with --plan-file. Builders
are never executed inside the InferCrane API process, and no optimized artifact qualifies without
exact setup, revision, digest, and semantic-quality evidence.
workload
Create a repository-local inference project, validate it without allocating a GPU, and use the same
DeploymentSpec for build, plan, and deployment:
workload init --recipe NAME --profile PROFILE pins a reviewed model commit and writes the selected
profile’s runtime version, immutable workload when present, GPU type and count, provider hint,
replica bounds, and routing policy. The default profile is
the recipe’s general starting point; profiles are not benchmark claims. workload build --tag IMAGE performs a
local Docker Buildx build for custom OCI projects; add --push to record a registry-confirmed
immutable digest. workload dev binds a custom OCI workload only to loopback. See
Inference projects.
plan
Preview semantic changes without provider or database mutation.
plan instead shows an
observed p50 from durable replica intent through runtime readiness. It shows p95 only after twenty
successes and labels the sample count and boundary. That is historical evidence, not a capacity
guarantee. For a DeploymentSpec pinned to an immutable model revision, plan can additionally show
per-stage observed p50 after three successes and p95 after twenty only when model commit, runtime
version and arguments, provider, region, GPU type/count, and compute mode all match. A stage with no
provider marker or insufficient samples remains absent/unknown. Once an operation starts, durable
events replace unknown stages with fresh provider and runtime observations where those boundaries
are actually exposed.
deploy and apply
deploy creates a cloud or existing-target deployment. apply declaratively converges a deployment using the same input shape.
The two self-serve destinations of a hosted workspace are
--infercrane-cloud and
--compute-connection NAME; they cannot be combined with each other, --targets, --backend, or
--compute serverless.
integrations
integrations list is the same command as integrations.
sandbox
Compose an externally operated sandbox with one stable inference endpoint:
connect and rotate reveal a credential once. It expires within 24 hours, can invoke only the
selected endpoint alias, and cannot use control-plane routes. revoke disables InferCrane access
without mutating the external sandbox. See External agent sandboxes.
training
Verify a signed, content-free artifact handoff from an external training system:
Send a request
Use the logical deployment name without assembling an HTTP request by hand:--output json returns the OpenAI-compatible choices and usage object for non-streaming requests.
The command uses the configured endpoint and credential; applications can continue to use any
OpenAI-compatible SDK directly. --protocol accepts chat, responses, embeddings,
completions, or batch; the endpoint must explicitly qualify the selected surface.
Inspect the persisted routing and timing evidence for a returned X-Request-Id without retrieving
prompt or output content:
Stable endpoints
Separate the name applications call from the deployment that currently serves it:endpoint plan makes the first plan active and stages later plans as candidates. Inspect and
promote explicitly:
--bindings is ordered and accepts optional bounded weights, for example
primary:80,overflow:20. See stable endpoints and serving plans.
Authenticated external APIs add provider-neutral policy flags to endpoint bind:
infercrane provider list shows reusable connection metadata without resolving credentials.
infercrane provider delete NAME removes only the reusable configuration; immutable endpoint
bindings and their release history are deliberately unchanged.
--from-env creates or reuses a secret reference by variable name, making a disconnected retry
idempotent without accepting the raw key. Use --secret-reference ID instead when secret references
are provisioned separately. The two flags are mutually exclusive.
Stage one environment’s active immutable plan as another endpoint’s candidate:
Connect an existing workload
The simple path verifies the upstream model list through the control-plane API and starts in observe-only mode:--model when the endpoint exposes more than one physical model. Use --manage-traffic only
when the endpoint has been qualified and should immediately enter the InferCrane logical route.
For a server started with an API key, --token-env VAR sends the key held in VAR once; the control
plane stores it encrypted and uses it for discovery and health checks (this needs
INFERCRANE_CREDENTIAL_ENCRYPTION_KEY on the control plane). connect refuses a loopback URL when
the control plane runs elsewhere, because the control plane would probe its own machine; use the
agent for that server.
The advanced ownership-compatible command remains available:
traffic-managed only when InferCrane should publish the healthy existing target. Neither mode
transfers provider lifecycle ownership. See Adopt and diagnose.
Connect servers on this machine
infercrane agent observes model servers the control plane cannot reach (on a laptop, behind NAT,
or in a private network) through an outbound-only agent. See
Run on this machine and InferCrane agent.
agent detect sends nothing. agent run uses the key in INFERCRANE_AGENT_KEY, then a key saved by an
earlier run, and otherwise creates one with the CLI’s own sign-in and saves it in the OS keychain (or
agent-credentials.json, mode 0600, next to the CLI config). It sends that key only over HTTPS,
except to a control plane on the same machine. A revoked agent stops at its next sync.
Existing targets and routing
route changes one deployment’s strategy. infercrane deployments lists the current strategy of
every deployment.
Supported strategies are round-robin, consistent-hash, power-of-two, and cache-aware, delegated to the pinned vLLM Router.
Observe and explain
operation list shows the newest 20 operations for your organization; --limit accepts 1 to 500.
When a page is full, it prints the --before command for the next older page.
Long operations belong to the control plane, not the terminal. deploy, apply, rollout, scale,
and deletion continue after the CLI disconnects. Pressing Ctrl-C stops only the local watcher. Run
infercrane operation watch ID later to resume from persisted progress; add --wait-timeout 20m
to bound only the new local watch. Explicit infercrane operation cancel ID is the separate action
that requests cooperative server-side cancellation and provider cleanup.
status deliberately separates two concerns. Serving answers whether the current route can
accept inference traffic. Convergence answers whether desired capacity, a rollout, or deletion is
still progressing. A deployment can therefore be serving · converging while a provider allocates
another replica; this is not reported as an outage. JSON output exposes the same stable fields under
lifecycle_status, including ready and desired replica counts, provisioning and draining counts,
candidate state, and the blocking durable operation.
Deterministic explanations use only persisted state and measurements:
Models and recipes
Explore the built-in reviewed catalog without contacting a control plane (models check is the one
subcommand that asks the control plane):
models inspect prints the immutable revision, license URL, protocol capabilities, serving
profiles, accelerator topology, immutable runtime workload when present, and the exact qualification boundary. Create a project with infercrane workload init --recipe NAME. Catalog profiles are configuration-verified, not benchmark, price, capacity, or
universal GPU-fit claims. See Verified Models.
models check OWNER/NAME asks the control plane to check any open Hugging Face model outside the
catalog: architecture support on the pinned vLLM and SGLang releases, size, the smallest priced GPU
layout for the load (--concurrency, --context-tokens), license, and access. Gated models need
--secret ID, a saved Hugging Face token reference. --spec FILE writes the proposed setup as a
DeploymentSpec for infercrane deploy FILE. See
Check any model automatically.
Benchmark
Run AIPerf and persist the exact reproduction metadata.--profile accepts balanced, buffered, interactive, long-context, long-generation,
overload, or throughput. Profiles are immutable workload shapes; explicit request,
concurrency, token, and streaming flags override their defaults. --revision accepts active,
candidate, or an explicit revision ID. Without a profile, defaults remain 100 requests,
concurrency 10, and seed 17. Declared TTFT, TPOT, or total-latency SLOs produce measured goodput.
Capture and search immutable recipes only after a matching benchmark exists:
interactive, latency, throughput, or cost-efficiency. InferCrane recommends only across
an exact common workload digest; mixed shapes remain UNRANKED. See
Recipes and Inference Lab for the provenance and missing-evidence contract.
Replay and capacity
--execute and --acknowledge-cost. See Replay and capacity intelligence.
Create or inspect a best-effort logical session hint when an application benefits from affinity:
Revisions and Release Guard
--model-revision, --runtime, --runtime-version, --runtime-args, --routing, and --region. --file preserves a complete validated DeploymentSpec, including advanced serving topology, and cannot be mixed with candidate override flags. Rolling out a revision with rollout promote remains policy-gated; no LLM decides the outcome. rollout rollback requires --reason.
rollout validate is explicit synthetic traffic, not shadowing. It runs the existing AIPerf adapter
against active and candidate revisions using the same workload and persisted hard bounds. The
acknowledgement is mandatory because both runs may incur provider cost.
Signed release evidence
passport keygen and passport verify work without a configured control-plane context. Issue and
list use the authenticated API. See Inference Passports.
Delete
Preview cleanup, then confirm it explicitly.--plan is side-effect-free. --yes is required for mutation, except in a terminal, where
delete without --yes shows the plan and asks you to type the deployment name instead. After a paid test, also verify the provider inventory reaches zero. Deletion also removes the endpoint InferCrane created for the deployment, unless another backend is bound to it. Once the operation finishes, the name can be deployed again.
Tenant administration
These commands require an admin/bootstrap credential.viewer, operator, or admin. Omitting --scopes
uses every action allowed by the selected role for compatibility.
Secret references
Register metadata that points to an injected environment value. InferCrane never accepts the raw secret on the command line or through its API.Signed alerts
Governed external fallback
External targets requiremanage_external. They are selected only when ordinary targets are
unhealthy and require an explicit privacy acknowledgement plus hard budgets.
SLO policy and recommendations
Define explicit fail-closed thresholds, then evaluate persisted benchmark evidence. Recommendations are advisory and never mutate the deployment.recommended identifies a qualified setup that satisfies the policy. no_match means the measured
setups violate it. unknown means required evidence is missing. See
Inference decisions.
Advisory Autopilot turns that evidence into a reviewable plan without mutating production: