Skip to main content

​
A model is enough to start

InferCrane turns a model reference and a serving plan into a durable deployment. The CLI submits the intent to the control plane and can disconnect safely while provider allocation, artifact transfer, runtime startup, readiness, and route publication continue.
Qwen3-8B is intentionally used as the release acceptance baseline: it is small enough for frequent single-GPU qualification and exercises the production lifecycle without making every test unusually expensive. It is not embedded in the InferCrane domain model.

​
Use the model your workload needs

The same model-to-endpoint path accepts other Hugging Face repository identities. These examples show portable input syntax; they do not claim that every model fits every GPU or that every adapter has been production-qualified for it.
plan validates InferCrane’s serving intent without creating resources. Before deploy, confirm the model card license or access grant, total and active parameter memory, quantization, tensor-parallel topology, runtime version, multimodal requirements, and provider capacity. Large MoE models such as Llama 4 and Kimi generally require materially different topology from an 8B dense model; the example GPU flag alone is not a sizing recommendation.
Model popularity changes faster than InferCrane releases. The source of truth is the immutable model artifact and the evidence attached to its serving plan—not a marketing list of “supported models.”
The short path uses configured defaults. Make infrastructure choices explicit when you need them:
plan is side-effect free. deploy returns a durable operation unless --wait is supplied. Closing the terminal stops only the local watch; use the returned operation ID to reconnect:
RunPod is one currently qualified adapter, not the InferCrane domain model. AWS BYOC, Kubernetes, existing targets, and future providers implement the same provider contract with independent qualification evidence. Check the capability matrix before selecting a backend.

​
Bring an immutable runtime when a model flag is not enough

For a qualified custom runtime, describe the OCI image, digest, startup contract, health contract, and protocol capabilities in a DeploymentSpec. InferCrane persists that configuration as an immutable revision and manages its lifecycle; it does not build images or execute arbitrary build scripts.
See Custom OCI workloads for the exact schema and safety requirements.

​
What “build” means here

This boundary keeps the product extensible: vLLM, SGLang, a custom OCI runtime, or another qualified runtime can change without changing the application-facing endpoint.

​
Separate development, staging, and production

Environments are first-class, tenant-scoped resources. Bind separate stable endpoints to the same logical model so application configuration and policy do not leak across stages:
Each endpoint has independent active and candidate serving plans. Request records preserve logical model, endpoint, and environment identity. Stage the exact serving plan from staging as a production candidate without changing production traffic:
The first command is a side-effect-free plan. The second atomically clones destination-scoped bindings and stages an immutable candidate. Production Release Guard remains the authority for activation. See Environment promotion.

​
Make the next revision safely

Use the same declarative path for updates:
InferCrane creates an immutable candidate and preserves the active revision until promotion policy is satisfied. Continue with the safe rollout showcase.