> ## Documentation Index
> Fetch the complete documentation index at: https://infercrane.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Build your inference deployment

> Go from a model or immutable OCI workload to a durable, OpenAI-compatible endpoint.

# A model is enough to start

InferCrane turns a model reference and a serving plan into a durable deployment. The CLI submits the
intent to the control plane and can disconnect safely while provider allocation, artifact transfer,
runtime startup, readiness, and route publication continue.

```bash theme={"theme":{"light":"github-light-default","dark":"vesper"}}
infercrane deploy Qwen/Qwen3-8B
```

Qwen3-8B is intentionally used as the release acceptance baseline: it is small enough for frequent
single-GPU qualification and exercises the production lifecycle without making every test unusually
expensive. It is not embedded in the InferCrane domain model.

## Use the model your workload needs

The same model-to-endpoint path accepts other Hugging Face repository identities. These examples show
portable input syntax; they do **not** claim that every model fits every GPU or that every adapter has
been production-qualified for it.

<CodeGroup>
  ```bash Qwen theme={"theme":{"light":"github-light-default","dark":"vesper"}}
  infercrane plan Qwen/Qwen3-8B --cloud runpod --gpu L40S
  ```

  ```bash Meta Llama theme={"theme":{"light":"github-light-default","dark":"vesper"}}
  infercrane plan meta-llama/Llama-4-Scout-17B-16E-Instruct \
    --cloud aws \
    --gpu H100
  ```

  ```bash Google Gemma theme={"theme":{"light":"github-light-default","dark":"vesper"}}
  infercrane plan google/gemma-3-4b-it \
    --cloud runpod \
    --gpu L40S
  ```

  ```bash Moonshot Kimi theme={"theme":{"light":"github-light-default","dark":"vesper"}}
  infercrane plan moonshotai/Kimi-K2-Instruct \
    --cloud aws \
    --gpu H100
  ```
</CodeGroup>

`plan` validates InferCrane's serving intent without creating resources. Before `deploy`, confirm the
model card license or access grant, total and active parameter memory, quantization, tensor-parallel
topology, runtime version, multimodal requirements, and provider capacity. Large MoE models such as
Llama 4 and Kimi generally require materially different topology from an 8B dense model; the example
GPU flag alone is not a sizing recommendation.

| Example family | Why teams use it                                         | Qualification concern                                                   |
| -------------- | -------------------------------------------------------- | ----------------------------------------------------------------------- |
| Qwen           | Compact general and coding deployments                   | Exact chat template, tool calling, and quantization                     |
| Meta Llama     | Broad ecosystem and open-weight deployment               | Meta license/access and multi-GPU topology for larger variants          |
| Google Gemma   | Small-to-large open-weight family                        | Google terms, multimodal/runtime support, and artifact access           |
| Moonshot Kimi  | Long-context, reasoning, coding, and multimodal variants | Very large MoE topology and version-specific runtime requirements       |
| Your fine-tune | Private or domain-specific behavior                      | Immutable artifact commit, tokenizer compatibility, and registry access |

<Note>
  Model popularity changes faster than InferCrane releases. The source of truth is the immutable model
  artifact and the evidence attached to its serving plan—not a marketing list of “supported models.”
</Note>

The short path uses configured defaults. Make infrastructure choices explicit when you need them:

```bash theme={"theme":{"light":"github-light-default","dark":"vesper"}}
infercrane plan Qwen/Qwen3-8B \
  --cloud runpod \
  --gpu L40S \
  --min 1 \
  --max 4

infercrane deploy Qwen/Qwen3-8B \
  --cloud runpod \
  --gpu L40S \
  --min 1 \
  --max 4
```

`plan` is side-effect free. `deploy` returns a durable operation unless `--wait` is supplied. Closing
the terminal stops only the local watch; use the returned operation ID to reconnect:

```bash theme={"theme":{"light":"github-light-default","dark":"vesper"}}
infercrane operation watch OPERATION_ID
infercrane status qwen3-8b
infercrane events qwen3-8b
```

<Note>
  RunPod is one currently qualified adapter, not the InferCrane domain model. AWS BYOC, Kubernetes,
  existing targets, and future providers implement the same provider contract with independent
  qualification evidence. Check the [capability matrix](/project-status) before selecting a backend.
</Note>

## Bring an immutable runtime when a model flag is not enough

For a qualified custom runtime, describe the OCI image, digest, startup contract, health contract,
and protocol capabilities in a DeploymentSpec. InferCrane persists that configuration as an
immutable revision and manages its lifecycle; it does not build images or execute arbitrary build
scripts.

```bash theme={"theme":{"light":"github-light-default","dark":"vesper"}}
infercrane plan deployment.yaml
infercrane apply deployment.yaml --idempotency-key release-2026-08-12
```

See [Custom OCI workloads](/features/custom-oci) for the exact schema and safety requirements.

## What “build” means here

| InferCrane owns                                     | Existing tools keep owning         |
| --------------------------------------------------- | ---------------------------------- |
| Serving intent and stable endpoint identity         | Model implementation               |
| Provider and runtime adapter selection              | Container registry and image build |
| Durable create, update, rollback, scale, and delete | CUDA/runtime internals             |
| Immutable revisions and artifact identity           | Inference engine execution         |
| Readiness, routing, telemetry, and policy evidence  | Provider infrastructure primitives |

This boundary keeps the product extensible: vLLM, SGLang, a custom OCI runtime, or another qualified
runtime can change without changing the application-facing endpoint.

## Separate development, staging, and production

Environments are first-class, tenant-scoped resources. Bind separate stable endpoints to the same
logical model so application configuration and policy do not leak across stages:

```bash theme={"theme":{"light":"github-light-default","dark":"vesper"}}
infercrane environment create development
infercrane environment create staging
infercrane environment create production

infercrane logical-model create coder --description "Stable coding model"
infercrane endpoint create coder-development --model coder --environment development
infercrane endpoint create coder-staging --model coder --environment staging
infercrane endpoint create coder-production --model coder --environment production
```

Each endpoint has independent active and candidate serving plans. Request records preserve logical
model, endpoint, and environment identity.

Stage the exact serving plan from staging as a production candidate without changing production
traffic:

```bash theme={"theme":{"light":"github-light-default","dark":"vesper"}}
infercrane environment promote coder-staging --to coder-production
infercrane environment promote coder-staging --to coder-production --yes
```

The first command is a side-effect-free plan. The second atomically clones destination-scoped
bindings and stages an immutable candidate. Production Release Guard remains the authority for
activation. See [Environment promotion](/features/environment-promotion).

## Make the next revision safely

Use the same declarative path for updates:

```bash theme={"theme":{"light":"github-light-default","dark":"vesper"}}
infercrane plan deployment-v2.yaml
infercrane apply deployment-v2.yaml --idempotency-key release-2026-08-13
infercrane rollout inspect qwen-prod
```

InferCrane creates an immutable candidate and preserves the active revision until promotion policy is
satisfied. Continue with the [safe rollout showcase](/showcase/safe-rollouts).