> ## Documentation Index
> Fetch the complete documentation index at: https://infercrane.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Stable endpoints and serving plans

> Keep application model names stable while changing concrete inference implementations.

# Stable endpoints and serving plans

An endpoint is the model name an application sends to InferCrane. It is deliberately separate from
a deployment, provider, and runtime:

```text theme={"theme":{"light":"github-light-default","dark":"vesper"}}
application model="coder-production"
              ↓
           endpoint
              ↓
      active serving plan
              ↓
        backend binding
              ↓
 concrete deployment or imported target
```

This lets an operator stage a different serving implementation without changing application code.
Existing v1 deployment aliases are migrated automatically to a `production` environment, a logical
model, a lifecycle-managed binding, and a one-binding active plan of the same name.

## Create an endpoint

```bash theme={"theme":{"light":"github-light-default","dark":"vesper"}}
infercrane logical-model create coder \
  --description "Stable coding model"

infercrane endpoint create coder-production \
  --model coder \
  --environment production

infercrane endpoint bind coder-production \
  --name primary \
  --deployment qwen-prod \
  --ownership lifecycle-managed

infercrane endpoint plan coder-production \
  --policy manual \
  --bindings primary
```

The first plan becomes active. Later plans are staged as candidates and receive no application
traffic until promoted:

```bash theme={"theme":{"light":"github-light-default","dark":"vesper"}}
infercrane endpoint plan coder-production \
  --policy primary-fallback \
  --bindings candidate,primary

infercrane endpoint inspect coder-production
infercrane endpoint guard coder-production --evaluate
infercrane endpoint promote coder-production PLAN_ID
```

Plan creation returns `route_refresh: converged` when the in-memory route was published before the
response. `pending` means durable state is committed and ordinary reconciliation will publish it.

## Routing policies

| Policy             | Behavior                                                                                           |
| ------------------ | -------------------------------------------------------------------------------------------------- |
| `manual`           | Exactly one selected binding.                                                                      |
| `primary-fallback` | First available binding in declared priority order. InferCrane does not replay an accepted stream. |
| `weighted`         | Deterministic bounded weighted selection among currently available bindings.                       |

Serving plans are immutable and SHA-256 identified. Promotion changes only future route
acquisitions. A buffered or streaming request remains pinned to the route generation it acquired
until the request finishes.

Promotion fails closed unless the latest persisted endpoint Release Guard evaluation is `PASS` for
the current active/candidate pair. `REJECT` and `INCONCLUSIVE` can be inspected but cannot be
promoted. Candidate traffic is never duplicated silently; Release Guard evaluates trustworthy telemetry
from the primary deployment-backed binding and reports insufficient evidence as `INCONCLUSIVE`.
It does not reuse those primary metrics for a weighted or otherwise unmeasured routing topology.
An append-only managed fallback is eligible only when it preserves the active routing prefix and
has immutable consent plus hard-budget policy; the persisted evaluation records that topology
classification explicitly.

## Ownership boundaries

Bindings record `observe-only`, `traffic-managed`, or `lifecycle-managed` ownership. InferCrane supports
fully compiled lifecycle-managed deployment bindings. Incremental import/adopt workflows and
customer-managed external bindings; creating a record does not imply that
InferCrane owns an external workload's lifecycle.

The inference hot path reads an immutable in-memory endpoint snapshot. It never queries PostgreSQL.
Request telemetry records the logical model, environment, endpoint, serving plan, binding,
deployment, revision, replica where known, provider, and runtime without storing prompt or output
content by default.

## Add an authenticated managed API

An OpenRouter or generic OpenAI-compatible API can be staged behind the same endpoint. Provider
credentials remain reference-only, external data transmission requires explicit consent, and hard
request/cost reservations fail closed before any bytes are sent:

```bash theme={"theme":{"light":"github-light-default","dark":"vesper"}}
export MANAGED_MODEL_API_KEY='...'
infercrane secret create managed-model-api \
  --from-env MANAGED_MODEL_API_KEY \
  --output json

infercrane target add managed-coder \
  --provider openai-compatible-external \
  --runtime openai-compatible \
  --url https://models.example.com/v1 \
  --upstream-model provider/coder-model

infercrane endpoint bind coder-production \
  --name managed-fallback \
  --target managed-coder \
  --ownership traffic-managed \
  --external-adapter openai-compatible-external \
  --secret-reference SECRET_REFERENCE_ID \
  --request-limit 1000 \
  --cost-limit-usd 25.00 \
  --max-request-cost-usd 0.10 \
  --acknowledge-external-data \
  --enable-external

infercrane endpoint plan coder-production \
  --policy primary-fallback \
  --bindings primary,managed-fallback
```

The new plan is a candidate. It receives no application traffic until Release Guard passes and an
operator explicitly promotes it. Adding a fallback never changes the application request:

```python theme={"theme":{"light":"github-light-default","dark":"vesper"}}
client.responses.create(model="coder-production", input="Explain this incident.")
```

The web console exposes the same staging workflow on the endpoint page. It never receives the
provider credential; it sends only the selected secret-reference ID to the control API.