Skip to main content

​
Stable endpoints and serving plans

An endpoint is the model name an application sends to InferCrane. It is deliberately separate from a deployment, provider, and runtime:
This lets an operator stage a different serving implementation without changing application code. Existing v1 deployment aliases are migrated automatically to a production environment, a logical model, a lifecycle-managed binding, and a one-binding active plan of the same name.

​
Create an endpoint

The first plan becomes active. Later plans are staged as candidates and receive no application traffic until promoted:
Plan creation returns route_refresh: converged when the in-memory route was published before the response. pending means durable state is committed and ordinary reconciliation will publish it.

​
Routing policies

Serving plans are immutable and SHA-256 identified. Promotion changes only future route acquisitions. A buffered or streaming request remains pinned to the route generation it acquired until the request finishes. Promotion fails closed unless the latest persisted endpoint Release Guard evaluation is PASS for the current active/candidate pair. REJECT and INCONCLUSIVE can be inspected but cannot be promoted. Candidate traffic is never duplicated silently; Release Guard evaluates trustworthy telemetry from the primary deployment-backed binding and reports insufficient evidence as INCONCLUSIVE. It does not reuse those primary metrics for a weighted or otherwise unmeasured routing topology. An append-only managed fallback is eligible only when it preserves the active routing prefix and has immutable consent plus hard-budget policy; the persisted evaluation records that topology classification explicitly.

​
Ownership boundaries

Bindings record observe-only, traffic-managed, or lifecycle-managed ownership. InferCrane supports fully compiled lifecycle-managed deployment bindings. Incremental import/adopt workflows and customer-managed external bindings; creating a record does not imply that InferCrane owns an external workload’s lifecycle. The inference hot path reads an immutable in-memory endpoint snapshot. It never queries PostgreSQL. Request telemetry records the logical model, environment, endpoint, serving plan, binding, deployment, revision, replica where known, provider, and runtime without storing prompt or output content by default.

​
Add an authenticated managed API

An OpenRouter or generic OpenAI-compatible API can be staged behind the same endpoint. Provider credentials remain reference-only, external data transmission requires explicit consent, and hard request/cost reservations fail closed before any bytes are sent:
The new plan is a candidate. It receives no application traffic until Release Guard passes and an operator explicitly promotes it. Adding a fallback never changes the application request:
The web console exposes the same staging workflow on the endpoint page. It never receives the provider credential; it sends only the selected secret-reference ID to the control API.