The hosted console is a company-operated service over the public InferCrane control contracts.
The Apache-2.0 core remains CLI-, API-, Terraform-, and terminal-first and does not require an
InferCrane account.
What the console covers
- fleet attention, stable endpoints, and concrete deployments;
- durable operations with reconnectable timelines and cooperative cancellation;
- Request Inspector and deterministic Doctor findings;
- the Workbench: send a prompt to an endpoint and read the streamed answer with its time to first
token, token counts, and request ID; compare two endpoints, or an endpoint and a run’s decided
setup, side by side; paste tool definitions and see whether the model’s tool call matches the
schema; and copy the request as cURL, Python, or TypeScript. The browser never holds an API key:
the console server calls
POST /api/v1/playground/chat/completionsas the signed-in member (Gateway); - endpoint monitoring for request rate, errors, fallback, TTFT, queueing, latency, and reported token throughput;
- typed freshness, source, sample count, and availability for every monitoring measurement;
- Release Guard, benchmark, replay, revision, and replica evidence where available;
- lifecycle overlays, signed alert policies, sourced FinOps reports, and signed output-quality evidence;
- provider/runtime capability inventory and secret references;
- private/gated model-access references and authenticated existing-endpoint discovery;
- content-free telemetry imports and workload replay history;
- scoped API-key lifecycle, organization roles, hosted SSO entry, and approval boundaries;
- CLI sign-in approval at
/cli/authorizeand Settings → CLI sessions to review and revoke CLI keys (Sign in the CLI); - a curated Model API shelf with InferCrane rates, capabilities, limits, and availability.
First-run paths
Create endpoint starts with the application identity the user wants, then asks how it should be served. The primary paths are ordered by intent:- Deploy an open model starts from a reviewed model configuration, exposes only the required serving choices, and keeps runtime, accelerator, region, and scaling under Advanced.
- Connect existing inference discovers vLLM, SGLang, LiteLLM, or another compatible endpoint and starts observe-only. The endpoint page then presents a separate, explicit ownership action before InferCrane can route traffic. Promotion changes routing ownership only; it does not grant lifecycle control over the external workload. Saved Baseten, Fireworks, OpenRouter, and generic OpenAI-compatible connections can authenticate the bounded discovery call without putting their bearer credential in the browser or endpoint URL.
- Use a Model API selects a hosted InferCrane product, creates one API key, and uses the same OpenAI-compatible endpoint across the catalog. The customer surface shows only the InferCrane rate, capability, limit, and availability contract. It does not name the upstream that serves a request or show its offer IDs and credentials. Prepaid reservation happens before a managed request can leave InferCrane.
workspace_owned. On your own control plane, set
INFERCRANE_OPERATOR_COMPUTE_TENANTS so the console can use the cluster or cloud account already
configured there; see
Let workspaces use operator compute.
The console navigation keeps stable application identities under Endpoints, lifecycle-managed
capacity under Deployments, live evidence under Monitoring, and provider/runtime setup under
Settings → Connections. Existing monitoring systems enter through Settings → Observability;
identity and policy remain under Settings → Access. Endpoint pages expose task tabs for
monitoring, requests, releases, routing, and settings. Deployment pages expose replicas, revisions,
autoscaling, benchmarks, and events without turning every capability into a global navigation item.
Optional evidence fails locally: a missing benchmark, alert, member directory, or adapter inventory
shows an explicit unavailable panel while the rest of the resource stays usable. Unknown values are
not converted to zero.
The monitoring page queries only the authenticated control API. PostgreSQL request evidence and
lifecycle events form the initial local evidence store; OpenTelemetry GenAI semantics define the
portable names and dimensions. Runtime-internal metrics remain unavailable until a qualified
runtime adapter reports them. GPU utilization, memory, temperature, power, and XID evidence appear
when a qualified collector posts a fresh revision-bound snapshot. The UI displays why evidence is
absent or stale rather than rendering an unknown value as zero.
Self-hosted interface boundary
The public repository ships the control API, OpenAI-compatible gateway, CLI, Python and TypeScript SDKs, Terraform provider, and terminal workspace. Those surfaces are sufficient to self-operate InferCrane and do not depend on the hosted console. The browser console source is maintained as a separate company-operated service and is not part of the Apache-2.0 repository.Hosted identity and workspace provisioning
Hosted mode uses Clerk for sign-in, sessions, and organization selection. The Go control plane then independently requires a mapped InferCrane user, organization membership, role/scopes, and activeweb_console_access entitlement.
Configure the Go verifier with the Clerk instance’s HTTPS issuer and public JWT verification key.
Use the inline value when the deployment platform stores environment secrets, or the file value
when it mounts secrets; never configure both:
Durable work
Closing the browser does not cancel deployment work. Reopen the console or resume from any terminal:Terminal equivalent
For headless, low-bandwidth, or incident use, the terminal surfaces remain first-class:npm run test:e2e:empty-control-plane journey starts with a new PostgreSQL
volume and verifies this first-run flow through the real control API. It is the regression boundary
for empty states and browser-to-control-plane wiring; the hosted identity provider and real GPU/cloud
boundaries remain separate release qualifications.