Skip to main content

Security

InferCrane authenticates the control and data planes with bearer credentials and tenant-scopes public API reads and writes. Use separate, least-privilege provider credentials and rotate them through your secret manager. Production PostgreSQL must use TLS; back up and restore it as the lifecycle source of truth. For the path from a local proof to a shared endpoint, pair this page with Share a development endpoint safely, endpoint admission and distributed tenant quotas, protocol qualification, and the Request Inspector contract. The identity boundary is a tenant. Principals are service accounts with a role ceiling and explicit action scopes. A scope can remove a role permission but cannot add one. Credentials are shown once, stored only as SHA-256 digests, and support rotation and revocation. Existing legacy principals are migrated to their previous explicit action set; new secret and external-capacity permissions are never granted implicitly during upgrade. infercrane login issues a CLI key only after a person approves the sign-in in the dashboard (Sign in the CLI). The two sign-in endpoints that the CLI calls before it has a key are unauthenticated and bounded: at most 500 sign-ins wait for approval at once, each expires after 10 minutes, polling faster than every 2 seconds is told to slow down, and finished requests are pruned after 24 hours. The CLI’s device code is a 256-bit secret stored only as a SHA-256 digest. Approval records the approver’s tenant, role, and scopes; the key is minted on the CLI’s next poll in the same transaction that consumes the request, so one approval yields at most one key. The key is an ordinary principal of kind cli_session with the approver’s role and scopes, never more, and expires after 90 days. Only dashboard principals (people, service accounts, and the bootstrap key) can approve; CLI keys and endpoint tokens cannot. Keys are revoked by infercrane logout or in Settings → CLI sessions. Last use is recorded on the control API at most once a minute per key and never on the inference path. Removing an organization member does not revoke their CLI keys; revoke them in Settings → CLI sessions. The CLI keeps the key in the OS keychain, written on standard input rather than the command line, and falls back to its 0600 config file. The InferCrane agent observes model servers on a customer’s own machines without any inbound access: it only makes outbound HTTPS calls to /api/v1/agent/sync and /api/v1/agent/servers, and sends its key only over HTTPS (plain HTTP only to a control plane on the same machine). An agent key is a principal of kind agent, minted with the deploy action and shown once. It is refused on every other control-plane route and on the inference gateway, so a leaked agent key can register at most 32 observe-only endpoints and report about them, nothing else. The control plane never dials, probes, or routes to a server it learned through an agent. The agent filters engine metrics on the machine to the families InferCrane parses and never sends prompts, completions, headers, or a server’s API key, which it sends only to the servers named with --server. Request hashes from its optional local gateway are keyed with a random secret created on the machine and never sent, independent of the agent key. Revoking an agent disables its principal and is checked on every agent request, so the agent stops at its next sync. Agent keys do not expire; rotate them by creating a new agent and revoking the old one. infercrane connect --token-env VAR sends a server’s API key to the control plane once. It is stored only as AES-256-GCM ciphertext bound to the workspace and endpoint, sealed with INFERCRANE_CREDENTIAL_ENCRYPTION_KEY, never returned, and used only for discovery and health probes of that server. Without that key the control plane refuses the credential, and the agent, which keeps the key on the server’s machine, is the path. Compute configured on the control-plane process (cloud workload identity, a kubeconfig context, SkyPilot manifests) is not tenant-scoped. Only the bootstrap operator may deploy or optimize on it without a tenant compute connection, unless INFERCRANE_OPERATOR_COMPUTE_TENANTS names the workspace or is *. The wildcard is refused when the control plane sells managed capacity or auto-provisions workspaces at sign-up. The grant never extends to InferCrane-managed prepaid capacity or the RunPod supply key. See Let workspaces use operator compute. Gateway authentication uses an in-memory last-known-good credential snapshot so an established data plane does not add PostgreSQL latency or fail immediately during a database outage. Rotation and revocation take effect on the next successful snapshot refresh. If immediate revocation is required while PostgreSQL is unreachable, restore authoritative database access or fence/restart the affected gateway instances; InferCrane cannot infer a revocation it cannot read. Secret objects are references, not a secret store. For example:
PostgreSQL stores the resolver (env) and reference (OPENROUTER_API_KEY), never the environment value. Resolved values stay in process memory and are excluded from API responses, logs, audit payloads and qualification evidence. Operators should inject referenced variables from their existing secret manager and restrict the control-plane process environment. An env reference reads the control-plane process environment, which also holds the operator’s own keys. So only the operator’s workspaces may create or resolve one: the workspace of the operator API key, the Model API operator workspace (INFERCRANE_MODEL_API_OPERATOR_TENANT_ID), and the workspaces named in INFERCRANE_ENV_SECRET_TENANTS. Any other workspace gets env_secret_reference_not_allowed when it creates one, and a reference it already holds stops resolving instead of returning a value. Left unset, INFERCRANE_ENV_SECRET_TENANTS is * on a control plane that serves one customer and empty where it sells managed capacity or provisions workspaces at sign-up; there, startup also refuses *. Workspaces on such a control plane keep their provider keys in their own compute connection.

Validate provider access without enabling traffic

Infrastructure adapters expose provider-specific read-only checks:
For an external model API, create only a reference-backed connection and inspect its redacted metadata:
Creating and listing the connection does not send a prompt, test request, or authorize spend. It proves that InferCrane can resolve the named reference, not that the upstream model can serve a real prompt. Connecting an existing endpoint can use that reference for one bounded authenticated /v1/models discovery request. This verifies reachability and metadata access without enabling traffic or spend; it is not a generation probe. Validate real serving through an explicitly approved, budgeted staging request. Never put a credential in a URL, CLI argument value, DeploymentSpec, ticket, log, or qualification artifact. Prompt and output content are not recorded by default. Request telemetry stores identifiers, dimensions, status, timing, and token counts. AIPerf uses metrics-only record exports; InferCrane does not upload benchmark data. Shadow traffic is not implemented, so requests are never duplicated silently. The built-in answer check of an optimization run sends a small fixed prompt suite to candidate and reference revisions and compares the answers in memory; answers are never stored. A customer eval set is the one place InferCrane keeps prompts you choose to give it: infercrane evaluation set upload stores the uploaded file in plaintext, bounded to 512 KiB and 200 prompts, scoped to your organization, and readable only by the worker. The API returns its metadata, never its prompts, and evaluation set delete removes it. Do not upload prompts containing personal data or secrets. Durable async inference is the explicit exception. It remains disabled until an operator injects INFERCRANE_ASYNC_ENCRYPTION_KEY, and every submission must acknowledge encrypted content storage. Request and result bodies are stored only as AES-256-GCM ciphertext bound to tenant and job identity. Completion webhooks are signed, HTTPS-only, redirect-free and resolved through an SSRF-hardened transport. Webhook payloads contain results, so their destinations are part of the application data boundary. Synchronous inference remains content-free by default. Tenant compute credentials are a second explicit encrypted boundary. Self-serve BYOC remains disabled until the operator injects separate INFERCRANE_CREDENTIAL_ENCRYPTION_KEY and INFERCRANE_WORKER_CREDENTIAL_KEY values. A submitted RunPod key is verified before storage, encrypted with AES-256-GCM, and bound through associated data to the tenant, provider, and connection name. API responses expose only connection metadata. Provider calls decrypt the credential in memory, while worker authentication uses a separate deterministic tenant-and-deployment-scoped credential. Operator-owned provider keys, volumes, and provider secret references are never copied into customer accounts. Optimization run notification destinations use the same key. A destination URL can itself be a bearer credential (a Slack incoming webhook is one), so the URL and the generated signing secret are both stored as AES-256-GCM ciphertext bound to the tenant and destination ID. API responses show only the host and the last characters of the path; the signing secret is returned once, at creation. Deliveries go only to public HTTPS addresses through the same checked transport as signed alerts, and carry run metadata and outcome summaries, never prompts, responses, or credentials. Changing a destination requires an admin principal with the manage_secrets scope. External fallback is disabled by default. Enabling it requires a persisted acknowledgement that prompt and output data can leave controlled infrastructure, plus atomic hard request and worst-case cost reservations. Selection happens before transmission; InferCrane does not replay a stream or retry after bytes may have reached an external provider. The complete commands are in Keep sensitive inputs self-hosted and Stable endpoints. A provider connection alone enables neither traffic nor spend. If data residency cannot be proven, keep a one-binding manual plan containing only self-hosted capacity. If trustworthy external pricing or a defensible worst-case per-request reservation is unavailable, do not pass --enable-external; cost limits are authorization ceilings, not provider invoice evidence. AWS BYOC uses STS role assumption for each provider call and passes temporary credentials only to the child AWS CLI process. EC2 receives no public IP. The worker retrieves its API key from AWS Secrets Manager through a narrowly scoped instance profile; the control plane stores the secret ARN, not the value. Require an external ID, restrict trust and permissions, and use tag conditions in production. The Kubernetes adapter always uses an explicit context and namespace. Its reference Role contains no wildcards and cannot read Secrets, mutate RBAC, or create Pods directly. Workers read one named Secret through their service account; the control plane stores only its name and key. Strict server-side apply preserves field ownership and never force-steals conflicts. InferCrane deletes only exact names with matching durable ownership labels and annotations. The control plane contacts huggingface.co over HTTPS when an authenticated principal searches Hugging Face models (GET /api/v1/catalog/hugging-face/search). It sends only the search text, the pipeline filter, and a fixed User-Agent, never a Hugging Face token or tenant data. If your network policy blocks that destination, the search returns a typed hugging_face_search_unavailable error and nothing else changes. See Search Hugging Face. When the operator sets INFERCRANE_HUGGING_FACE_METADATA_ENABLED=true, the control plane also reads model info, config.json, and the safetensors summary from huggingface.co to check a model outside the reviewed catalog (planning, GET /api/v1/catalog/hugging-face/check, and optimization proposals). It is off by default. For a gated or private model, the workspace’s saved Hugging Face token reference is resolved at that read and sent only as an Authorization header to the same host: redirects to another host are refused, and the token is never logged, cached, stored, or returned. Reads made with a token are cached per workspace and credential. See Check any model automatically. Run the container as its non-root user, pin immutable image tags, restrict network access, and protect /metrics according to your environment. Report vulnerabilities through the repository’s private GitHub security-advisory flow; do not open a public issue.