Security
InferCrane authenticates the control and data planes with bearer credentials and tenant-scopes public API reads and writes. Use separate, least-privilege provider credentials and rotate them through your secret manager. Production PostgreSQL must use TLS; back up and restore it as the lifecycle source of truth. For the path from a local proof to a shared endpoint, pair this page with Share a development endpoint safely, endpoint admission and distributed tenant quotas, protocol qualification, and the Request Inspector contract. The identity boundary is a tenant. Principals are service accounts with a role ceiling and explicit action scopes. A scope can remove a role permission but cannot add one. Credentials are shown once, stored only as SHA-256 digests, and support rotation and revocation. Existing legacy principals are migrated to their previous explicit action set; new secret and external-capacity permissions are never granted implicitly during upgrade.infercrane login issues a CLI key only after a person approves the sign-in in the dashboard
(Sign in the CLI). The two sign-in endpoints that the CLI calls before it
has a key are unauthenticated and bounded: at most 500 sign-ins wait for approval at once, each
expires after 10 minutes, polling faster than every 2 seconds is told to slow down, and finished
requests are pruned after 24 hours. The CLI’s device code is a 256-bit secret stored only as a
SHA-256 digest. Approval records the approver’s tenant, role, and scopes; the key is minted on the
CLI’s next poll in the same transaction that consumes the request, so one approval yields at most one
key. The key is an ordinary principal of kind cli_session with the approver’s role and scopes,
never more, and expires after 90 days. Only dashboard principals (people, service accounts, and the
bootstrap key) can approve; CLI keys and endpoint tokens cannot. Keys are revoked by
infercrane logout or in Settings → CLI sessions. Last use is recorded on the control API at most
once a minute per key and never on the inference path. Removing an organization member does not
revoke their CLI keys; revoke them in Settings → CLI sessions. The CLI keeps the key in the OS
keychain, written on standard input rather than the command line, and falls back to its 0600
config file.
The InferCrane agent observes model servers on a customer’s own machines
without any inbound access: it only makes outbound HTTPS calls to /api/v1/agent/sync and
/api/v1/agent/servers, and sends its key only over HTTPS (plain HTTP only to a control plane on
the same machine). An agent key is a principal of kind agent, minted with the deploy action and
shown once. It is refused on every other control-plane route and on the inference gateway, so a
leaked agent key can register at most 32 observe-only endpoints and report about them, nothing
else. The control plane never dials, probes, or routes to a server it learned through an agent. The
agent filters engine metrics on the machine to the families InferCrane parses and never sends
prompts, completions, headers, or a server’s API key, which it sends only to the servers named
with --server. Request hashes from its optional local gateway are keyed with a random secret
created on the machine and never sent, independent of the agent key. Revoking an agent disables
its principal and is checked on every agent request, so the agent stops at its next sync. Agent keys do not expire; rotate them by creating a new agent and revoking the
old one.
infercrane connect --token-env VAR sends a server’s API key to the control plane once. It is
stored only as AES-256-GCM ciphertext bound to the workspace and endpoint, sealed with
INFERCRANE_CREDENTIAL_ENCRYPTION_KEY, never returned, and used only for discovery and health
probes of that server. Without that key the control plane refuses the credential, and the agent,
which keeps the key on the server’s machine, is the path.
Compute configured on the control-plane process (cloud workload identity, a kubeconfig context,
SkyPilot manifests) is not tenant-scoped. Only the bootstrap operator may deploy or optimize on it
without a tenant compute connection, unless INFERCRANE_OPERATOR_COMPUTE_TENANTS names the
workspace or is *. The wildcard is refused when the control plane sells managed capacity or
auto-provisions workspaces at sign-up. The grant never extends to InferCrane-managed prepaid
capacity or the RunPod supply key. See
Let workspaces use operator compute.
Gateway authentication uses an in-memory last-known-good credential snapshot so an established data
plane does not add PostgreSQL latency or fail immediately during a database outage. Rotation and
revocation take effect on the next successful snapshot refresh. If immediate revocation is required
while PostgreSQL is unreachable, restore authoritative database access or fence/restart the affected
gateway instances; InferCrane cannot infer a revocation it cannot read.
Secret objects are references, not a secret store. For example:
env) and reference (OPENROUTER_API_KEY), never the environment
value. Resolved values stay in process memory and are excluded from API responses, logs, audit
payloads and qualification evidence. Operators should inject referenced variables from their
existing secret manager and restrict the control-plane process environment.
An env reference reads the control-plane process environment, which also holds the operator’s own
keys. So only the operator’s workspaces may create or resolve one: the workspace of the operator API
key, the Model API operator workspace (INFERCRANE_MODEL_API_OPERATOR_TENANT_ID), and the workspaces
named in INFERCRANE_ENV_SECRET_TENANTS. Any other workspace gets env_secret_reference_not_allowed
when it creates one, and a reference it already holds stops resolving instead of returning a value.
Left unset, INFERCRANE_ENV_SECRET_TENANTS is * on a control plane that serves one customer and
empty where it sells managed capacity or provisions workspaces at sign-up; there, startup also refuses
*. Workspaces on such a control plane keep their provider keys in their own compute connection.
Validate provider access without enabling traffic
Infrastructure adapters expose provider-specific read-only checks:/v1/models discovery request. This verifies reachability and metadata access without enabling
traffic or spend; it is not a generation probe. Validate real serving through an explicitly approved,
budgeted staging request. Never put a credential in a URL, CLI argument value, DeploymentSpec,
ticket, log, or qualification artifact.
Prompt and output content are not recorded by default. Request telemetry stores identifiers, dimensions, status, timing, and token counts. AIPerf uses metrics-only record exports; InferCrane does not upload benchmark data. Shadow traffic is not implemented, so requests are never duplicated silently.
The built-in answer check of an optimization run sends a small fixed prompt suite to candidate and
reference revisions and compares the answers in memory; answers are never stored. A customer eval
set is the one place InferCrane keeps prompts you choose to give it: infercrane evaluation set upload stores the uploaded file in plaintext, bounded to 512 KiB and 200 prompts, scoped to your
organization, and readable only by the worker. The API returns its metadata, never its prompts, and
evaluation set delete removes it. Do not upload prompts containing personal data or secrets.
Durable async inference is the explicit exception. It remains disabled until an operator injects
INFERCRANE_ASYNC_ENCRYPTION_KEY, and every submission must acknowledge encrypted content storage.
Request and result bodies are stored only as AES-256-GCM ciphertext bound to tenant and job identity.
Completion webhooks are signed, HTTPS-only, redirect-free and resolved through an SSRF-hardened
transport. Webhook payloads contain results, so their destinations are part of the application data
boundary. Synchronous inference remains content-free by default.
Tenant compute credentials are a second explicit encrypted boundary. Self-serve BYOC remains
disabled until the operator injects separate INFERCRANE_CREDENTIAL_ENCRYPTION_KEY and
INFERCRANE_WORKER_CREDENTIAL_KEY values. A submitted RunPod key
is verified before storage, encrypted with AES-256-GCM, and bound through associated data to the
tenant, provider, and connection name. API responses expose only connection metadata. Provider
calls decrypt the credential in memory, while worker authentication uses a separate deterministic
tenant-and-deployment-scoped credential. Operator-owned provider keys, volumes, and provider secret
references are never copied into customer accounts.
Optimization run notification destinations use the same key. A destination URL can itself be a
bearer credential (a Slack incoming webhook is one), so the URL and the generated signing secret are
both stored as AES-256-GCM ciphertext bound to the tenant and destination ID. API responses show only
the host and the last characters of the path; the signing secret is returned once, at creation.
Deliveries go only to public HTTPS addresses through the same checked transport as signed alerts, and
carry run metadata and outcome summaries, never prompts, responses, or credentials. Changing a
destination requires an admin principal with the manage_secrets scope.
External fallback is disabled by default. Enabling it requires a persisted acknowledgement that
prompt and output data can leave controlled infrastructure, plus atomic hard request and worst-case
cost reservations. Selection happens before transmission; InferCrane does not replay a stream or
retry after bytes may have reached an external provider.
The complete commands are in Keep sensitive inputs self-hosted
and Stable endpoints. A provider connection
alone enables neither traffic nor spend. If data residency cannot be proven, keep a one-binding
manual plan containing only self-hosted capacity. If trustworthy external pricing or a defensible
worst-case per-request reservation is unavailable, do not pass --enable-external; cost limits are
authorization ceilings, not provider invoice evidence.
AWS BYOC uses STS role assumption for each provider call and passes temporary credentials only to
the child AWS CLI process. EC2 receives no public IP. The worker retrieves its API key from AWS
Secrets Manager through a narrowly scoped instance profile; the control plane stores the secret ARN,
not the value. Require an external ID, restrict trust and permissions, and use tag conditions in
production.
The Kubernetes adapter always uses an explicit context and namespace. Its reference Role contains no
wildcards and cannot read Secrets, mutate RBAC, or create Pods directly. Workers read one named Secret
through their service account; the control plane stores only its name and key. Strict server-side
apply preserves field ownership and never force-steals conflicts. InferCrane deletes only exact names
with matching durable ownership labels and annotations.
The control plane contacts huggingface.co over HTTPS when an authenticated principal searches
Hugging Face models (GET /api/v1/catalog/hugging-face/search). It sends only the search text, the
pipeline filter, and a fixed User-Agent, never a Hugging Face token or tenant data. If your network
policy blocks that destination, the search returns a typed hugging_face_search_unavailable error
and nothing else changes. See Search Hugging Face.
When the operator sets INFERCRANE_HUGGING_FACE_METADATA_ENABLED=true, the control plane also reads
model info, config.json, and the safetensors summary from huggingface.co to check a model outside
the reviewed catalog (planning, GET /api/v1/catalog/hugging-face/check, and optimization
proposals). It is off by default. For a gated or private model, the workspace’s saved Hugging Face
token reference is resolved at that read and sent only as an Authorization header to the same host:
redirects to another host are refused, and the token is never logged, cached, stored, or returned.
Reads made with a token are cached per workspace and credential. See
Check any model automatically.
Run the container as its non-root user, pin immutable image tags, restrict network access, and protect /metrics according to your environment. Report vulnerabilities through the repository’s private GitHub security-advisory flow; do not open a public issue.