Skip to main content

Connect LiteLLM without replacing it

LiteLLM and InferCrane own different problems. LiteLLM translates provider protocols and holds its upstream credentials. InferCrane owns the stable application endpoint, adoption state, persisted request evidence, and safe release decisions.

Connect in observe-only mode

InferCrane verifies the OpenAI-compatible discovery surface before persisting the connection. It does not install, configure, upgrade, or fork LiteLLM.
The private console exposes the same path under New inference → Connect existing inference. Choose LiteLLM as the connector. Connections start observe-only; traffic ownership never transfers silently.

Promote traffic ownership explicitly

After you have inspected health and evidence:
The application continues calling:

Ownership and qualification

The OpenAI-compatible behavior is locally qualified with hermetic fixtures. Real LiteLLM versions, plugins, upstream providers, and credential policy still require qualification in your environment.

Why not bundle LiteLLM?

Bundling would make InferCrane responsible for another gateway’s security patches, license boundary, provider catalog, and upgrades. The composition contract keeps that dependency replaceable and lets teams use LiteLLM, another gateway, or InferCrane’s native data plane without changing the logical endpoint contract.