vLLM
vLLM is InferCrane’s default qualified runtime profile. InferCrane owns deployment state and lifecycle; vLLM owns model execution. A supervised vLLM Router process distributes requests across healthy standalone replicas.Connect existing workers
Each worker must expose the OpenAI-compatible vLLM API and be reachable from the InferCrane gateway.Responsibility boundary
InferCrane does not implement an inference engine or distributed KV cache. SGLang and custom OCI
workloads use the same versioned runtime boundary and have independent qualification states.
Gateway and routing
Follow an OpenAI request from alias resolution to a healthy vLLM replica.