Python SDK

The zero-dependency Python SDK uses the authenticated control-plane API. It does not connect to PostgreSQL, invoke infrastructure providers, or implement deployment lifecycle logic in the client.

Install the public beta

python -m pip install \
  'https://github.com/infercrane/infercrane/releases/download/v1.0.0-rc.1/infercrane-1.0.0rc1-py3-none-any.whl'

Use the SDK version from the same InferCrane release as the control plane.

Submit durable work

from infercrane import InferCrane

client = InferCrane(
    api_key="...",
    base_url="https://infercrane.internal",
)

operation = client.deploy(
    name="qwen-prod",
    model="Qwen/Qwen3-8B",
    cloud="runpod",  # use a provider qualified by your installation
    gpu="L40S",
    min_replicas=1,
    max_replicas=4,
    idempotency_key="qwen-prod-initial",
)

ready = client.wait(operation.id, timeout=900)

A wait timeout raises OperationTimeout and includes the operation ID. It does not call the cancel endpoint. A later process can resume with client.wait(operation_id).

for event in client.stream_chat(
    "qwen-prod",
    [{"role": "user", "content": "Summarize durable operations."}],
):
    print(event)

from infercrane import AsyncInferCrane

client = AsyncInferCrane(api_key="...", base_url="https://infercrane.internal")
operation = await client.deploy(
    name="qwen-prod",
    model="Qwen/Qwen3-8B",
    cloud="runpod",
    gpu="L40S",
)
ready = await client.wait(operation.id, timeout=900)

The stream parser yields each SSE JSON event once and requires [DONE]. Closing the iterator closes the HTTP response; it never replays a partially transmitted inference request.

Evaluate inference evidence

client.set_slo_policy(
    "qwen-prod",
    max_ttft_p95_ms=250,
    max_error_rate=0.01,
)
recommendation = client.recommend("qwen-prod")
history = client.recommendations("qwen-prod", limit=20)

The async client exposes the same SLO and recommendation methods. Recommendations are persisted, advisory decisions; these calls never create or resize provider resources.

Errors

Error	Meaning
`APIError`	Typed status, code, retryability, and remediation from the control API
`OperationFailed`	Persisted durable operation reached `failed`
`OperationCancelled`	Persisted operation reached `cancelled`
`OperationTimeout`	This client stopped waiting; server-side work continues
`StreamError`	Malformed or incomplete SSE stream

The generated low-level client is available as client.api. Generated files carry a do-not-edit header and are checked against openapi.json during qualification.

Start here

Gateways and external systems

SDKs and automation

Providers

Runtimes and evidence tools

Install the public beta

Submit durable work

Evaluate inference evidence

Errors

​Install the public beta

​Submit durable work

​Evaluate inference evidence

​Errors

Install the public beta

Submit durable work

Evaluate inference evidence

Errors