Python SDK

The zero-dependency Python SDK uses the authenticated control-plane API. It does not connect to PostgreSQL, invoke infrastructure providers, or implement deployment lifecycle logic in the client.

Install from the repository

Install the SDK from the same checked-out InferCrane tag as your control plane until a matching published Python distribution is listed in the release notes.

python -m pip install ./sdk/python

Submit durable work

from infercrane import InferCrane

client = InferCrane(
    api_key="...",
    base_url="https://infercrane.internal",
)

operation = client.deploy(
    name="qwen-prod",
    model="Qwen/Qwen3-8B",
    cloud="runpod",  # use a provider qualified by your installation
    gpu="L40S",
    min_replicas=1,
    max_replicas=4,
    idempotency_key="qwen-prod-initial",
)

ready = client.wait(operation.id, timeout=900)

A wait timeout raises OperationTimeout and includes the operation ID. It does not call the cancel endpoint. A later process can resume with client.wait(operation_id).

for event in client.stream_chat(
    "qwen-prod",
    [{"role": "user", "content": "Summarize durable operations."}],
):
    print(event)

from infercrane import AsyncInferCrane

client = AsyncInferCrane(api_key="...", base_url="https://infercrane.internal")
operation = await client.deploy(
    name="qwen-prod",
    model="Qwen/Qwen3-8B",
    cloud="runpod",
    gpu="L40S",
)
ready = await client.wait(operation.id, timeout=900)

The stream parser yields each SSE JSON event once and requires [DONE]. Closing the iterator closes the HTTP response; it never replays a partially transmitted inference request.

Evaluate inference evidence

client.set_slo_policy(
    "qwen-prod",
    max_ttft_p95_ms=250,
    max_error_rate=0.01,
)
recommendation = client.recommend("qwen-prod")
history = client.recommendations("qwen-prod", limit=20)

The async client exposes the same SLO and recommendation methods. Recommendations are persisted, advisory decisions; these calls never create or resize provider resources.

Errors

Error	Meaning
`APIError`	Typed status, code, retryability, and remediation from the control API
`OperationFailed`	Persisted durable operation reached `failed`
`OperationCancelled`	Persisted operation reached `cancelled`
`OperationTimeout`	This client stopped waiting; server-side work continues
`StreamError`	Malformed or incomplete SSE stream

The generated low-level client is available as client.api. Generated files carry a do-not-edit header and are checked against openapi.json during qualification.

GET STARTED

BUILD

OPERATE

INTELLIGENCE

INTEGRATE

REFERENCE

PROJECT

Install from the repository

Submit durable work

Evaluate inference evidence

Errors

​Install from the repository

​Submit durable work

​Evaluate inference evidence

​Errors

Install from the repository

Submit durable work

Evaluate inference evidence

Errors