Skip to main content

​
Treat model locality as evidence

Large model artifacts often dominate readiness time. InferCrane records cache state through a neutral contract while each provider adapter retains its native mechanism: a volume or provider cache, an AWS volume or snapshot, a Kubernetes volume or node cache, or another qualified implementation.
The result separates observations from requested work and shows whether an observation is still fresh.

​
Request prefetch

This persists durable intent. When an adapter for the named provider is configured, InferCrane ensures that intent with a stable idempotency key and checkpoints the provider operation identity. If a provider response is lost, a retry may repeat the API call with the same key and adopt the same logical operation; it must not create a second cache resource. If no adapter is configured, the response says execution: not_configured instead of implying that warming began. Submission is not proof that the artifact is present. The adapter must publish a fresh observation before a planner can rely on locality.

​
Record an adapter observation

Valid states are present, prefetching, missing, and unknown. Expiry prevents stale provider state from becoming a permanent planning assumption.
Cache population remains adapter-specific and requires qualification on the selected infrastructure. requested or running describes work, while only a fresh present observation describes locality.

​
Why prefetch is not a universal download command

Providers expose different ownership boundaries. For example, RunPod Serverless configures one cached model on an endpoint, while an EC2 or Kubernetes deployment commonly obtains locality from a persistent volume, snapshot, node cache, or image strategy owned by that workload. Those are not interchangeable APIs. InferCrane therefore does not download model weights through a custom protocol and does not mark a prefetch request as a cache hit. An adapter may consume the durable intent using its native mechanism, but it must publish a bounded fresh observation before planning or placement can rely on locality. Until then, infercrane plan reports artifact cache and startup time as unknown and unavailable.