Replay and capacity intelligence
Inference Replay records the timing and token-count shape of production traffic without storing prompt or output bodies. Session, parent-session, and shared-prefix identities are tenant-secret HMACs, not raw identifiers. A trace records its window, sample count, schema, canonical digest, and evidence class.--max-requests requests, newest first. When the window held more, the
trace sets request_limit_reached to true: its requests cover less time than the window. It is
false when every request in the window was kept, and null for a trace captured before this was
recorded. An optimization report uses it to project monthly cost over the time the kept requests
cover.
This command only captures persisted observations. It sends no workload and creates no infrastructure.
To approximate the captured mean token sizes and peak concurrency with AIPerf, opt in explicitly:
mooncake_trace, and it keeps requests that share system messages, or continue a session, sharing
their leading prompt blocks. Prompt content is still synthetic and arrival timing is still not replayed.
Capacity intelligence
Artifact placement
The artifact cache contract records short-lived provider-native observations (present, prefetching,
missing, or unknown). Prefetch requests are durable and idempotent, but execution belongs to the
selected infrastructure adapter. InferCrane does not implement a model CDN or download protocol.
The AWS adapter verifies immutable encrypted EBS snapshots. The GCP adapter verifies zonal
Persistent Disks whose description binds the exact model-identity digest. The Kubernetes adapter
verifies bound, multi-reader PVCs with the same identity binding. All three attach or mount storage
read-only; real provider cache-hit timing and node-local Kubernetes cache execution remain separate
qualification work.