Skip to main content

​
ADR 0033: Replaceable external composition contracts

Status: Accepted Supersedes the implementation-status and verification details of ADR 0032. The ownership boundary established there remains unchanged.

​
Context

Inference teams already use systems such as LiteLLM for provider translation, sandbox products for agent execution, and MLflow or Kubeflow-connected pipelines for training. Requiring those teams to replace working systems would slow adoption. Treating those systems as InferCrane providers or runtimes would also make the lifecycle core depend on unrelated vendor semantics. The integration must therefore be useful today while preserving one concern per owner.

​
Decision

InferCrane registers a versioned infercrane.composition/v1 contract alongside, but separate from, provider and runtime contracts. A composition profile declares:
  • its kind: gateway, sandbox, or training;
  • the external owner and exact ownership boundary;
  • supported, unsupported, and unknown capabilities;
  • local and real-system qualification evidence.
The first three profiles are: An inference token issued for a sandbox:
  • can call and enumerate only its allowed endpoint alias;
  • cannot access the control API;
  • expires within 24 hours;
  • is shown only when issued or rotated;
  • can be revoked without deleting or mutating the external sandbox.
A training handoff:
  • uses the signed evidence envelope and Ed25519 verification boundary;
  • contains provenance and digests, not dataset, prompt, output, log, or checkpoint bytes;
  • binds to exactly one existing immutable deployment revision;
  • becomes a ModelArtifact only after signature and tenant checks pass;
  • cannot replace a different artifact already bound to that revision.

​
Consequences

  • Teams can adopt InferCrane without replacing LiteLLM, their sandbox vendor, or their training platform.
  • Application code continues using one stable InferCrane endpoint while external implementations remain replaceable.
  • The console can manage InferCrane-side access and evidence without receiving provider credentials, training keys, commands, or model content.
  • Sandbox lifecycle and training execution remain explicitly unsupported. Supporting them later requires a separate adapter contract and qualification decision.
  • Real LiteLLM behavior, provider secret injection, and registry availability remain system-specific qualification tasks; local tests do not upgrade those claims.

​
Alternatives rejected

  • Forking LiteLLM would create licensing, security-update, and lifecycle ownership that InferCrane does not need.
  • Building a sandbox runtime would duplicate specialist isolation systems and create a dangerous code-execution boundary.
  • Building a training scheduler would turn InferCrane into a workflow engine.
  • Issuing a tenant-wide token to every sandbox violates least privilege.
  • Accepting unsigned checkpoint URLs allows mutable or cross-tenant artifact substitution.

​
Verification

  • TestEndpointRestrictedInferenceCredentialCannotEnumerateOrInvokeOtherAlias proves endpoint allow-list enforcement in the gateway.
  • TestSandboxReferenceIssuesEndpointRestrictedExpiringCredential proves expiry, rotation, revocation, nested content rejection, and tenant-scoped persistence.
  • TestTrainingHandoffIsRevisionBoundImmutableAndTenantSafe proves immutable, idempotent, tenant-safe attachment.
  • TestSignedHandoffRoundTripAndTamperRejection proves signature and payload tamper rejection.
  • scripts/product-acceptance.sh modules exercises the CLI, public gateway, control API, credential lifecycle, signed handoff, and list surfaces as a black-box product journey.
See LiteLLM, External sandboxes, and Training artifact handoffs.