Skip to main content

​
AWS EC2 BYOC

The AWS adapter provisions one EC2 instance per durable replica intent through Provider Contract V1. InferCrane assumes a customer role, uses explicit private networking, and adopts resources by idempotency token and ownership tags after uncertain responses. This is intentionally narrow. It is not EKS, SageMaker, automatic instance selection, public-IP bootstrap, or a general AWS abstraction.

​
Prerequisites

  • AWS CLI v2 on the control-plane host
  • a source identity permitted to call sts:AssumeRole
  • a role trust policy scoped to that source identity and, preferably, an external ID
  • an explicit private subnet and security group path between the control plane and worker port 8000
  • a GPU-compatible AMI containing NVIDIA drivers, Docker, and AWS CLI v2
  • an EC2 instance profile permitted to read exactly the worker API-key secret
  • an OCI runtime image pinned by sha256 digest
The assumed control-plane role needs the narrow EC2 describe, run, tag, and terminate permissions for InferCrane-owned instances. Restrict resources and tag conditions according to your AWS account policy.

​
Control-plane configuration

Configure the complete set. Partial configuration fails startup rather than silently disabling safety controls.
The Secrets Manager value referenced by INFERCRANE_AWS_WORKER_SECRET_ARN must contain the same worker credential configured as INFERCRANE_API_KEY on this self-hosted control plane. InferCrane uses that value for private worker health checks, routing, and explicit candidate validation; it persists only the secret ARN in provider metadata. INFERCRANE_AWS_IMAGE_DIGEST is the configured vLLM default and remains required for the adapter’s complete startup configuration. An SGLang or custom OCI revision supplies its own immutable image and argv; the EC2 adapter uses that revision workload instead of the vLLM default. It still uses the same private network, instance profile, worker secret and ownership tags. Validate role assumption without creating a resource:

​
Deploy

The requested region and GPU must exactly match the configured, qualified instance profile:
Closing the terminal does not stop provisioning. Resume with the operation ID printed by deploy. Deletion terminates only instances carrying InferCrane ownership tags and the persisted replica key. For the simulated portable-runtime path, apply examples/sglang.yaml or replace the placeholder image in examples/custom-oci.yaml. Real GPU compatibility remains a deferred manual qualification gate.

​
Security and accounting behavior

STS credentials are short lived and exist only in the child AWS CLI process environment. InferCrane does not persist or return them. Workers retrieve their API key directly from Secrets Manager through the instance profile. EC2 is launched without a public IP. Cost is reported as unknown. InferCrane does not ship a live AWS pricing catalog and will not infer cost from an instance-type name.

​
Qualification state

Hermetic contract tests cover create-response loss, adoption, replay, deletion, tag-scoped inventory, private networking, immutable images, and credential redaction. Real AWS provisioning remains deferred until consolidated manual qualification; inspect infercrane integrations for the exact state.