Deployment guide
Deployment guide
Section titled “Deployment guide”This guide covers deploying ABCA into an AWS account, including compute backend choices, scale-to-zero characteristics, and the complete AWS service inventory. For day-to-day development workflow, see the Developer guide. For a quick first deployment, see the Quick start. For least-privilege IAM deployment roles, see DEPLOYMENT_ROLES.md.
Architecture overview
Section titled “Architecture overview”ABCA deploys from the backgroundagent-dev application stack with nested stacks for selected subsystems. Networking stays in that stack by default; networkTopology=split gives it a separate top-level stack. A deployment can provision one or more compute backends:
| Aspect | AgentCore (default) | ECS Fargate (opt-in) | Lambda MicroVMs (experimental) |
|---|---|---|---|
| Compute | Bedrock AgentCore Runtime (Firecracker MicroVMs) | ECS Fargate containers | AWS Lambda MicroVMs |
| Resources | 2 vCPU, 8 GB RAM, 2 GB max image size | 2 vCPU, 4 GB RAM | 8 GB baseline / 32 GB peak memory |
| Orchestration | Durable Lambda (checkpoint/replay) | Same durable Lambda via ComputeStrategy | Same durable Lambda via ComputeStrategy |
| Agent mode | FastAPI server (HTTP invocation) | Batch (run-to-completion) | FastAPI server (lifecycle hooks) |
| Startup | ~10s (warm MicroVM) | ~60-180s (Fargate cold start) | ~6s to RUNNING (live-measured) |
| Max duration | 8 hours (AgentCore service limit) | 9 hours (orchestrator executionTimeout) | 8 hours (maximumDurationInSeconds) |
All backends are orchestrated by the same durable Lambda function. The ComputeStrategy interface abstracts startSession(), pollSession(), and stopSession() — the ECS strategy calls ecs:RunTask / ecs:DescribeTasks / ecs:StopTask directly from the Lambda. No Step Functions are used.
AgentCore is the default. compute_types selects a comma-separated list (or a JSON array in cdk/cdk.json); its first entry is the repository default. For example, -c compute_types=agentcore,ecs deploys both, while -c compute_types=ecs deploys only ECS. Repository overrides can select any deployed backend. Memory, Gateway, Registry and the Linear vault remain independently configurable.
Without compute_types, legacy compute_type=ecs or compute_type=lambda-microvm keeps AgentCore alongside that backend, including on upgrade. An unchanged legacy context therefore does not remove AgentCore. Use the updated CLI for lists whose default is not AgentCore or that omit AgentCore; it reads the ordered ComputeTypes output. See the backend transition procedure before deliberately removing a backend or changing the default.
Network stack topology
Section titled “Network stack topology”networkTopology=inline is the default. Use networkTopology=split for a new environment to put the VPC, subnets, endpoints, flow logs and DNS firewall in ${stackName}-network. The application stack retains its name, data stores, compute resources and shared Task API. It depends on network exports, so CDK deploys the network first. The complete VPC/subnet/security-group export set stays present across compute-backend changes. Keep the same topology context on subsequent synth, diff and deploy commands.
Every local and pipeline synthesis enforces a 490-resource ceiling per parent or nested template, including operator configurations outside the census. CDK fails synthesis with the stack name, resource count and ceiling when a template exceeds it. @aws-cdk/core:stackResourceLimit accepts a stricter integer from 1 to 490, as either a JSON number or CLI string; it cannot raise the production ceiling.
The budget covers complete configurations, including multiple backends, Gateway, Registry, the Linear vault, managed MicroVM images, alert email, a fork Blueprint, explicit three-zone pins and expanded model allowlists. Wider inline combinations exceed 490 and fail synthesis; their split counterparts fit in the sampled matrix. The budget guard never changes topology automatically. The offline census reports counts for each supported profile.
For a new installation:
MISE_EXPERIMENTAL=1 mise //cdk:deploy -- --all \ -c networkTopology=split -c compute_types=agentcoreReplace the compute list as needed and supply image settings for MicroVM. Persist the selected context in cdk/cdk.json for subsequent synth, diff and deploy commands. The split preserves the supported-AZ policy, HTTPS egress, endpoints and DNS observation mode. Configure additional Blueprint domains in cdk/src/blueprints/definitions.ts; both stacks consume those inputs before repository resources are constructed.
Existing inline-to-split migration is deferred. Do not flip networkTopology on an existing inline deployment. An ordinary deploy creates a new VPC and removes old resources; identical logical IDs in different stacks do not preserve physical identity. Keep the existing topology until a separately reviewed migration has established resource-type eligibility, source retention and cleanup behavior, an ownership mapping, physical-ID/data preservation, and rollback on a populated disposable deployment. The cdk refactor --unstable=refactor criterion in #852 remains open; it is not waived or satisfied by synthesis tests.
This change does not install broad retention, change the Blueprint provider, create an ownership ledger, or replace the guardrail versioning scheme. Those prerequisites belong in separate releases. CloudFormation exports still prevent removing a network used by the application; switching back to inline is not an automatic rollback. See ADR-023’s deferred work.
Stacks used to test earlier revisions with blueprintProvisioning=prepare|adopt|managed or the broad retention aspect require a separate recovery plan before adopting this narrowed version. Returning those stacks directly to the original Blueprint provider can overwrite or soft-delete repository rows. Synthesis rejects the removed blueprintProvisioning and guardrailVersionMigration context keys instead of silently ignoring them; removing those keys does not make an experimental stack safe to update. Preserve its deployed templates, repository data and ledger inventory; the removed experimental modes are not an upgrade path supported by this release.
Reducing AZs in an existing split network
Section titled “Reducing AZs in an existing split network”A normal --all deployment updates the network first, so removing an AZ can fail because the old application still imports its private-subnet export. Reducing the count also shifts CDK’s private-subnet CIDRs unless the vacated address slot stays reserved. Use the following staged procedure for a three-to-two-zone reduction that keeps the first two existing AZs in their original order. Replacing or reordering AZs requires a separate network migration.
-
Pause automated deployments, task submissions, webhooks and scheduled work. Drain running and suspended sessions. Record the deployed templates, AZ order, subnet CIDRs, physical IDs and network exports. Keep the same account, region, stack identity, backend, image and Blueprint configuration throughout.
-
Persist the target AZ list in the existing
cdk/cdk.jsoncontext, keepingnetworkTopology=split. IncreasenetworkReservedAzsby the number of removed trailing AZs, so active plus reserved slots remains constant. For three active zones with no reservations, the target is two active zones and one reserved slot. These example names must match the deployment’s first two AZs:"agentcore:availabilityZones": ["us-east-1a", "us-east-1b"],"networkReservedAzs": 1networkReservedAzsaccepts an integer from 0 to 6, as a JSON number or CLI string; the default is 0. It reserves address space and creates no AWS resources. Keep this setting in every subsequent synth/deploy, including automation. -
Set
APP_STACKto the existing application stack name and review both target templates. The remaining subnets must keep their logical IDs, CIDRs and AZs; the target application must stop importing the removed subnet. Stop if the diff changes a retained subnet or any unrelated configuration.Terminal window APP_STACK=backgroundagent-devMISE_EXPERIMENTAL=1 mise //cdk:diff -- --all --method template -
Deploy only the application, leaving the existing three-zone network in place.
--exclusivelyprevents CDK from deploying its network dependency:Terminal window MISE_EXPERIMENTAL=1 mise //cdk:deploy -- "$APP_STACK" --exclusively -
Copy each removed private-subnet export’s exact name from the deployed network outputs. Verify that
list-importsreturns[]before changing the network. Any additional consumer stack must also release that export.Terminal window aws cloudformation describe-stacks --stack-name "${APP_STACK}-network" \--query 'Stacks[0].Outputs[].{ExportName:ExportName,Value:OutputValue}' --output tableREMOVED_SUBNET_EXPORT='<exact removed private-subnet export name>'aws cloudformation list-imports --export-name "$REMOVED_SUBNET_EXPORT" \--query Imports --output json -
Deploy both stacks with the same persisted target context. The network can now remove the unused export and AZ resources. Verify the remaining subnet physical IDs and CIDRs, DNS/egress behavior and a task before resuming producers and automation.
Terminal window MISE_EXPERIMENTAL=1 mise //cdk:deploy -- --all
To restore the previous three-zone layout, restore its AZ list and reservation count, then deploy the network before the application (--all uses this order). The network must recreate the third subnet and export before the application imports it again. Keep active plus reserved slots constant during this recovery too.
Local synthesis tests verify the import ordering and unchanged remaining subnet properties for all three backends. This procedure still requires a disposable AWS rehearsal before a production network update.
Lambda MicroVMs backend (experimental)
Section titled “Lambda MicroVMs backend (experimental)”Not for production.
lambda-microvmcarries no smoke-parity guarantee for an unattended deployment. Keep production repositories onagentcoreorecs. Synth emits an unsuppressible warning to this effect whenever the backend is selected. Design detail: COMPUTE.md and ADR-021.
Include it in the deployment’s backend list, preserving any backends already in use:
mise //cdk:deploy -- --context compute_types=agentcore,lambda-microvmYou must re-bootstrap first. This is the single most common way this backend fails, and the failure does not look like a configuration problem:
-
Check the bootstrap policy bundle already deployed in the account:
Terminal window aws cloudformation describe-stacks --stack-name CDKToolkit \--query "Stacks[0].Outputs[?OutputKey=='BootstrapPolicyVersion'].OutputValue | [0]" --output text -
If it is below 1.6.0, re-bootstrap with the backend included in
ComputeTypes.cdk bootstrapcannot pass template parameters, so the parameter goes on theCDKToolkitstack directly:Terminal window aws cloudformation deploy \--template-file cdk/bootstrap/bootstrap-template.yaml \--stack-name CDKToolkit --capabilities CAPABILITY_NAMED_IAM \--parameters ParameterKey=ComputeTypes,ParameterValue=agentcore\,lambda-microvmBundle 1.6.0 adds the
MicrovmPassRolesstatement, without which the CDK-managed MicroVM image deploy fails with aniam:PassRoleAccessDenied on the build role — an IAM error that reads like a code bug. Rationale and the live evidence for the missingiam:PassedToServicecondition: DEPLOYMENT_ROLES.md. -
Regional availability is limited (5 Regions at launch). Synth fails fast with the supported list if the stack’s Region is not among them;
bgagent doctorprobes it live.
Operational notes specific to this backend:
- Nothing self-terminates. A MicroVM whose task finished, crashed, or hung stays
RUNNINGand billing until the 8-hour cap. The orchestrator callsTerminateMicrovmon finalize, and the heartbeat-staleness check catches a hung guest inside a healthy VM — but a leaked handle is a cost incident. The one exception: the service reaps a VM whose/runhook returns 4xx (~12s). - Logs land in
/aws/lambda-microvms/<image-name>. Guest stdout goes there too, which is the fallback path when the agent cannot reach the application log group. - Deployment identifiers are not baked into the image. The snapshot carries no configuration; table names, secret ARNs, and the per-task session-role ARN arrive in the
/runpayload as aplatform_configblock. A version-skewed orchestrator that does not send it is refused rather than run with tenant scoping disabled.
Optional Agent Registry
Section titled “Optional Agent Registry”Without a deployment profile, AWS Agent Registry is enabled. In an account or Region where the service is unavailable or prohibited, set agentRegistry to false in abca.deployment.json, or use the context switch, to omit the registry, its REST API, IAM grants, environment wiring and stack outputs:
cdk deploy --context enableAgentRegistry=falseBlueprints without registry:// asset references continue to work. A remaining registry reference fails task startup rather than silently skipping the asset.
The string form is case-sensitive: use lowercase true or false. Any other value fails synthesis with an actionable validation error.
This context is an infrastructure switch, not a pause control. Applying it to an existing enabled deployment deletes the CloudFormation-managed registry and its records; re-enabling creates an empty registry that must be republished. The optional network split preserves that lifecycle; registry retention remains deferred. See REGISTRY.md for the catalog migration and runtime behavior.
Deployment profiles
Section titled “Deployment profiles”ABCA’s core always deploys:
- the AgentCore runtime with VPC/DNS egress controls;
- the Task API, with Cognito and API-key authentication;
- task state, admission and concurrency control, and budgets;
- repository onboarding and the GitHub task/PR workflow;
- Cedar HITL approvals and audit events;
- attachments, reconcilers, alarms and diagnostics.
Six features are optional and can be enabled independently (ADR-024):
Feature (abca.deployment.json key) | Context switch | Root resources (enabled alone) | Region-limited by |
|---|---|---|---|
Slack (slack) | enableSlack | ~51 | — |
Linear (linear) | enableLinear | ~45 | — |
Jira (jira) | enableJira | ~35 | — |
Deploy-preview screenshots (deployPreviewScreenshots) | enableDeployPreviewScreenshots | ~23 | AgentCore Built-in Tools (Browser) |
AWS Agent Registry (agentRegistry) | enableAgentRegistry | 2 (plus nested stacks) | AWS Agent Registry |
Tool Gateway (toolGateway, experimental) | enableToolGateway | ~7 | AgentCore Gateway |
Linear and Jira share the stranded-orchestration sweep and the iteration heartbeat, so enabling both costs slightly less than the sum. The minimal profile deploys about 317 root resources; the legacy topology deploys about 463.
Choosing features with bgagent setup
Section titled “Choosing features with bgagent setup”Run the wizard from your ABCA checkout. The examples use the short bgagent form. Without a global install, run node lib/bin/bgagent.js setup … from cli/ instead, as the Quick Start Step 3 does. Setup prints its next steps in whichever form you launched it with.
bgagent setup # interactivebgagent setup --features slack,jira --yes # scriptedbgagent setup --features none --no-deploy # only write the profilebgagent setup -c networkTopology=split # pass the context you deploy withThe wizard works in this order:
-
Resolves the Region (
--region, configured region,AWS_REGION) and rejects Regions without AgentCore Runtime and Memory. It shows the AWS account and identity your credentials resolve to (STSGetCallerIdentity) and asks you to confirm it. Before deploying it also checks for a running container engine, because the agent image is built locally:CDK_DOCKERif set, otherwise Docker, otherwise Finch (passed to CDK asCDK_DOCKER=finch).- Which deployment: without
--stack-name, it lists the ABCA deployments already in the account and Region. You can update one, or create a new deployment under a name you choose (the suggestion avoids existing names). With none yet, it asks for a name (defaultbackgroundagent-dev). - Stack names: ABCA’s least-privilege bootstrap scopes its deploy role to
backgroundagent-devandbackgroundagent-dev-<suffix>. When the account uses it (or setup installs it), setup only creates stacks with those names, for examplebackgroundagent-dev-team-b; a name outside that scope would fail partway through the create and leave a stack inROLLBACK_FAILED. A standard CDK bootstrap with administrator access deploys any name. A stack that exists but is not a working deployment (for example after a failed first create) is not listed; setup tells you to delete it or pass--stack-name.
- Which deployment: without
-
Reads the existing stack (
--stack-name, defaultbackgroundagent-dev). It preselects the features listed in the stack’sDeploymentFeaturesoutput, or infers them for stacks that predate profiles. Accepting the defaults never removes anything. A new stack defaults to the minimal profile.- The wizard passes only
stackName, the profile and any-c/--context key=valueyou supply to CDK, alongside whatevercdk/cdk.jsonalready holds. - It refuses to continue if a
${stackName}-networkstack exists andnetworkTopology=splitis missing from--contextandcdk/cdk.json, because dropping it would rebuild the VPC rather than change features. Compute comes from the stack itself (see step 3).
- The wizard passes only
-
New installations only: asks where agents run: AgentCore Runtime (recommended), ECS Fargate, or both (
--compute agentcore|ecs|agentcore,ecswhen scripted). The first backend is the repository default. An existing stack keeps its deployed backends (ComputeTypesoutput; stacks from before it that report a non-AgentCoreComputeSubstratemust pass--compute).--computemay add a backend, and setup warns before changing which one is first, but it refuses to remove one (see the backend transition procedure). Lambda MicroVMs are experimental and are selected only with--compute. -
Marks features that the Region does not support as unavailable and does not offer them. A scripted request for one fails with the supported Regions.
-
Plans any one-time account preparation and lists it on the review screen:
- No
CDKToolkitstack: installs ABCA’s least-privilege CDK bootstrap (mise //cdk:bootstrap). For ECS it also sets the bootstrap’sComputeTypesparameter, whichcdk bootstrapcannot set. - An older ABCA bootstrap: updates it to the version this CLI requires, and adds missing backends to
ComputeTypes. - A standard CDK bootstrap (no ABCA policy outputs): left unchanged, because other apps in the account may depend on it.
mise //cdk:deploystarts with a check that fails on a standard bootstrap, so setup passesABCA_SKIP_BOOTSTRAP_PREFLIGHT=1to that deploy, and the review screen says so. To switch to the least-privilege bootstrap instead, runmise //cdk:bootstrapbefore setup.
Preparation runs only after you confirm the deploy.
- No
-
Shows a review screen of what you chose, then runs
cdk diffagainst a temporary copy of the profile and summarizes it in one line (for example317 to create · 0 to update · 0 to remove). The full change list goes to~/.bgagent/logs/<stack>/setup-changes.log. Nothing in the repository changes until you confirm. -
Requires stronger confirmation if the diff deletes, replaces or orphans state-bearing resources (DynamoDB tables, secrets, buckets, log groups, user pools, VPCs, nested stacks): type the stack name interactively, or pass
--accept-data-losstogether with--yes. Declining, or closing the terminal at any prompt, exits with code 130 and leaves the repository unchanged. -
Prepares the account, writes
abca.deployment.json(orabca.deployment.<stack>.json), and deploys withmise //cdk:deploy -- -c stackName=<name> -c deploymentProfile=<file> --all. If preparation fails, nothing is written.- While deploying, it shows a progress bar and one line per component (Networking, Task API, Agent runtime, each enabled feature), with a check mark as each finishes. Raw CDK output goes to
~/.bgagent/logs/<stack>/setup-deploy.log. - Your confirmation replaces CDK’s own IAM approval prompt; the change summary says how many IAM roles and policies are included.
- On success it configures the CLI from the stack outputs and prints next steps, in the same command form you launched setup with. A new installation lists storing a GitHub token, creating a user, signing in and onboarding a repository. Any Slack, Linear or Jira feature added by this run lists its own setup command. If the CLI was previously signed in to another stack’s user pool, the cached tokens are removed; run
bgagent loginagain. On failure it names the failed component and CloudFormation’s reason, and keeps your selection so you can rerun.
- While deploying, it shows a progress bar and one line per component (Networking, Task API, Agent runtime, each enabled feature), with a check mark as each finishes. Raw CDK output goes to
--no-deploy writes the profile without a diff. If it removes a stateful feature from an existing stack, it asks for the same data-loss confirmation, because the next mise //cdk:deploy or CI deploy applies the removal with no wizard in the loop.
Commit abca.deployment.json. Each additional deployment from the same checkout gets its own abca.deployment.<stackName>.json, so deploying one never changes another’s features. Synthesis uses only the file matching -c stackName=…: the default stack backgroundagent-dev uses abca.deployment.json, and any other stack without its own file keeps the legacy topology rather than inheriting the default stack’s features.
Deployments in one account share only the CDK bootstrap, with one exception: Bedrock model-invocation logging is an account-wide setting, configured by whichever deployment was deployed last. Every synthesis reads it, including mise //cdk:synth, mise //cdk:deploy and the CI build, so local and pipeline deployments match.
Profile file and precedence
Section titled “Profile file and precedence”{ "schemaVersion": 1, "features": { "slack": false, "linear": false, "jira": false, "deployPreviewScreenshots": false, "agentRegistry": false, "toolGateway": false }, "compute": ["agentcore"]}Every feature must be listed explicitly. compute is optional: the ordered list of backends, first = repository default. It applies unless compute_types or compute_type context is set, so a later plain mise //cdk:deploy keeps the backends bgagent setup deployed. Without it, today’s compute context rules apply. Unknown keys, a different schemaVersion or a non-boolean value fail synthesis. For each feature, the first source that sets it wins:
- the per-feature context switch (for example
-c enableSlack=true); - the profile;
- the legacy topology.
-c deploymentProfile=<path> selects another file; relative paths resolve against the repository root, not cdk/. -c deploymentProfile=none ignores the profile. The wizard refuses to deploy while ~/.cdk.json, cdk/cdk.json or cdk/cdk.context.json sets a switch that contradicts your selection.
No profile means no change. Without abca.deployment.json, Slack, Linear, Jira, screenshots and the Agent Registry deploy as before, and the Tool Gateway stays opt-in. The only template change for an existing stack is the new DeploymentFeatures output.
What disabling a feature removes
Section titled “What disabling a feature removes”A disabled feature synthesizes no API routes, WAF body-size exemptions, Lambdas, IAM grants, tables, secrets, fan-out wiring, alarms or outputs. That includes the compute roles (AgentCore, ECS, MicroVM): they can read a tracker’s OAuth secrets only when that tracker is deployed. Screenshot-to-Linear or screenshot-to-Jira posting exists only when screenshots and that tracker are both enabled. The orchestration table and live reconciler remain core because the reconciler also owns per-task budget rollups. The task table keeps its LinearIssueIndex and JiraIssueIndex indexes, which do nothing while unused.
Removing a feature from an existing stack deletes its data: Slack installations and mappings, Linear and Jira workspace registries, project and user mappings, webhook secrets, the screenshot bucket, or the Agent Registry and its records. Re-enabling it creates empty resources, and there is no import or recovery path. Rerun the feature’s setup command (bgagent slack setup, bgagent linear setup, bgagent jira setup, bgagent github set-webhook-secret), and republish registry assets. Per-workspace OAuth secrets created by the CLI (bgagent-linear-oauth-*, bgagent-jira-oauth-*, bgagent/slack/*) are not managed by CloudFormation and survive removal. Delete them separately if you no longer need them.
CLI commands for an omitted feature (for example bgagent slack setup) fail with a message directing you to bgagent setup. They check the stack named by --stack-name, or else the stack the CLI was configured from (bgagent setup or bgagent configure --stack-name); with neither, the command runs its own checks.
Regional availability
Section titled “Regional availability”contracts/deployment-capabilities.json records the services each capability needs, the Regions that support it, and the source and date of the last verification. CDK synthesis enforces it for a concrete Region, so a CI or manual deploy fails before CloudFormation runs, not only in the wizard. Environment-agnostic synthesis is not checked.
To update the data when AWS expands coverage:
- Check the supported AgentCore Regions page.
- Edit
regionsand setlastVerifiedincontracts/deployment-capabilities.json. - Mirror the change into
cli/src/deployment-capabilities.ts. - Run
mise //cdk:testandmise //cli:build. The parity test fails if the mirror drifts, and the census fails if a footprint estimate drifts by more than 15%.
A stale entry blocks a newly supported Region until it is updated. The check fails closed by design, and there is no context override: editing the catalog is the only way to unblock a Region. Stacks that synthesize without a concrete Region (for example CI without credentials) are not checked; synthesis fails before any AWS lookup when a Region is known.
Optional operator console
Section titled “Optional operator console”The operator console is a read-only web UI for tasks, approvals and agent health. It deploys as a separate stack, ${stackName}-console: a private S3 bucket served through CloudFront, plus a Cognito app client and managed-login domain on the platform’s existing user pool. Enable it with:
MISE_EXPERIMENTAL=1 mise //cdk:deploy -- --all -c operatorConsole=true--all is required because the app now has more than one stack. The deploy task builds the bundle (mise //web:bundle) first. Open the ConsoleUrl stack output and sign in with the same Cognito user as the bgagent CLI.
The static app shell is public, like any single-page app on a CDN. It holds no data or secrets. Every request for data goes to the REST API, which requires a Cognito ID token. The console stack also deploys a small console-only API for the issue picker. It lists the open issues of onboarded repositories, using the stored GitHub token on the server side; the browser never receives the token. That API requires a Cognito ID token on every route, and its CORS rules allow only the console’s own address. To sign in from a local vite dev server against a deployed stack, add -c operatorConsoleDevOrigins=http://localhost:5173. Bootstrap policy bundle 1.8.0 or later is required for the managed-login domain; mise //cdk:preflight reports a stale bundle.
Bedrock inference geography
Section titled “Bedrock inference geography”The shipped cdk/cdk.json sets bedrockGeoRegion: "global", so inference routes to
any supported commercial Region — better throughput and resilience under burst, which
matters for tasks that run for hours.
If you have a data-residency requirement, set the geography before upgrading:
mise //cdk:deploy -- -c bedrockGeoRegion=us # or eu, apac, jp, au, us-govOne value drives every Bedrock grant and both injected model env vars, so the geography cannot disagree with what the agent calls. Changing it needs a redeploy, because the IAM grants are scoped to explicit profile ARNs resolved at synth. Full detail, including the per-layer model precedence, is in Model configuration.
Scale-to-zero analysis
Section titled “Scale-to-zero analysis”Components that scale to zero (pay-per-use)
Section titled “Components that scale to zero (pay-per-use)”| Component | Billing Model | Idle Cost |
|---|---|---|
| DynamoDB (8 core tables; integrations add more) | PAY_PER_REQUEST | $0 |
| Lambda (all functions) | Per invocation | $0 |
| API Gateway REST | Per request | $0 |
| ECS Fargate tasks (when enabled) | Per running task | $0 (cluster is free) |
| AgentCore Runtime | Per session | $0 |
| Bedrock inference | Per token | $0 |
| AgentCore Memory | Proportional to usage | ~$0 |
| Cognito | Free tier (50K MAU) | $0 |
Components that do not scale to zero (always-on)
Section titled “Components that do not scale to zero (always-on)”| Component | Est. Monthly Idle Cost | Why |
|---|---|---|
| NAT Gateway (1x) | ~$32 | $0.045/hr fixed charge |
| VPC Interface Endpoints (7x, 2 AZs) | ~$102 | $0.01/hr × 7 endpoints × 2 AZs × 730 hrs |
| WAF v2 Web ACL | ~$5 | Base monthly charge |
| CloudWatch Dashboard | ~$3 | Per-dashboard charge |
| Secrets Manager (1+ secrets) | ~$0.40/secret | Per-secret monthly |
| CloudWatch Alarms | ~$0.10/alarm | Per standard alarm |
| CloudWatch Logs retention | ~$1-5 | Storage for retained logs |
| Total always-on baseline | ~$140-150/month |
The dominant idle cost is VPC networking: 7 interface endpoints across 2 AZs ($102/month) plus the NAT Gateway ($32/month).
For the full cost model including per-task costs, see COST_MODEL.md.
Incremental cost-control cost
Section titled “Incremental cost-control cost”Monthly user/team controls add one DynamoDB on-demand table with PITR and three standard CloudWatch alarms. The table, API/Lambda reads, stream rollups, and SNS notifications are usage-based; the existing list Lambda and TaskTable reconciler perform the work, so there is no additional always-running compute.
At public US East (N. Virginia) first-tier list rates verified in August 2026, the three alarms are approximately $0.30/month total. The three possible custom metric series can add up to approximately $0.90/month when active. CloudWatch’s account-wide free tier includes 10 custom metrics and 10 alarm metrics, so the incremental CloudWatch charge may be $0 when that allowance is still available. DynamoDB storage/PITR and request charges depend on task volume and the number of team scopes. See Setting up cost controls for the operational overhead and pricing links.
AWS services inventory
Section titled “AWS services inventory”Compute
Section titled “Compute”| Service | Used By | Scales to Zero |
|---|---|---|
| Bedrock AgentCore Runtime (MicroVMs) | Agent sessions (default) | Yes |
| ECS Fargate (when enabled) | Agent sessions (opt-in) | Yes |
| AWS Lambda MicroVMs (when enabled) | Agent sessions (experimental, include lambda-microvm in compute_types) | Yes |
| Lambda (Node.js 24, ARM64) | Orchestrator, API handlers, fanout consumer, reconcilers, custom resources | Yes |
| Service | Used By | Scales to Zero |
|---|---|---|
| Bedrock (Claude Opus 5.5, Haiku 5.5, plus Sonnet 4.6, Opus 4.8, Opus 5, Haiku 4.5 for pinned repos) | Agent reasoning, cross-region inference profiles | Yes |
| Bedrock Guardrails | Prompt injection detection on task input | Yes |
| Bedrock AgentCore Memory | Semantic + episodic extraction strategies | Yes |
| AWS Agent Registry (preview, default-on) | Versioned agent asset catalog and governance | N/A (managed preview service) |
Networking
Section titled “Networking”| Service | Used By | Scales to Zero |
|---|---|---|
| VPC (public + private subnets, 2 AZs) | All compute | N/A (no direct cost) |
| NAT Gateway (1x) | Private subnet internet egress | No (~$32/mo) |
| VPC Interface Endpoints (7x, 2 AZs) | AWS service connectivity from private subnets | No (~$102/mo) |
| VPC Gateway Endpoints (2x: S3, DynamoDB) | S3 and DynamoDB connectivity | Yes (free) |
| Security Groups | HTTPS-only egress | N/A |
| Route 53 Resolver DNS Firewall | Domain allowlisting for agent egress | Minimal |
Storage / Database
Section titled “Storage / Database”| Service | Used By | Scales to Zero |
|---|---|---|
| DynamoDB (8 core tables, PAY_PER_REQUEST) | Task state, events, nudges, concurrency, monthly budgets, webhooks, repo config, approvals. Enabling integrations adds their mapping, registry, and deduplication tables. | Yes |
| DynamoDB Streams | TaskEventsTable → FanOut Consumer; TaskTable → combined orchestration/budget reconciler | Yes |
| S3 | CDK asset bucket, ECR image layers, FUSE session storage, trace artifacts (7-day lifecycle) | Minimal |
| SQS (DLQ) | FanOut, approval metrics, screenshot, and orchestration/budget reconciler dead-letter queues | Yes |
| Secrets Manager | GitHub PAT, webhook HMAC secrets | No (~$0.40/secret/mo) |
API / Auth
Section titled “API / Auth”| Service | Used By | Scales to Zero |
|---|---|---|
| API Gateway (REST) | Task REST API | Yes |
| Cognito User Pool | CLI/API authentication | Yes (free tier) |
| WAF v2 | API Gateway protection (managed rules + rate limiting) | No (~$5/mo base) |
Scheduling
Section titled “Scheduling”| Service | Used By | Scales to Zero |
|---|---|---|
| EventBridge (scheduled rule) | Stranded task reconciler (every 5 min) | Yes (rule is free; Lambda invocation is the cost) |
Observability
Section titled “Observability”| Service | Used By | Scales to Zero |
|---|---|---|
| CloudWatch Logs (multiple log groups) | Application, usage, model invocation, VPC flow, DNS query logs | No (storage) |
| CloudWatch Dashboard | Operational metrics visualization | No (~$3/mo) |
| CloudWatch Alarms | Operational failures plus 80%/100% monthly-budget thresholds | No (~$0.10/alarm) |
| X-Ray | AgentCore Runtime tracing | Yes |
Infrastructure / Deployment
Section titled “Infrastructure / Deployment”| Service | Used By | Scales to Zero |
|---|---|---|
| CloudFormation | Stack deployment, custom resources | N/A |
| ECR | Container image storage | Minimal |
| IAM | Roles and policies for all components | N/A |
Reference
Section titled “Reference”CI/CD pipeline (deploy.yml)
Section titled “CI/CD pipeline (deploy.yml)”The repository includes a two-stage CI/CD pipeline:
Stage 1: Build (build.yml)
Section titled “Stage 1: Build (build.yml)”Triggers on every PR and push to main. Runs mise run build (compile, test, lint, synth) and uploads the synthesized cdk.out/ as a deploy-intent artifact. The intent file declares whether a deploy should happen and for which compute types.
Stage 2: Deploy (deploy.yml)
Section titled “Stage 2: Deploy (deploy.yml)”Triggers via workflow_run when build.yml completes successfully. The pipeline:
- Skips fork PRs —
head_repository.full_name == github.repositoryprevents forks from entering the deploy flow. This is a security measure: an untrusted fork could modifybuild.ymlto produce a deploy-intent artifact, which would otherwise prompt maintainers for approval unnecessarily. - Downloads
deploy-intent.jsonfrom the triggering build run. - Resolves targets — Determines which compute types to deploy:
intent: "-"→ no-op (most PRs)intent: "labels"→ reads PR labels against an allowlistintent: "<type>"→ deploys the specified type (e.g.,agentcore)
- Requires approval — The
deployjob uses a GitHub Environment with required reviewers. Approvals are logged and the self-review rule prevents unilateral deploys. - Deploys via OIDC — Assumes an IAM role via GitHub OIDC federation (no long-lived credentials). The role is scoped to the
cdk deployaction with least-privilege policies per DEPLOYMENT_ROLES.md.
Security controls
Section titled “Security controls”| Control | Purpose |
|---|---|
Fork exclusion (head_repository check) | Prevents fork PRs from triggering deploy approval prompts |
| Environment approval | Human gate before any deploy reaches AWS |
| OIDC federation | No stored AWS credentials; tokens are request-scoped |
| Compute type allowlist | Only pre-approved types can be deployed |
| Non-cancellable concurrency | Deploy can’t be interrupted mid-flight |
For administrators
Section titled “For administrators”- Enable deploys: Set the
deployEnvironment in repo settings with required reviewers. - Configure OIDC: Set
AWS_ROLE_TO_ASSUMEsecret andAWS_REGIONvariable. - Allowlist compute types: Edit
ALLOWED_COMPUTE_TYPESindeploy.yml. - Deploy via PR label: Add the
deploy:<type>label to a PR (e.g.,deploy:agentcore).
Known deployment issues
Section titled “Known deployment issues”Log-delivery rename on upgrade (pin-table removal, #703)
Section titled “Log-delivery rename on upgrade (pin-table removal, #703)”Affects: any stack — whatever its name — whose live AWS::Logs::Delivery* logical ids contain CDKSource or CdkLogGroup. In practice that is any stack last deployed before the CDK library switch in #339 (2026-06-13), plus any stack deployed while the pin table held those ids. The legacy names come from the previous library version, not from the pin table, so a custom stack name confers no immunity. A currently-working deployment is not evidence of being unaffected: the failure only appears on the first deploy after upgrading past the pin removal.
Check any stack in one command — it deletes nothing:
STACK_NAME=<stack> mise //cdk:preflight:log-delivery -- --check-onlySymptom: The update fails on a new AWS::Logs::DeliverySource with AlreadyExists (“This ResourceId has already been used in another Delivery Source in this account”) and the whole stack rolls back. The stack keeps running on its previous template.
Resolution: None needed if you deploy with mise //cdk:deploy. The preflight is the first command of that task, so it detects the legacy resources on the live stack and applies the one-time migration — deleting exactly those resources, scoped to that stack, in the same account and region as the deploy, because the task forwards its own arguments to it.
The GitHub Actions pipeline does not migrate automatically. Its deploy job calls cdk deploy directly and so inherits none of that task’s steps. What it does do is report: the read-only diff job runs the preflight with --check-only and writes the result into the job summary, so a stack needing migration is visible in the run summary before the approval gate rather than discovered as a rollback afterwards. Migrate it with STACK_NAME=<stack> mise //cdk:deploy, or the manual sequence below, then re-run the pipeline.
A bare cdk deploy (no mise, no pipeline) does not run the preflight. On that path, or after the rollback above, run the check command shown under Affects and then follow the manual steps in Observability — AgentCore log delivery.
AgentCore unsupported Availability Zones
Section titled “AgentCore unsupported Availability Zones”Affects: Fresh deploys in accounts whose default Availability Zones don’t line up with the zones AgentCore supports for the region.
Symptom: The AWS::BedrockAgentCore::Runtime resource fails to stabilize (NotStabilized — “subnets are in unsupported availability zones”) and the stack rolls back.
Root cause: AgentCore Runtime only places its network interfaces in a subset of each region’s Availability Zones, published as physical zone IDs (e.g. use1-az1, use1-az2, use1-az4 for us-east-1). Zone IDs are stable across accounts, but zone names (us-east-1a) are aliased per-account — so us-east-1a can map to a different physical zone in your account than in another. Left to its default, CDK picks zones by name and can land the Runtime subnets in an unsupported zone. See the AWS Supported Availability Zones table for the per-region set; the same table is snapshotted as AGENTCORE_SUPPORTED_AZ_IDS in cdk/src/constructs/agentcore-azs.ts.
Which deploy paths are protected
Section titled “Which deploy paths are protected”| Path | Auto-pinned? | What you must do |
|---|---|---|
Local cdk deploy / mise //cdk:deploy (credentials resolve at synth) | Yes — for regions in the built-in map | Nothing, unless synth reports an [AgentCore AZs] error |
CI/CD (build.yml → deploy.yml) | No — the assembly is synthesized credential-less | Set the AGENTCORE_AVAILABILITY_ZONES repo/environment variable (below) |
| Any region absent from the built-in map | No | Set the context override (below) |
Auto-pin (local deploys). With a concrete account and region, synth confirms the credentials belong to that account (sts:GetCallerIdentity), reads the account’s zone name-to-ID mapping (ec2:DescribeAvailabilityZones), and pins the VPC to the first two AZ names — sorted, so the pin is stable across synths — whose zone IDs are AgentCore-supported. Two zones matches AgentVpc’s default maxAzs, so enabling this does not widen an already-working topology.
If auto-pin is attempted and cannot finish — lookup denied or throttled, credentials pointing at a different account, or fewer than two supported zones — synth fails with an [AgentCore AZs] error rather than quietly falling back. That is deliberate: a silent fallback produces a template that looks pinned but is not, which is the failure this section exists to prevent. Fix the cause or set the override.
CI/CD deploys are not auto-pinned. Auto-pin needs a bound account at synth time and the pipeline has none: build.yml synthesizes cdk.out without credentials (env-agnostic) and deploy.yml deploys that pre-built assembly (--app cdk/cdk.out). Two consequences worth being explicit about:
- Passing
-c 'agentcore:availabilityZones=...'tocdk deployhas no effect on the pipeline path — the template is already synthesized by then. Context only matters at synth. cdk/cdk.context.jsonis not a durable place to set this: it is gitignored, andbuild.ymlregenerates the whole file.
Instead, set the repo (or environment) variable AGENTCORE_AVAILABILITY_ZONES to a JSON array of zone names. build.yml’s “Generate CDK context” step folds it into the context that the uploaded assembly is synthesized with, and an unset variable simply leaves the stack unpinned (synth logs an [AgentCore AZs] warning):
AGENTCORE_AVAILABILITY_ZONES = ["us-east-1b","us-east-1c"]Choosing the values.
- Discover your account’s zone name-to-ID mapping:
Terminal window aws ec2 describe-availability-zones --region <region> \--query 'AvailabilityZones[].[ZoneName,ZoneId]' --output text - Pick at least two zone names (column 1) whose zone IDs (column 2) appear in the AgentCore-supported set for that region.
- Set the value — the pipeline variable above, or for a local synth either
cdk/cdk.jsoncontext:or the CLI at synth time:{ "context": { "agentcore:availabilityZones": ["us-east-1b", "us-east-1c"] } }Terminal window cdk deploy -c 'agentcore:availabilityZones=["us-east-1b","us-east-1c"]'
The override is validated at synth time, and both the JSON-array and -c string forms behave identically. Synth fails with a message naming the key when the value is not an array, has an empty/non-string entry, lists fewer than two distinct zones, contains zone IDs instead of names (use1-az2 — a common column mix-up), or names zones outside the target region. When the account’s mapping is knowable, the override is additionally cross-checked against the supported set, and unsupported or nonexistent zones fail synth.
Auto-pin selects two supported zones; an explicit override uses all the zones supplied. Pins above two zones remain subject to the 490-resource production ceiling. Configurations that exceed it need split networking; see Network stack topology for the measured boundaries and migration requirements.
Upgrading an existing stack. Auto-pin is on by default, so a local cdk deploy against a stack created before this change may select different zones than the deployed subnets use. Subnet.AvailabilityZone is create-only, so that is a replacement of the subnets and the resources bound to them (route tables, NAT gateway/EIP, VPC endpoints). Run mise //cdk:diff first. If the diff shows subnet replacement and you would rather keep the current topology, pin the override to the zones already deployed:
aws ec2 describe-subnets --filters "Name=vpc-id,Values=<vpc-id>" \ --query 'Subnets[].[SubnetId,AvailabilityZone,AvailabilityZoneId]' --output textTeardown blocked by AgentCore network interfaces
Section titled “Teardown blocked by AgentCore network interfaces”Deleting an AgentCore Runtime does not immediately release its service-managed network interfaces. The live review of #912 observed agentic_ai interfaces attached by amazon-aws still in use after five hours. They can block RuntimeSG, subnet and VPC deletion and leave a stack in DELETE_FAILED. This also affects ordinary destroys from main; it is not a fixed 20–40 minute delay.
Inspect the failed stack and its VPC before retrying:
aws cloudformation list-stack-resources --stack-name <failed-stack> \ --query 'StackResourceSummaries[?ResourceStatus==`DELETE_FAILED`].[LogicalResourceId,PhysicalResourceId,ResourceType]' \ --output tableaws ec2 describe-network-interfaces --filters Name=vpc-id,Values=<vpc-id>Let AgentCore release its interfaces; do not try to force-detach interfaces owned by amazon-aws. To finish deleting a stack already in DELETE_FAILED, inventory the blocked network resources and their dependencies, then retain those exact logical IDs in a deletion retry:
aws cloudformation delete-stack --stack-name <failed-stack> \ --retain-resources <blocked-logical-id> <dependent-logical-id>For split deployments, identify whether the failed resources belong to the application or the -network stack and target that stack. Keep physical IDs for every retained resource and track their cost. After the service releases the ENIs, clean up retained security groups, subnets and other VPC dependencies before deleting the VPC. --retain-resources does not perform that later cleanup or make the resources reusable by a same-name reinstall. Reconcile any retained resources from earlier experimental builds separately.
DNS Query Log Config replacement cascade (upgrading from pre-v0.5)
Section titled “DNS Query Log Config replacement cascade (upgrading from pre-v0.5)”Affects: Stacks deployed before the tag-exclusion fix (#222). Stacks created after this fix are not affected.
Symptom: UPDATE_FAILED on AWS::Route53Resolver::ResolverQueryLoggingConfigAssociation with error InvalidRequest: Cannot create association — one already exists for this VPC.
Root cause: The ResolverQueryLoggingConfig resource is create-only in CloudFormation — any property change (including Tags) triggers a full replacement. Pre-fix stacks have github:sha and other tags on this resource. Although the new code excludes it from future tag applications, CloudFormation still attempts to remove the now-excluded tags from the existing resource during the update, triggering the replacement cascade:
- Config is replaced → new physical resource ID
- Association detects
ResolverQueryLogConfigIdchanged → triggers its own replacement - CloudFormation attempts Create-before-Delete on the association → Route53 Resolver rejects (one association per VPC) →
InvalidRequest
Resolution — choose one:
Option A: AWS CLI disassociation (recommended)
Section titled “Option A: AWS CLI disassociation (recommended)”Fastest, scriptable, no console access required. Replace <vpc-id> with the agent VPC ID and <region> with your stack’s region.
- List the association for your VPC to get the
ResolverQueryLogConfigId:Terminal window aws route53resolver list-resolver-query-log-config-associations \--region <region> \--query "ResolverQueryLogConfigAssociations[?ResourceId=='<vpc-id>']" - Disassociate using the
Idfrom step 1:Terminal window aws route53resolver disassociate-resolver-query-log-config \--resolver-query-log-config-id <rqlc-id> \--resource-id <vpc-id> \--region <region> - Run
mise //cdk:deploy— CloudFormation recreates both the config and association without the orphan tags. The pre-existingResolverQueryLoggingConfigis replaced as part of the same update, so an explicitdelete-resolver-query-log-configis not required.
Option B: Two-phase deploy (comment-out / re-add)
Section titled “Option B: Two-phase deploy (comment-out / re-add)”- In
cdk/src/stacks/agent.ts, comment out theDnsFirewallconstruct instantiation (~line 197):// new DnsFirewall(this, 'DnsFirewall', {// vpc: agentVpc.vpc,// additionalAllowedDomains: additionalDomains,// observationMode: true,// }); - Deploy:
mise //cdk:deploy— this deletes the query log config, association, firewall rules, and related resources - Uncomment the
DnsFirewallblock - Deploy again:
mise //cdk:deploy— resources are recreated cleanly without tags
Option B is more disruptive (two deploys, brief DNS logging gap) but requires no AWS API access beyond cdk deploy.
Option C: Manual disassociation via AWS Console
Section titled “Option C: Manual disassociation via AWS Console”For users without AWS CLI access.
- Open the Route 53 Resolver console
- Select the query logging configuration named
agent-dns-query-log - Under Associated VPCs, disassociate the VPC
- Delete the query logging configuration
- Run
mise //cdk:deploy(orcdk deploy) — CloudFormation will recreate both resources without tags
Related docs
Section titled “Related docs”- Quick start — Zero-to-first-PR in 6 steps.
- Developer guide — Local development, testing, repository onboarding.
- User guide — API reference, CLI usage, task management.
- DEPLOYMENT_ROLES.md — Least-privilege IAM policies for CloudFormation execution.
- COST_MODEL.md — Per-task costs, cost guardrails, cost at scale.
- COST_ATTRIBUTION.md — Operator FinOps setup for per-user/per-repo Bedrock chargeback (Cost Explorer / CUR 2.0, invocation-log forensics).
- COMPUTE.md — Compute backend architecture and trade-offs.
- ADR-021 — Lambda MicroVMs backend decision, phased rollout, and live-verification evidence.