This document provides a comprehensive reference for all 208 security checks performed by the AI/ML Security Assessment framework (94 core checks across Amazon Bedrock, Amazon SageMaker AI, Amazon Bedrock AgentCore, and AWS Agent Registry, 38 Agentic AI Security checks, 64 Responsible AI GRC checks, and 12 OWASP Top 10 for LLM checks).
Sources differ by bucket and are not interchangeable: the core Bedrock, SageMaker, AgentCore, and AWS Agent Registry checks derive from the AWS Well-Architected Generative AI Lens security best practices (gensec*) and service security documentation; the Agentic AI Security checks from the AWS Well-Architected Agentic AI Lens; the FS-* Responsible AI GRC checks from the AWS GRC User Guide; and the OW-* checks from the OWASP Top 10 for LLM. The AWS Well-Architected Responsible AI Lens is not a source for any of them — see Responsible AI GRC — scope, sources, and compatibility.
The 64 Responsible AI GRC checks occupy 69 FS-* numbers: 64 ship as standalone checks and 5 are merged into upstream Bedrock/SageMaker checks. The framework also emits BR-00, SM-00, AC-00, AR-00, FS-00, and OW-00 operational marker rows at runtime; these are not controls and are excluded from the 208-check total. Per-control provenance, including which controls are project extensions rather than guide-derived, is recorded in provenance.json.
The counts above describe the full catalog. Core service assessments are enabled
by default and can be selected independently with the four
Enable*Assessment switches.
Deselected services produce no findings and appear as Not selected in the
report; this is not an N/A finding or a compliant result. Agentic AI and OWASP
mapping coverage decreases when their direct-service sources are deselected.
The framework evaluates your AI/ML workloads against AWS security best practices across four services:
| Service | Number of Checks | Focus Areas |
|---|---|---|
| Amazon SageMaker AI | 29 | Security Hub controls, encryption, network isolation, GuardDuty AI Protection, HyperPod, IAM, MLOps, Model Registry policy exposure |
| Amazon Bedrock | 40 | Guardrails, prompt-attack/image filters, retention, inference profiles, automated reasoning and Marketplace endpoint governance, encryption, networking, IAM, logging, monitoring, and evaluation |
| Amazon Bedrock AgentCore | 17 | Runtime/tool VPC isolation, encryption, browser recording, observability, resource policies, Identity token vaults, and online evaluation |
| AWS Agent Registry | 8 | IAM access, approval governance, discovery authorization, encryption, organization auto-detection, record lifecycle, and provenance |
| Agentic AI Security | 38 | Bounded autonomy, agent identity, tool authorization, Registry governance and provenance, guardrail enforcement, prompt/input protection, memory privacy, auditability, continuous assurance, abuse protection |
| Responsible AI GRC | 64 | Unbounded consumption, excessive agency, supply chain, training data poisoning, vector weaknesses, non-compliant output, misinformation, harmful output, biased output, PII disclosure, hallucination, prompt injection, improper output handling, off-topic output, out-of-date training data |
| OWASP Top 10 for LLM | 12 | LLM01 Prompt Injection, LLM02 Sensitive Info Disclosure, LLM03 Supply Chain, LLM04 Data/Model Poisoning, LLM05 Improper Output Handling, LLM06 Excessive Agency, LLM07 System Prompt Leakage, LLM08 Vector/Embedding Weaknesses, LLM09 Misinformation, LLM10 Unbounded Consumption |
Each security check has a unique identifier with a service prefix:
| Prefix | Service | Example |
|---|---|---|
| SM-XX | Amazon SageMaker | SM-01, SM-30 (SM-29 reserved) |
| BR-XX | Amazon Bedrock | BR-01, BR-40 |
| AC-XX | Amazon Bedrock AgentCore | AC-01, AC-17 |
| AR-XX | AWS Agent Registry | AR-01, AR-08 |
| AG-XX | Agentic AI Security | AG-01, AG-38 |
| FS-XX | Responsible AI GRC | FS-01, FS-69 |
| OW-XX | OWASP Top 10 for LLM | OW-01, OW-12 |
The *-00 rows below make assessment coverage and execution problems visible in
CSV and HTML reports. They are operational markers rather than security
controls, do not increase the published check counts, and must not be treated as
evidence that a control passed or failed.
| Marker | Runtime meaning | Normal status / severity |
|---|---|---|
BR-00 |
Amazon Bedrock is unavailable or not enabled in the target region, so regional Bedrock checks were not run. | N/A / Informational |
SM-00 |
Amazon SageMaker AI is unavailable or not enabled in the target region, so regional SageMaker checks were not run. | N/A / Informational |
AC-00 |
Amazon Bedrock AgentCore is unavailable in the target region, or the Runtime availability probe rejected the assessment credentials before regional checks could run. Unexpected errors inside individual checks use their affected AC-* or AG-* control IDs instead. |
N/A / Informational |
AR-00 |
AWS Agent Registry is unavailable in the target region, so regional AR-03 through AR-08 checks were not run. |
N/A / Informational |
FS-00 |
No regional Bedrock, AgentCore, or SageMaker resource footprint was found, so Responsible AI GRC was not applicable to that region. | N/A / Informational |
OW-00 |
A required upstream assessment CSV was missing, so one or more mapping-derived OWASP rows could not be generated. | N/A / Informational |
When a Bedrock API is access-denied or an AgentCore check raises an unexpected
execution error, the affected control ID is reported as informational N/A
with an incomplete-assessment message. These rows remain visible for
troubleshooting but are excluded from scoring. A control is Failed only when
the scanner successfully observes evidence that violates its baseline.
FS-00 is described in more detail in
Responsible AI GRC Checks,
and OW-00 in
OWASP Top 10 for LLM Security Checks.
Pass rates are calculated from unique direct-service Check_ID values, not
from report-row counts. Findings for resources, Regions, or accounts are
aggregated into one result per control: any assessable Failed row makes the
control fail, and a control passes only when all assessable rows pass.
Informational and N/A rows are excluded from the score. Agentic AI and
compliance-mapping rows are contextual views of source evidence and are also
excluded to prevent double counting. Resource-level rows remain visible for
investigation and remediation.
| Severity | Description | Action Required |
|---|---|---|
| High | Critical security issues that could lead to data exposure, unauthorized access, or compliance violations | Immediate remediation recommended |
| Medium | Important security improvements that strengthen your security posture | Address in next maintenance window |
| Low | Minor optimizations and best practice recommendations | Address when convenient |
| Informational | Advisory information about your configuration | No action required |
| Status | Description |
|---|---|
| Failed | Security issue identified that requires remediation |
| Passed | Checked resources met the assessed best practice at time of scan |
| N/A | The check was not applicable, advisory-only, unavailable in the region, or could not be assessed (for example, because no resources exist or access was denied). |
N/A incomplete-assessment row rather than a
compliant result.AI_PROTECTION feature is ENABLED. No detector is N/A because SM-04 separately reports GuardDuty enablement.InstanceStorageConfigs; therefore, an absent root-volume storage configuration fails this CMK baseline rather than producing N/A.OverrideVpcConfig before the cluster-level VpcConfig.SM-29 is reserved for SageMaker Unified Studio private networking. It is not currently emitted because the available domain APIs do not expose a sufficient domain-level networking configuration.
AIML_APPROVED_EXTERNAL_ACCOUNT_IDS / AIML_APPROVED_ORG_IDS boundaries. Configure those boundaries through the ApprovedExternalAccountIds and ApprovedOrganizationIds deployment parameters, respectively; both default to empty. Wildcard principals constrained by exact aws:PrincipalAccount or aws:PrincipalOrgID values, fixed-account aws:PrincipalArn patterns, or fixed-organization aws:PrincipalOrgPaths patterns are treated as bounded. Wildcard account/organization identifiers remain public; ForAllValues organization-path conditions count as boundaries only when a matching Null: false condition requires the key to be present. Because AWS supports NotPrincipal only with Deny, an Allow statement containing NotPrincipal is reported as unsupported and N/A rather than silently passing or being treated as public. Valid Deny statements do not create exposure and are ignored. If sts:GetCallerIdentity is unavailable, public wildcard statements are still reported, but account principals that cannot be distinguished as same-account or external produce N/A instead of an external-access finding. This is a conservative heuristic, not a complete IAM authorization simulator.BR-01, BR-02, BR-03, BR-08, BR-10, and BR-21 depend on the shared IAM
permissions cache. If that prerequisite is missing, unreadable, or malformed,
each affected control is reported as informational N/A; an empty replacement
inventory is never treated as evidence of compliance.
Global region in multi-region scans.BEDROCK_POLICY policy type) for centralized safety control enforcement across all accounts. Checks if running in the AWS Organizations management account, validates the Bedrock policy type is enabled at the organization root, and verifies that Bedrock policies are attached.STANDARD content-filter tier (vs the CLASSIC tier) for enhanced protection and broader language support. Lists all guardrails in the region and inspects each guardrail’s contentPolicy.tier.tierName. The STANDARD tier requires cross-Region inference.validate_flow_definition API before deployment to prevent misconfigured flows. Lists all flows in the region, checks for validation records or status, identifies unvalidated flows, and reports flows deployed without validation.type (VECTOR | KENDRA | SQL | MANAGED) to decide how to assess each KB: for MANAGED knowledge bases it reads knowledgeBaseConfiguration.managedKnowledgeBaseConfiguration.serverSideEncryptionConfiguration.kmsKeyArn and fails KBs encrypted with an AWS-owned key; for custom vector stores (OpenSearch, RDS, Pinecone, etc.) the encryption key lives on the underlying storage resource and cannot be read from the KB API, so those are reported as N/A for manual review. If a MANAGED KB’s encryption block is missing from the API response (deployed botocore older than 1.43.32, which silently drops the unmodeled field), the KB is reported as N/A “indeterminate” rather than a false-positive failure.GetGuardrail.sensitiveInformationPolicy and reports guardrails that have no PII entity types (piiEntities) or custom regex patterns (regexes) configured, leaving prompts and responses unscreened for sensitive data.GetGuardrail.contextualGroundingPolicy.filters and reports guardrails with no enabled grounding/relevance filters. Complements BR-25 (RAG evaluation) with a runtime control.guardrailConfiguration from the agent summaries returned by ListAgents and reports agents with no guardrail attached.GetAgent.idleSessionTTLInSeconds and reports agents whose TTL exceeds a conservative ceiling (3600 seconds).GetImportedModel.modelKmsKeyArn, reporting models encrypted with AWS-owned keys instead of a customer-managed key.outputDataConfig.s3OutputDataConfig.s3EncryptionKeyId from the job summaries returned by ListModelInvocationJobs and reports jobs without a customer-managed output key.AWS/Bedrock namespace) to detect abuse, denial-of-wallet, sustained throttling, and content-filter spikes. Uses DescribeAlarms and matches alarms that target the AWS/Bedrock namespace directly or via a metric-math expression. Only assessed in regions that have Bedrock resources.lambda) and Lambda code scanning (lambdaCode) are both enabled so those in-scope functions and their dependencies are scanned for vulnerable packages and hardcoded secrets. Calls lambda:ListFunctions for scoping and inspector2:BatchGetAccountStatus for Inspector status. Reports Failed only when in-scope Lambda functions exist and either resourceState.lambda.status or resourceState.lambdaCode.status is not ENABLED. No in-scope Lambda functions, access denied, and region-unavailable states resolve to N/A.PROMPT_ATTACK input filter with inputEnabled=true, inputAction=BLOCK, and a non-NONE input strength. Standard tier is reported as a strengthening note.HATE, INSULTS, SEXUAL, and VIOLENCE as the cross-region image-filter baseline. Because AWS documents MISCONDUCT image filtering as region-dependent, its absence is not reported as a gap, but a configured MISCONDUCT filter is reported when its input or output modalities omit IMAGE. Complete coverage is Passed; advisory gaps are N/A/Informational because the scanner cannot infer whether protected applications accept or produce images.provider_data_share, passes none, and reports default/inherit as informational unless the RequireBedrockZeroDataRetention deployment parameter is true (REQUIRE_BEDROCK_ZERO_DATA_RETENTION in the Lambda), in which case those modes fail.kmsKeyArn.kmsEncryptionKey with kms:DescribeKey and requires KeyMetadata.KeyManager to be CUSTOMER; AWS-managed keys do not pass. The RequireMarketplaceEndpointCMK deployment parameter defaults to true (REQUIRE_MARKETPLACE_ENDPOINT_CMK in the Lambda). Set it to false to make a missing or AWS-managed key an N/A/Informational hardening advisory rather than a failure. An inconclusive KMS lookup is always N/A/Informational.Allow/NotAction allow-except statements that still grant the AgentCore namespace when they apply to all resources. Only the valid bedrock-agentcore IAM namespace is evaluated; overly permissive agent-registry grants are reported by AR-01 instead. Service-agnostic administrator-style grants are out of scope in both forms: a bare Action: "*" and a NotAction whose exclusions name no platform namespace are treated alike and not reported as AgentCore-specific grants. A missing, unreadable, or malformed permissions cache is reported as informational N/A. If an individual cached policy document cannot be parsed, valid findings from other policies are retained and an additional informational N/A row marks the control incomplete; the unparsed policy cannot produce a compliant pass.Allow and NotAction grants in attached and inline policy documents before querying IAM service-last-accessed history. Only the bedrock-agentcore namespace is evaluated; agent-registry grants are reported by AR-02 instead. As in AC-02, a NotAction whose exclusions name no platform namespace is a service-agnostic administrator grant and is not treated as an AgentCore-specific permission. Attached policy names alone are never treated as proof of access. IAM last-accessed jobs are polled within the Lambda deadline; a job that does not complete in time is reported as an indeterminate N/A rather than a failed control. A missing, unreadable, or malformed permissions cache is also reported as informational N/A. This identifies candidate grants from the cached policy documents; it is not a complete effective-permissions simulation across boundaries, session policies, or organization controls.recording.enabled=true with a non-empty S3 recording bucket.CustomerManagedKey with a KMS key ARN. Set the AgentCoreTokenVaultId deployment parameter to override the default vault ID (AGENTCORE_TOKEN_VAULT_ID in the Lambda).VPC network mode with non-empty subnets and security groups.VPC network mode with non-empty subnets and security groups. Shares browser inventory with AC-06.RequireAgentCoreOnlineEvaluation deployment parameter to true (REQUIRE_AGENTCORE_ONLINE_EVALUATION in the Lambda) to make incomplete coverage fail.AWS Agent Registry checks use the AR-XX namespace and run in a dedicated
regional Lambda that writes its own CSV artifact and HTML report area. They are
included with the default assessment.
AR-01 and AR-02 are account-scoped IAM checks that read the shared
permission cache and are reported once under the Global region. AR-03
through AR-08 are regional and use the generally available
agent-registry-control API. Registry detail is read once per registry and
shared across AR-03 through AR-06; record inventory is shared between
AR-07 and AR-08.
Record inventory is bounded to 1,000 records and paginates within the Lambda
deadline. When the cap or the deadline is reached, AR-07 and AR-08 report a
single informational N/A incomplete-assessment row and continue assessing
the records already collected. A registry that is not READY, a registry
whose detail call fails, an access-denied response, and a region where AWS
Agent Registry is unavailable all resolve to informational N/A with
error-specific remediation rather than to a failure.
Allow/NotAction allow-except statements that still grant the agent-registry namespace when they apply to all resources. Only the valid agent-registry IAM namespace is evaluated; bedrock-agentcore grants are reported by AC-02 instead. Service-agnostic administrator-style grants are out of scope in both forms: a bare Action: "*" and a NotAction whose exclusions name no platform namespace are treated alike and not reported as Registry-specific grants. An empty permission cache is an informational N/A tooling condition, not a failure.agent-registry namespace, either through an Allow action or through a NotAction allow-except statement that does not fully cover the namespace. Attached policy names alone are never treated as proof of access. It uses IAM service-last-accessed jobs to identify access older than 60 days and principals with no Registry usage evidence. IAM job errors, timeouts, and inaccessible principals are indeterminate informational N/A findings rather than failures.READY registry requires manual review for submitted records. A registry whose approvalConfiguration carries autoApprovalRules approves submitted records automatically and is informational by default; set RequireAgentRegistryManualApproval to true (REQUIRE_AGENT_REGISTRY_MANUAL_APPROVAL in the Lambda) to make automatic approval fail, which also switches the remediation text from advisory to actionable. A registry with no auto-approval rules passes. approvalConfiguration is optional in the GA response; a registry that omits it is reported as informational N/A because manual review was never observed, not as a pass.READY registry. A custom JWT authorizer without both an OpenID Connect discovery URL and at least one caller constraint (allowedAudience, allowedClients, allowedScopes, or customClaims) fails. Every other outcome is informational N/A pending review, because the authorizer configuration alone does not establish which callers hold effective discovery access: AWS_IAM requires an effective-policy review, a constrained custom JWT authorizer requires comparing the approved audiences, clients, scopes, and claims against intended consumers, and an absent or unrecognized discoveryConfiguration establishes no authorization fact either way.GetRegistry.encryptionConfiguration.kmsKeyArn. Registries with a customer-managed KMS key pass. Registries using the default AWS owned key are informational by default because AWS Agent Registry still encrypts them at rest. Set RequireAgentRegistryCMK to true (REQUIRE_AGENT_REGISTRY_CMK in the Lambda) to make the AWS owned key configuration fail. The registry encryption key is immutable after creation, so remediation requires a replacement registry and record migration.READY registry reports auto-detection that is enabled, scoped to ORGANIZATION, and ACTIVE. Disabled, account-scoped, or INACTIVE configurations are informational N/A because the feature is optional. An omitted or incomplete optional autoDetection block, and a registry that has not reached READY, are also informational N/A because the control state could not be established.ListRegistryRecords across every accessible registry and reports the lifecycle state returned in each record summary as an advisory N/A observation, because occupying a documented service state does not by itself prove a security control. Review failed or unknown lifecycle states operationally. Per-registry listing failures are reported individually with error-specific remediation so one inaccessible registry does not hide the rest. AR-07 does not affect the score unless a future baseline defines a genuine noncompliant lifecycle state.DETECTED_FROM provenance summary whose sourceId is a bedrock-agentcore ARN matching its declared sourceType: a runtime/... resource for AWS::BedrockAgentCore::Runtime or a gateway/... resource for AWS::BedrockAgentCore::Gateway. A record whose declared lineage does not match fails, and it continues to fail even when another provenance entry omits its own source type. Optional origin-mode, creator-attribution, provenance, and source-type metadata are reported as informational N/A rather than as operator-remediable failures.Agentic AI Security checks use the AG-XX namespace and are included with the
default assessment. They follow a hybrid model:
These checks reference the AWS Well-Architected Agentic AI Lens, with scope limited to the Security pillar.
ListGateways and GetGatewayNONE authorizers. Passes AWS_IAM and CUSTOM_JWT. AUTHENTICATE_ONLY passes only when an AgentCore policy engine is attached in ENFORCE mode, because the gateway authenticates the SigV4 caller but does not make an authorization decision for that authorizer type.GetGateway.policyEngineConfiguration plus ListPoliciesENFORCE, or with no ACTIVE policy whose enforcement mode is ACTIVE. A mix of enforcing and LOG_ONLY/inactive policies passes with an advisory.GetGateway.exceptionLevelDEBUG-level exception detail.GetGateway.webAclArnInvokeGuardrailChecks / ApplyGuardrail are per-request runtime APIs rather than a persistent configuration surface. The assessment therefore does not emit a pass/fail finding for their use; applications should validate these calls through runtime architecture review, telemetry, and testing.
These 64 standalone checks (FS-XX) extend the framework with cross-industry AI governance, risk, and compliance controls derived from the AWS User Guide to Governance, Risk, and Compliance for Responsible AI Adoption. An additional 5 FS checks are contributed as extensions to existing SM-07, SM-22, SM-23, BR-04, and BR-06 (see in-file extension notes).
The full catalog is in SECURITY_CHECKS_RESPONSIBLE_AI_GRC.md,
organized into three parts:
The same document includes the shared intro, severity rubric, validation note, upstream-overlap table, and the compliance framework mapping table (SR 11-7, FFIEC CAT, NYDFS 500.06, PCI-DSS 12.3.2, DORA Art.6, MAS TRM 9, ISO 27001 A.12, ECOA, OWASP LLM Top 10).
These 12 checks (OW-XX) map the AI/ML Security Assessment findings to the
OWASP Top 10 for LLM 2025 categories.
OW-01..OW-10 are derived by mapping from existing BR/SM/AC/FS findings.
The OWASP Lambda itself does not call AWS APIs for mapped rows, but enabling
OWASP can auto-run Responsible AI GRC to produce FS-* source findings when
Responsible AI GRC is otherwise disabled. OW-11 and OW-12 are net-new checks
that address LLM07 (System Prompt Leakage), which the existing checks do not
directly cover.
If a required source CSV is missing, the OWASP Lambda emits an informational
OW-00 completeness row rather than silently dropping derived rows.
Opt-in. OWASP checks run only when the EnableOWASPAssessment deployment
parameter is true and the Step Functions execution includes "enableOWASP": "true".
Rendered under a new “By Compliance Standard” sidebar section of the HTML report, alongside future NIST AI RMF and EU AI Act sections.
The full catalog is in SECURITY_CHECKS_OWASP.md,
organized by OWASP category:
Preliminary and illustrative. OWASP mappings have not been reviewed by external auditors. Validate mappings with your Security/Compliance team before using as audit evidence.