sample-aiml-security-assessment

Security Checks Reference

This document provides a comprehensive reference for all 174 security checks performed by the AI/ML Security Assessment framework (71 core checks across Amazon Bedrock, Amazon SageMaker AI, and Amazon Bedrock AgentCore, 27 Agentic AI Security checks, 64 Financial Services GenAI Risk checks, and 12 OWASP Top 10 for LLM checks).

Table of Contents


Overview

The framework evaluates your AI/ML workloads against AWS security best practices across three services:

Service Number of Checks Focus Areas
Amazon SageMaker AI 25 Security Hub controls, encryption, network isolation, IAM, MLOps
Amazon Bedrock 33 Guardrails, content filters, sensitive-information/PII filters, contextual grounding, automated reasoning, encryption (custom, imported, knowledge base, batch inference output), VPC endpoints, IAM permissions, agent guardrail association and least privilege, logging, CloudWatch alarms, cross-account policies, model evaluation, prompt flow validation, RAG evaluation, service quotas, Lambda code scanning (Amazon Inspector)
Amazon Bedrock AgentCore 13 VPC configuration, encryption, observability, resource policies
Agentic AI Security 27 Bounded autonomy, agent identity, tool authorization, guardrail enforcement, prompt/input protection, memory privacy, auditability, abuse protection
Financial Services GenAI Risk 64 Unbounded consumption, excessive agency, supply chain, training data poisoning, vector weaknesses, non-compliant output, misinformation, harmful output, biased output, PII disclosure, hallucination, prompt injection, improper output handling, off-topic output, out-of-date training data
OWASP Top 10 for LLM 12 LLM01 Prompt Injection, LLM02 Sensitive Info Disclosure, LLM03 Supply Chain, LLM04 Data/Model Poisoning, LLM05 Improper Output Handling, LLM06 Excessive Agency, LLM07 System Prompt Leakage, LLM08 Vector/Embedding Weaknesses, LLM09 Misinformation, LLM10 Unbounded Consumption

Check ID Convention

Each security check has a unique identifier with a service prefix:

Prefix Service Example
SM-XX Amazon SageMaker SM-01, SM-25
BR-XX Amazon Bedrock BR-01, BR-33
AC-XX Amazon Bedrock AgentCore AC-01, AC-13
AG-XX Agentic AI Security AG-01, AG-27
FS-XX Financial Services GenAI Risk FS-01, FS-69
OW-XX OWASP Top 10 for LLM OW-01, OW-12

Severity Levels

Severity Description Action Required
High Critical security issues that could lead to data exposure, unauthorized access, or compliance violations Immediate remediation recommended
Medium Important security improvements that strengthen your security posture Address in next maintenance window
Low Minor optimizations and best practice recommendations Address when convenient
Informational Advisory information about your configuration No action required
N/A Check not applicable (no resources to assess) No action required

Status Values

Status Description
Failed Security issue identified that requires remediation
Passed Checked resources met the assessed best practice at time of scan
N/A No resources exist to check (for example, no notebooks, no guardrails configured)

Amazon SageMaker AI Security Checks (25)

SM-01: Internet Access

SM-02: AWS IAM Permissions

SM-03: Data Protection

SM-04: Amazon GuardDuty Integration

SM-05: MLOps Features

SM-06: Clarify Usage

SM-07: Model Monitor

SM-08: Model Registry

SM-09: Notebook Root Access

SM-10: Notebook Amazon VPC Deployment

SM-11: Model Network Isolation

SM-12: Endpoint Instance Count

SM-13: Monitoring Network Isolation

SM-14: Model Container Repository

SM-15: Feature Store Encryption

SM-16: Data Quality Encryption

SM-17: Processing Job Encryption

SM-18: Transform Job Encryption

SM-19: Hyperparameter Tuning Encryption

SM-20: Compilation Job Encryption

SM-21: AutoML Network Isolation

SM-22: Model Approval Workflow

SM-23: Model Drift Detection

SM-24: A/B Testing and Shadow Deployment

SM-25: ML Lineage Tracking


Amazon Bedrock Security Checks (33)

BR-01: AWS IAM Least Privilege

BR-02: Amazon VPC Endpoint Configuration

BR-03: Marketplace Subscription Access

BR-04: Model Invocation Logging

BR-05: Guardrail Configuration

BR-06: AWS CloudTrail Logging

BR-07: Prompt Management

BR-08: Agent AWS IAM Configuration

BR-09: Knowledge Base Encryption

BR-10: Guardrail AWS IAM Enforcement

BR-11: Custom Model Encryption

BR-12: Invocation Log Encryption

BR-13: Flows Guardrails

BR-14: Stale Bedrock Access

BR-15: Cross-Account Guardrails Enforcement

BR-16: Guardrail Tier Validation

BR-17: Custom Model Customer-Managed KMS Encryption

BR-18: Model Evaluation Implementation

BR-19: Prompt Flow Validation

BR-20: Knowledge Base Encryption Enhancement

BR-21: Agent Action Group IAM Least Privilege

BR-22: Model Invocation Throttling Limits

BR-23: Guardrail Content Filter Coverage

BR-24: Automated Reasoning Policy Implementation

BR-25: RAG Evaluation Jobs

BR-26: Guardrail Sensitive Information Filter

BR-27: Guardrail Contextual Grounding Check

BR-28: Agent Guardrail Association

BR-29: Agent Idle Session TTL

BR-30: Imported Model Customer-Managed KMS Encryption

BR-31: Batch Inference Output Encryption

BR-32: CloudWatch Alarms on Bedrock Metrics

BR-33: Amazon Inspector Lambda Code Scanning


Amazon Bedrock AgentCore Security Checks (13)

AC-01: Runtime Amazon VPC Configuration

AC-02: AWS IAM Full Access

AC-03: Stale Access

AC-04: Observability

AC-05: Amazon ECR Repository Encryption

AC-06: Browser Tool Recording

AC-07: Memory Encryption

AC-08: Amazon VPC Endpoints

AC-09: Service-Linked Role

AC-10: Resource-Based Policies

AC-11: Policy Engine Encryption

AC-12: Gateway Encryption

AC-13: Gateway Configuration


Agentic AI Security Checks (27)

Agentic AI Security checks use the AG-XX namespace and are included with the default assessment. They follow a hybrid model:

These checks reference the AWS Well-Architected Agentic AI Lens, with scope limited to the Security pillar.

AG-01: Agent Guardrail Association

AG-02: Harmful Content Guardrail Coverage

AG-03: Sensitive Information Protection

AG-04: Automated Reasoning Guardrails

AG-05: Grounding Controls

AG-06: Tool Execution Least Privilege

AG-07: Model Invocation Logging

AG-08: API Audit Trail

AG-09: Guardrail Enforcement Boundary

AG-10: Adversarial Evaluation Coverage

AG-11: Prompt Flow Validation

AG-12: Invocation Abuse Controls

AG-13: Session Boundary

AG-14: Operational Abuse Alarms

AG-15: Runtime Network Boundary

AG-16: AgentCore Least Privilege

AG-17: Stale AgentCore Access

AG-18: AgentCore Observability

AG-19: Memory Data Protection

AG-20: Private AgentCore Connectivity

AG-21: Resource Policy Boundary

AG-22: Policy Engine Data Protection

AG-23: Gateway Data Protection

AG-24: Gateway Inbound Authorization

AG-25: Gateway Tool Policy Enforcement

AG-26: Gateway Error Detail Exposure

AG-27: Gateway WAF Protection


Additional Resources


Financial Services GenAI Risk Checks (64 additional, 5 upstream extensions)

These 64 standalone checks (FS-XX) extend the framework with Financial Services risk-management controls derived from the AWS User Guide to Governance, Risk, and Compliance for Responsible AI Adoption. An additional 5 FS checks are contributed as extensions to existing SM-07, SM-22, SM-23, BR-04, and BR-06 (see in-file extension notes).

The full catalog is in SECURITY_CHECKS_FINSERV.md, organized into three parts:

The same document includes the shared intro, severity rubric, validation note, upstream-overlap table, and the compliance framework mapping table (SR 11-7, FFIEC CAT, NYDFS 500.06, PCI-DSS 12.3.2, DORA Art.6, MAS TRM 9, ISO 27001 A.12, ECOA, OWASP LLM Top 10).


OWASP Top 10 for LLM Checks (12)

These 12 checks (OW-XX) map the AI/ML Security Assessment findings to the OWASP Top 10 for LLM 2025 categories. OW-01..OW-10 are derived by mapping from existing BR/SM/AC/FS findings. The OWASP Lambda itself does not call AWS APIs for mapped rows, but enabling OWASP can auto-run FinServ to produce FS-* source findings when FinServ is otherwise disabled. OW-11 and OW-12 are net-new checks that address LLM07 (System Prompt Leakage), which the existing checks do not directly cover. If a required source CSV is missing, the OWASP Lambda emits an informational OW-00 completeness row rather than silently dropping derived rows.

Opt-in. OWASP checks run only when the EnableOWASPAssessment deployment parameter is true and the Step Functions execution includes "enableOWASP": "true".

Rendered under a new “By Compliance Standard” sidebar section of the HTML report, alongside future NIST AI RMF and EU AI Act sections.

The full catalog is in SECURITY_CHECKS_OWASP.md, organized by OWASP category:

Preliminary and illustrative. OWASP mappings have not been reviewed by external auditors. Validate mappings with your Security/Compliance team before using as audit evidence.