Getting Started

Note: All http://localhost URLs in this documentation refer to the local development server and are not exposed to the public internet.

This guide walks you through setting up and running the MedML Training Dashboard — a FastAPI web application for managing medical image classification and segmentation workflows on AWS.

Prerequisites

Before you begin, ensure you have the following:

Requirement	Details
Python	3.11 or higher
AWS Account	With permissions for S3, SageMaker, and ECR
AWS CLI	Installed and configured (~/.aws/credentials or environment variables)
Docker	Optional — required only for the multi-service setup (dashboard + MLflow + TensorBoard)
Git	For cloning the repository

AWS Permissions

Your AWS credentials need access to:

• S3 — read/write for training data, model artifacts, and capture logs

• SageMaker — create/manage training jobs, endpoints, processing jobs, and monitoring schedules

• ECR — pull/push Docker images (only needed if building custom training containers)

---

⚠️ Medical AI Safety Notice

This platform is designed for research and development purposes only.

Models trained and deployed through this system are not validated for clinical use. Production deployment of medical imaging AI requires:

1. Regulatory clearance — FDA 510(k), De Novo, PMA, CE marking, or equivalent regulatory pathway for your jurisdiction

2. Clinical validation studies — Prospective trials demonstrating safety and efficacy on the target patient population

3. Human-in-the-loop oversight — Qualified healthcare professionals must review and interpret all AI predictions

4. Bias and fairness evaluation — Subgroup analysis across demographic groups to ensure equitable performance

5. Continuous monitoring — Post-deployment surveillance for model drift and performance degradation

AI predictions must not be used as the sole basis for medical decisions.

Consult with regulatory affairs, clinical validation teams, and your institution’s IRB/ethics board before any patient-facing deployment. For guidance on responsible AI practices in healthcare, see the AWS Responsible AI documentation and AWS Healthcare AI compliance resources.

---

Installation

1. Clone the repository

git clone <repository-url>
cd Kiro-ML-model

2. Create a virtual environment

python3 -m venv .venv
source .venv/bin/activate

3. Install dependencies

pip install -r requirements.txt

This installs FastAPI, boto3, PyTorch, MONAI, MLflow, and all other required packages.

4. Copy environment files

cp .env.example .env
cp .env.local.example .env.local

Edit both files with your configuration (see Configuration below).

Quick Setup

Once .env.local is configured with your AWS_ACCOUNT_ID, run the setup script to create the SageMaker IAM role and build/push all Docker images in one step:

./setup.sh

Or run individual steps:

./setup.sh --role    # Only create the SageMaker execution role
./setup.sh --images  # Only build and push Docker images

The role creation script (create-sagemaker-role.sh) mirrors the Python get_or_create_role() utility from utils.py. It creates a role named AmazonSageMaker-ExecutionRole-sgm with the same managed policies and inline MLflow policy, so the behavior is identical whether you run the bash script or use the Python helper in notebooks.

# Equivalent standalone usage:
./create-sagemaker-role.sh                          # Uses default name
./create-sagemaker-role.sh MyCustomRoleName         # Custom name

The setup script reads all account-specific values from .env.local — no positional arguments needed.

Configuration

The application loads environment variables in two layers:

1. .env — General application configuration (loaded first)

2. .env.local — Account-specific overrides (loaded second, takes priority)

.env.local is gitignored and should never be committed — it contains your AWS account-specific values.

.env — General config

Variable	Description	Default
AWS_ACCESS_KEY_ID	AWS access key (optional if using ~/.aws/credentials)	—
AWS_SECRET_ACCESS_KEY	AWS secret key (optional if using ~/.aws/credentials)	—
AWS_DEFAULT_REGION	AWS region for all services	us-east-1
MLFLOW_TRACKING_URI	MLflow server URI for experiment tracking	http://localhost:5000
MLFLOW_EXPERIMENT_NAME	Default MLflow experiment name	medical_imaging_experiments
TENSORBOARD_LOG_DIR	Directory for TensorBoard log files	./logs/tensorboard

.env.local — Account-specific config

Variable	Description	Example
AWS_ACCOUNT_ID	Your 12-digit AWS account ID	123456789012
SAGEMAKER_ROLE_ARN	IAM role ARN that SageMaker assumes for training and deployment	arn:aws:iam::123456789012:role/AmazonSageMaker-ExecutionRole-sgm
TRAINING_IMAGE_CLASSIFICATION	ECR URI for the classification training container	123456789012.dkr.ecr.us-east-1.amazonaws.com/sm-training-byoc-classification:latest
TRAINING_IMAGE_SEGMENTATION	ECR URI for the segmentation training container	123456789012.dkr.ecr.us-east-1.amazonaws.com/sm-training-byoc-segmentation:latest
INFERENCE_IMAGE_CLASSIFICATION	ECR URI for the classification inference container	123456789012.dkr.ecr.us-east-1.amazonaws.com/sm-inference-byoc-classification:latest
INFERENCE_IMAGE_SEGMENTATION	ECR URI for the segmentation inference container	123456789012.dkr.ecr.us-east-1.amazonaws.com/sm-inference-byoc-segmentation:latest
DEFAULT_DATA_S3_URI	Default S3 URI for training data (pre-populates the UI form)	s3://your-bucket/training-data/
TEST_IMAGE_S3_URI	S3 URI of a test DICOM image for integration tests	s3://your-bucket/test/image.dcm
TEST_SEGMENTATION_S3_URI	S3 URI of a test NIfTI volume for segmentation tests	s3://your-bucket/test/volume.nii.gz

Tip: You only need AWS_ACCOUNT_ID and SAGEMAKER_ROLE_ARN to get started. The ECR image URIs are required when launching SageMaker training jobs or deploying endpoints.

First Launch

Start the application:

python main.py

The dashboard starts at http://localhost:8000.

Open it in your browser — you’ll see the main dashboard with tabs for Training, Deployment, Inference, Monitoring, and more.

On first launch with empty data stores, the local job and deployment lists will be empty. See the Initial Setup section below to populate them from AWS.

Initial Setup

Option A: Configure AWS via environment variables

If your .env or ~/.aws/credentials already has valid AWS credentials, the app connects automatically on startup.

Option B: Configure AWS via the UI

1. Open http://localhost:8000

2. Navigate to the AWS section

3. Enter your Access Key ID, Secret Access Key, and Region

4. Click Connect

Sync existing resources from AWS

Once connected, pull your existing SageMaker resources into the dashboard:

1. Go to the Training tab → click Sync from AWS to import existing training jobs

2. Go to the Deployment tab → click Sync from AWS to import existing endpoints

3. Go to the Evaluation tab → click Sync from AWS to import existing evaluation jobs

This populates the local data stores (data/sagemaker_jobs.json, data/deployments.json) so subsequent loads are fast and don’t require AWS calls.

Docker Setup

For a full development environment with MLflow and TensorBoard, use Docker Compose:

docker-compose up --build

This starts three services:

Service	URL	Purpose
Dashboard	http://localhost:8000	Main application UI and API
MLflow	http://localhost:5000	Experiment tracking and model registry
TensorBoard	http://localhost:6006	Training visualization

Alternatively, use the startup script to run all services without Docker:

chmod +x start.sh
./start.sh

This starts MLflow, TensorBoard, and the FastAPI app together in the background.

Quick Start Workflow

Here’s a complete end-to-end workflow: connect to AWS, prepare data, train a model, deploy it, and run inference.

Step 1 — Connect to AWS

Ensure your AWS credentials are configured (via .env, environment variables, or the UI). Verify the connection by browsing S3 buckets in the AWS tab.

Step 2 — Prepare training data in S3

Upload your medical imaging dataset to S3 in the expected folder structure:

s3://your-bucket/training-data/
├── class_a/
│   ├── image001.dcm
│   ├── image002.dcm
│   └── ...
├── class_b/
│   ├── image003.dcm
│   └── ...
└── class_c/
    └── ...

Each subfolder name becomes a class label. The number of classes is determined automatically from the folder structure.

For segmentation tasks, organize data with paired images and labels.

Step 3 — Train a model

Option A: SageMaker training (recommended for GPU workloads)

1. Go to the Training tab

2. Select SageMaker Training

3. Configure:

   • Model architecture — ResNet50, VGG16, EfficientNet, DenseNet121, ViT, or custom

   • Data S3 URI — path to your training data (pre-filled from DEFAULT_DATA_S3_URI)

   • Instance type — e.g. ml.g5.xlarge for GPU training

   • Epochs, learning rate, batch size — training hyperparameters

   • MLflow tracking URI — optional, for experiment tracking

4. Click Start Training

The job runs on SageMaker. Monitor progress in the Training tab — active jobs auto-refresh their status.

Option B: Local training

1. Select Local Training in the Training tab

2. Choose your model and dataset

3. Training runs on your local machine with MLflow and TensorBoard tracking

Step 4 — Deploy the model

Once training completes:

1. Go to the Deployment tab

2. Click Deploy on a completed training job, or create a new deployment:

   • Endpoint type — choose Real-time or Async (radio buttons)

   • Model artifacts — S3 URI of the model.tar.gz produced by training

   • Inference image — ECR URI of the inference container (from .env.local)

   • Instance type — e.g. ml.g5.xlarge

   • Role ARN — SageMaker execution role (from .env.local)

   • Output S3 URI — S3 prefix for inference results (required for async, optional for real-time)

3. Click Deploy

The endpoint progresses from Creating → InService. This typically takes 5–10 minutes.

Once deployed, endpoint cards display scaling controls: Scale up (+), Scale down (-), and Scale to 0 (async only — real-time endpoints require a minimum of 1 instance). An active endpoint count badge shows endpoint names and instance counts at a glance.

Step 5 — Run inference

Once the endpoint is InService:

1. Go to the Inference tab

2. Select the endpoint and task type (classification or segmentation)

3. Provide input:

   • S3 URI — point to an image in S3

   • File upload — upload a DICOM or NIfTI file directly from your browser

4. Click Invoke

The response shows predicted classes with confidence scores (classification) or a binary mask with volume statistics (segmentation).

Step 6 — Monitor (optional)

Set up Model Monitor to track prediction drift:

1. Enable Data Capture on the endpoint (Monitoring tab → Data Capture)

2. Run a Baseline job against your training data

3. Create a Monitoring Schedule to compare live traffic against the baseline

---

Next Steps

• Evaluation — run a SageMaker Processing Job to evaluate endpoint accuracy on a labeled test set

• Fairness Analysis — scan DICOM metadata for demographic distributions and bias auditing

• Fine-tuning — resume training from a checkpoint using checkpoint_s3_uri

• Report Generation — generate academic paper sections from completed training and evaluation jobs

• Custom Containers — build and push BYOC training/inference images (see README for build_and_push.sh instructions)

Troubleshooting

Issue	Solution
ValueError: AWS credentials not configured	Set credentials in .env, environment, or ~/.aws/credentials
SageMaker job fails immediately	Check that SAGEMAKER_ROLE_ARN has the necessary permissions (run ./create-sagemaker-role.sh to create/fix it) and the ECR image URI is correct
Endpoint stuck in Creating	Wait up to 10 minutes; check the SageMaker console for errors
MLflow not tracking	Ensure MLFLOW_TRACKING_URI points to a running MLflow server (or use Docker Compose)
Docker build fails with 403 Forbidden	Run aws ecr get-login-password to authenticate with the AWS Deep Learning Containers ECR (see README)
SageMaker rejects image with Unsupported manifest media type	Your Docker image was built in OCI format. Rebuild with docker buildx build --provenance=false or use the provided build_and_push.sh scripts which handle this automatically
Real-time endpoint times out on first request	This is a cold start — the model loads into GPU memory on the first invocation. The first request may exceed the 60s API gateway timeout, but subsequent requests are fast (<30s). Send a warm-up request after deployment, or use async endpoints for segmentation which have no timeout limit
Real-time works on second request but not first	Cold start. The model and MONAI transforms are initialized on the first predict_fn call. After that, the model stays in GPU memory and subsequent requests are fast
Empty job list after restart	Data stores auto-rebuild from data/training_reports/ on first boot; click Sync from AWS to pull fresh data