EKS Upgrade Readiness Report¶
Cluster: eks-review-cluster¶
| Field | Value |
|---|---|
| Current Version | 1.30 (EXTENDED_SUPPORT), Platform: eks.64 |
| Target Version | 1.33 (STANDARD_SUPPORT) |
| Region | us-east-1 |
| Assessment Date | 2026-04-06 |
| Upgrade Readiness | ❌ NOT READY |
| Upgrade Path | 1.30 → 1.31 → 1.32 → 1.33 (3 sequential hops) |
| Checks Run | 38 |
| Passed | 24 |
| Blockers | 3 |
| Warnings | 11 |
Go / No-Go Decision¶
NO-GO. The cluster has 3 blockers that must be resolved before any upgrade can proceed. (1) 31 live resources use the flowcontrol.apiserver.k8s.io/v1beta3 API which is removed in Kubernetes 1.32 — these will break during the 1.31→1.32 hop (U24, U4). (2) The karpenter-controller node group uses AL2 AMI type (AL2_x86_64), which is removed in Kubernetes 1.33 — nodes cannot be launched after the 1.32→1.33 hop (U9). Additionally, Karpenter v1.1.0 is incompatible with Kubernetes 1.32+ (requires ≥1.2) and 1.33+ (requires ≥1.5) per the official compatibility matrix (U19). All blockers and the Karpenter version gap must be addressed before beginning the multi-hop upgrade sequence.
Check Results Summary¶
| ID | Check | Category | Severity | Status | Timing | Impacted Resources |
|---|---|---|---|---|---|---|
| U1 | Cluster Version and Support Status | Control Plane | High | ⚠️ WARNING | — | 1.30 (EXTENDED_SUPPORT) |
| U2 | Target Version Availability | Control Plane | — | ✅ PASS | — | — |
| U3 | Multi-Version Hop Detection | Control Plane | High | ⚠️ WARNING | Before | 3 sequential upgrades required |
| U4 | EKS Upgrade Insights | Control Plane | Critical | 🚫 BLOCKER | Before | 3 deprecated API paths detected |
| U5 | EKS Managed Addon Compatibility | Addons | — | ✅ PASS | — | — |
| U6 | Self-Managed Addon Detection | Addons | — | ✅ PASS | — | — |
| U7 | Addon Health Issues | Addons | — | ✅ PASS | — | — |
| U8 | Node Kubelet Version Skew | Data Plane | Critical | 🔶 POST-UPGRADE | After | 3 nodes (v1.30.14-eks-ecaa3a6) |
| U9 | Node Group AMI Type | Data Plane | Critical | 🚫 BLOCKER | Before | karpenter-controller (AL2_x86_64) |
| U10 | Node Group Pending Updates | Data Plane | — | ✅ PASS | — | — |
| U11 | Subnet IP Availability | Networking | — | ✅ PASS | — | — |
| U12 | PodDisruptionBudget Coverage | Workload | High | ⚠️ WARNING | Before | 50 workloads in default namespace |
| U13 | Single-Replica Workloads | Workload | — | ✅ PASS | — | — |
| U14 | Missing Readiness Probes | Workload | High | ⚠️ WARNING | Before | 80 containers across workloads |
| U15 | Pod Topology and Anti-Affinity | Workload | Medium | ⚠️ WARNING | Before | 50 deployments in default namespace |
| U16 | MinReadySeconds Configuration | Workload | — | ✅ PASS | — | — |
| U17 | TerminationGracePeriodSeconds Zero | Workload | — | ✅ PASS | — | — |
| U18 | Cluster Autoscaler Version Compatibility | Components | — | ✅ PASS | — | — |
| U19 | Karpenter Version Compatibility | Components | High | ⚠️ WARNING | Before | Karpenter v1.1.0 (kube-system) |
| U20 | Third-Party Controller Inventory | Components | Medium | ⚠️ WARNING | Before | aws-load-balancer-controller v3.1.0 |
| U21 | Control Plane Logging | Observability | High | ⚠️ WARNING | Before | Audit logging disabled |
| U22 | Cluster Health Issues | Control Plane | — | ✅ PASS | — | — |
| U23 | kube-proxy Version Skew | Addons | Critical | 🔶 POST-UPGRADE | After | kube-proxy 1.30.14-eksbuild.28 |
| U24 | Deprecated APIs in Live Resources | API Compatibility | Critical | 🚫 BLOCKER | Before | 31 resources (v1beta3) |
| U25 | Deprecated APIs in Helm Releases | API Compatibility | — | ✅ PASS | — | — |
| U26 | Deprecated APIs Warning (Future Versions) | API Compatibility | — | ✅ PASS | — | — |
| U27 | Node Subnet IP Availability | Networking | — | ✅ PASS | — | — |
| U28 | Pod Subnet IP Availability (Custom Networking) | Networking | — | ✅ PASS | — | — |
| U29 | EC2 Instance Service Limits | Capacity | — | ✅ PASS | — | — |
| U30 | EBS GP2 Volume Service Limits | Capacity | — | ✅ PASS | — | — |
| U31 | EBS GP3 Volume Service Limits | Capacity | — | ✅ PASS | — | — |
| U32 | Self-Managed Node Group Launch Template Updates | Data Plane | Medium | ⚠️ WARNING | After | 1 self-managed node |
| U33 | kube-proxy IPVS Mode Deprecation | Networking | — | ✅ PASS | — | — |
| U34 | Ingress NGINX Controller Retirement | Components | — | ✅ PASS | — | — |
| U35 | Docker Socket Mounts | Workload | — | ✅ PASS | — | — |
| U36 | Workloads Scaled to Zero | Workload | — | ✅ PASS | — | — |
| U37 | EKS Configuration Insights | Control Plane | — | ✅ PASS | — | — |
| U38 | Third-Party API Deprecations | API Compatibility | — | ✅ PASS | — | — |
Blockers (Must Fix Before Upgrade)¶
1. Deprecated APIs in Live Resources (U24)¶
| Field | Detail |
|---|---|
| Severity | Critical |
| Category | API Compatibility |
| Upgrade Timing | Before (blocks 1.31 → 1.32 hop) |
Finding: 31 live resources use flowcontrol.apiserver.k8s.io/v1beta3, which is removed in Kubernetes 1.32. These resources will fail to be served after upgrading past 1.31. This is corroborated by EKS Upgrade Insights (U4), which detected active API calls from clients awslabs and kubectl to these deprecated endpoints within the last 30 days.
Impacted Resources (31 total):
FlowSchemas (20+):
- catch-all FlowSchema
- eks-exempt FlowSchema
- eks-leader-election FlowSchema
- eks-monitoring FlowSchema
- eks-workload-high FlowSchema
- endpoint-controller FlowSchema
- exempt FlowSchema
- gameday-flow-1 FlowSchema
- gameday-flow-2 FlowSchema
- gameday-flow-3 FlowSchema
- global-default FlowSchema
- kube-controller-manager FlowSchema
- kube-scheduler FlowSchema
- kube-system-service-accounts FlowSchema
- probes FlowSchema
- service-accounts FlowSchema
- system-leader-election FlowSchema
- system-node-high FlowSchema
- system-nodes FlowSchema
- workload-leader-election FlowSchema
PriorityLevelConfigurations (10):
- catch-all PriorityLevelConfiguration
- eks-monitoring PriorityLevelConfiguration
- exempt PriorityLevelConfiguration
- gameday-priority-1 PriorityLevelConfiguration
- gameday-priority-2 PriorityLevelConfiguration
- global-default PriorityLevelConfiguration
- leader-election PriorityLevelConfiguration
- node-high PriorityLevelConfiguration
- system PriorityLevelConfiguration
- workload-high PriorityLevelConfiguration
Additional resource from U4:
- test-flow-schema FlowSchema
Remediation:
Many of the eks-*, system-*, and built-in FlowSchemas/PriorityLevelConfigurations (e.g., catch-all, exempt, global-default, kube-controller-manager, kube-scheduler, probes, service-accounts) are EKS-managed or Kubernetes system resources. These are automatically migrated by EKS during the control plane upgrade to 1.32. You do not need to manually migrate these.
For custom/user-created resources (gameday-flow-1, gameday-flow-2, gameday-flow-3, gameday-priority-1, gameday-priority-2, test-flow-schema), you must manually migrate them:
# Step 1: Export custom FlowSchemas using the v1 API
kubectl get flowschema gameday-flow-1 -o yaml | \
sed 's|flowcontrol.apiserver.k8s.io/v1beta3|flowcontrol.apiserver.k8s.io/v1|g' > gameday-flow-1-v1.yaml
kubectl get flowschema gameday-flow-2 -o yaml | \
sed 's|flowcontrol.apiserver.k8s.io/v1beta3|flowcontrol.apiserver.k8s.io/v1|g' > gameday-flow-2-v1.yaml
kubectl get flowschema gameday-flow-3 -o yaml | \
sed 's|flowcontrol.apiserver.k8s.io/v1beta3|flowcontrol.apiserver.k8s.io/v1|g' > gameday-flow-3-v1.yaml
kubectl get flowschema test-flow-schema -o yaml | \
sed 's|flowcontrol.apiserver.k8s.io/v1beta3|flowcontrol.apiserver.k8s.io/v1|g' > test-flow-schema-v1.yaml
# Step 2: Export custom PriorityLevelConfigurations using the v1 API
kubectl get prioritylevelconfiguration gameday-priority-1 -o yaml | \
sed 's|flowcontrol.apiserver.k8s.io/v1beta3|flowcontrol.apiserver.k8s.io/v1|g' > gameday-priority-1-v1.yaml
kubectl get prioritylevelconfiguration gameday-priority-2 -o yaml | \
sed 's|flowcontrol.apiserver.k8s.io/v1beta3|flowcontrol.apiserver.k8s.io/v1|g' > gameday-priority-2-v1.yaml
# Step 3: Apply the migrated resources
kubectl apply -f gameday-flow-1-v1.yaml
kubectl apply -f gameday-flow-2-v1.yaml
kubectl apply -f gameday-flow-3-v1.yaml
kubectl apply -f test-flow-schema-v1.yaml
kubectl apply -f gameday-priority-1-v1.yaml
kubectl apply -f gameday-priority-2-v1.yaml
# Step 4: Verify all resources now use v1
kubectl get flowschemas.v1.flowcontrol.apiserver.k8s.io
kubectl get prioritylevelconfigurations.v1.flowcontrol.apiserver.k8s.io
Note: On Kubernetes 1.30, both
v1beta3andv1are served. Applying viav1will update the stored version. Thev1beta3API is still available on 1.30 and 1.31 but is removed in 1.32.
Considerations: FlowSchemas and PriorityLevelConfigurations control API priority and fairness. Incorrectly migrated resources could affect API server throttling behavior. Test the migration in a non-production cluster first. Also update any IaC (Terraform, Helm, manifests) that references v1beta3 to prevent drift.
2. EKS Upgrade Insights — Deprecated API Usage (U4)¶
| Field | Detail |
|---|---|
| Severity | Critical |
| Category | Control Plane |
| Upgrade Timing | Before (blocks 1.31 → 1.32 hop) |
Finding: EKS Upgrade Insights detected active deprecated API calls within the last 30 days. 1 of 6 insight checks is not passing. Clients awslabs (4 requests) and kubectl (5 requests) are actively calling flowcontrol.apiserver.k8s.io/v1beta3 endpoints. This confirms the deprecated APIs flagged in U24 are actively in use, not just stored.
Impacted API Paths:
- /apis/flowcontrol.apiserver.k8s.io/v1beta3/flowschemas → replace with /apis/flowcontrol.apiserver.k8s.io/v1/flowschemas (removed in 1.32)
- /apis/flowcontrol.apiserver.k8s.io/v1beta3/prioritylevelconfigurations → replace with /apis/flowcontrol.apiserver.k8s.io/v1/prioritylevelconfigurations (removed in 1.32)
- /apis/flowcontrol.apiserver.k8s.io/v1beta3/flowschemas/test-flow-schema (live resource)
Active Clients: | Client | API Path | Requests (30d) | |--------|----------|----------------| | awslabs | flowschemas (v1beta3) | 2 | | kubectl | flowschemas (v1beta3) | 5 | | awslabs | prioritylevelconfigurations (v1beta3) | 2 |
Remediation:
- Migrate stored resources — see U24 remediation above.
- Update client tooling:
awslabsclient: Identify the tool/controller making these calls. Update to a version that usesflowcontrol.apiserver.k8s.io/v1.kubectlclient: Ensure all operators/CI pipelines use kubectl v1.32+ which defaults to thev1API.
# Identify which tool "awslabs" refers to
kubectl get pods -A -o jsonpath='{range .items[*]}{.metadata.namespace}/{.metadata.name}: {.spec.containers[*].image}{"\n"}{end}' | grep awslabs
- Enable audit logging (see U21) to continuously monitor for deprecated API usage before each hop.
Considerations: Even after migrating stored resources, if clients continue calling v1beta3 endpoints, those calls will fail after the 1.32 upgrade. Both stored resources AND client behavior must be updated.
3. Node Group AMI Type — AL2 Removal (U9)¶
| Field | Detail |
|---|---|
| Severity | Critical |
| Category | Data Plane |
| Upgrade Timing | Before (blocks 1.32 → 1.33 hop) |
Finding: The karpenter-controller node group uses AMI type AL2_x86_64. Amazon Linux 2 (AL2) AMI support is removed in Kubernetes 1.33+. Nodes using AL2 cannot be launched or replaced after upgrading to 1.33.
Impacted Resources:
- karpenter-controller (AMI type: AL2_x86_64)
Remediation:
Since this is a Karpenter-managed node group, the AMI family is configured in the EC2NodeClass resource:
# Step 1: Identify the EC2NodeClass(es) using AL2
kubectl get ec2nodeclasses -o yaml | grep -A5 "amiFamily"
# Step 2: Update the EC2NodeClass to use AL2023
# TEMPLATE — replace <NODECLASS_NAME> with the actual EC2NodeClass name
kubectl patch ec2nodeclass <NODECLASS_NAME> --type=merge \
-p '{"spec":{"amiFamily":"AL2023"}}'
# Step 3: If using a managed node group for Karpenter controller itself,
# update via AWS CLI or console:
# TEMPLATE — replace <NODEGROUP_NAME> with actual node group name
aws eks update-nodegroup-config \
--cluster-name eks-review-cluster \
--nodegroup-name karpenter-controller \
--region us-east-1
# Note: AMI type change requires creating a new node group or updating the launch template
For managed node groups, AMI type changes typically require creating a new node group with AL2023 and migrating workloads:
# Option: Create new node group with AL2023 and drain the old one
# TEMPLATE — replace values as needed
aws eks create-nodegroup \
--cluster-name eks-review-cluster \
--nodegroup-name karpenter-controller-al2023 \
--ami-type AL2023_x86_64_STANDARD \
--region us-east-1 \
# ... (copy remaining config from existing node group)
# After new nodes are ready, cordon and drain old nodes
kubectl cordon <old-node-name>
kubectl drain <old-node-name> --ignore-daemonsets --delete-emptydir-data
Considerations:
- AL2023 uses dnf instead of yum, and has different default kernel parameters. Test workload compatibility.
- AL2023 uses systemd cgroup driver by default (same as EKS-optimized AMIs).
- If custom user data scripts reference AL2-specific paths or packages, they must be updated for AL2023.
- This migration must be completed before the 1.32→1.33 hop but can be done at any point during the upgrade sequence (AL2 is supported through 1.32).
- The Karpenter controller itself runs on this node group — plan for brief controller unavailability during node replacement.
Warnings (Recommended)¶
1. Cluster Version and Support Status (U1)¶
| Field | Detail |
|---|---|
| Severity | High |
| Upgrade Timing | — |
Finding: Current version 1.30 is in EXTENDED_SUPPORT, which incurs higher costs. Standard support for 1.30 ended 2025-07-23. Target version 1.33 is in STANDARD_SUPPORT.
Recommendation: Proceed with the upgrade to return to standard support pricing. Each hop should be completed promptly to minimize extended support costs.
2. Multi-Version Hop Detection (U3)¶
| Field | Detail |
|---|---|
| Severity | High |
| Upgrade Timing | Before |
Finding: EKS only supports upgrading one minor version at a time. The 1.30→1.33 upgrade requires 3 sequential control plane upgrades: 1.30→1.31, 1.31→1.32, 1.32→1.33. Each hop takes 15–25 minutes for the control plane alone.
Recommendation: Plan for approximately 45–75 minutes of control plane upgrade time across all 3 hops, plus additional time for node group updates and addon upgrades at each step. See the Upgrade Plan section below for the detailed sequence.
3. Karpenter Version Compatibility (U19)¶
| Field | Detail |
|---|---|
| Severity | High |
| Upgrade Timing | Before |
Finding: Karpenter v1.1.0 is installed (namespace: kube-system, image: public.ecr.aws/karpenter/controller:1.1.0). Per the Karpenter compatibility matrix:
| Kubernetes Version | Minimum Karpenter Version | Current (v1.1.0) Compatible? |
|---|---|---|
| 1.30 | ≥ 0.37 | ✅ Yes |
| 1.31 | ≥ 1.0.5 | ✅ Yes |
| 1.32 | ≥ 1.2 | ❌ No |
| 1.33 | ≥ 1.5 | ❌ No |
Karpenter v1.1.0 is compatible through the 1.30→1.31 hop but must be upgraded to ≥1.2 before the 1.31→1.32 hop and ≥1.5 before the 1.32→1.33 hop.
Recommendation: Upgrade Karpenter incrementally alongside each Kubernetes hop:
# Before 1.31→1.32 hop: Upgrade Karpenter to v1.2+
# Check Karpenter upgrade guide for breaking changes between 1.1.0 and 1.2+
helm upgrade karpenter oci://public.ecr.aws/karpenter/karpenter \
--version 1.2.1 \
--namespace kube-system \
--reuse-values
# Before 1.32→1.33 hop: Upgrade Karpenter to v1.5+
helm upgrade karpenter oci://public.ecr.aws/karpenter/karpenter \
--version 1.5.0 \
--namespace kube-system \
--reuse-values
Important: Review the Karpenter upgrade guide for breaking changes at each major version boundary. Karpenter follows semver — minor version bumps in 0.x and major version bumps in 1.x+ may contain breaking changes.
4. Control Plane Logging (U21)¶
| Field | Detail |
|---|---|
| Severity | High |
| Upgrade Timing | Before |
Finding: Control plane audit logging is NOT enabled. No log types are currently enabled. EKS Upgrade Insights relies on audit logs to detect deprecated API usage from controllers and operators. Without audit logs, the deprecated API detection in U4 may be incomplete.
Recommendation: Enable audit logging immediately, before beginning the upgrade sequence:
aws eks update-cluster-config \
--name eks-review-cluster \
--region us-east-1 \
--logging '{"clusterLogging":[{"types":["audit","api","authenticator","controllerManager","scheduler"],"enabled":true}]}'
Wait at least 24–48 hours after enabling audit logging before proceeding with the first upgrade hop. This allows EKS Upgrade Insights to collect sufficient data to detect any additional deprecated API usage not captured in the current assessment.
5. PodDisruptionBudget Coverage (U12)¶
| Field | Detail |
|---|---|
| Severity | High |
| Upgrade Timing | Before |
Finding: 50 workloads with multiple replicas in the default namespace lack PodDisruptionBudget (PDB) coverage. Without PDBs, all pods can be evicted simultaneously during node rolling updates, causing service disruption.
Impacted Resources (sample from default namespace):
- nginx-1 (Deployment, 3 replicas)
- nginx-10 (Deployment, 3 replicas)
- nginx-11 (Deployment, 3 replicas)
- ... and 47 more
Recommendation: Create PDBs for all multi-replica workloads before starting the upgrade:
# Example PDB for a deployment with 3 replicas
# TEMPLATE — replace <DEPLOYMENT_NAME> and <LABEL_SELECTOR> for each workload
kubectl create pdb <DEPLOYMENT_NAME>-pdb \
--namespace default \
--selector=app=<LABEL_SELECTOR> \
--min-available=1
# Verify PDB coverage
kubectl get pdb -n default
6. Missing Readiness Probes (U14)¶
| Field | Detail |
|---|---|
| Severity | High |
| Upgrade Timing | Before |
Finding: 80 containers (50 reported + 30 more) are missing readiness probes. Traffic may be routed to unready pods after node replacement during rolling updates.
Impacted Resources (sample from default namespace):
- ReplicaSet/nginx-1-7f679f8d4c container: nginx
- ReplicaSet/nginx-10-768f774ff container: nginx
- ReplicaSet/nginx-11-56df9f6659 container: nginx
- ... and 17 more in default namespace (plus additional in other namespaces)
Recommendation: Add readiness probes to all containers before the upgrade. For nginx workloads:
readinessProbe:
httpGet:
path: /
port: 80
initialDelaySeconds: 5
periodSeconds: 10
7. Pod Topology and Anti-Affinity (U15)¶
| Field | Detail |
|---|---|
| Severity | Medium |
| Upgrade Timing | Before |
Finding: 50 deployments lack topology spread constraints or pod anti-affinity rules. All replicas could be scheduled on the same node, creating a single point of failure during node replacement.
Impacted Resources (sample from default namespace):
- nginx-1 (3 replicas)
- nginx-10 (3 replicas)
- nginx-11 (3 replicas)
- ... and 12 more in default namespace
Recommendation: Add topology spread constraints to distribute pods across nodes:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: kubernetes.io/hostname
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: <app-label>
8. Third-Party Controller Inventory (U20)¶
| Field | Detail |
|---|---|
| Severity | Medium |
| Upgrade Timing | Before |
Finding: AWS Load Balancer Controller v3.1.0 detected (namespace: kube-system, image: public.ecr.aws/eks/aws-load-balancer-controller:v3.1.0).
Compatibility Verification: Per the AWS Load Balancer Controller installation docs, the supported Kubernetes versions are: - v2.4.0+ requires Kubernetes 1.19+ - v2.5.0+ requires Kubernetes 1.22+ - v3.1.0 is the current latest release
AWS Load Balancer Controller v3.1.0 is compatible with Kubernetes 1.33. ✅ No action required for this component.
9. kube-proxy Version Skew — Post-Upgrade (U23)¶
| Field | Detail |
|---|---|
| Severity | Critical (post-upgrade) |
| Upgrade Timing | After each hop |
Finding: kube-proxy v1.30.14-eksbuild.28 will be 3 minor versions behind target 1.33. kube-proxy must be updated after each control plane hop to stay within the supported 1-version skew.
Impacted Resources:
- kube-proxy 1.30.14-eksbuild.28 (skew=3 with target 1.33)
Recommendation: Update kube-proxy as an EKS managed addon after each control plane upgrade:
# After each control plane hop, update kube-proxy
# The exact version will be determined by the target K8s version at each hop
aws eks update-addon \
--cluster-name eks-review-cluster \
--addon-name kube-proxy \
--resolve-conflicts PRESERVE \
--region us-east-1
10. Node Kubelet Version Skew — Post-Upgrade (U8)¶
| Field | Detail |
|---|---|
| Severity | Critical (post-upgrade) |
| Upgrade Timing | After each hop |
Finding: 3 nodes running kubelet v1.30.14-eks-ecaa3a6 will be 3 minor versions behind after upgrading to 1.33. Kubernetes supports a maximum skew of 3 minor versions between kubelet and the API server, but best practice is to keep them aligned.
Impacted Nodes:
- ip-192-168-26-84.ec2.internal (kubelet v1.30.14-eks-ecaa3a6)
- ip-192-168-40-195.ec2.internal (kubelet v1.30.14-eks-ecaa3a6)
- ip-192-168-87-204.ec2.internal (kubelet v1.30.14-eks-ecaa3a6) — also a self-managed node (U32)
Recommendation: Update node groups after each control plane hop. For managed node groups, use aws eks update-nodegroup-version. For the self-managed node ip-192-168-87-204.ec2.internal, manually update the AMI and replace the instance.
11. Self-Managed Node Group Launch Template Updates (U32)¶
| Field | Detail |
|---|---|
| Severity | Medium |
| Upgrade Timing | After each hop |
Finding: 1 self-managed node detected. Self-managed nodes are not automatically upgraded by EKS. You must manually update their AMIs and replace them after each control plane upgrade.
Impacted Resources:
- ip-192-168-87-204.ec2.internal
Recommendation: After each control plane hop: 1. Update the launch template with the new EKS-optimized AMI for the target version 2. Cordon and drain the old node 3. Terminate the old instance and let the Auto Scaling Group launch a replacement
# After each control plane upgrade
kubectl cordon ip-192-168-87-204.ec2.internal
kubectl drain ip-192-168-87-204.ec2.internal --ignore-daemonsets --delete-emptydir-data
# Then terminate the instance via AWS Console or CLI after updating the launch template
Upgrade Plan¶
⚠️ This is a 3-hop upgrade (1.30 → 1.31 → 1.32 → 1.33). Execute all phases in order. Do not skip hops.
Phase 0: Pre-Upgrade Preparation (Before Any Hop)¶
| # | Action | Command | Notes |
|---|---|---|---|
| 0a | Enable control plane audit logging | aws eks update-cluster-config --name eks-review-cluster --region us-east-1 --logging '{"clusterLogging":[{"types":["audit","api","authenticator","controllerManager","scheduler"],"enabled":true}]}' |
Wait 24–48h for Upgrade Insights to collect data (U21) |
| 0b | Migrate custom FlowSchemas to v1 | See U24 remediation commands above | Migrate gameday-flow-1/2/3, test-flow-schema (U24) |
| 0c | Migrate custom PriorityLevelConfigurations to v1 | See U24 remediation commands above | Migrate gameday-priority-1/2 (U24) |
| 0d | Verify deprecated API migration | kubectl get flowschemas.v1.flowcontrol.apiserver.k8s.io |
Confirm all custom resources use v1 |
| 0e | Update client tooling | Update awslabs tool and kubectl to versions using v1 APIs |
Stop deprecated API calls (U4) |
| 0f | Create PDBs for multi-replica workloads | Create PDBs for 50 workloads in default namespace | Prevents simultaneous pod eviction (U12) |
| 0g | Add readiness probes | Update 80 containers with readiness probes | Prevents traffic to unready pods (U14) |
| 0h | Add topology spread constraints | Update 50 deployments with topology constraints | Prevents single-node scheduling (U15) |
Phase 1: Upgrade 1.30 → 1.31¶
| # | Action | Command | Est. Duration |
|---|---|---|---|
| 1a | Upgrade control plane to 1.31 | aws eks update-cluster-version --name eks-review-cluster --kubernetes-version 1.31 --region us-east-1 |
15–25 min |
| 1b | Monitor upgrade | aws eks describe-update --name eks-review-cluster --update-id <UPDATE_ID> --region us-east-1 |
— |
| 1c | Update kube-proxy addon | aws eks update-addon --cluster-name eks-review-cluster --addon-name kube-proxy --resolve-conflicts PRESERVE --region us-east-1 |
2–5 min |
| 1d | Update managed node groups | aws eks update-nodegroup-version --cluster-name eks-review-cluster --nodegroup-name <NODEGROUP_NAME> --region us-east-1 |
10–20 min per group |
| 1e | Update self-managed node | Update launch template AMI to 1.31, cordon/drain/replace ip-192-168-87-204.ec2.internal |
5–10 min |
| 1f | Verify | kubectl get nodes -o wide && kubectl get pods -n kube-system |
— |
Phase 2: Upgrade 1.31 → 1.32¶
| # | Action | Command | Est. Duration |
|---|---|---|---|
| 2a | Upgrade Karpenter to ≥1.2 | helm upgrade karpenter oci://public.ecr.aws/karpenter/karpenter --version 1.2.1 --namespace kube-system --reuse-values |
2–5 min |
| 2b | Verify Karpenter health | kubectl get pods -n kube-system -l app.kubernetes.io/name=karpenter |
— |
| 2c | Upgrade control plane to 1.32 | aws eks update-cluster-version --name eks-review-cluster --kubernetes-version 1.32 --region us-east-1 |
15–25 min |
| 2d | Monitor upgrade | aws eks describe-update --name eks-review-cluster --update-id <UPDATE_ID> --region us-east-1 |
— |
| 2e | Verify system FlowSchemas migrated | kubectl get flowschemas -o jsonpath='{range .items[*]}{.metadata.name}: {.apiVersion}{"\n"}{end}' |
EKS auto-migrates system resources |
| 2f | Update kube-proxy addon | aws eks update-addon --cluster-name eks-review-cluster --addon-name kube-proxy --resolve-conflicts PRESERVE --region us-east-1 |
2–5 min |
| 2g | Update managed node groups | aws eks update-nodegroup-version --cluster-name eks-review-cluster --nodegroup-name <NODEGROUP_NAME> --region us-east-1 |
10–20 min per group |
| 2h | Update self-managed node | Update launch template AMI to 1.32, cordon/drain/replace | 5–10 min |
| 2i | Verify | kubectl get nodes -o wide && kubectl get pods -n kube-system |
— |
Phase 3: Upgrade 1.32 → 1.33¶
| # | Action | Command | Est. Duration |
|---|---|---|---|
| 3a | Migrate AL2 → AL2023 | Migrate karpenter-controller node group from AL2_x86_64 to AL2023_x86_64_STANDARD (see U9 remediation) |
15–30 min |
| 3b | Upgrade Karpenter to ≥1.5 | helm upgrade karpenter oci://public.ecr.aws/karpenter/karpenter --version 1.5.0 --namespace kube-system --reuse-values |
2–5 min |
| 3c | Update EC2NodeClass amiFamily | kubectl patch ec2nodeclass <NODECLASS_NAME> --type=merge -p '{"spec":{"amiFamily":"AL2023"}}' |
Ensure Karpenter provisions AL2023 nodes |
| 3d | Verify Karpenter health | kubectl get pods -n kube-system -l app.kubernetes.io/name=karpenter |
— |
| 3e | Upgrade control plane to 1.33 | aws eks update-cluster-version --name eks-review-cluster --kubernetes-version 1.33 --region us-east-1 |
15–25 min |
| 3f | Monitor upgrade | aws eks describe-update --name eks-review-cluster --update-id <UPDATE_ID> --region us-east-1 |
— |
| 3g | Update kube-proxy addon | aws eks update-addon --cluster-name eks-review-cluster --addon-name kube-proxy --resolve-conflicts PRESERVE --region us-east-1 |
2–5 min |
| 3h | Update managed node groups | aws eks update-nodegroup-version --cluster-name eks-review-cluster --nodegroup-name <NODEGROUP_NAME> --region us-east-1 |
10–20 min per group |
| 3i | Update self-managed node | Update launch template AMI to 1.33, cordon/drain/replace | 5–10 min |
| 3j | Verify | kubectl get nodes -o wide && kubectl get pods -n kube-system |
— |
Phase 4: Post-Upgrade Verification¶
# Verify cluster version
kubectl version
# Verify all nodes are at target version
kubectl get nodes -o wide
# Verify kube-system pods are healthy
kubectl get pods -n kube-system
# Verify no pods in error state
kubectl get pods -A --field-selector status.phase!=Running,status.phase!=Succeeded
# Verify Karpenter is functioning
kubectl get nodeclaims
kubectl get nodepools
# Verify AWS Load Balancer Controller
kubectl get pods -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller
# Verify no deprecated API usage remains
kubectl get --raw /metrics | grep apiserver_requested_deprecated_apis
Estimated Total Duration¶
| Phase | Duration |
|---|---|
| Phase 0: Pre-upgrade fixes | 1–4 hours (depending on PDB/probe work) |
| Phase 1: 1.30 → 1.31 | 30–60 min |
| Phase 2: 1.31 → 1.32 | 35–65 min (includes Karpenter upgrade) |
| Phase 3: 1.32 → 1.33 | 50–90 min (includes AL2→AL2023 + Karpenter upgrade) |
| Phase 4: Verification | 15–30 min |
| Total | ~2.5–5 hours (excluding Phase 0 prep work) |
Passed Checks¶
| ID | Check | Category |
|---|---|---|
| U2 | Target Version Availability | Control Plane |
| U5 | EKS Managed Addon Compatibility | Addons |
| U6 | Self-Managed Addon Detection | Addons |
| U7 | Addon Health Issues | Addons |
| U10 | Node Group Pending Updates | Data Plane |
| U11 | Subnet IP Availability | Networking |
| U13 | Single-Replica Workloads | Workload |
| U16 | MinReadySeconds Configuration | Workload |
| U17 | TerminationGracePeriodSeconds Zero | Workload |
| U18 | Cluster Autoscaler Version Compatibility | Components |
| U22 | Cluster Health Issues | Control Plane |
| U25 | Deprecated APIs in Helm Releases | API Compatibility |
| U26 | Deprecated APIs Warning (Future Versions) | API Compatibility |
| U27 | Node Subnet IP Availability | Networking |
| U28 | Pod Subnet IP Availability (Custom Networking) | Networking |
| U29 | EC2 Instance Service Limits | Capacity |
| U30 | EBS GP2 Volume Service Limits | Capacity |
| U31 | EBS GP3 Volume Service Limits | Capacity |
| U33 | kube-proxy IPVS Mode Deprecation | Networking |
| U34 | Ingress NGINX Controller Retirement | Components |
| U35 | Docker Socket Mounts | Workload |
| U36 | Workloads Scaled to Zero | Workload |
| U37 | EKS Configuration Insights | Control Plane |
| U38 | Third-Party API Deprecations | API Compatibility |
Appendix: Third-Party Components¶
| Component | Version | Namespace | Compatibility with K8s 1.33 | Source |
|---|---|---|---|---|
| Karpenter | v1.1.0 | kube-system | ❌ Requires ≥ v1.5 | Karpenter Compatibility Matrix |
| AWS Load Balancer Controller | v3.1.0 | kube-system | ✅ Compatible (v2.5.0+ supports K8s 1.22+) | AWS LBC Installation Docs |
Appendix: Related Finding Correlations¶
| Correlation | Findings | Impact |
|---|---|---|
| Deprecated API usage (stored + active) | U24 + U4 | Same root cause — flowcontrol.apiserver.k8s.io/v1beta3 resources and API calls must both be migrated |
| Self-managed node + kubelet skew | U32 + U8 | ip-192-168-87-204.ec2.internal is both self-managed and has kubelet skew — requires manual AMI update |
| Karpenter + AL2 AMI | U19 + U9 | karpenter-controller node group uses AL2 and runs Karpenter — both must be updated before 1.33 |
| Workload resilience cluster | U12 + U14 + U15 | 50 deployments in default namespace lack PDBs, readiness probes, and topology constraints — address together |