This page is generated from devops-agent/eks-recon/references/workloads.md. Edit the source, not this page.
Module: Workloads
Part of: eks-recon Purpose: Detect running workloads — deployments, services, ingresses, jobs
Note: This is an optional module for migration assessments or workload inventory. Skip for basic infrastructure recon.
Table of Contents
- Access Model
- Detection Strategy
- Detection Capabilities
- Summary Statistics
- Output Schema
- Additional Fact Collection
- Edge Cases
Access Model
This module reads facts almost entirely from the Kubernetes API — deployments, statefulsets, daemonsets, cronjobs/jobs, services, ingresses, HPAs, PDBs, priority classes, VPAs, and the live-resource apiVersion inventory are all in-cluster reads. Both sources are read-only:
- Kubernetes API (via the Agent Space EKS access entry) — all workload, networking, and
autoscaling resources below. Requires
authenticationModeto includeAPIand theAmazonAIOpsAssistantPolicyaccess entry to be present. RBAC verbs needed:get,list. - AWS control-plane APIs — this module makes no AWS-API calls of its own; the shared
cluster:block is sourced byreferences/cluster-basics.md.
If the Kubernetes API is unreachable (access entry absent), this module can produce almost
nothing — mark every workload sub-fact as unconfirmed in the report's Coverage section,
never as false/count: 0.
Reference pseudocode note. Code blocks labeled reference pseudocode (kubernetes client) below illustrate the resource, fields, and RBAC verbs for each K8s-API read. They are not executable in the Agent Space and are not an operational path — do not emit
kubectl ... | jqpipelines. The agent reads these resources through its Kubernetes-API capability, then applies the described selection/aggregation logic.
Detection Strategy
Workload inventory covers the following resource types. Start with Deployments (most common), then expand based on what you find:
1. Deployments -> Long-running applications
2. StatefulSets -> Stateful applications
3. DaemonSets -> Node-level agents
4. CronJobs/Jobs -> Batch workloads
5. Services -> Network exposure
6. Ingresses -> External access
7. HPAs -> Horizontal autoscaling configuration
8. PDBs -> Pod disruption budgets
9. PriorityClasses -> Scheduling priority definitions
10. VPAs -> Vertical autoscaling configuration (if CRD present)
11. API versions -> apiVersions live resources use (raw fact list)
Detection Capabilities
1. Deployments
Run this first to identify the main application workloads. Deployments are the most common workload type and reveal the core services running in the cluster.
Via Kubernetes API — list Deployments across all namespaces:
- Resource:
Deployment, group/versionapps/v1, all namespaces. - Fields to extract:
metadata.namespace,metadata.name,spec.replicas,status.readyReplicas, allspec.template.spec.containers[].image(not justcontainers[0]— a workload may run sidecars/proxies),spec.template.spec.initContainers[].name(empty list when none),metadata.labels. - Selection: exclude namespaces beginning with
kube-. - RBAC verbs:
get,listondeployments.apps.
Iterate all containers[].image (a workload may run multiple containers — sidecars,
proxies), not just containers[0]. Record init_containers (names) per workload; empty
list when none.
Example (one deployment):
{
"namespace": "production",
"name": "api-gateway",
"replicas": 3,
"ready": 3,
"images": [
"123456789012.dkr.ecr.us-west-2.amazonaws.com/api-gateway:v2.1.0",
"docker.io/envoyproxy/envoy:v1.28.0"
],
"init_containers": ["migrate-db"],
"labels": {"app": "api-gateway", "team": "platform"}
}
2. StatefulSets
Check StatefulSets when you need to understand stateful applications like databases, message queues, or caches. These require special handling during migrations due to persistent storage.
Via Kubernetes API — list StatefulSets across all namespaces:
- Resource:
StatefulSet, group/versionapps/v1, all namespaces. - Fields to extract:
metadata.namespace,metadata.name,spec.replicas,status.readyReplicas,spec.volumeClaimTemplates[].metadata.name(→volume_claims),spec.volumeClaimTemplates[].spec.storageClassName(→storage_class),spec.template.spec.initContainers[].name. - RBAC verbs:
get,listonstatefulsets.apps.
Capture the volumeClaimTemplate → storageClassName linkage: each StatefulSet volume
claim template names a storage class the pods provision volumes from. Record the storage
class per StatefulSet (null when the template omits it and the cluster default applies).
Example (one statefulset):
{
"namespace": "production",
"name": "redis-cluster",
"replicas": 3,
"volume_claims": ["data"],
"storage_class": "gp3"
}
3. DaemonSets
List DaemonSets to identify node-level agents (monitoring, logging, security). These run on every node and reveal operational tooling in use.
Via Kubernetes API — list DaemonSets across all namespaces:
- Resource:
DaemonSet, group/versionapps/v1, all namespaces. - Fields to extract:
metadata.namespace,metadata.name,status.desiredNumberScheduled(→desired),status.numberReady(→ready). - RBAC verbs:
get,listondaemonsets.apps.
Example (one daemonset):
{
"namespace": "kube-system",
"name": "aws-node",
"desired": 5,
"ready": 5
}
4. CronJobs and Jobs
Check batch workloads to understand scheduled tasks (backups, reports) and one-time jobs. Important for understanding maintenance windows and resource usage patterns.
Via Kubernetes API — list CronJobs across all namespaces:
- Resource:
CronJob, group/versionbatch/v1, all namespaces. - Fields to extract:
metadata.namespace,metadata.name,spec.schedule,spec.suspend,status.lastScheduleTime(→last_schedule, null if never run). - RBAC verbs:
get,listoncronjobs.batch.
Via Kubernetes API — list active/incomplete Jobs across all namespaces:
- Resource:
Job, group/versionbatch/v1, all namespaces. - Fields to extract:
metadata.namespace,metadata.name,status.active,status.failed. - Selection: count/list only jobs that have not completed successfully (i.e. not
status.succeeded == 1). - RBAC verbs:
get,listonjobs.batch.
Example (CronJob):
{
"namespace": "production",
"name": "db-backup",
"schedule": "0 2 * * *",
"suspend": false,
"lastSchedule": "2026-04-22T02:00:00Z"
}
5. Services
List Services to understand network exposure. Services reveal how applications communicate internally (ClusterIP) and externally (LoadBalancer, NodePort).
Via Kubernetes API — list Services across all namespaces:
- Resource:
Service, group/versionv1(core), all namespaces. - Fields to extract:
metadata.namespace,metadata.name,spec.type,spec.ports[](format each as"<port>/<protocol>"),spec.selector. - Selection: exclude namespaces beginning with
kube-. - RBAC verbs:
get,listonservices.
Record each service as a full entry (name, type, ports, selector), not only an
aggregate count by type.
Note: The
services.by_typefield (counts perClusterIP/NodePort/LoadBalancer) is aggregated by the agent from the enumerated services list above — no separate counting read is needed.
Example (one service):
{
"namespace": "production",
"name": "api-gateway",
"type": "LoadBalancer",
"ports": ["443/TCP", "80/TCP"],
"selector": {"app": "api-gateway"}
}
6. Ingresses
Check Ingresses to identify external access points and routing rules. Ingresses show domain mappings, TLS configuration, and which services are publicly accessible.
Via Kubernetes API — list Ingresses across all namespaces:
- Resource:
Ingress, group/versionnetworking.k8s.io/v1, all namespaces. - Fields to extract:
metadata.namespace,metadata.name,spec.ingressClassName(→class),spec.rules[].host(→hosts), andtls_enabled=truewhenspec.tlshas one or more entries, elsefalse. - RBAC verbs:
get,listoningresses.networking.k8s.io.
tls_enabled = true when the ingress declares one or more spec.tls entries, else false.
Example (one ingress):
{
"namespace": "production",
"name": "api-ingress",
"class": "alb",
"hosts": ["api.example.com", "api-internal.example.com"],
"tls_enabled": true
}
7. HPAs
List Horizontal Pod Autoscalers to understand autoscaling configuration. HPAs reveal which workloads scale automatically and their scaling thresholds.
Via Kubernetes API — list HPAs across all namespaces:
- Resource:
HorizontalPodAutoscaler, group/versionautoscaling/v2, all namespaces. - Fields to extract:
metadata.namespace,metadata.name,spec.scaleTargetRef.name(→target),spec.minReplicas,spec.maxReplicas,status.currentReplicas,spec.metrics[].type(→metrics). - RBAC verbs:
get,listonhorizontalpodautoscalers.autoscaling.
Example (one HPA):
{
"namespace": "production",
"name": "api-gateway-hpa",
"target": "api-gateway",
"minReplicas": 2,
"maxReplicas": 10,
"currentReplicas": 4,
"metrics": ["Resource", "External"]
}
8. Pod Disruption Budgets
Enumerate PodDisruptionBudgets (PDBs) across namespaces. A PDB records the minimum
availability guarantee for the pods matched by its selector. Record existence, the
minAvailable / maxUnavailable value, and the label selector (the workloads it covers).
Also record status.disruptionsAllowed — the point-in-time count of voluntary evictions the
Eviction API currently permits for the pods this PDB selects. 0 means it currently rejects
voluntary eviction of covered healthy pods, which can stall a drain of the nodes hosting them
(a PDB selecting zero pods also reports 0). Always record spec.unhealthyPodEvictionPolicy
(→ unhealthy_pod_eviction_policy) — it governs whether running-but-not-ready (unhealthy)
covered pods can be evicted, independent of the disruptions_allowed counter
(AlwaysAllow = evict such pods regardless of budget health; IfHealthyBudget, the unset
default, = only while the budget is met, currentHealthy >= desiredHealthy) — this is what
lets a drain make progress when a covered workload is unhealthy. Emit the value verbatim; use null only when
the field is genuinely unset — never drop the key.
Via Kubernetes API — list PDBs across all namespaces (VERIFIED live: PDBs expose MIN AVAILABLE / MAX UNAVAILABLE):
- Resource:
PodDisruptionBudget, group/versionpolicy/v1, all namespaces. - Fields to extract:
metadata.namespace,metadata.name,spec.minAvailable,spec.maxUnavailable,status.disruptionsAllowed(→disruptions_allowed),spec.unhealthyPodEvictionPolicy(→unhealthy_pod_eviction_policy),spec.selector.matchLabels(→selector). - RBAC verbs:
get,listonpoddisruptionbudgets.policy.
At most one of min_available / max_unavailable is set per PDB (both may be null for a
selector-only PDB). The selector matchLabels identify the covered workloads.
Example (one PDB):
{
"namespace": "production",
"name": "api-gateway-pdb",
"min_available": 1,
"max_unavailable": null,
"disruptions_allowed": 2,
"unhealthy_pod_eviction_policy": "AlwaysAllow",
"selector": {"app": "api-gateway"}
}
9. PriorityClasses
List PriorityClasses (cluster-scoped) to record scheduling priority definitions.
Via Kubernetes API — list PriorityClasses (cluster-scoped):
- Resource:
PriorityClass, group/versionscheduling.k8s.io/v1. - Fields to extract:
metadata.name,value,globalDefault(→global_default). - RBAC verbs:
get,listonpriorityclasses.scheduling.k8s.io.
Example (one PriorityClass):
{
"name": "high-priority",
"value": 1000000,
"global_default": false
}
10. Vertical Pod Autoscalers
List VerticalPodAutoscalers only if the CRD is present. VPA is an add-on CRD
(verticalpodautoscalers.autoscaling.k8s.io); guard the read so its absence is a clean
fact, not an error.
Via Kubernetes API — detect the CRD first, then list VPAs if present:
- CRD check:
CustomResourceDefinition, group/versionapiextensions.k8s.io/v1, nameverticalpodautoscalers.autoscaling.k8s.io. Absence is a fact — recorddetected: falsefor the whole block and skip the list read. - Resource (if CRD present):
VerticalPodAutoscaler, group/versionautoscaling.k8s.io/v1, all namespaces. - Fields to extract:
metadata.namespace,metadata.name,spec.targetRef.name(→target),spec.updatePolicy.updateMode(→update_mode). - RBAC verbs:
get,listoncustomresourcedefinitions.apiextensions.k8s.ioandverticalpodautoscalers.autoscaling.k8s.io.
Record detected: false for the whole block when the CRD is absent.
Example (one VPA):
{
"namespace": "production",
"name": "api-gateway-vpa",
"target": "api-gateway",
"update_mode": "Auto"
}
11. API Versions In Use
Enumerate the distinct apiVersion values that live resources in the cluster use, as a
raw fact list. This is inventory only.
Scope note: Report the apiVersions verbatim. Do not flag any as deprecated, removed, or upgrade-blocking, and do not cross-reference them against a target Kubernetes version — that deprecation math belongs to the
eks-upgrade-checkskill, not recon.
Via Kubernetes API — read live resources across the common workload/config kinds and
collect their apiVersion values:
- Resources:
Deployment,StatefulSet,DaemonSet,ReplicaSet(apps/v1);CronJob,Job(batch/v1);Ingress,NetworkPolicy(networking.k8s.io/v1);HorizontalPodAutoscaler(autoscaling/v2);PodDisruptionBudget(policy/v1) — all namespaces. - Aggregation: collect
metadata-levelapiVersion(the served version each object was read as) across all items, then de-duplicate and sort into a distinct list. This is a raw union of the versions observed in use, nothing more. - RBAC verbs:
get,liston each resource above.
Reference pseudocode (kubernetes client), not executable:
# Collect the served apiVersion of every live object across the common kinds,
# then de-duplicate. Raw inventory only — no deprecation classification.
versions = set()
for obj in live_workload_and_config_objects(): # deploy/sts/ds/rs/cronjob/job/ingress/hpa/pdb/netpol
versions.add(obj["apiVersion"])
api_versions_in_use = sorted(versions)
Example output:
apps/v1
autoscaling/v2
batch/v1
networking.k8s.io/v1
policy/v1
Summary Statistics
A high-level view of workload distribution across namespaces, for initial assessment before
diving into details. This is aggregated by the agent from the per-type Kubernetes-API
reads above (Deployments, StatefulSets, DaemonSets, Jobs) — no separate read is needed.
Group the collected objects by metadata.namespace, then within each namespace count total
objects and counts per kind.
Example output:
[
{
"namespace": "production",
"total": 12,
"by_kind": {"Deployment": 8, "StatefulSet": 2, "DaemonSet": 2}
},
{
"namespace": "staging",
"total": 6,
"by_kind": {"Deployment": 5, "Job": 1}
}
]
Per-Namespace Rollup
Aggregate the per-type listings collected above into the by_namespace schema block — one
row per namespace, with a count column per workload type. Do this explicitly: the schema
advertises per-namespace hpas/pdbs counts and the deployments/services kube-* handling,
but no other step produces them, so on a complete run they are otherwise dropped. Roll up from
the already-collected lists (no additional read) — count Deployments, StatefulSets, Services,
Ingresses, HPAs, and PDBs per namespace.
deployments and services are null for kube-* namespaces — their §1/§5 listings scope
kube-* out, so a count here would not be corroborated by the detailed lists (null ≠ 0:
null = column scoped out for that row). statefulsets, ingresses, hpas, and pdbs
include kube-*. A row exists for any namespace containing at least one of the six workload
types, so rows may be partial.
Reference pseudocode (agent-side aggregation), not executable:
# Roll the already-collected per-type lists up into by_namespace rows.
by_namespace = []
for ns in sorted(namespaces_with_any_workload):
sys = ns.startswith("kube-")
by_namespace.append({
"namespace": ns,
"deployments": None if sys else count(deployments, ns), # §1 scopes kube-* out
"statefulsets": count(statefulsets, ns),
"services": None if sys else count(services, ns), # §5 scopes kube-* out
"ingresses": count(ingresses, ns),
"hpas": count(hpas, ns),
"pdbs": count(pdbs, ns),
})
Example output:
[
{"namespace": "kube-system", "deployments": null, "statefulsets": 0, "services": null, "ingresses": 0, "hpas": 0, "pdbs": 1},
{"namespace": "production", "deployments": 8, "statefulsets": 2, "services": 5, "ingresses": 1, "hpas": 3, "pdbs": 2}
]
Output Schema
This is the single canonical schema for the workloads module — it carries every
workloads fact. The workloads-recon agent emits exactly this shape (plus the shared
cluster: block from references/cluster-basics.md). Use null where a fact was not
detected; never omit a key. Aggregate containers use the {count, list} wrapper. Every key
must trace to a collection step above — including derived/rolled-up keys (summary,
by_namespace, services.by_type); a schema key with no driving step is a defect (the
Per-Namespace Rollup step drives by_namespace).
PVCs are owned by the storage module. This schema does not carry a
storage.pvcsblock — see the storage module for PVC inventory (status, storage class, capacity).
workloads:
summary: # agent rolls these up from the per-type section counts above;
# namespaces_with_workloads = distinct namespaces across those lists
deployments: int # kube-*-scoped total (§1 excludes kube-*); a true 0 is a valid fact
statefulsets: int
daemonsets: int
cronjobs: int
jobs: int
services: int # kube-*-scoped total (§5 excludes kube-*); a true 0 is a valid fact
ingresses: int
hpas: int
pdbs: int
namespaces_with_workloads: int
# kube-* scope split across these columns: ONLY deployments and services exclude kube-*
# namespaces (their §1/§5 listings filter kube-* out). statefulsets, ingresses, hpas, and pdbs
# INCLUDE kube-* (their listings apply no namespace filter). A row exists wherever ANY of the six
# workload types is present, so rows may be partial. For a scoped-out column emit `null`, NOT 0 (schema
# header rule: null = fact not detected/collected) — e.g. a kube-system row shows `null` for
# deployments and services (column scoped out, not "zero found"; kube-system always runs CoreDNS
# Deployments + kube-dns Services) but real counts for statefulsets/ingresses/hpas/pdbs.
by_namespace:
- namespace: string
deployments: int|null # null for kube-* rows: §1 scopes kube-* out, so not counted here (never 0)
statefulsets: int
services: int|null # null for kube-* rows: §5 scopes kube-* out, so not counted here (never 0)
ingresses: int
hpas: int # per-namespace HPA count (from the Per-Namespace Rollup step)
pdbs: int # per-namespace PDB count (from the Per-Namespace Rollup step)
deployments:
count: int
list:
- namespace: string
name: string
replicas: int
ready: int # status.readyReplicas
images: list # ALL containers[].image
init_containers: list # initContainers[].name, [] when none
labels: object
statefulsets:
count: int
list:
- namespace: string
name: string
replicas: int
ready: int
volume_claims: list # volumeClaimTemplates[].metadata.name
storage_class: string # volumeClaimTemplates[].spec.storageClassName, null if unset
init_containers: list
daemonsets:
count: int
list:
- namespace: string
name: string
desired: int # status.desiredNumberScheduled
ready: int # status.numberReady
cronjobs:
count: int
list:
- namespace: string
name: string
schedule: string
suspend: bool
last_schedule: string # status.lastScheduleTime, null if never run
jobs:
count: int # active/incomplete jobs
list:
- namespace: string
name: string
active: int
failed: int
services:
count: int
by_type:
cluster_ip: int
load_balancer: int
node_port: int
external_name: int
list:
- namespace: string
name: string
type: string # ClusterIP | NodePort | LoadBalancer | ExternalName
ports: list # ["443/TCP", ...]
selector: object
ingresses:
count: int
list:
- namespace: string
name: string
class: string # spec.ingressClassName
hosts: list
tls_enabled: bool # true when spec.tls has >=1 entry
hpas:
count: int
list:
- namespace: string
name: string
target: string # spec.scaleTargetRef.name
min_replicas: int
max_replicas: int
current_replicas: int # status.currentReplicas
metrics: list # spec.metrics[].type
pdbs:
count: int
list:
- namespace: string
name: string
min_available: int|string # spec.minAvailable — plain int or "N%" string (K8s IntOrString); at most one is set (both may be null for a selector-only PDB)
max_unavailable: int|string # spec.maxUnavailable — plain int or "N%" string (K8s IntOrString)
disruptions_allowed: int # status.disruptionsAllowed — voluntary evictions the Eviction API currently permits for
# covered pods; 0 rejects eviction of covered healthy pods (also 0 when it selects no pods); null if status unpopulated
unhealthy_pod_eviction_policy: string # spec.unhealthyPodEvictionPolicy — emit verbatim; null when unset (API defaults behavior to IfHealthyBudget)
selector: object # spec.selector.matchLabels — covered workloads
priority_classes:
count: int
list:
- name: string
value: int
global_default: bool
vpas:
detected: bool # verticalpodautoscalers CRD present
count: int
list:
- namespace: string
name: string
target: string # spec.targetRef.name
update_mode: string # spec.updatePolicy.updateMode
api_versions_in_use: list # RAW distinct apiVersions used by live resources.
# NOT flagged deprecated/removed — see Scope note in §11.
images:
unique:
count: int
list: list # unique image strings across pods
by_registry:
- registry: string # normalized: bare "nginx" -> docker.io/library
count: int
Additional Fact Collection
Facts that feed the schema blocks above beyond the primary per-type listings. All are Kubernetes-API reads with agent-side selection/aggregation.
Pod Phase and Replica Counts
Enumerate pod phase and replica-vs-desired counts.
Via Kubernetes API — pods not in a healthy phase, and deployments not fully ready:
- Resource:
Pod, group/versionv1(core), all namespaces. Select pods whosestatus.phaseis neitherRunningnorSucceeded; report<namespace>/<name>: <phase>. - Resource:
Deployment, group/versionapps/v1, all namespaces. Select deployments wherestatus.readyReplicas != spec.replicas; report<namespace>/<name>: <ready>/<desired>. - RBAC verbs:
get,listonpodsanddeployments.apps.
Example output:
production/api-gateway: 2/3
staging/worker: 0/2
Resource Requests and Limits
Inventory pods that declare no resource requests (count).
Via Kubernetes API — count pods missing resource requests:
- Resource:
Pod, group/versionv1(core), all namespaces. - Selection: a pod counts when any container has
resources.requests == null; de-duplicate by<namespace>/<name>and count. - RBAC verbs:
get,listonpods.
Example output:
23
Inventory Container Images
Enumerate unique images and image counts by registry.
Via Kubernetes API — read pod container images, then aggregate:
- Resource:
Pod, group/versionv1(core), all namespaces. Collect everyspec.containers[].image. - Unique images: de-duplicate and sort the collected image strings.
- Images by registry: bucket each image by its registry, applying Docker Hub normalization (below).
- RBAC verbs:
get,listonpods.
Registry normalization rule (agent-side selection logic): a bare image reference like
nginx resolves to the Docker Hub library namespace, so bucket it under docker.io/library
(not under the literal first path segment). A two-segment reference whose first segment has
no . or : (e.g. bitnami/redis) is also a Docker Hub reference → docker.io. Otherwise
the first path segment is the registry host.
Reference pseudocode (kubernetes client), not executable:
# Bucket collected pod container images by registry with Docker Hub normalization.
def registry(image):
parts = image.split("/")
if len(parts) == 1: # bare "nginx"
return "docker.io/library"
if len(parts) == 2 and not any(c in parts[0] for c in ".:"): # "bitnami/redis"
return "docker.io"
return parts[0] # registry host present
images = [c.image for pod in pods for c in pod.spec.containers]
by_registry = Counter(registry(i) for i in images)
Example output (images by registry):
[
{"registry": "123456789012.dkr.ecr.us-west-2.amazonaws.com", "count": 15},
{"registry": "docker.io", "count": 3},
{"registry": "docker.io/library", "count": 2},
{"registry": "quay.io", "count": 2}
]
PVCs (see storage module)
PVC inventory (status, storage class, capacity) is owned by the storage module — see
references/storage.md. This module does not emit a storage.pvcs block. The StatefulSet
volume_claims / storage_class linkage above is the only volume fact workloads carries
(it records the volumeClaimTemplate → storage class wiring, not the bound PVCs).
Edge Cases
Exclude System Namespaces
Filter out system namespaces to focus on user workloads. Typically exclude:
kube-systemkube-publickube-node-leaseamazon-cloudwatchargocd(unless specifically interested)
Handle Large Clusters
For clusters with many workloads, avoid overwhelming output:
- Paginate results
- Focus on user namespaces
- Provide summary first, details on request
Identify Orphaned Resources
Check for Services without matching pods to find potential cleanup opportunities or misconfiguration.
Via Kubernetes API — list Services with a selector that may lack backing pods:
- Resource:
Service, group/versionv1(core), all namespaces. - Selection: services where
spec.selector != nullandspec.type != "ExternalName"; reportnamespace,name,selector. Cross-reference against pods to find selectors with no matching pods. - RBAC verbs:
get,listonservicesandpods.