Skip to main content
Source

This page is generated from devops-agent/eks-recon/references/workloads.md. Edit the source, not this page.

Module: Workloads

Part of: eks-recon Purpose: Detect running workloads — deployments, services, ingresses, jobs

Note: This is an optional module for migration assessments or workload inventory. Skip for basic infrastructure recon.


Table of Contents​


Access Model​

This module reads facts almost entirely from the Kubernetes API — deployments, statefulsets, daemonsets, cronjobs/jobs, services, ingresses, HPAs, PDBs, priority classes, VPAs, and the live-resource apiVersion inventory are all in-cluster reads. Both sources are read-only:

  • Kubernetes API (via the Agent Space EKS access entry) — all workload, networking, and autoscaling resources below. Requires authenticationMode to include API and the AmazonAIOpsAssistantPolicy access entry to be present. RBAC verbs needed: get, list.
  • AWS control-plane APIs — this module makes no AWS-API calls of its own; the shared cluster: block is sourced by references/cluster-basics.md.

If the Kubernetes API is unreachable (access entry absent), this module can produce almost nothing — mark every workload sub-fact as unconfirmed in the report's Coverage section, never as false/count: 0.

Reference pseudocode note. Code blocks labeled reference pseudocode (kubernetes client) below illustrate the resource, fields, and RBAC verbs for each K8s-API read. They are not executable in the Agent Space and are not an operational path — do not emit kubectl ... | jq pipelines. The agent reads these resources through its Kubernetes-API capability, then applies the described selection/aggregation logic.


Detection Strategy​

Workload inventory covers the following resource types. Start with Deployments (most common), then expand based on what you find:

1. Deployments -> Long-running applications
2. StatefulSets -> Stateful applications
3. DaemonSets -> Node-level agents
4. CronJobs/Jobs -> Batch workloads
5. Services -> Network exposure
6. Ingresses -> External access
7. HPAs -> Horizontal autoscaling configuration
8. PDBs -> Pod disruption budgets
9. PriorityClasses -> Scheduling priority definitions
10. VPAs -> Vertical autoscaling configuration (if CRD present)
11. API versions -> apiVersions live resources use (raw fact list)

Detection Capabilities​

1. Deployments​

Run this first to identify the main application workloads. Deployments are the most common workload type and reveal the core services running in the cluster.

Via Kubernetes API — list Deployments across all namespaces:

  • Resource: Deployment, group/version apps/v1, all namespaces.
  • Fields to extract: metadata.namespace, metadata.name, spec.replicas, status.readyReplicas, all spec.template.spec.containers[].image (not just containers[0] — a workload may run sidecars/proxies), spec.template.spec.initContainers[].name (empty list when none), metadata.labels.
  • Selection: exclude namespaces beginning with kube-.
  • RBAC verbs: get, list on deployments.apps.

Iterate all containers[].image (a workload may run multiple containers — sidecars, proxies), not just containers[0]. Record init_containers (names) per workload; empty list when none.

Example (one deployment):

{
"namespace": "production",
"name": "api-gateway",
"replicas": 3,
"ready": 3,
"images": [
"123456789012.dkr.ecr.us-west-2.amazonaws.com/api-gateway:v2.1.0",
"docker.io/envoyproxy/envoy:v1.28.0"
],
"init_containers": ["migrate-db"],
"labels": {"app": "api-gateway", "team": "platform"}
}

2. StatefulSets​

Check StatefulSets when you need to understand stateful applications like databases, message queues, or caches. These require special handling during migrations due to persistent storage.

Via Kubernetes API — list StatefulSets across all namespaces:

  • Resource: StatefulSet, group/version apps/v1, all namespaces.
  • Fields to extract: metadata.namespace, metadata.name, spec.replicas, status.readyReplicas, spec.volumeClaimTemplates[].metadata.name (→ volume_claims), spec.volumeClaimTemplates[].spec.storageClassName (→ storage_class), spec.template.spec.initContainers[].name.
  • RBAC verbs: get, list on statefulsets.apps.

Capture the volumeClaimTemplate → storageClassName linkage: each StatefulSet volume claim template names a storage class the pods provision volumes from. Record the storage class per StatefulSet (null when the template omits it and the cluster default applies).

Example (one statefulset):

{
"namespace": "production",
"name": "redis-cluster",
"replicas": 3,
"volume_claims": ["data"],
"storage_class": "gp3"
}

3. DaemonSets​

List DaemonSets to identify node-level agents (monitoring, logging, security). These run on every node and reveal operational tooling in use.

Via Kubernetes API — list DaemonSets across all namespaces:

  • Resource: DaemonSet, group/version apps/v1, all namespaces.
  • Fields to extract: metadata.namespace, metadata.name, status.desiredNumberScheduled (→ desired), status.numberReady (→ ready).
  • RBAC verbs: get, list on daemonsets.apps.

Example (one daemonset):

{
"namespace": "kube-system",
"name": "aws-node",
"desired": 5,
"ready": 5
}

4. CronJobs and Jobs​

Check batch workloads to understand scheduled tasks (backups, reports) and one-time jobs. Important for understanding maintenance windows and resource usage patterns.

Via Kubernetes API — list CronJobs across all namespaces:

  • Resource: CronJob, group/version batch/v1, all namespaces.
  • Fields to extract: metadata.namespace, metadata.name, spec.schedule, spec.suspend, status.lastScheduleTime (→ last_schedule, null if never run).
  • RBAC verbs: get, list on cronjobs.batch.

Via Kubernetes API — list active/incomplete Jobs across all namespaces:

  • Resource: Job, group/version batch/v1, all namespaces.
  • Fields to extract: metadata.namespace, metadata.name, status.active, status.failed.
  • Selection: count/list only jobs that have not completed successfully (i.e. not status.succeeded == 1).
  • RBAC verbs: get, list on jobs.batch.

Example (CronJob):

{
"namespace": "production",
"name": "db-backup",
"schedule": "0 2 * * *",
"suspend": false,
"lastSchedule": "2026-04-22T02:00:00Z"
}

5. Services​

List Services to understand network exposure. Services reveal how applications communicate internally (ClusterIP) and externally (LoadBalancer, NodePort).

Via Kubernetes API — list Services across all namespaces:

  • Resource: Service, group/version v1 (core), all namespaces.
  • Fields to extract: metadata.namespace, metadata.name, spec.type, spec.ports[] (format each as "<port>/<protocol>"), spec.selector.
  • Selection: exclude namespaces beginning with kube-.
  • RBAC verbs: get, list on services.

Record each service as a full entry (name, type, ports, selector), not only an aggregate count by type.

Note: The services.by_type field (counts per ClusterIP/NodePort/LoadBalancer) is aggregated by the agent from the enumerated services list above — no separate counting read is needed.

Example (one service):

{
"namespace": "production",
"name": "api-gateway",
"type": "LoadBalancer",
"ports": ["443/TCP", "80/TCP"],
"selector": {"app": "api-gateway"}
}

6. Ingresses​

Check Ingresses to identify external access points and routing rules. Ingresses show domain mappings, TLS configuration, and which services are publicly accessible.

Via Kubernetes API — list Ingresses across all namespaces:

  • Resource: Ingress, group/version networking.k8s.io/v1, all namespaces.
  • Fields to extract: metadata.namespace, metadata.name, spec.ingressClassName (→ class), spec.rules[].host (→ hosts), and tls_enabled = true when spec.tls has one or more entries, else false.
  • RBAC verbs: get, list on ingresses.networking.k8s.io.

tls_enabled = true when the ingress declares one or more spec.tls entries, else false.

Example (one ingress):

{
"namespace": "production",
"name": "api-ingress",
"class": "alb",
"hosts": ["api.example.com", "api-internal.example.com"],
"tls_enabled": true
}

7. HPAs​

List Horizontal Pod Autoscalers to understand autoscaling configuration. HPAs reveal which workloads scale automatically and their scaling thresholds.

Via Kubernetes API — list HPAs across all namespaces:

  • Resource: HorizontalPodAutoscaler, group/version autoscaling/v2, all namespaces.
  • Fields to extract: metadata.namespace, metadata.name, spec.scaleTargetRef.name (→ target), spec.minReplicas, spec.maxReplicas, status.currentReplicas, spec.metrics[].type (→ metrics).
  • RBAC verbs: get, list on horizontalpodautoscalers.autoscaling.

Example (one HPA):

{
"namespace": "production",
"name": "api-gateway-hpa",
"target": "api-gateway",
"minReplicas": 2,
"maxReplicas": 10,
"currentReplicas": 4,
"metrics": ["Resource", "External"]
}

8. Pod Disruption Budgets​

Enumerate PodDisruptionBudgets (PDBs) across namespaces. A PDB records the minimum availability guarantee for the pods matched by its selector. Record existence, the minAvailable / maxUnavailable value, and the label selector (the workloads it covers). Also record status.disruptionsAllowed — the point-in-time count of voluntary evictions the Eviction API currently permits for the pods this PDB selects. 0 means it currently rejects voluntary eviction of covered healthy pods, which can stall a drain of the nodes hosting them (a PDB selecting zero pods also reports 0). Always record spec.unhealthyPodEvictionPolicy (→ unhealthy_pod_eviction_policy) — it governs whether running-but-not-ready (unhealthy) covered pods can be evicted, independent of the disruptions_allowed counter (AlwaysAllow = evict such pods regardless of budget health; IfHealthyBudget, the unset default, = only while the budget is met, currentHealthy >= desiredHealthy) — this is what lets a drain make progress when a covered workload is unhealthy. Emit the value verbatim; use null only when the field is genuinely unset — never drop the key.

Via Kubernetes API — list PDBs across all namespaces (VERIFIED live: PDBs expose MIN AVAILABLE / MAX UNAVAILABLE):

  • Resource: PodDisruptionBudget, group/version policy/v1, all namespaces.
  • Fields to extract: metadata.namespace, metadata.name, spec.minAvailable, spec.maxUnavailable, status.disruptionsAllowed (→ disruptions_allowed), spec.unhealthyPodEvictionPolicy (→ unhealthy_pod_eviction_policy), spec.selector.matchLabels (→ selector).
  • RBAC verbs: get, list on poddisruptionbudgets.policy.

At most one of min_available / max_unavailable is set per PDB (both may be null for a selector-only PDB). The selector matchLabels identify the covered workloads.

Example (one PDB):

{
"namespace": "production",
"name": "api-gateway-pdb",
"min_available": 1,
"max_unavailable": null,
"disruptions_allowed": 2,
"unhealthy_pod_eviction_policy": "AlwaysAllow",
"selector": {"app": "api-gateway"}
}

9. PriorityClasses​

List PriorityClasses (cluster-scoped) to record scheduling priority definitions.

Via Kubernetes API — list PriorityClasses (cluster-scoped):

  • Resource: PriorityClass, group/version scheduling.k8s.io/v1.
  • Fields to extract: metadata.name, value, globalDefault (→ global_default).
  • RBAC verbs: get, list on priorityclasses.scheduling.k8s.io.

Example (one PriorityClass):

{
"name": "high-priority",
"value": 1000000,
"global_default": false
}

10. Vertical Pod Autoscalers​

List VerticalPodAutoscalers only if the CRD is present. VPA is an add-on CRD (verticalpodautoscalers.autoscaling.k8s.io); guard the read so its absence is a clean fact, not an error.

Via Kubernetes API — detect the CRD first, then list VPAs if present:

  • CRD check: CustomResourceDefinition, group/version apiextensions.k8s.io/v1, name verticalpodautoscalers.autoscaling.k8s.io. Absence is a fact — record detected: false for the whole block and skip the list read.
  • Resource (if CRD present): VerticalPodAutoscaler, group/version autoscaling.k8s.io/v1, all namespaces.
  • Fields to extract: metadata.namespace, metadata.name, spec.targetRef.name (→ target), spec.updatePolicy.updateMode (→ update_mode).
  • RBAC verbs: get, list on customresourcedefinitions.apiextensions.k8s.io and verticalpodautoscalers.autoscaling.k8s.io.

Record detected: false for the whole block when the CRD is absent.

Example (one VPA):

{
"namespace": "production",
"name": "api-gateway-vpa",
"target": "api-gateway",
"update_mode": "Auto"
}

11. API Versions In Use​

Enumerate the distinct apiVersion values that live resources in the cluster use, as a raw fact list. This is inventory only.

Scope note: Report the apiVersions verbatim. Do not flag any as deprecated, removed, or upgrade-blocking, and do not cross-reference them against a target Kubernetes version — that deprecation math belongs to the eks-upgrade-check skill, not recon.

Via Kubernetes API — read live resources across the common workload/config kinds and collect their apiVersion values:

  • Resources: Deployment, StatefulSet, DaemonSet, ReplicaSet (apps/v1); CronJob, Job (batch/v1); Ingress, NetworkPolicy (networking.k8s.io/v1); HorizontalPodAutoscaler (autoscaling/v2); PodDisruptionBudget (policy/v1) — all namespaces.
  • Aggregation: collect metadata-level apiVersion (the served version each object was read as) across all items, then de-duplicate and sort into a distinct list. This is a raw union of the versions observed in use, nothing more.
  • RBAC verbs: get, list on each resource above.

Reference pseudocode (kubernetes client), not executable:

# Collect the served apiVersion of every live object across the common kinds,
# then de-duplicate. Raw inventory only — no deprecation classification.
versions = set()
for obj in live_workload_and_config_objects(): # deploy/sts/ds/rs/cronjob/job/ingress/hpa/pdb/netpol
versions.add(obj["apiVersion"])
api_versions_in_use = sorted(versions)

Example output:

apps/v1
autoscaling/v2
batch/v1
networking.k8s.io/v1
policy/v1

Summary Statistics​

A high-level view of workload distribution across namespaces, for initial assessment before diving into details. This is aggregated by the agent from the per-type Kubernetes-API reads above (Deployments, StatefulSets, DaemonSets, Jobs) — no separate read is needed. Group the collected objects by metadata.namespace, then within each namespace count total objects and counts per kind.

Example output:

[
{
"namespace": "production",
"total": 12,
"by_kind": {"Deployment": 8, "StatefulSet": 2, "DaemonSet": 2}
},
{
"namespace": "staging",
"total": 6,
"by_kind": {"Deployment": 5, "Job": 1}
}
]

Per-Namespace Rollup​

Aggregate the per-type listings collected above into the by_namespace schema block — one row per namespace, with a count column per workload type. Do this explicitly: the schema advertises per-namespace hpas/pdbs counts and the deployments/services kube-* handling, but no other step produces them, so on a complete run they are otherwise dropped. Roll up from the already-collected lists (no additional read) — count Deployments, StatefulSets, Services, Ingresses, HPAs, and PDBs per namespace.

deployments and services are null for kube-* namespaces — their §1/§5 listings scope kube-* out, so a count here would not be corroborated by the detailed lists (null ≠ 0: null = column scoped out for that row). statefulsets, ingresses, hpas, and pdbs include kube-*. A row exists for any namespace containing at least one of the six workload types, so rows may be partial.

Reference pseudocode (agent-side aggregation), not executable:

# Roll the already-collected per-type lists up into by_namespace rows.
by_namespace = []
for ns in sorted(namespaces_with_any_workload):
sys = ns.startswith("kube-")
by_namespace.append({
"namespace": ns,
"deployments": None if sys else count(deployments, ns), # §1 scopes kube-* out
"statefulsets": count(statefulsets, ns),
"services": None if sys else count(services, ns), # §5 scopes kube-* out
"ingresses": count(ingresses, ns),
"hpas": count(hpas, ns),
"pdbs": count(pdbs, ns),
})

Example output:

[
{"namespace": "kube-system", "deployments": null, "statefulsets": 0, "services": null, "ingresses": 0, "hpas": 0, "pdbs": 1},
{"namespace": "production", "deployments": 8, "statefulsets": 2, "services": 5, "ingresses": 1, "hpas": 3, "pdbs": 2}
]

Output Schema​

This is the single canonical schema for the workloads module — it carries every workloads fact. The workloads-recon agent emits exactly this shape (plus the shared cluster: block from references/cluster-basics.md). Use null where a fact was not detected; never omit a key. Aggregate containers use the {count, list} wrapper. Every key must trace to a collection step above — including derived/rolled-up keys (summary, by_namespace, services.by_type); a schema key with no driving step is a defect (the Per-Namespace Rollup step drives by_namespace).

PVCs are owned by the storage module. This schema does not carry a storage.pvcs block — see the storage module for PVC inventory (status, storage class, capacity).

workloads:
summary: # agent rolls these up from the per-type section counts above;
# namespaces_with_workloads = distinct namespaces across those lists
deployments: int # kube-*-scoped total (§1 excludes kube-*); a true 0 is a valid fact
statefulsets: int
daemonsets: int
cronjobs: int
jobs: int
services: int # kube-*-scoped total (§5 excludes kube-*); a true 0 is a valid fact
ingresses: int
hpas: int
pdbs: int
namespaces_with_workloads: int

# kube-* scope split across these columns: ONLY deployments and services exclude kube-*
# namespaces (their §1/§5 listings filter kube-* out). statefulsets, ingresses, hpas, and pdbs
# INCLUDE kube-* (their listings apply no namespace filter). A row exists wherever ANY of the six
# workload types is present, so rows may be partial. For a scoped-out column emit `null`, NOT 0 (schema
# header rule: null = fact not detected/collected) — e.g. a kube-system row shows `null` for
# deployments and services (column scoped out, not "zero found"; kube-system always runs CoreDNS
# Deployments + kube-dns Services) but real counts for statefulsets/ingresses/hpas/pdbs.
by_namespace:
- namespace: string
deployments: int|null # null for kube-* rows: §1 scopes kube-* out, so not counted here (never 0)
statefulsets: int
services: int|null # null for kube-* rows: §5 scopes kube-* out, so not counted here (never 0)
ingresses: int
hpas: int # per-namespace HPA count (from the Per-Namespace Rollup step)
pdbs: int # per-namespace PDB count (from the Per-Namespace Rollup step)

deployments:
count: int
list:
- namespace: string
name: string
replicas: int
ready: int # status.readyReplicas
images: list # ALL containers[].image
init_containers: list # initContainers[].name, [] when none
labels: object

statefulsets:
count: int
list:
- namespace: string
name: string
replicas: int
ready: int
volume_claims: list # volumeClaimTemplates[].metadata.name
storage_class: string # volumeClaimTemplates[].spec.storageClassName, null if unset
init_containers: list

daemonsets:
count: int
list:
- namespace: string
name: string
desired: int # status.desiredNumberScheduled
ready: int # status.numberReady

cronjobs:
count: int
list:
- namespace: string
name: string
schedule: string
suspend: bool
last_schedule: string # status.lastScheduleTime, null if never run

jobs:
count: int # active/incomplete jobs
list:
- namespace: string
name: string
active: int
failed: int

services:
count: int
by_type:
cluster_ip: int
load_balancer: int
node_port: int
external_name: int
list:
- namespace: string
name: string
type: string # ClusterIP | NodePort | LoadBalancer | ExternalName
ports: list # ["443/TCP", ...]
selector: object

ingresses:
count: int
list:
- namespace: string
name: string
class: string # spec.ingressClassName
hosts: list
tls_enabled: bool # true when spec.tls has >=1 entry

hpas:
count: int
list:
- namespace: string
name: string
target: string # spec.scaleTargetRef.name
min_replicas: int
max_replicas: int
current_replicas: int # status.currentReplicas
metrics: list # spec.metrics[].type

pdbs:
count: int
list:
- namespace: string
name: string
min_available: int|string # spec.minAvailable — plain int or "N%" string (K8s IntOrString); at most one is set (both may be null for a selector-only PDB)
max_unavailable: int|string # spec.maxUnavailable — plain int or "N%" string (K8s IntOrString)
disruptions_allowed: int # status.disruptionsAllowed — voluntary evictions the Eviction API currently permits for
# covered pods; 0 rejects eviction of covered healthy pods (also 0 when it selects no pods); null if status unpopulated
unhealthy_pod_eviction_policy: string # spec.unhealthyPodEvictionPolicy — emit verbatim; null when unset (API defaults behavior to IfHealthyBudget)
selector: object # spec.selector.matchLabels — covered workloads

priority_classes:
count: int
list:
- name: string
value: int
global_default: bool

vpas:
detected: bool # verticalpodautoscalers CRD present
count: int
list:
- namespace: string
name: string
target: string # spec.targetRef.name
update_mode: string # spec.updatePolicy.updateMode

api_versions_in_use: list # RAW distinct apiVersions used by live resources.
# NOT flagged deprecated/removed — see Scope note in §11.

images:
unique:
count: int
list: list # unique image strings across pods
by_registry:
- registry: string # normalized: bare "nginx" -> docker.io/library
count: int

Additional Fact Collection​

Facts that feed the schema blocks above beyond the primary per-type listings. All are Kubernetes-API reads with agent-side selection/aggregation.

Pod Phase and Replica Counts​

Enumerate pod phase and replica-vs-desired counts.

Via Kubernetes API — pods not in a healthy phase, and deployments not fully ready:

  • Resource: Pod, group/version v1 (core), all namespaces. Select pods whose status.phase is neither Running nor Succeeded; report <namespace>/<name>: <phase>.
  • Resource: Deployment, group/version apps/v1, all namespaces. Select deployments where status.readyReplicas != spec.replicas; report <namespace>/<name>: <ready>/<desired>.
  • RBAC verbs: get, list on pods and deployments.apps.

Example output:

production/api-gateway: 2/3
staging/worker: 0/2

Resource Requests and Limits​

Inventory pods that declare no resource requests (count).

Via Kubernetes API — count pods missing resource requests:

  • Resource: Pod, group/version v1 (core), all namespaces.
  • Selection: a pod counts when any container has resources.requests == null; de-duplicate by <namespace>/<name> and count.
  • RBAC verbs: get, list on pods.

Example output:

23

Inventory Container Images​

Enumerate unique images and image counts by registry.

Via Kubernetes API — read pod container images, then aggregate:

  • Resource: Pod, group/version v1 (core), all namespaces. Collect every spec.containers[].image.
  • Unique images: de-duplicate and sort the collected image strings.
  • Images by registry: bucket each image by its registry, applying Docker Hub normalization (below).
  • RBAC verbs: get, list on pods.

Registry normalization rule (agent-side selection logic): a bare image reference like nginx resolves to the Docker Hub library namespace, so bucket it under docker.io/library (not under the literal first path segment). A two-segment reference whose first segment has no . or : (e.g. bitnami/redis) is also a Docker Hub reference → docker.io. Otherwise the first path segment is the registry host.

Reference pseudocode (kubernetes client), not executable:

# Bucket collected pod container images by registry with Docker Hub normalization.
def registry(image):
parts = image.split("/")
if len(parts) == 1: # bare "nginx"
return "docker.io/library"
if len(parts) == 2 and not any(c in parts[0] for c in ".:"): # "bitnami/redis"
return "docker.io"
return parts[0] # registry host present

images = [c.image for pod in pods for c in pod.spec.containers]
by_registry = Counter(registry(i) for i in images)

Example output (images by registry):

[
{"registry": "123456789012.dkr.ecr.us-west-2.amazonaws.com", "count": 15},
{"registry": "docker.io", "count": 3},
{"registry": "docker.io/library", "count": 2},
{"registry": "quay.io", "count": 2}
]

PVCs (see storage module)​

PVC inventory (status, storage class, capacity) is owned by the storage module — see references/storage.md. This module does not emit a storage.pvcs block. The StatefulSet volume_claims / storage_class linkage above is the only volume fact workloads carries (it records the volumeClaimTemplate → storage class wiring, not the bound PVCs).


Edge Cases​

Exclude System Namespaces​

Filter out system namespaces to focus on user workloads. Typically exclude:

  • kube-system
  • kube-public
  • kube-node-lease
  • amazon-cloudwatch
  • argocd (unless specifically interested)

Handle Large Clusters​

For clusters with many workloads, avoid overwhelming output:

  • Paginate results
  • Focus on user namespaces
  • Provide summary first, details on request

Identify Orphaned Resources​

Check for Services without matching pods to find potential cleanup opportunities or misconfiguration.

Via Kubernetes API — list Services with a selector that may lack backing pods:

  • Resource: Service, group/version v1 (core), all namespaces.
  • Selection: services where spec.selector != null and spec.type != "ExternalName"; report namespace, name, selector. Cross-reference against pods to find selectors with no matching pods.
  • RBAC verbs: get, list on services and pods.