Kubernetes Autoscaling Patterns at Scale

Autoscaling is one of Kubernetes' most compelling features, but also one of the most frequently misconfigured. Getting it right requires understanding how the different autoscaling mechanisms interact, what their failure modes are, and how to tune them for your specific workload characteristics.

The Three Dimensions of Kubernetes Autoscaling

Kubernetes offers autoscaling across three dimensions:

• Horizontal Pod Autoscaler (HPA) — adds or removes pod replicas based on metrics
• Vertical Pod Autoscaler (VPA) — adjusts CPU and memory requests/limits for existing pods
• Cluster Autoscaler (CA) — adds or removes nodes when pods cannot be scheduled

A fourth mechanism, KEDA (Kubernetes Event-Driven Autoscaler), extends HPA with support for external metrics like queue depth, Kafka lag, and custom business metrics.

Horizontal Pod Autoscaler (HPA)

HPA is the right tool for stateless workloads that scale out naturally. It queries the Metrics API every 15 seconds and adjusts the replica count to keep the target metric at the specified value.

CPU-Based HPA with Behavior Tuning

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-server-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api-server
  minReplicas: 3
  maxReplicas: 50
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 60
      policies:
        - type: Percent
          value: 100
          periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 10
          periodSeconds: 60

The behavior section is critical. Without it, HPA uses aggressive defaults that cause oscillation. The 300-second scale-down stabilization window prevents premature scale-downs that would immediately require a scale-up again.

KEDA: Event-Driven Autoscaling

KEDA extends HPA with external metrics and enables scaling to zero — something native HPA cannot do.

Kafka Consumer Scaling

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-processor-scaler
spec:
  scaleTargetRef:
    name: order-processor
  minReplicaCount: 0
  maxReplicaCount: 30
  cooldownPeriod: 120
  triggers:
    - type: kafka
      metadata:
        bootstrapServers: kafka.production:9092
        consumerGroup: order-processor-v3
        topic: orders.order.placed.v1
        lagThreshold: "100"

With minReplicaCount: 0, the deployment scales to zero during off-peak hours — critical for cost control on batch processing workloads.

Vertical Pod Autoscaler (VPA)

VPA addresses pods with wrong resource requests. Start with Off mode and observe recommendations before applying:

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: order-processor-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: order-processor
  updatePolicy:
    updateMode: "Off"
  resourcePolicy:
    containerPolicies:
      - containerName: "*"
        minAllowed:
          cpu: "100m"
          memory: "128Mi"
        maxAllowed:
          cpu: "4"
          memory: "8Gi"

Critical warning: Do not run VPA in Auto mode alongside HPA targeting CPU utilization. They fight each other. VPA raises the CPU request, changing the HPA utilization denominator, triggering scale-down, which causes VPA to lower the request again. Use VPA for memory and HPA for CPU, or use VPA in Initial mode only.

Cluster Autoscaler and Node Pools

Use dedicated node pools with taints for different workload types:

# Prevent non-API pods from landing on API pool nodes
taint:
  key: workload-type
  value: api
  effect: NO_SCHEDULE

Protect critical workloads from eviction during scale-down with PodDisruptionBudgets:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: api-server-pdb
spec:
  minAvailable: "70%"
  selector:
    matchLabels:
      app: api-server

Pre-Scaling for Predictable Traffic

React-based autoscaling always lags behind demand. For predictable traffic patterns, use KEDA's Cron trigger to pre-scale before traffic arrives:

triggers:
  - type: cron
    metadata:
      timezone: America/New_York
      start: "0 8 * * 1-5"
      end: "0 20 * * 1-5"
      desiredReplicas: "10"
  - type: kafka
    metadata:
      lagThreshold: "100"

KEDA uses the maximum across all trigger values, so the cron trigger provides a capacity floor during business hours while Kafka lag handles additional bursts.

Tuning Checklist

Before deploying autoscaling to production, verify:

• Resource requests are set on every container (HPA requires them)
• Readiness probes are configured so HPA does not route traffic to unready pods
• PodDisruptionBudgets protect critical deployments from cluster autoscaler evictions
• Scale-down stabilization windows are conservative for slow-starting workloads
• VPA recommendations have been observed for at least one full traffic cycle before applying
• Cost alerts are configured on maxReplicas to prevent runaway scaling

Autoscaling done well is invisible. Autoscaling done poorly oscillates, over-provisions, or leaves you scrambling during traffic spikes. The difference is in the configuration details.