Autoscaling with HPA and VPA

By Michael Samuel Naeem · March 1, 2026 · 5 min read

Traffic spiked every weekday at nine; CPU stayed at forty percent while request latency doubled—threads blocked on JDBC pool size, not CPU. HPA on CPU added pods that contended for the same database. VPA recommendations showed requests set to 100m CPU but actual use at 800m; HPA on a custom http_requests_in_flight metric with right-sized requests from VPA fixed both problems.

Kubernetes offers Horizontal Pod Autoscaler (HPA) for replica count and Vertical Pod Autoscaler (VPA) for per-container CPU/memory requests. Node scaling (Cluster Autoscaler, Karpenter) sits below; these sit at the workload layer.

HPA on CPU

Requires metrics-server:

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: checkout-api
  namespace: checkout
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout-api
  minReplicas: 2
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 60
      policies:
        - type: Percent
          value: 100
          periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 10
          periodSeconds: 60

behavior prevents flapping—slow scale-down, cautious scale-up.

Ensure requests are set—HPA utilization is usage/requests, not limits.

HPA on custom metrics

Prometheus adapter exposes metrics:

metrics:
  - type: Pods
    pods:
      metric:
        name: http_requests_per_second
      target:
        type: AverageValue
        averageValue: "1000"

Install prometheus-adapter with rules mapping PromQL to custom.metrics.k8s.io.

Queue depth via external metrics:

  - type: External
    external:
      metric:
        name: sqs_queue_length
        selector:
          matchLabels:
            queue: orders
      target:
        type: AverageValue
        averageValue: "30"

VPA installation and modes

kubectl apply -f https://github.com/kubernetes/autoscaler/releases/latest/download/vertical-pod-autoscaler-release.yaml

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: checkout-api-vpa
  namespace: checkout
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout-api
  updatePolicy:
    updateMode: "Auto"  # Off | Initial | Recreate | Auto
  resourcePolicy:
    containerPolicies:
      - containerName: api
        minAllowed:
          cpu: 100m
          memory: 256Mi
        maxAllowed:
          cpu: 2
          memory: 2Gi

Off — recommendations only in VPA object status.

Initial — apply at pod creation.

Auto — evict and recreate pods with new resources (disruptive).

Combined strategy

Recommended production pattern:

1. VPA in Off mode for two weeks—collect recommendations
2. Apply recommendations to Deployment requests manually or switch to Initial
3. HPA on CPU or custom metric for replica scaling
4. PDB ensures minimum availability during VPA evictions

Avoid HPA CPU + VPA Auto on identical Deployment without resourcePolicy exclusions—race conditions on pod spec.

KEDA for event-driven scale

KEDA scales from zero based on Kafka lag, cron, Prometheus queries:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-processor
spec:
  scaleTargetRef:
    name: order-processor
  minReplicaCount: 0
  maxReplicaCount: 50
  triggers:
    - type: kafka
      metadata:
        bootstrapServers: kafka:9092
        consumerGroup: order-processor
        topic: orders
        lagThreshold: "100"

Complements HPA for batch and queue workers.

Observability

Alert on HPA hitting maxReplicas, VPA recommendations far above requests (cost risk), and scaling loop oscillation. Dashboard: desired vs current replicas, metric values, VPA recommendation vs actual.

Cooldown during deploys

HPA may scale up during rolling update when old and new pods coexist briefly—raise minReplicas during deploy or use Flagger/Argo Rollouts for controlled traffic shift.

Memory-based HPA caution

Memory utilization HPA lags JVM heap growth—custom metrics (GC pause, pool wait time) often scale better than raw memory percent.

What to measure after rollout

Track error rates, tail latency, and resource utilization for two weeks after changes land—most regressions appear under real traffic mixes, not in staging smoke tests. Keep a rollback path documented: feature flags, Helm revision, or Git revert with known good digest. Review on-call pages tied to the topic quarterly; delete alerts that never fire and add thresholds that would have caught your last incident.

Run a short blameless postmortem if production surprised you, even for minor issues. The goal is updating this runbook section with one concrete lesson per quarter so the next engineer inherits context, not just configuration snippets.

Documentation your team should maintain

Maintain a one-page runbook link from your main service README: prerequisites, owner rotation, last drill date, and known sharp edges. Link to vendor docs in the Resources section below but capture org-specific decisions (CIDR ranges, cluster names, approval gates) in internal docs that stay current. New hires should deploy a safe canary within a week using only that runbook—if they cannot, the doc is incomplete.

Pre-production checklist

Before promoting to production, walk through this list with someone who was not the primary author—fresh eyes catch assumptions.

• Staging parity: The staging environment exercises the same code paths as production, including failure modes you expect to handle (timeouts, retries, partial outages).
• Observability: Dashboards and alerts exist for the metrics and log patterns discussed above; on-call knows where to look first.
• Rollback: You can revert to the previous known-good state in one documented step without improvising.
• Access control: Only the principals that need access have it; audit logs are enabled where the topic touches secrets or infrastructure APIs.
• Load test: You have evidence—not intuition—about behavior at expected peak plus headroom.

If any item is "we will do that later," treat it as a release blocker for tier-1 services.

Common questions from reviewers

Reviewers and auditors often ask whether this approach scales with team growth and whether it fails safely. Answer explicitly in your design doc: what happens when dependencies are down, when credentials expire, and when traffic doubles overnight. Prefer defaults that deny or degrade gracefully over defaults that fail open. Document known limits (throughput ceilings, supported versions, regions) in the same place operators look during incidents—avoid scattering critical constraints across Slack threads.

Resources

• Horizontal Pod Autoscaler documentation — metrics and behavior API
• Vertical Pod Autoscaler FAQ — modes and HPA interaction
• KEDA documentation — event-driven autoscaling
• Prometheus adapter setup — custom metrics for HPA