Kubernetes Requests and Limits Explained: 8% CPU Used, Resize GA in 1.35

Every Kubernetes cost review I have sat in eventually lands on the same two YAML fields, requests and limits, and on the same surprise: most people who set them every day cannot say which component reads which one. Kubernetes requests and limits look symmetrical in a manifest and behave completely differently at runtime. One is a promise the scheduler books against a node; the other is a ceiling the Linux kernel enforces, and it enforces CPU and memory ceilings in opposite ways (one slows you down, the other kills you). This post explains what each field really does in 2026, how QoS classes decide who gets killed first, why CPU limits cause throttling, what cgroup v2 changed about OOM kills, where in-place pod resize and VPA stand after Kubernetes 1.35 to 1.37, and what the public rightsizing data says about how badly we all guess.

What do Kubernetes requests and limits actually do?

A request is a scheduling input. The scheduler only places a pod on a node whose allocatable capacity minus the sum of existing requests can fit it. After placement, the kubelet passes requests and limits to the container runtime, which writes them into cgroups. The official resource management docs spell out the split: the CPU request becomes a weighting used only when CPUs are contended, the CPU limit becomes a hard ceiling checked every scheduling period, and the memory limit becomes the cgroup's memory cap, enforced by the kernel's OOM killer.

The memory request is the odd one out. For years it did almost nothing at runtime except feed the scheduler and the OOM score. That is changing, slowly, through Memory QoS (more below).

Field	Read by	Linux cgroup v2 control	What happens when exceeded	Source
CPU request	Scheduler, then kernel	cpu.weight (relative share)	Nothing; you just lose contention fights to others	K8s docs
CPU limit	Kernel	cpu.max as quota per period (default period 100000 µs)	Throttled until the next period	Kernel cgroup v2
Memory request	Scheduler, OOM score, eviction ranking	memory.min / memory.low only with Memory QoS tiered reservation	Pod becomes an eviction candidate under node pressure	Node-pressure eviction
Memory limit	Kernel	memory.max (plus optional memory.high)	OOM kill, exit code 137	K8s docs

Two consequences follow. First, a node can be "full" by requests while its CPUs sit idle, because requests are bookings, not usage. Second, a container can use more memory than it requested for as long as the node has spare memory, and nothing complains until the node runs short. That gap between booking and use is exactly where both waste and incidents live.

Kubernetes QoS classes: who gets killed first

Kubernetes derives a Quality of Service class from your requests and limits; you cannot set it directly.

• Guaranteed: every container has CPU and memory requests equal to limits.
• Burstable: at least one container has a request or limit, but the pod does not meet the Guaranteed bar.
• BestEffort: no requests or limits anywhere.

The class matters at two moments. When the whole node runs out of memory before the kubelet can react, the kernel's OOM killer picks a victim, and the kubelet has already biased that choice through oom_score_adj. Per the node-pressure eviction docs, Guaranteed containers get -997, BestEffort get 1000, and Burstable get a value between 2 and 999 computed from the memory request as a fraction of node memory. A Burstable pod with a tiny memory request is therefore nearly as killable as BestEffort.

The second moment is kubelet eviction, which kicks in at a soft or hard threshold (the default hard threshold on Linux is memory.available<100Mi). The docs are careful here: the kubelet does not rank by QoS class. It ranks by whether usage exceeds requests, then by Priority, then by how far over the request a pod is. QoS is just a good predictor of that order. The practical takeaway is the same either way: an honest memory request is your best protection from both the OOM killer and the evictor.

CPU limits and CFS throttling: should you set CPU limits?

A CPU limit is a time budget. With the default 100 ms period, a 500m limit means 50 ms of CPU time per 100 ms window across all threads in the container. A process with eight busy threads can burn that budget in roughly 6 ms and then sit frozen for the remaining 94 ms, even if the node has idle cores. Averaged over a minute, the dashboard says the pod used 40% of its limit. The p99 latency graph says something ruder.

This is the core of the CPU-limits debate, and in 2026 it is less a debate than a set of agreed exceptions. An October 2025 paper, CPU-Limits kill Performance, argues that limits harm latency-sensitive applications and cost more than they help, while remaining useful for background jobs. The kernel does have cpu.max.burst to let a group bank unused quota, but Kubernetes does not expose it as a pod field, and the kubelet's cpuCFSQuotaPeriod knob still sits behind the CustomCPUCFSQuotaPeriod feature gate per the kubelet configuration reference.

My position, which I hold loosely: set CPU requests on everything, because requests drive scheduling and contention shares. Omit CPU limits for latency-sensitive services. Keep them where a hard cap is the point: multi-tenant platforms selling fixed CPU, batch jobs you want to stop from starving neighbours, and Guaranteed pods that need exclusive cores from the static CPU manager policy. If you run agents or untrusted code, isolation is a separate question from CPU fairness; I wrote about that in container vs microVM vs gVisor.

To see whether throttling is hurting you, compare the cAdvisor counters container_cpu_cfs_throttled_periods_total and container_cpu_cfs_periods_total. A ratio above a few percent on a request path is worth a look before anyone blames the code.

Memory limits, OOMKilled and what cgroup v2 changed

Memory is not compressible. When a container's charged memory reaches memory.max and the kernel cannot reclaim enough, the OOM killer fires and the container exits with code 137. Two 2026-era changes affect what that looks like.

First, the OOM unit is now the container, not a process. On cgroup v2 nodes the kubelet defaults singleProcessOOMKill to false and sets memory.oom.group on each container cgroup, so all processes in the container die together, as the Kubernetes blog's cgroup v2 shift post of 6 October 2026 explains. Multi-process containers that used to survive a child being killed now restart outright. That surprised more than one team upgrading node images.

Second, cgroup v1 is on its way out. Since v1.35 failCgroupV1 defaults to true, so the kubelet refuses to start on cgroup v1 nodes without an override, and the same post says the fallback is scheduled for removal in v1.38. Memory QoS, which can use memory.high to throttle a container before it reaches its hard limit and memory.min/memory.low to protect requested memory, graduated to beta in v1.37 with the gate on by default. Note the fine print: memoryThrottlingFactor now defaults to null and memoryReservationPolicy to None, so nothing changes until you opt in, and the reservation policy is node-wide.

The other half of memory trouble lives inside the process. The JVM sizes its default max heap at 25% of the container limit (MaxRAMPercentage, per the JDK 25 java man page), which wastes memory in small containers and invites native-memory OOMs if you crank it to 90%. Node.js now has --max-old-space-size-percentage (Node CLI docs), and Go has GOMEMLIMIT since Go 1.19 (Go GC guide). The step-by-step diagnosis lives in my companion guide, how to fix OOMKilled and exit code 137.

In-place pod resize: GA in Kubernetes 1.35, extended in 1.36 and 1.37

Until recently, changing a pod's resources meant recreating it. In-place pod resize reached GA in Kubernetes 1.35 after alpha in v1.27 and beta in v1.33. The 1.35 release also allowed memory limit decreases, with a best-effort check that current usage is below the new limit. Since then, v1.36 made pod-level resource resize beta and on by default, and v1.37 added alpha scheduler preemption so a high-priority pod's deferred resize can evict lower-priority neighbours.

You resize through the resize subresource, and resizePolicy decides whether a container restarts:

# deployment.yaml: Kubernetes 1.35+ (in-place resize GA), kubectl 1.32+ for --subresource
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 3
  selector:
    matchLabels: { app: api }
  template:
    metadata:
      labels: { app: api }
    spec:
      containers:
      - name: api
        image: registry.k8s.io/pause:3.10   # placeholder image; swap for your app
        resources:
          requests:
            cpu: "250m"        # drives scheduling and cpu.weight
            memory: "512Mi"    # drives scheduling, OOM score, eviction rank
          limits:
            memory: "512Mi"    # memory.max: equal to request avoids overcommit surprises
            # no CPU limit: avoid CFS throttling on a latency-sensitive path
        resizePolicy:
        - resourceName: cpu
          restartPolicy: NotRequired      # CPU changes apply live
        - resourceName: memory
          restartPolicy: RestartContainer # most runtimes size heaps at start
---
# Resize one running pod without recreating it (Deployment template is unchanged):
# kubectl patch pod api-7d9c8-abcde --subresource resize --patch \
#   '{"spec":{"containers":[{"name":"api","resources":{"requests":{"cpu":"500m"}}}]}}'
# Watch status.conditions for PodResizePending (Deferred/Infeasible) or PodResizeInProgress.

The resize task docs list the limits that bite: the QoS class cannot change through a resize, Windows pods are excluded, pods under static CPU or memory manager policies cannot resize in place, and resizing a pod does not change its Deployment, so the next rollout reverts it. Resize is a tool for controllers, not a replacement for fixing the template.

VPA in 2026: in-place updates are finally the default path

The Vertical Pod Autoscaler is the controller that makes resize useful. According to the VPA features doc, the InPlaceOrRecreate update mode went alpha in VPA 1.4.0, beta in 1.5.0 and GA in 1.6.0; it tries an in-place resize first and evicts only if that fails. A stricter InPlace mode, which never evicts, is alpha since 1.7.0. The current release is VPA 1.8.0, published 23 September 2026, which among other fixes stops a recent OOM being discarded as "too old" at an aggregation boundary.

The old rule still applies: do not let VPA and HPA both act on CPU or memory for the same workload, because they will fight. Horizontal scaling is the more common habit; Datadog's State of Containers and Serverless puts HPA use at over 64% of Kubernetes organisations in Q3 2025. A sensible split is HPA on a traffic or queue metric and VPA in Initial or recommendation-only mode to tell you what requests should be. My autoscaling explainer covers the HPA side.

How overprovisioned are Kubernetes clusters in 2026?

Badly, and getting worse by some measures. The figure plots the two most-cited public datasets.

Cast AI's 2026 State of Kubernetes Optimization Report, covering calendar 2025 across AWS, GCP and Azure, puts average CPU utilization at 8% (down from 10%), memory at 20% (down from 23%), CPU overprovisioning at 69% (up from 40%) and memory overprovisioning at 79%. The prior edition, covering 2024 across more than 2,100 organisations, reported the 10% CPU and 23% memory figures. Datadog's 2025 report says most workloads use less than half their requested memory and less than 25% of their requested CPU.

Treat these with the scepticism you would apply to any vendor that sells the fix. The direction, though, matches every cluster I have looked at: requests are set once, padded "to be safe" and never revisited. The money side, including OpenCost and Karpenter, is in Kubernetes cost optimization in 2026, and the wider token-and-GPU version of the same problem is in FinOps for AI.

A sane default policy for requests and limits

This is the policy I would put in a platform team's paved road, enforced by admission policy rather than a wiki page:

1. Every container sets CPU and memory requests. Base them on observed p95 usage over at least a week, not a guess. VPA in recommendation mode gives you the number for free.
2. Memory limit equals memory request for most services. It makes the pod's footprint predictable and removes overcommit from the node's memory story. Raise both together when needed.
3. No CPU limit by default for request-serving workloads; set one deliberately for batch, tenants and exclusive-core pods.
4. Tell the runtime about the limit. JVM MaxRAMPercentage, Node's heap flag, GOMEMLIMIT. A container limit the runtime does not know about is a trap.
5. Review quarterly. Rightsizing is a recurring job, which is why VPA's GA in-place mode matters more than any one-off audit. In the AI era, where agents and inference sidecars spin up workloads nobody hand-tuned, this discipline matters more, not less.

FAQ

What is the difference between requests and limits in Kubernetes?

Requests are what the scheduler reserves for a container on a node and what the kernel uses to share CPU under contention. Limits are hard ceilings enforced by the kernel: exceeding a CPU limit causes throttling, and exceeding a memory limit causes an OOM kill with exit code 137.

Should I set CPU limits in Kubernetes?

Usually not for latency-sensitive services, because CFS quota throttling can stall a pod even when the node has idle CPU. Set CPU requests on everything, and use CPU limits deliberately for batch jobs, multi-tenant tiers and Guaranteed pods that need exclusive cores.

Can I change pod CPU and memory without restarting?

Yes. In-place pod resize is GA since Kubernetes 1.35: patch the pod's resize subresource with kubectl 1.32 or newer. CPU changes usually apply live; memory changes restart the container if its resizePolicy says RestartContainer, and the QoS class can never change.

Which QoS class should production pods use?

Guaranteed for critical stateful or latency-sensitive workloads that must not be evicted early, Burstable with honest requests for most services, and BestEffort only for disposable work. Guaranteed pods get an oom_score_adj of -997, so they are the last candidates when a node runs out of memory.

This quarter I would do three things on any cluster I inherited: turn on VPA in recommendation mode everywhere to get real numbers, remove CPU limits from the top five latency-critical services and watch the throttling ratio, and check every node pool is on cgroup v2 before v1.38 removes the escape hatch. None of it is new technology. It is just the boring loop of measure, set, review, which is the part of DevOps that never got automated away.

Related: Kubernetes as the Agent Control Plane in 2026, FinOps for AI in 2026 and DevOps Still Matters in 2026.