How to Fix Kubernetes CrashLoopBackOff: Causes and Commands (2026)

Oct 20, 2026 · 13 min · Ajay Kumar

This guide is for anyone staring at CrashLoopBackOff in kubectl get pods and wondering where to start. By the end you will be able to read the restart state, get the logs from the crash that actually happened, decode the exit code, match it to one of six common causes, and debug a container that dies before you can exec into it. Commands and field names are checked against the Kubernetes v1.37 documentation (the current release, with 1.35 and 1.36 also supported) and work with kubectl 1.32 or newer. You need kubectl access to the namespace with permission to read Pods, logs and events, and to create ephemeral containers for the debugging step.

What CrashLoopBackOff means

CrashLoopBackOff is not an error. It is the kubelet telling you it is waiting before restarting a container that keeps exiting. The Pod lifecycle docs describe the sequence: the first crash gets an immediate restart, later crashes get an exponential delay of 10s, 20s, 40s and so on, capped at 300 seconds (5 minutes), and once a container runs for 10 minutes without a problem the timer resets. While the kubelet is sitting in that delay, the container's state is Waiting with reason CrashLoopBackOff.

So the real question is always "why did the container exit?", and the answer lives in the previous container instance, not the current one.

Two newer knobs change the timing, and they matter if you are wondering why one cluster retries faster than another:

  • KubeletCrashLoopBackOffMax was alpha in 1.32 to 1.34 and has been beta and enabled by default since 1.35. It lets a node administrator lower the 5 minute cap per node with crashLoopBackOff.maxContainerRestartPeriod in the kubelet configuration, anywhere from "1s" to "300s".
  • ReduceDefaultCrashLoopBackOffDecay has been alpha and off by default since 1.33. When enabled it changes the cluster default to start at 1s and cap at 60s, as proposed in KEP-4603.

On managed clusters (EKS, GKE, AKS) you usually cannot change kubelet configuration on managed nodes, so assume the 10s to 300s default unless you run your own node pools.

1. kubectl describe pod Last State, Reason, Exit Code 2. kubectl logs --previous output of the crashed instance Logs empty? kubectl debug --copy-to 3. Branch on exit code 137 OOMKilled raise memory limit or fix the leak 137/143 + probe Liveness probe failed event: add startupProbe 126 / 127 command not runnable or not found 1 or other app error, often missing config or deps 0 Completed process exits OK: use a Job Backoff between restarts: 10s, 20s, 40s ... capped at 300s; resets after 10 minutes of clean running. Since 1.35 a node can lower the cap with crashLoopBackOff.maxContainerRestartPeriod (1s to 300s).
The debugging order used in this guide. Backoff values are from the Kubernetes Pod lifecycle documentation; exit code meanings follow the Linux convention of 128 plus the signal number.

Step 1: Read the Pod's last state with kubectl describe

Start by confirming which container is crashing and how many times it has restarted.

kubectl get pods -n shop
# NAME                       READY   STATUS             RESTARTS      AGE
# checkout-7d9f8b6c5-x2kqp   0/1     CrashLoopBackOff   6 (82s ago)   9m

Then describe it. Ignore most of the output and look for three things under the container: State, Last State and the Events at the bottom.

kubectl describe pod checkout-7d9f8b6c5-x2kqp -n shop

The part you care about looks like this:

    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Tue, 20 Oct 2026 09:14:02 +0000
      Finished:     Tue, 20 Oct 2026 09:14:03 +0000
    Restart Count:  6
...
Events:
  Warning  BackOff  12s (x25 over 5m)  kubelet  Back-off restarting failed container checkout in pod checkout-7d9f8b6c5-x2kqp_shop(...)

Last State is the crash. Reason and Exit Code tell you which branch of the diagram you are on. Started and Finished tell you how long it lived: one second means it died during startup, several minutes usually means a probe or memory limit killed it.

For scripts or multi-container Pods, pull the same data with JSONPath:

kubectl get pod checkout-7d9f8b6c5-x2kqp -n shop -o jsonpath='{range .status.containerStatuses[*]}{.name}{"\t"}{.restartCount}{"\t"}{.lastState.terminated.reason}{"\t"}{.lastState.terminated.exitCode}{"\n"}{end}'
# checkout   6   Error   1

Check initContainerStatuses the same way if the Pod status shows Init:CrashLoopBackOff: the failing container is an init container, and the main container has not started at all.

Step 2: Get the crashed container's logs with --previous

Plain kubectl logs shows the current instance, which during backoff is either not running or has just started and printed nothing useful. You want the previous instance, as the debugging docs show:

# Logs from the instance that crashed
kubectl logs checkout-7d9f8b6c5-x2kqp -n shop -c checkout --previous

# Last 50 lines only, with timestamps
kubectl logs checkout-7d9f8b6c5-x2kqp -n shop -c checkout --previous --tail=50 --timestamps

# Every container in the Pod, including sidecars
kubectl logs checkout-7d9f8b6c5-x2kqp -n shop --all-containers --previous

If the app writes its fatal error to a file rather than stdout, or exits before flushing logs, set terminationMessagePolicy: FallbackToLogsOnError on the container. Kubernetes then copies the last chunk of the log into the termination message, which shows up in describe as Message: under Last State (docs).

containers:
  - name: checkout
    image: ghcr.io/example/checkout:2.4.1
    terminationMessagePolicy: FallbackToLogsOnError

Step 3: Decode the exit code

Exit codes above 128 mean the process was killed by a signal: subtract 128 to get the signal number.

Exit code Meaning Usual Kubernetes cause
0 Clean exit Process finished; with restartPolicy: Always it restarts forever
1 Generic application error Unhandled exception, missing config, failed dependency
126 Command found but not executable Wrong permissions, script without exec bit, wrong architecture
127 Command not found Typo in command, binary missing from a slim image
137 SIGKILL (128 + 9) OOMKilled, or liveness probe kill after grace period
139 SIGSEGV (128 + 11) Native crash, often image built for the wrong CPU
143 SIGTERM (128 + 15) Graceful kill, usually by a failed liveness probe

The reason field disambiguates 137: OOMKilled means the kernel killed it for exceeding the memory limit; Error with a Liveness probe failed event means the kubelet killed it.

Step 4: Check events for probe and scheduling clues

Events expire (the API server keeps them for one hour by default), so check them early, or ship them to your log pipeline as I describe in Observability in 2026.

kubectl events -n shop --for pod/checkout-7d9f8b6c5-x2kqp --types=Warning
# LAST SEEN  TYPE     REASON     OBJECT                         MESSAGE
# 40s        Warning  Unhealthy  Pod/checkout-7d9f8b6c5-x2kqp   Liveness probe failed: Get "http://10.0.3.17:8080/healthz": dial tcp 10.0.3.17:8080: connect: connection refused
# 40s        Warning  BackOff    Pod/checkout-7d9f8b6c5-x2kqp   Back-off restarting failed container checkout ...

Unhealthy with Liveness probe failed before every restart is the strongest single signal you will get. If you see FailedMount or CreateContainerConfigError instead, you do not have a crash loop: the container never started. That is a different problem, covered below.

Step 5: Match the cause and apply the fix

Missing config, secrets or environment variables

The most common exit-code-1 crash is an app that validates its config at startup and exits. The log line will say so (DATABASE_URL is not set, no such file /etc/app/config.yaml). Compare what the Pod gets with what the app expects:

# Environment as rendered in the Pod spec (values from Secrets show as references)
kubectl get pod checkout-7d9f8b6c5-x2kqp -n shop -o jsonpath='{.spec.containers[0].env}' | jq .

# Does the referenced Secret and key exist?
kubectl get secret checkout-db -n shop -o jsonpath='{.data}' | jq 'keys'

A useful distinction: if a referenced Secret or key is missing entirely, the kubelet usually refuses to start the container and the Pod shows CreateContainerConfigError, not CrashLoopBackOff. A crash loop means the variable existed but had the wrong value, or the app needed something you never wired up.

Liveness probe kills a slow-starting app

If the app takes 40 seconds to warm up and the liveness probe starts failing after 30, the kubelet kills it, it restarts, and it never finishes starting. The fix is a startup probe, which disables the liveness probe until it succeeds:

containers:
  - name: checkout
    image: ghcr.io/example/checkout:2.4.1
    startupProbe:
      httpGet:
        path: /healthz
        port: 8080
      failureThreshold: 30   # 30 x 10s = up to 300s to start
      periodSeconds: 10
    livenessProbe:
      httpGet:
        path: /healthz
        port: 8080
      periodSeconds: 10
      failureThreshold: 3

Also check the liveness endpoint does not call your database. A liveness probe that fails when a dependency is down turns a dependency outage into a restart storm across every replica.

OOMKilled (exit code 137)

Reason: OOMKilled means the container exceeded its memory limit. Check what it was allowed and what it used just before dying:

kubectl get pod checkout-7d9f8b6c5-x2kqp -n shop -o jsonpath='{.spec.containers[0].resources}'
kubectl top pod checkout-7d9f8b6c5-x2kqp -n shop --containers   # requires metrics-server

Raise the limit if the workload genuinely needs it, or cap the runtime's heap (JVM -XX:MaxRAMPercentage, Node --max-old-space-size) so it stays under the limit. The dedicated OOMKilled exit code 137 guide covers this in depth, and requests and limits explained covers how to size them.

Wrong command, entrypoint or architecture (126, 127, 139)

These die within a second and often print a single line such as exec /app/server: no such file or directory or exec format error. Check what the Pod actually runs versus what the image expects:

kubectl get pod checkout-7d9f8b6c5-x2kqp -n shop -o jsonpath='{.spec.containers[0].command} {.spec.containers[0].args}'
# Inspect the image's own entrypoint and platform without running it (crane v0.22)
crane config ghcr.io/example/checkout:2.4.1 | jq '{entrypoint: .config.Entrypoint, cmd: .config.Cmd, arch: .architecture}'

exec format error almost always means an arm64 image on amd64 nodes or the reverse. Build multi-arch images or pin node affinity.

The process exits 0 (Completed)

A container that does its work and exits cleanly, under a Deployment's restartPolicy: Always, will be restarted forever and land in CrashLoopBackOff with Reason: Completed. Either it should be a Job or CronJob, or the process is daemonising into the background (a shell script that starts a server with & and then ends). Run the server in the foreground with exec.

Permissions and security context

Hardened Pods (runAsNonRoot, readOnlyRootFilesystem: true) break images that write to /tmp or their own directory, with logs such as permission denied or read-only file system. Mount an emptyDir at the path the app writes to rather than loosening the security context.

Step 6: Debug a container that crashes too fast to exec into

kubectl exec needs a running container, which a crash-looping one rarely is. Ephemeral containers attach to a live Pod, so they help when the container stays up for a while between crashes:

# Add a busybox shell that shares the target container's process namespace
kubectl debug -it checkout-7d9f8b6c5-x2kqp -n shop --image=busybox:1.37 --target=checkout --profile=general

When it dies on startup, make a copy of the Pod with a different command, so the container stays up and you can run the real entrypoint by hand:

# Copy the Pod, replace the checkout container's command with a shell
kubectl debug checkout-7d9f8b6c5-x2kqp -n shop -it --copy-to=checkout-debug --container=checkout -- sh

# Inside the copy: check env, files and run the binary to see the real error
env | sort
ls -la /app /etc/app
/app/server --config /etc/app/config.yaml

# Distroless image with no shell? Add a debug container to a copy that shares
# the process namespace, then browse the app's filesystem via /proc/<pid>/root
kubectl debug checkout-7d9f8b6c5-x2kqp -n shop -it --image=busybox:1.37 --share-processes --copy-to=checkout-debug2

# Clean up the copies afterwards
kubectl delete pod checkout-debug checkout-debug2 -n shop

Note that --copy-to creates a standalone Pod with the same spec, Secrets and service account. Treat it like production, and delete it when done.

Since 1.35, container restart rules are beta and on by default: a container can set restartPolicy plus restartPolicyRules to restart only on specific exit codes. That is useful for batch work where exit code 42 means "retry" and anything else means "stop", but it does not fix a crash loop; it only changes which exits trigger one.

Troubleshooting

Symptom Likely cause Fix
Exit code 1, log says a variable or file is missing Config not wired into the Pod Add the env var, ConfigMap or Secret key; check spelling and namespace
Exit 137, Reason OOMKilled Memory limit too low or a leak Raise resources.limits.memory or cap the runtime heap
Exit 137 or 143, Liveness probe failed events Probe too aggressive for startup time Add a startupProbe; keep liveness checks local to the process
Exit 127, executable file not found Wrong command or missing binary Fix command/args; check the image entrypoint with crane config
exec format error Image architecture does not match node Build a multi-arch image or set node affinity
Reason Completed, exit 0 Short-lived process under a Deployment Use a Job/CronJob, or run the server in the foreground with exec
Init:CrashLoopBackOff An init container is failing kubectl logs POD -c INIT_NAME --previous
--previous logs are empty Crash before logging, or logs written to a file Set terminationMessagePolicy: FallbackToLogsOnError; use kubectl debug --copy-to
permission denied or read-only file system runAsNonRoot or read-only root filesystem Mount an emptyDir where the app writes; fix file ownership in the image
Shows CreateContainerConfigError, not a crash loop Referenced Secret or ConfigMap key does not exist Create the object or key; then the Pod starts without a restart

If the Pod never gets to a node at all, see Pod Pending troubleshooting; if the image cannot be pulled, see ImagePullBackOff and ErrImagePull.

FAQ

How long does CrashLoopBackOff wait between restarts?

By default the kubelet waits 10 seconds after the second crash and doubles the delay each time up to 5 minutes. The delay resets after the container runs cleanly for 10 minutes. Since Kubernetes 1.35 a node administrator can lower the cap with crashLoopBackOff.maxContainerRestartPeriod in the kubelet configuration.

How do I see logs from a container in CrashLoopBackOff?

Use kubectl logs POD -c CONTAINER --previous. The --previous flag returns the output of the instance that crashed rather than the one currently waiting to start. If that is empty, the process died before logging, so use kubectl debug --copy-to with a shell.

Will deleting the Pod fix CrashLoopBackOff?

Usually not. The controller creates a new Pod from the same spec, which crashes for the same reason, though it does reset the backoff timer. It only helps when the cause was transient, such as a dependency that has since recovered.

What is the difference between CrashLoopBackOff and Error?

Error is the reason recorded for a single non-zero exit. CrashLoopBackOff means the container has exited repeatedly and the kubelet is now waiting before the next restart. You will usually see Error under Last State and CrashLoopBackOff under State on the same container.

In production, the first thing I check is not the logs but Last State: the exit code and how long the container lived narrow it to one or two causes before I read a line of output. The second is whether the liveness probe touches anything outside the process, because that is the crash loop that takes down every replica at once. If you are weighing whether all of this operational surface is worth it for your team, my sibling post on whether Kubernetes is still worth it in 2026 covers that. Official references: Pod lifecycle and container restarts, Debug running Pods, Determine the reason for Pod failure and Configure liveness, readiness and startup probes.

Related: Kubernetes as the Agent Control Plane in 2026, Agent Observability in 2026 and It's 2 AM. Do You Know What Your AI Agent Is Doing?.

I'm Ajay Kumar — I build and operate PandaStack, an open-source Firecracker microVM cloud for AI agents, and write about the infrastructure and tooling around it.

Need this kind of infrastructure work? See what I do or email hello@ajayk.sh.


Related