How to Fix a Kubernetes CronJob Not Running or Running Twice (2026)
This guide is for anyone staring at a CronJob whose LAST SCHEDULE column says <none>, whose backup ran at 03:00 instead of 02:00, or whose report email arrived twice. By the end you will have a repeatable order of checks, the exact events and conditions that point to each cause, and a safe way to rehearse the fix without waiting for the next tick. Everything here was checked against the Kubernetes v1.37 documentation and the v1.37.2 controller source (the current release, published 28 October 2026); the fields used have all been GA since v1.27 or earlier, so it applies to any supported cluster. You need kubectl with read access to the namespace and, for the fixes, permission to patch CronJobs and create Jobs. If you want the background on how Jobs and CronJobs work first, read the companion explainer, Kubernetes Jobs and CronJobs explained.
Step 1: See what the CronJob controller thinks happened
Start with the CronJob itself, not the Pods. kubectl get shows the schedule, the time zone, whether it is suspended, how many Jobs are active and when it last fired.
# Replace NS and NAME. The TIMEZONE column is printed server-side by v1.30+ API servers.
NS=batch
NAME=nightly-backup
kubectl get cronjob "$NAME" -n "$NS"
# illustrative output
# NAME SCHEDULE TIMEZONE SUSPEND ACTIVE LAST SCHEDULE AGE
# nightly-backup 30 2 * * * Europe/London False 0 21h 40d
# The two timestamps that matter, straight from status
kubectl get cronjob "$NAME" -n "$NS" \
-o jsonpath='{.status.lastScheduleTime}{" "}{.status.lastSuccessfulTime}{"\n"}'
# Events recorded against the CronJob (kept 1 hour by default)
kubectl get events -n "$NS" \
--field-selector involvedObject.kind=CronJob,involvedObject.name="$NAME" \
--sort-by=.lastTimestamp
The event reasons come straight from the CronJob controller source, and they map almost one to one onto the causes below:
SuccessfulCreate: a Job was created. If you see this, the CronJob worked and your problem is in the Job or Pod (Cause 5).MissSchedule: the run was skipped because it was later thanstartingDeadlineSeconds(Cause 3).TooManyMissedTimes: more than 100 schedules were missed (Cause 3).JobAlreadyActive:concurrencyPolicy: Forbidskipped this run (Cause 4).UnknownTimeZone,UnparseableSchedule,InvalidSchedule: the spec is wrong (Cause 1).FailedCreate: the Job could not be created, usually quota, admission policy or RBAC.
Events expire after one hour unless your cluster overrides the kube-apiserver --event-ttl flag, so check soon after a missed run or ship events to your log store.
Cause 1: The schedule or timeZone is not what you think
The most common "it never runs" is "it runs, at a time you did not expect". Without .spec.timeZone, the CronJob docs say the schedule is interpreted in the kube-controller-manager's local time zone. On managed clusters you cannot see that process, so treat an unset timeZone as a bug. The field has been stable since v1.27 and takes IANA names.
# Set an explicit time zone (IANA name, not an abbreviation like BST or IST)
kubectl patch cronjob "$NAME" -n "$NS" --type=merge \
-p '{"spec":{"timeZone":"Europe/London"}}'
Things that get rejected or misread:
CRON_TZ=Europe/London 30 2 * * *orTZ=...in the schedule string. The docs say this was never supported and now fails validation on create or update; the controller also emits anUnsupportedSchedulewarning for any old object that still carries it. Use the field.- Six-field schedules with seconds, copied from Quartz or Spring. Kubernetes uses five fields: minute, hour, day of month, month, day of week.
- Day of week
7. The documented range is 0 to 6 (Sunday to Saturday) orsuntosat. - Expecting
0 0 31 * *every month. It only fires in months with 31 days.
Also check the name. The controller appends 11 characters to the CronJob name to build the Job name, and Job names are limited to 63, so the docs cap CronJob names at 52 characters.
Cause 2: The CronJob is suspended
SUSPEND True in the get output means nothing new will start. Teams suspend CronJobs during incidents or migrations and forget, or a GitOps tool keeps reapplying suspend: true from Git. Check where the value came from before patching it, or your fix will be reverted on the next sync.
# Who set suspend? managedFields shows the field manager (argocd, helm, kubectl...)
kubectl get cronjob "$NAME" -n "$NS" --show-managed-fields -o yaml \
| grep -B2 -A12 'f:suspend'
# Resume scheduling
kubectl patch cronjob "$NAME" -n "$NS" --type=merge -p '{"spec":{"suspend":false}}'
Know what happens when you unsuspend. Per the docs, schedules missed while suspended count as missed, and if there is no startingDeadlineSeconds, a Job for the most recent missed schedule starts immediately. That is one of the classic "why did it run at 14:07?" reports. If an off-schedule run would be harmful, set startingDeadlineSeconds before you unsuspend.
Cause 3: startingDeadlineSeconds and the 100 missed schedules rule
startingDeadlineSeconds is how late a run may start. Each time the controller syncs, if the scheduled time plus the deadline is already in the past, it skips the run and records MissSchedule. Two traps:
- Very small values. The docs warn that under 10 seconds the run may never be scheduled, because the controller works on a 10-second check cycle. A value like
5looks strict and is actually "sometimes never". Use minutes, not seconds. - Controller downtime. The controller counts missed schedules since the last run, or since
now - startingDeadlineSecondsif set. In the v1.37.2 source, more than 100 raisesTooManyMissedTimes. The docs, clarified on 31 October 2026, state that the controller still starts one Job for the most recent missed time. It never backfills the rest.
The practical upshot: a per-minute CronJob after a two-hour control plane outage will log the warning and run once. Setting a deadline narrows the counting window, so the warning goes away, and you choose whether a late run is still useful.
# Allow a backup to start up to 1 hour late, never later
spec:
startingDeadlineSeconds: 3600
If you also see the warning on a healthy cluster, compare node and control plane clocks; the event text itself suggests checking clock skew.
Cause 4: concurrencyPolicy skips runs (or allows overlap)
With concurrencyPolicy: Forbid, a run that comes due while the previous Job is still active is skipped with a JobAlreadyActive event, and it counts as missed. A nightly job that slowly grows from 20 minutes to 25 hours will silently skip every other night.
# Is a Job still active, and how long has it been running?
kubectl get jobs -n "$NS" --sort-by=.status.startTime | grep "$NAME"
kubectl get cronjob "$NAME" -n "$NS" -o jsonpath='{.status.active[*].name}{"\n"}'
The fix is to bound the run, not to switch policy blindly. Put activeDeadlineSeconds on the jobTemplate.spec so a stuck Job fails with DeadlineExceeded and the next run can start. Choose the policy by what is safe: Forbid for anything that must not overlap (backups, migrations), Replace when only the newest run matters (cache warmers), Allow only when runs are independent. Note the combination the docs call out: under Forbid, when the long run finishes, a missed schedule still inside startingDeadlineSeconds can start right away. That looks like a second run.
Cause 5: The Job was created but failed (BackoffLimitExceeded, DeadlineExceeded)
If the events show SuccessfulCreate, the CronJob did its job. Look at the Job's conditions.
JOB=$(kubectl get jobs -n "$NS" --sort-by=.metadata.creationTimestamp -o name | grep "$NAME" | tail -1)
kubectl get "$JOB" -n "$NS" \
-o jsonpath='{range .status.conditions[*]}{.type}={.status} {.reason}: {.message}{"\n"}{end}'
# Failed=True BackoffLimitExceeded: Job has reached the specified backoff limit
# Logs from every attempt (works best with restartPolicy: Never)
kubectl logs -n "$NS" -l batch.kubernetes.io/job-name="${JOB#job.batch/}" --prefix --tail=50
The reasons you will see, from the Job API types: BackoffLimitExceeded (retries used up, default limit 6), DeadlineExceeded (activeDeadlineSeconds hit), PodFailurePolicy (a FailJob rule matched), and for Indexed Jobs FailedIndexes or MaxFailedIndexesExceeded. From there it is a Pod problem: CrashLoopBackOff, ImagePullBackOff, or Pods stuck in Pending because of quota or requests.
Cause 6: History limits and TTL deleted the evidence
"It never ran" sometimes means "it ran and failed, and the failed Job is gone". CronJobs keep 3 successful and 1 failed Job by default. If someone set failedJobsHistoryLimit: 0, or the template sets a short ttlSecondsAfterFinished, there is nothing left to inspect. status.lastScheduleTime updating while status.lastSuccessfulTime does not is the giveaway.
kubectl patch cronjob "$NAME" -n "$NS" --type=merge \
-p '{"spec":{"failedJobsHistoryLimit":3,"successfulJobsHistoryLimit":3}}'
Why does my Kubernetes CronJob run twice?
The controller names each Job <cronjob>-<scheduled minute since epoch>, so it will not create two Jobs for the same tick by itself. The docs still warn that two Jobs, or none, can occasionally be created and that Jobs must be idempotent. In practice, double runs come from:
- Pod retries. A Pod that did the work and then crashed on exit, or was evicted after finishing the side effect, gets retried by the Job. Your code ran twice inside one Job.
restartPolicy: OnFailure. The container restarts in place, re-running the work.kubectl create cronjobdefaults toOnFailure; I checked withkubectl create cronjob --dry-run=client -o yaml.- Two CronJobs. The same chart installed in two namespaces or two clusters, or a renamed CronJob whose old copy still exists. Search by image:
kubectl get cronjobs -A -o wide | grep IMAGE. - Catch-up runs after unsuspend or under
Forbid, as above. - DST. With a time zone that observes daylight saving, a schedule inside the repeated hour can fire twice on the changeover day.
- Someone ran it by hand with
kubectl create job --from.
Compare the batch.kubernetes.io/cronjob-scheduled-timestamp annotation (on by default for CronJob-created Jobs since v1.28, GA in v1.32) on the two Jobs to see which case you are in.
Step 2: Rehearse the fix with kubectl create job --from
Do not wait until 02:30 to find out. kubectl create job --from=cronjob/NAME creates a one-off Job from the CronJob's current template, so it tests your image, command, Secrets and RBAC exactly as the next scheduled run would.
kubectl create job "${NAME}-manual-$(date +%s)" -n "$NS" --from=cronjob/"$NAME"
kubectl get jobs -n "$NS" -w
# illustrative output
# NAME STATUS COMPLETIONS DURATION AGE
# nightly-backup-manual-1762750000 Running 0/1 4s 4s
# nightly-backup-manual-1762750000 Complete 1/1 31s 31s
It does not test the schedule, time zone or concurrency settings, so after a schedule change, temporarily set a schedule a few minutes ahead and watch LAST SCHEDULE update.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
LAST SCHEDULE is <none>, no events |
Suspended, or name or schedule invalid | kubectl get cronjob; check SUSPEND, fix spec |
| Runs one hour or more off | No timeZone; controller uses its local zone |
Set .spec.timeZone to an IANA name |
| Create fails with schedule validation error | CRON_TZ= or TZ= in schedule, or 6 fields |
Use timeZone field and 5-field cron |
MissSchedule warning |
Started later than startingDeadlineSeconds |
Raise the deadline; avoid values under 10s |
TooManyMissedTimes warning |
More than 100 missed ticks (outage, skew) | Set a deadline, check clocks; one catch-up run occurs |
JobAlreadyActive, runs skipped |
Forbid and the previous Job still running |
Add activeDeadlineSeconds, speed up the job |
Job Failed, BackoffLimitExceeded |
Container exits non-zero repeatedly | Read logs per attempt; add podFailurePolicy |
Job Failed, DeadlineExceeded |
activeDeadlineSeconds too short or job hung |
Raise the deadline or fix the hang |
| Ran at an odd time after a change | Unsuspend started the most recent missed run | Set startingDeadlineSeconds before unsuspending |
| Work done twice | Pod retry, OnFailure, duplicate CronJob |
Make the job idempotent; search all namespaces |
FAQ
How do I trigger a Kubernetes CronJob manually?
Run kubectl create job my-test --from=cronjob/NAME -n NAMESPACE. It creates a Job from the CronJob's template immediately, without changing the schedule. Delete it afterwards or let the template's ttlSecondsAfterFinished clean it up.
What time zone does a Kubernetes CronJob use by default?
The local time zone of the kube-controller-manager process. On managed services you cannot easily see it, so set .spec.timeZone explicitly, for example Etc/UTC or America/New_York. The field is GA since v1.27.
What does "too many missed start times" mean?
The controller found more than 100 schedules it should have run since the last run, usually after control plane downtime or clock skew. It records the warning, starts one Job for the most recent schedule, and skips the rest. Setting startingDeadlineSeconds limits the window it counts over.
Why is my CronJob not creating Jobs even though it is not suspended?
Check the CronJob's events within an hour of the missed time. MissSchedule means the deadline passed, JobAlreadyActive means Forbid blocked it, FailedCreate means quota, admission or RBAC rejected the Job, and schedule or time zone errors point to the spec.
In production the first thing I check is the gap between lastScheduleTime and lastSuccessfulTime, because it separates "the scheduler never fired" from "it fired and failed" in one command, and I alert on that gap rather than on Pod failures. Then I make sure every CronJob has timeZone, startingDeadlineSeconds, activeDeadlineSeconds and a non-zero failedJobsHistoryLimit. The official references are the CronJob concept page, the Job concept page and the Running automated tasks with a CronJob task.
Related: How to Fix Kubernetes Pods Stuck in Pending: Causes and Commands (2026), Fix Kubernetes OOMKilled and Exit Code 137: Step-by-Step Guide (2026) and How to Fix Kubernetes RBAC Forbidden: User Cannot List Resource (2026).
Spot an error or something out of date? Email me and I will fix it and log it on the corrections page.
I'm Ajay Kumar — I build and operate PandaStack, an open-source Firecracker microVM cloud for AI agents, and write about the infrastructure and tooling around it.
Need this kind of infrastructure work? See what I do or email hello@ajayk.sh.
Related
How to Fix AWS ALB Unhealthy Targets and 502/503 Errors (2026)
Fix AWS Application Load Balancer unhealthy targets and 502, 503 and 504 errors step by step: read the health reason code, fix health check paths and success codes, security groups, ip vs instance targets, EKS readiness gates, deregistration delay, keep-alive timeouts and access logs.
13 minNov 10, 2026How to Fix a Kubernetes Service Connection Refused or Timeout (2026)
Step-by-step fix for a Kubernetes Service that is not reachable: tell connection refused from timeout, then check selectors and labels, empty EndpointSlices, targetPort vs containerPort, readiness, NetworkPolicy and kube-proxy, using netshoot and kubectl debug on Kubernetes 1.37.
11 minNov 10, 2026How to Upgrade a kubeadm Cluster to Kubernetes 1.37: Step by Step (2026)
Upgrade a kubeadm cluster from Kubernetes 1.36 to 1.37.2 safely: back up etcd, scan for deprecated APIs with Pluto, switch the pkgs.k8s.io repo, run kubeadm upgrade plan and apply, then drain, upgrade and uncordon every node, with a troubleshooting table.
13 min