Kubernetes Operators Explained (2026): CRDs, Reconcile Loops, When to Write One

Every Kubernetes cluster I have inherited in the last few years had more operators in it than the team could name. cert-manager, a Postgres operator, the Prometheus Operator, something from the cloud vendor, and usually one homegrown controller written by someone who has since left. Operators are how most serious software now gets installed on Kubernetes, and they are also how a cluster quietly grows a second control plane nobody owns. This post explains what a Kubernetes operator actually is (a CRD plus a controller running a reconcile loop), how the capability levels work, what CEL validation changed for CRDs, when Helm is the better tool, and what operator sprawl costs you. If you want to build one after reading, the companion guide walks through building a Kubernetes operator with Kubebuilder step by step.

What is a Kubernetes operator?

The term came from CoreOS, which announced operators on 3 November 2016 with two examples, an etcd operator and a Prometheus operator. The idea was to take the runbook a human operator follows (create the cluster, add a member when one dies, take a backup before an upgrade) and encode it as software that runs inside Kubernetes and talks to the API like any other controller. The Kubernetes docs still describe the operator pattern in those terms: capturing the knowledge of someone who manages a specific application.

Mechanically, an operator is two things:

1. A CustomResourceDefinition (CRD), which teaches the API server a new type. After you install it, kubectl get postgresclusters works the same way kubectl get deployments does, with schema validation, RBAC and kubectl explain.
2. A controller, a long-running process (usually a Deployment in the cluster) that watches objects of that type and makes the world match them.

A CRD on its own is just structured storage in etcd. A controller without a CRD is just a controller; Kubernetes itself is made of dozens of those, watching built-in types. People say "operator" when the controller encodes application-specific operational knowledge, such as how to fail over a Postgres primary. Nothing in the API distinguishes them. It is a description of intent, not a feature.

How the reconcile loop works

The heart of every controller is Reconcile: a function that receives the name of one object, reads its desired state (spec), looks at the actual state of the world, and takes one step to close the gap. Then it writes what it observed into status and returns.

Three properties make this loop work, and every bug I have seen in homegrown controllers breaks one of them.

It is level-triggered, not edge-triggered. The controller is not told "replicas changed from 2 to 3". It is told "look at default/my-site" and has to work out everything from current state. Events get coalesced, dropped during restarts and replayed on resync, so a reconcile that depends on seeing a specific transition will eventually miss one.

It must be idempotent. Reconcile will run many times for the same object, often with nothing changed. Running it a tenth time must be as safe as running it once. That is why most controllers use create-or-update helpers rather than "create if this is a new object".

State lives in the API, not in the process. The controller can be killed at any moment and a new leader takes over. Anything it needs to remember goes into status, annotations, or child objects it can rediscover with labels and owner references.

In controller-runtime, the library under both Kubebuilder and Operator SDK, the return value controls what happens next: return an error and the key is retried with rate-limited backoff, return RequeueAfter to poll later, or return a TerminalError to log without retrying. The old Requeue: true flag is now marked deprecated in the reconcile package in favour of RequeueAfter.

Two other primitives show up in every real operator. Owner references link a child (a Deployment) to its parent (your custom resource), so the garbage collector deletes children when the parent goes, and so the controller is woken when a child changes. Finalizers are strings on the parent's metadata that block deletion until the controller has cleaned up something Kubernetes cannot see, such as a DNS record, a cloud bucket or a database user. Forget to remove the finalizer and kubectl delete hangs forever, which is the most common operator support ticket in existence.

Operator capability levels, and what they really measure

The Operator Framework publishes a capability model with five levels, and OperatorHub listings show it:

Level	Name	What the operator handles	Typical example
1	Basic Install	Provisioning and configuration from a custom resource	Most homegrown operators
2	Seamless Upgrades	Upgrading the operator and the operand, older operand versions	Operators that roll a new app version safely
3	Full Lifecycle	Backup, restore, failure recovery, app-aware scaling	Database operators with backup CRDs
4	Deep Insights	Metrics, alerts, events about operator and operand	Operators that ship dashboards and alerts
5	Auto Pilot	Automatic scaling, healing, tuning, anomaly detection	Rare, and worth reading the fine print

Source: Operator Framework capability levels.

Treat the levels as a checklist for questions, not a quality grade. A level is self-declared by the publisher. The useful exercise is to ask, for any operator you are about to install, "what happens on upgrade, on backup, on node loss?" and check whether the answer lives in the operator or in a runbook you still have to write. A Level 1 operator for a stateless app is fine. A Level 1 operator for a database means you own every hard part yourself, with an extra layer of indirection in the way.

CEL validation changed how CRDs are written

For years the only way to enforce rules like "minReplicas must not exceed maxReplicas" or "this field is immutable" was a validating admission webhook: another HTTPS server, another certificate, another thing that can take the API down if it is unavailable. That changed when CRD validation rules using CEL (x-kubernetes-validations) became stable in Kubernetes v1.29. The API server evaluates the expression itself, in process.

Two follow-ups made it practical for long-lived APIs. Transition rules reference oldSelf, so you can write immutability and allowed-transition rules without a webhook (they never apply on create). And validation ratcheting, stable since v1.33 and on by default, means tightening a rule no longer breaks updates to existing objects whose invalid fields were not touched. In Kubebuilder you write these as markers and controller-gen emits the schema:

// api/v1alpha1/website_types.go (Kubebuilder 4.16, controller-tools v0.22.0)
// The struct-level rule runs on every create and update.
// +kubebuilder:validation:XValidation:rule="!self.image.endsWith(':latest')",message="pin the image; :latest is not allowed"
type WebsiteSpec struct {
	// MaxLength keeps the CEL cost estimate bounded; the API server
	// rejects CRDs whose rules could exceed the cost budget.
	// +kubebuilder:validation:MinLength=1
	// +kubebuilder:validation:MaxLength=253
	Image string `json:"image"`

	// Transition rule: self == oldSelf makes the field immutable after create.
	// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="port is immutable"
	// +kubebuilder:validation:Minimum=1
	// +kubebuilder:validation:Maximum=65535
	// +kubebuilder:default=8080
	// +optional
	Port int32 `json:"port,omitempty"`
}

The cost budget is the catch. The docs show the error you get when a rule is estimated to be too expensive, and the fix is always the same: put maxItems, maxLength and maxProperties on anything a rule iterates over. My rule of thumb now is that if a validation does not need to call something outside the object, it belongs in CEL, not a webhook. Cluster-wide policy across many types is a different job; see Kyverno vs Gatekeeper vs ValidatingAdmissionPolicy for that layer.

Kubernetes operator vs Helm: when should you write one?

This is the question I get most, and the answer is usually "not yet". Helm renders templates and applies them once per helm upgrade. An operator runs continuously and reacts. If your application's operational life is "deploy a new version, change some config", Helm or Kustomize does that with far less code to own; Helm vs Kustomize covers that choice. Helm also has a sharp edge with operators specifically: its documentation states that there is no support for upgrading or deleting CRDs placed in a chart's crds/ directory, so CRD schema changes need their own rollout plan whichever way you install.

Write an operator when at least one of these is true:

• The app needs actions over time, not just at deploy: failover, backup schedules, certificate renewal, rebalancing partitions, rotating credentials.
• You are offering a platform API to other teams, and want them to request "a Postgres database" or "a preview environment" with a small, validated spec instead of copying 400 lines of YAML.
• Something outside the cluster must be kept in sync with something inside it, such as DNS, cloud resources or SaaS accounts, and you need deletion to clean up via a finalizer.

Do not write one when a CronJob, a Helm chart, or an existing operator does the job. Before you scaffold anything, search OperatorHub: the community-operators repository had 476 operator directories when I counted with the GitHub API on 10 November 2026. Also consider whether the thing you want is really infrastructure provisioning, where Crossplane or Terraform already solved the reconcile-cloud-resources problem.

Popular Kubernetes operators in 2026, and the toolchain

Most clusters I see run some subset of these. Versions are the latest GitHub releases as of 10 November 2026:

Operator	What it manages	Latest release	GitHub stars	Source
cert-manager	TLS certificates, ACME, issuers	v1.21.3	14,151	releases
Prometheus Operator	Prometheus, Alertmanager, scrape config CRDs	v0.95.1	9,993	releases
CloudNativePG	PostgreSQL clusters, backups, failover	v1.30.1	9,568	releases
Strimzi	Apache Kafka clusters, topics, users	1.3.1	5,976	releases

For building your own, the Go toolchain has consolidated. Kubebuilder v4.16.0 shipped on 10 September 2026; its scaffold targets Go 1.26, k8s.io/api v0.37.0 and controller-runtime v0.25. controller-runtime v0.25.0 moved to the Kubernetes 1.37 libraries, added a new EventRecorder interface (a breaking change for anyone upgrading) and an experimental read-your-own-writes client, which addresses the stale-cache reads that have bitten every controller author at least once; v0.25.2 followed on 1 October. Operator SDK builds its Go projects on Kubebuilder's plugins and adds OLM bundle tooling, plus Helm and Ansible operator types; its latest release is v1.42.3 from 26 June 2026. If you are not shipping to OperatorHub or OpenShift, I would start with plain Kubebuilder. Kubebuilder has also added an alpha --ssa flag on create api that scaffolds server-side apply support.

The risks of operator sprawl

Every operator you install is a privileged program with cluster-wide reach, written by someone else, on its own release cycle. The costs add up in ways a Helm chart's do not:

• Cluster-scoped CRDs are shared state. Two teams cannot run different major versions of the same operator's CRDs in one cluster. CRD upgrades are a cluster event, not a namespace event.
• RBAC is usually broad. Many operators ask for cluster-wide get/list/watch on Secrets or the ability to create pods anywhere. Read the ClusterRole before you install, the same way you would review Kubernetes RBAC for a human.
• Webhooks sit in the API request path. A conversion or validating webhook with failurePolicy: Fail whose pods are down blocks writes to that type, and sometimes to everything it matches. This is the strongest argument for moving validation into CEL.
• Upgrades chain. The operator version supports a range of Kubernetes versions, so a cluster upgrade becomes an upgrade of every operator first. Your Kubernetes release plan needs an operator column.
• Orphaned finalizers. Uninstall an operator before deleting its custom resources and every object with its finalizer will hang in Terminating, along with the namespace that holds it.

This is ordinary platform discipline: an inventory, owners, a version matrix, staged upgrades. It matters more now that AI tooling makes it trivial to generate a controller in an afternoon. Generating the code was never the expensive part. Owning a second control plane for five years is.

FAQ

What is the difference between a Kubernetes operator and a controller?

Every operator is a controller, but not every controller is an operator. A controller is any loop that watches API objects and reconciles state; the Deployment and Job controllers are examples. "Operator" usually means a controller plus a custom resource that encodes knowledge about running one specific application.

Is a CRD the same as an operator?

No. A CRD only adds a new type to the API server and stores objects of that type. Without a controller watching them, creating a custom resource does nothing except put a record in etcd.

Should I use Kubebuilder or Operator SDK in 2026?

For a Go operator that you deploy yourself, Kubebuilder is the simpler choice and Operator SDK uses its plugins anyway. Choose Operator SDK when you need OLM bundles, OperatorHub publishing, or a Helm or Ansible based operator.

Do I still need validating webhooks for CRDs?

Mostly not for single-object rules. CEL validation rules have been stable since Kubernetes v1.29 and handle cross-field checks, immutability and transition rules in the API server. You still need a webhook for checks that call external systems or look at other objects, and for conversion between stored versions with different schemas.

This quarter, if I were taking over a platform team, I would list every CRD in each cluster with kubectl get crd, map each one to an operator, an owner and a supported Kubernetes range, and delete anything nobody claims. Then I would move any simple validating webhooks in our own controllers to CEL rules, and only after that consider writing a new operator, following the Kubebuilder operator guide with envtest coverage from day one.

Related: Helm 4 vs Helm 3: What Changed, Should You Run Databases on Kubernetes in 2026? and Kubernetes as the Agent Control Plane in 2026.