How to Build a Kubernetes Operator With Kubebuilder 4.16 (2026)

This guide is for engineers who know Go and Kubernetes basics and want to write their first real operator, not a hello-world. By the end you will have a Website custom resource with CEL validation, a controller that creates and owns a Deployment and Service, status conditions that kubectl get shows, a finalizer for cleanup, an envtest suite, and the operator running in a local kind cluster. Everything here was checked against Kubebuilder v4.16.0 (10 September 2026), whose scaffold pins Go 1.26, controller-runtime v0.25, k8s.io/api v0.37.0 and controller-tools v0.22.0, plus kind v0.33.0 with its default Kubernetes v1.37.0 node image. You need Go 1.26, Docker (or Podman), kubectl and kind. If you want the concepts first (what a reconcile loop is, and when Helm is enough), read Kubernetes operators explained and come back.

Step 1: Install Kubebuilder and create a kind cluster

Pin the binary rather than using the latest URL from the quick start, so your scaffold matches this guide.

# Kubebuilder v4.16.0 from the official GitHub release
curl -L -o kubebuilder \
  "https://github.com/kubernetes-sigs/kubebuilder/releases/download/v4.16.0/kubebuilder_$(go env GOOS)_$(go env GOARCH)"
chmod +x kubebuilder && sudo mv kubebuilder /usr/local/bin/
kubebuilder version   # first line: KubeBuilder: v4.16.0

go version            # go1.26.x

# Local cluster; kind v0.33.0 defaults to kindest/node:v1.37.0
kind create cluster --name website-dev
kubectl config current-context   # expect: kind-website-dev

Check the context every time. make install, make run and make deploy all use whatever your kubeconfig's current context points at, and the generated Makefile will happily install CRDs into production if that is where you were last.

Step 2: Initialise the project

mkdir website-operator && cd website-operator
kubebuilder init --domain example.com --repo example.com/website-operator

--domain becomes the suffix of your API group (web.example.com in the next step), and --repo is the Go module path, required outside GOPATH. The scaffold gives you cmd/main.go (the manager), a Makefile, a Dockerfile, Kustomize manifests under config/, a PROJECT file that tracks what was generated, and a test/e2e suite. Commit now so you can diff what each later command changes.

git init && git add -A && git commit -m "kubebuilder init (v4.16.0)"
grep -E "controller-runtime|k8s.io/api " go.mod
# expect controller-runtime v0.25.0 and k8s.io/api v0.37.0 (listed as indirect until you import it)

Step 3: Create the API and controller

kubebuilder create api --group web --version v1alpha1 --kind Website \
  --resource --controller

Both flags skip the interactive prompts. This writes api/v1alpha1/website_types.go, internal/controller/website_controller.go, a test file, a sample in config/samples/, then runs go mod tidy and make generate for you (you run make manifests yourself). Start at v1alpha1: you are allowed to break an alpha API, and you will want to.

Step 4: Define the CRD types with markers

Replace the generated WebsiteSpec and WebsiteStatus in api/v1alpha1/website_types.go. Markers are comments that controller-gen turns into OpenAPI schema, CEL rules, defaults and printer columns.

// api/v1alpha1/website_types.go

// WebsiteSpec defines the desired state of Website.
// Struct-level CEL rule, evaluated by the API server on create and update.
// +kubebuilder:validation:XValidation:rule="!self.image.endsWith(':latest')",message="pin the image; :latest is not allowed"
type WebsiteSpec struct {
	// image is the container image to serve, for example nginxinc/nginx-unprivileged:1.29.
	// MaxLength bounds the CEL cost estimate.
	// +kubebuilder:validation:MinLength=1
	// +kubebuilder:validation:MaxLength=253
	// +required
	Image string `json:"image"`

	// replicas is the number of pods.
	// +kubebuilder:validation:Minimum=0
	// +kubebuilder:validation:Maximum=10
	// +kubebuilder:default=1
	// +optional
	Replicas *int32 `json:"replicas,omitempty"`

	// port is the container port. Immutable after creation (transition rule).
	// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="port is immutable"
	// +kubebuilder:validation:Minimum=1
	// +kubebuilder:validation:Maximum=65535
	// +kubebuilder:default=8080
	// +optional
	Port int32 `json:"port,omitempty"`
}

// WebsiteStatus defines the observed state of Website.
type WebsiteStatus struct {
	// observedGeneration is the .metadata.generation last reconciled.
	// +optional
	ObservedGeneration int64 `json:"observedGeneration,omitempty"`

	// readyReplicas mirrors the Deployment's ready pod count.
	// +optional
	ReadyReplicas int32 `json:"readyReplicas,omitempty"`

	// conditions follow the Kubernetes API conventions.
	// +listType=map
	// +listMapKey=type
	// +optional
	Conditions []metav1.Condition `json:"conditions,omitempty"`
}

Then add printer columns above the existing // +kubebuilder:object:root=true and // +kubebuilder:subresource:status markers on the Website type (keep those two; the status subresource is what lets the controller write status separately from spec):

// +kubebuilder:printcolumn:name="Image",type=string,JSONPath=`.spec.image`
// +kubebuilder:printcolumn:name="Ready",type=string,JSONPath=`.status.conditions[?(@.type=="Available")].status`
// +kubebuilder:printcolumn:name="Pods",type=integer,JSONPath=`.status.readyReplicas`
// +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp`

Regenerate and look at the result:

make generate manifests
grep -A3 "x-kubernetes-validations" config/crd/bases/web.example.com_websites.yaml

# expected excerpt
              x-kubernetes-validations:
              - message: port is immutable
                rule: self == oldSelf

CEL validation rules have been stable since Kubernetes v1.29, and rules that reference oldSelf never run on create, so the immutability rule only bites on update.

Step 5: Write the Reconcile function

The controller below follows the standard shape: fetch, handle deletion, ensure the finalizer, converge children with CreateOrUpdate, then write status. The flow is easier to see as a diagram first.

Replace internal/controller/website_controller.go with:

package controller

import (
	"context"
	"fmt"

	appsv1 "k8s.io/api/apps/v1"
	corev1 "k8s.io/api/core/v1"
	"k8s.io/apimachinery/pkg/api/meta"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/apimachinery/pkg/runtime"
	"k8s.io/apimachinery/pkg/util/intstr"
	ctrl "sigs.k8s.io/controller-runtime"
	"sigs.k8s.io/controller-runtime/pkg/client"
	"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
	logf "sigs.k8s.io/controller-runtime/pkg/log"

	webv1alpha1 "example.com/website-operator/api/v1alpha1"
)

const websiteFinalizer = "web.example.com/finalizer"

// WebsiteReconciler reconciles a Website object.
type WebsiteReconciler struct {
	client.Client
	Scheme *runtime.Scheme
}

// RBAC markers: controller-gen turns these into config/rbac/role.yaml.
// +kubebuilder:rbac:groups=web.example.com,resources=websites,verbs=get;list;watch;create;update;patch;delete
// +kubebuilder:rbac:groups=web.example.com,resources=websites/status,verbs=get;update;patch
// +kubebuilder:rbac:groups=web.example.com,resources=websites/finalizers,verbs=update
// +kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete
// +kubebuilder:rbac:groups=core,resources=services,verbs=get;list;watch;create;update;patch;delete

func (r *WebsiteReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
	log := logf.FromContext(ctx)

	// 1. Fetch. NotFound means it was deleted after the event was queued.
	site := &webv1alpha1.Website{}
	if err := r.Get(ctx, req.NamespacedName, site); err != nil {
		return ctrl.Result{}, client.IgnoreNotFound(err)
	}

	// 2. Deletion: run cleanup while our finalizer still blocks removal.
	if !site.DeletionTimestamp.IsZero() {
		if controllerutil.ContainsFinalizer(site, websiteFinalizer) {
			if err := r.cleanupExternal(ctx, site); err != nil {
				return ctrl.Result{}, err // retried; object stays until cleanup succeeds
			}
			controllerutil.RemoveFinalizer(site, websiteFinalizer)
			if err := r.Update(ctx, site); err != nil {
				return ctrl.Result{}, err
			}
		}
		return ctrl.Result{}, nil
	}

	// 3. Ensure the finalizer. AddFinalizer reports whether it changed anything.
	if controllerutil.AddFinalizer(site, websiteFinalizer) {
		if err := r.Update(ctx, site); err != nil {
			return ctrl.Result{}, err
		}
	}

	labels := map[string]string{
		"app.kubernetes.io/name":       "website",
		"app.kubernetes.io/instance":   site.Name,
		"app.kubernetes.io/managed-by": "website-operator",
	}

	// 4a. Deployment. Mutate only the fields we own so server defaults survive.
	dep := &appsv1.Deployment{ObjectMeta: metav1.ObjectMeta{Name: site.Name, Namespace: site.Namespace}}
	depOp, err := controllerutil.CreateOrUpdate(ctx, r.Client, dep, func() error {
		dep.Spec.Replicas = site.Spec.Replicas
		if dep.Spec.Selector == nil { // selector is immutable; set it once
			dep.Spec.Selector = &metav1.LabelSelector{MatchLabels: labels}
		}
		dep.Spec.Template.Labels = labels
		if len(dep.Spec.Template.Spec.Containers) == 0 {
			dep.Spec.Template.Spec.Containers = []corev1.Container{{Name: "web"}}
		}
		c := &dep.Spec.Template.Spec.Containers[0]
		c.Image = site.Spec.Image
		c.Ports = []corev1.ContainerPort{{Name: "http", ContainerPort: site.Spec.Port, Protocol: corev1.ProtocolTCP}}
		return controllerutil.SetControllerReference(site, dep, r.Scheme)
	})
	if err != nil {
		return ctrl.Result{}, fmt.Errorf("reconcile deployment: %w", err)
	}

	// 4b. Service on port 80 pointing at the container port.
	svc := &corev1.Service{ObjectMeta: metav1.ObjectMeta{Name: site.Name, Namespace: site.Namespace}}
	if _, err := controllerutil.CreateOrUpdate(ctx, r.Client, svc, func() error {
		svc.Spec.Selector = labels
		svc.Spec.Ports = []corev1.ServicePort{{
			Name: "http", Port: 80, Protocol: corev1.ProtocolTCP,
			TargetPort: intstr.FromInt32(site.Spec.Port),
		}}
		return controllerutil.SetControllerReference(site, svc, r.Scheme)
	}); err != nil {
		return ctrl.Result{}, fmt.Errorf("reconcile service: %w", err)
	}

	// 5. Status. dep.Status was read by CreateOrUpdate's initial Get.
	want := int32(1)
	if site.Spec.Replicas != nil {
		want = *site.Spec.Replicas
	}
	cond := metav1.Condition{Type: "Available", ObservedGeneration: site.Generation}
	if dep.Status.ReadyReplicas >= want {
		cond.Status, cond.Reason = metav1.ConditionTrue, "DeploymentReady"
		cond.Message = fmt.Sprintf("%d/%d pods ready", dep.Status.ReadyReplicas, want)
	} else {
		cond.Status, cond.Reason = metav1.ConditionFalse, "Progressing"
		cond.Message = fmt.Sprintf("%d/%d pods ready", dep.Status.ReadyReplicas, want)
	}
	meta.SetStatusCondition(&site.Status.Conditions, cond)
	site.Status.ReadyReplicas = dep.Status.ReadyReplicas
	site.Status.ObservedGeneration = site.Generation
	if err := r.Status().Update(ctx, site); err != nil {
		return ctrl.Result{}, err // conflicts are normal; the retry re-reads fresh state
	}

	log.Info("reconciled", "deployment", depOp, "ready", dep.Status.ReadyReplicas)
	return ctrl.Result{}, nil
}

// cleanupExternal is where you delete things Kubernetes cannot garbage
// collect: DNS records, buckets, database users. It must be idempotent.
func (r *WebsiteReconciler) cleanupExternal(ctx context.Context, site *webv1alpha1.Website) error {
	logf.FromContext(ctx).Info("cleaning up external resources", "website", site.Name)
	return nil
}

// SetupWithManager watches Websites and the children they own.
func (r *WebsiteReconciler) SetupWithManager(mgr ctrl.Manager) error {
	return ctrl.NewControllerManagedBy(mgr).
		For(&webv1alpha1.Website{}).
		Owns(&appsv1.Deployment{}).
		Owns(&corev1.Service{}).
		Named("website").
		Complete(r)
}

A few choices worth explaining. CreateOrUpdate does a Get, runs your mutate function, and only issues an Update if the object changed, which keeps the loop idempotent. Setting the whole container slice each time would wipe server-defaulted fields like terminationMessagePath and cause an Update on every pass, so the mutate function edits the first container in place. SetControllerReference gives the children an owner reference, so deleting the Website garbage-collects them, and Owns() means a Deployment change (pods becoming ready) re-triggers the reconcile that updates status. There is no polling. If you need it for something external, return ctrl.Result{RequeueAfter: time.Minute}; the older Requeue: true field is deprecated in controller-runtime. For errors that retrying cannot fix, wrap them in reconcile.TerminalError(err).

Run make manifests again so the RBAC markers land in config/rbac/role.yaml. Forgetting this is the most common cause of forbidden errors after deploy.

Step 6: Run the controller locally against kind

make install                 # applies the CRD to the current context
kubectl get crd websites.web.example.com
make run                     # runs the manager on your laptop; leave it running

In a second terminal, edit config/samples/web_v1alpha1_website.yaml:

apiVersion: web.example.com/v1alpha1
kind: Website
metadata:
  name: demo
  namespace: default
spec:
  image: nginxinc/nginx-unprivileged:1.29
  replicas: 2
  port: 8080

kubectl apply -f config/samples/web_v1alpha1_website.yaml
kubectl get websites
# expected output (illustrative):
# NAME   IMAGE                              READY   PODS   AGE
# demo   nginxinc/nginx-unprivileged:1.29   True    2      40s

# Prove the CEL rules work:
kubectl patch website demo --type=merge -p '{"spec":{"port":9090}}'
# expected: The Website "demo" is invalid: spec.port: Invalid value: 9090: port is immutable
kubectl patch website demo --type=merge -p '{"spec":{"image":"nginx:latest"}}'
# expected: ... spec: Invalid value: pin the image; :latest is not allowed

Delete the Website and watch the make run log print the cleanup line before the Deployment and Service disappear.

Step 7: Test with envtest

envtest starts a real kube-apiserver and etcd (no kubelet, no controller-manager) and the Makefile downloads binaries matching the k8s.io/api minor version, so 1.37 here. Extend the generated internal/controller/website_controller_test.go. In BeforeEach, give the resource a valid spec, because the scaffolded one is empty and will fail your MinLength rule:

resource := &webv1alpha1.Website{
	ObjectMeta: metav1.ObjectMeta{Name: resourceName, Namespace: "default"},
	Spec:       webv1alpha1.WebsiteSpec{Image: "nginxinc/nginx-unprivileged:1.29", Port: 8080},
}

Then assert on the children inside the It block:

It("creates an owned Deployment and Service", func() {
	r := &WebsiteReconciler{Client: k8sClient, Scheme: k8sClient.Scheme()}
	_, err := r.Reconcile(ctx, reconcile.Request{NamespacedName: typeNamespacedName})
	Expect(err).NotTo(HaveOccurred())

	dep := &appsv1.Deployment{}
	Expect(k8sClient.Get(ctx, typeNamespacedName, dep)).To(Succeed())
	Expect(dep.Spec.Template.Spec.Containers[0].Image).To(Equal("nginxinc/nginx-unprivileged:1.29"))
	Expect(dep.OwnerReferences).To(HaveLen(1))

	site := &webv1alpha1.Website{}
	Expect(k8sClient.Get(ctx, typeNamespacedName, site)).To(Succeed())
	Expect(site.Finalizers).To(ContainElement(websiteFinalizer))
	// No kubelet in envtest, so pods never become ready:
	Expect(meta.IsStatusConditionFalse(site.Status.Conditions, "Available")).To(BeTrue())
})

Add appsv1 "k8s.io/api/apps/v1" and "k8s.io/apimachinery/pkg/api/meta" to the imports. Because the finalizer is now set, the scaffolded AfterEach delete leaves the object in Terminating; call Reconcile once more after Delete so the finalizer is removed. Then:

make test
# ok   example.com/website-operator/internal/controller   7.522s   coverage: 81.8% of statements

Two envtest facts trip people up: there is no garbage collector, so owned Deployments are not deleted when the parent is, and there is no kubelet, so nothing ever becomes ready. Test your logic, not Kubernetes. The v4.16.0 release also changed the scaffolded AfterSuite to check testEnv.Stop() once instead of retrying it, so a suite that leaks kube-apiserver processes now fails loudly.

Step 8: Build the image and deploy into the cluster

export IMG=website-operator:0.1.0
make docker-build IMG=$IMG
kind load docker-image $IMG --name website-dev   # no registry needed for kind
make deploy IMG=$IMG

kubectl -n website-operator-system rollout status deploy/website-operator-controller-manager
kubectl -n website-operator-system logs deploy/website-operator-controller-manager -c manager -f

make deploy runs Kustomize over config/default, which puts everything in the website-operator-system namespace with a website-operator- name prefix. Stop the make run process first, or two controllers will fight over the same objects. For a real cluster, push to a registry with make docker-push and consider make build-installer, which writes a single dist/install.yaml. If you ship it via Helm, remember that Helm does not upgrade CRDs in a chart's crds/ directory.

Troubleshooting

Symptom	Likely cause	Fix
no matches for kind "Website" in version "web.example.com/v1alpha1"	CRD not installed in this cluster, or stale	make install, then kubectl get crd websites.web.example.com
... is forbidden: User "system:serviceaccount:website-operator-system:..." cannot list resource "deployments"	RBAC markers missing or make manifests not rerun before deploy	Add the +kubebuilder:rbac marker, make manifests deploy IMG=$IMG
the object has been modified; please apply your changes to the latest version	Optimistic concurrency conflict between two writes	Normal under load; return the error and let the retry re-read. Persistent: two controllers running (stop make run)
Website stuck in Terminating	Finalizer never removed: controller down, uninstalled, or cleanup keeps failing	Fix and restart the controller. Last resort: kubectl patch website demo --type=merge -p '{"metadata":{"finalizers":null}}' and clean up by hand
Reconcile runs constantly, CPU high	Update every pass (whole-struct overwrite) or status written with a changing timestamp	Mutate only owned fields; meta.SetStatusCondition only bumps the transition time when status changes
Status never updates, no error	+kubebuilder:subresource:status removed, or using r.Update for status	Keep the marker; write status with r.Status().Update
New field defaults (such as port: 8080) not applied	CRD in the cluster is older than your markers	make manifests install, then re-apply the sample
CRD apply fails: CEL rule exceeded budget	Rule iterates over unbounded strings or lists	Add MaxLength, MaxItems, MaxProperties markers
make test fails with unable to start control plane	envtest binaries missing or wrong version	make setup-envtest; check bin/k8s/ exists and matches the go.mod minor
Pod ErrImagePull after make deploy on kind	Image not loaded into kind, or tag is latest so policy is Always	kind load docker-image $IMG --name website-dev with a non-latest tag

FAQ

How long does it take to build a Kubernetes operator with Kubebuilder?

The scaffold and a basic Level 1 operator like this one take an afternoon. The expensive part is everything after: upgrades of your own CRD schema, conversion between API versions, and the operational cases (backup, failover) that make an operator worth having over a Helm chart.

Should I use CreateOrUpdate or server-side apply?

CreateOrUpdate is the simpler starting point and what this guide uses. Server-side apply avoids read-modify-write conflicts and handles field ownership properly when several actors touch an object; Kubebuilder v4.16.0 has an alpha --ssa flag on create api that scaffolds apply configuration support.

Do I need a webhook for validation?

Not for rules about a single object. CEL markers like the two in Step 4 run in the API server. Use a webhook only for checks against other objects or external systems, or for conversion between API versions.

How do I upgrade a Kubebuilder project to a newer version?

The release notes recommend kubebuilder alpha update or the AutoUpdate plugin, which re-scaffold the project with the new version and reapply your changes. Both need cliVersion in your PROJECT file.

In production, the first thing I check on any operator, mine or someone else's, is the generated ClusterRole and whether a finalizer can block deletion when the controller is gone. Then I check that observedGeneration is in status, because without it nobody can tell whether Available=True refers to the current spec. For the bigger picture of when to build one at all, see Kubernetes operators explained. Official docs: the Kubebuilder book, controller-runtime releases, and the Kubernetes page on CustomResourceDefinitions.

Related: How to Fix Kubernetes RBAC Forbidden, How to Run Postgres on Kubernetes with CloudNativePG 1.30 and Deploy Your First App on Kubernetes With kind.