How to Build a k3s Cluster at Home: HA etcd, MetalLB, Upgrades (2026)

This guide is for anyone who wants a real multi-node Kubernetes cluster on their own hardware: a few mini PCs, old laptops or Raspberry Pis with SSDs. At the end you will have three k3s servers running embedded etcd, optional agent nodes, a kubeconfig on your workstation, MetalLB handing out LAN IPs, Traefik serving ingress, persistent storage, automated upgrades and etcd snapshots you have actually restored. Everything is pinned and checked against the current releases: k3s v1.37.2+k3s1 (Kubernetes 1.37.2), MetalLB v0.16.2, Longhorn 1.13.0 and system-upgrade-controller v0.21.0. You need 64-bit Linux on each node (Ubuntu 24.04 or Debian 13 are easiest), static or DHCP-reserved IPs, SSH access, and kubectl plus Helm on your workstation. If you are still choosing hardware or wondering whether to use Talos instead, read the companion post, Homelab for DevOps in 2026, first.

The examples use these addresses. Change them to match your network.

Host	IP	Role
k3s-1	192.168.1.11	server (first, cluster-init)
k3s-2	192.168.1.12	server
k3s-3	192.168.1.13	server
k3s-a1	192.168.1.21	agent (optional)
MetalLB pool	192.168.1.200 to 192.168.1.220	LoadBalancer IPs, outside your DHCP range

Step 1: Prepare every node

The k3s requirements are small: 2 cores and 2GB of RAM for a server, 1 core and 512MB for an agent. The part people skip is storage. The docs say etcd is write-intensive and that SD cards and eMMC cannot handle the I/O, so every server node needs an SSD.

Open the ports k3s uses between nodes. With ufw on Ubuntu:

# On every node. Adjust 192.168.1.0/24 to your LAN.
sudo ufw allow from 192.168.1.0/24 to any port 6443 proto tcp       # API server and supervisor
sudo ufw allow from 192.168.1.0/24 to any port 2379:2380 proto tcp  # embedded etcd, servers only
sudo ufw allow from 192.168.1.0/24 to any port 10250 proto tcp      # kubelet
sudo ufw allow from 192.168.1.0/24 to any port 8472 proto udp       # Flannel VXLAN
sudo ufw allow from 192.168.1.0/24 to any port 7946                 # MetalLB memberlist, tcp and udp
# k3s pod and service CIDRs must also be allowed, or pod traffic is dropped:
sudo ufw allow from 10.42.0.0/16 to any
sudo ufw allow from 10.43.0.0/16 to any

On Raspberry Pi OS, k3s will not start until memory cgroups are enabled. Append the flags to the single line in the kernel command line and reboot:

# Raspberry Pi OS (Bookworm and later). On Debian 11 era images the file is /boot/cmdline.txt.
sudo sed -i '$ s/$/ cgroup_memory=1 cgroup_enable=memory/' /boot/firmware/cmdline.txt
sudo reboot

Give every node a unique hostname and make sure they can resolve each other (DNS on your router, or /etc/hosts). Create a DNS name for the API, here k3s.home.arpa, pointing at k3s-1. It is your fixed registration address.

Step 2: Install the first server with embedded etcd

Use a config file rather than a long command line; k3s reads /etc/rancher/k3s/config.yaml at start, and the file goes into your notes or Git. Generate a token once and reuse it on every node.

# On k3s-1
openssl rand -hex 32 > ~/k3s-token   # keep this safe; it lets machines join the cluster
sudo mkdir -p /etc/rancher/k3s
sudo tee /etc/rancher/k3s/config.yaml >/dev/null <<EOF
cluster-init: true
token: "$(cat ~/k3s-token)"
write-kubeconfig-mode: "0640"
tls-san:
  - "k3s.home.arpa"
  - "192.168.1.11"
disable:
  - servicelb          # MetalLB replaces it in Step 6
EOF

curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION="v1.37.2+k3s1" sh -s - server

cluster-init starts embedded etcd instead of the default SQLite datastore, as described in the HA embedded etcd docs. Check it:

sudo k3s kubectl get nodes -o wide
# Expected output (illustrative, columns trimmed):
# NAME    STATUS   ROLES                       AGE   VERSION
# k3s-1   Ready    control-plane,etcd          45s   v1.37.2+k3s1

(The ROLES column may include master on some versions; it is the same thing.)

Step 3: Join two more servers

etcd needs a majority to accept writes. With three servers you can lose one; with two you can lose none, which is worse than one. The k3s docs say an HA embedded etcd cluster must have an odd number of server nodes.

Some flags must match on every server: the network flags (cluster-cidr, service-cidr, cluster-dns, cluster-domain), every disable entry and secrets-encryption. Copy the same file, swap cluster-init for server:

# On k3s-2 and k3s-3 (copy ~/k3s-token from k3s-1 first)
sudo mkdir -p /etc/rancher/k3s
sudo tee /etc/rancher/k3s/config.yaml >/dev/null <<EOF
server: "https://192.168.1.11:6443"
token: "$(cat ~/k3s-token)"
write-kubeconfig-mode: "0640"
tls-san:
  - "k3s.home.arpa"
disable:
  - servicelb
EOF

curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION="v1.37.2+k3s1" sh -s - server

Join them one at a time and wait for each to show Ready before starting the next.

sudo k3s kubectl get nodes
# Expected output (illustrative):
# k3s-1   Ready   control-plane,etcd   6m    v1.37.2+k3s1
# k3s-2   Ready   control-plane,etcd   2m    v1.37.2+k3s1
# k3s-3   Ready   control-plane,etcd   40s   v1.37.2+k3s1

Step 4: Add agent nodes (optional)

Servers run workloads too, so a three-node lab can stop here. Agents are worth adding for practising node drains and for a Raspberry Pi to give you an ARM node.

# On each agent
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION="v1.37.2+k3s1" \
  K3S_TOKEN="$(cat ~/k3s-token)" \
  sh -s - agent --server https://k3s.home.arpa:6443

Step 5: Get a kubeconfig on your workstation

k3s writes the admin kubeconfig to /etc/rancher/k3s/k3s.yaml with the server set to 127.0.0.1. Copy it and point it at the DNS name you put in tls-san:

# On your workstation
ssh k3s-1 'sudo cat /etc/rancher/k3s/k3s.yaml' > ~/.kube/k3s-home.yaml
chmod 600 ~/.kube/k3s-home.yaml
sed -i.bak 's#https://127.0.0.1:6443#https://k3s.home.arpa:6443#' ~/.kube/k3s-home.yaml
export KUBECONFIG=~/.kube/k3s-home.yaml
kubectl get nodes

This file is cluster-admin. Treat it like a root password, and if you want a less powerful identity for daily use, the RBAC guide shows how bindings work.

Step 6: Install MetalLB for LoadBalancer IPs

k3s ships ServiceLB (klipper-lb), which exposes LoadBalancer Services through host ports on the nodes. It is fine for one node. For a cluster you want a real LAN IP per Service that moves when a node dies, which is what MetalLB in L2 mode does. The k3s networking docs say to start every server with --disable=servicelb before using another load balancer; we did that in Steps 2 and 3.

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.16.2/config/manifests/metallb-native.yaml
kubectl -n metallb-system rollout status deploy/controller --timeout=120s

Then give it a pool of addresses your DHCP server will never hand out:

# metallb-pool.yaml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: home-pool
  namespace: metallb-system
spec:
  addresses:
    - 192.168.1.200-192.168.1.220
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: home-l2
  namespace: metallb-system
spec:
  ipAddressPools:
    - home-pool

kubectl apply -f metallb-pool.yaml
kubectl -n kube-system get svc traefik
# Expected output (illustrative):
# NAME      TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)
# traefik   LoadBalancer   10.43.x.x      192.168.1.200   80:3xxxx/TCP,443:3xxxx/TCP

Traefik's Service was pending until now; it should pick up the first address in the pool.

Step 7: Use the bundled Traefik ingress

k3s v1.37.2+k3s1 bundles Traefik v3.7.15 and the Gateway API CRDs, per the release notes. Test it with a small app and a standard Ingress:

# whoami.yaml
apiVersion: apps/v1
kind: Deployment
metadata: { name: whoami, namespace: default }
spec:
  replicas: 2
  selector: { matchLabels: { app: whoami } }
  template:
    metadata: { labels: { app: whoami } }
    spec:
      containers:
        - name: whoami
          image: traefik/whoami:v1.12.0
          ports: [{ containerPort: 80 }]
---
apiVersion: v1
kind: Service
metadata: { name: whoami, namespace: default }
spec:
  selector: { app: whoami }
  ports: [{ port: 80, targetPort: 80 }]
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata: { name: whoami, namespace: default }
spec:
  ingressClassName: traefik
  rules:
    - host: whoami.home.arpa
      http:
        paths:
          - path: /
            pathType: Prefix
            backend: { service: { name: whoami, port: { number: 80 } } }

kubectl apply -f whoami.yaml
curl -s -H 'Host: whoami.home.arpa' http://192.168.1.200/ | head -3
# Expected (illustrative):
# Hostname: whoami-...

To change Traefik's settings, do not edit /var/lib/rancher/k3s/server/manifests/traefik.yaml; k3s overwrites it at start. Drop a HelmChartConfig next to it on a server instead. This one turns on the Gateway API provider so you can practise HTTPRoutes, which is where the ecosystem went after Ingress NGINX was retired:

# /var/lib/rancher/k3s/server/manifests/traefik-config.yaml (on a server node)
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: traefik
  namespace: kube-system
spec:
  valuesContent: |-
    providers:
      kubernetesGateway:
        enabled: true

Step 8: Set up persistent storage

k3s installs the local-path provisioner (v0.0.37 in this release) as the default StorageClass. It creates a directory on whichever node the pod lands on. That is fast and simple, and the volume dies with the node. Fine for caches and scratch data; not fine for a database you care about.

kubectl get storageclass
# Expected output (illustrative, columns trimmed):
# NAME                   PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE
# local-path (default)   rancher.io/local-path   Delete          WaitForFirstConsumer

For replicated volumes, install Longhorn. Longhorn 1.13.0 requires Kubernetes 1.34 or later, and its requirements include open-iscsi with iscsid running, an NFSv4 client, and ext4 or XFS. The docs warn against open-iscsi 2.1.12 specifically.

# On every node (Ubuntu/Debian)
sudo apt-get install -y open-iscsi nfs-common cryptsetup
sudo systemctl enable --now iscsid

# From the workstation: optional preflight with longhornctl v1.13.0, then install
./longhornctl check preflight
helm repo add longhorn https://charts.longhorn.io
helm repo update
helm install longhorn longhorn/longhorn --namespace longhorn-system \
  --create-namespace --version 1.13.0
kubectl -n longhorn-system get pods

Once it is healthy, you can make longhorn the default class and remove the default annotation from local-path. The background on why this matters is in Kubernetes storage explained.

Step 9: Automate upgrades with system-upgrade-controller

The automated upgrades docs use Rancher's system-upgrade-controller, which runs upgrade Jobs node by node from Plan resources. Install a pinned release:

kubectl apply -f https://github.com/rancher/system-upgrade-controller/releases/download/v0.21.0/crd.yaml \
  -f https://github.com/rancher/system-upgrade-controller/releases/download/v0.21.0/system-upgrade-controller.yaml
kubectl -n system-upgrade get deploy

Then two Plans: servers one at a time, agents after the servers finish. Pin version instead of following a channel, so the upgrade happens when you change a line in Git, not when Rancher publishes.

# upgrade-plans.yaml
apiVersion: upgrade.cattle.io/v1
kind: Plan
metadata:
  name: server-plan
  namespace: system-upgrade
spec:
  concurrency: 1
  cordon: true
  nodeSelector:
    matchExpressions:
      - { key: node-role.kubernetes.io/control-plane, operator: In, values: ["true"] }
  serviceAccountName: system-upgrade
  upgrade:
    image: rancher/k3s-upgrade
  version: v1.37.2+k3s1      # bump this, one minor at a time
---
apiVersion: upgrade.cattle.io/v1
kind: Plan
metadata:
  name: agent-plan
  namespace: system-upgrade
spec:
  concurrency: 1
  cordon: true
  nodeSelector:
    matchExpressions:
      - { key: node-role.kubernetes.io/control-plane, operator: DoesNotExist }
  prepare:
    image: rancher/k3s-upgrade
    args: ["prepare", "server-plan"]   # wait for the server plan to finish
  serviceAccountName: system-upgrade
  upgrade:
    image: rancher/k3s-upgrade
  version: v1.37.2+k3s1

kubectl apply -f upgrade-plans.yaml
kubectl -n system-upgrade get plans,jobs

Take an etcd snapshot (next step) before every version bump. The Kubernetes version skew policy does not let the control plane skip minor versions, so go 1.36 to 1.37, never 1.35 to 1.37.

Step 10: Take and restore etcd snapshots

k3s snapshots etcd at 00:00 and 12:00 and keeps 5 per server by default, in /var/lib/rancher/k3s/server/db/snapshots, according to the etcd-snapshot docs. Take one on demand and copy it off the box:

sudo k3s etcd-snapshot save --name pre-upgrade
sudo k3s etcd-snapshot ls
scp k3s-1:/var/lib/rancher/k3s/server/db/snapshots/pre-upgrade-* ./backups/

Restore drill (do this once in the lab, on purpose):

# On every server
sudo systemctl stop k3s
# On k3s-1 only
sudo k3s server --cluster-reset \
  --cluster-reset-restore-path=/var/lib/rancher/k3s/server/db/snapshots/<snapshot-file>
sudo systemctl start k3s
# On k3s-2 and k3s-3: remove the old etcd data, then start so they rejoin
sudo rm -rf /var/lib/rancher/k3s/server/db/
sudo systemctl start k3s

etcd snapshots hold cluster objects, not the data in your volumes. For that, add Velero or Longhorn backups to a target outside the cluster.

Troubleshooting

Symptom	Likely cause	Fix
k3s.service fails on a Raspberry Pi with a cgroup error	Memory cgroups not enabled	Append cgroup_memory=1 cgroup_enable=memory to /boot/firmware/cmdline.txt and reboot
Second server never joins, logs show token or CA errors	Different token, or server: points at the wrong host	Use the exact token from the first server (/var/lib/rancher/k3s/server/token), check server: URL and port 6443
Join hangs at etcd, context deadline exceeded	Ports 2379 and 2380 blocked between servers	Open them on every server; check sudo journalctl -u k3s -f
etcd warnings about slow fsync, API timeouts	SD card or slow USB disk	Move server nodes to SSD; the k3s docs warn against SD and eMMC
traefik Service stuck at <pending> EXTERNAL-IP	ServiceLB disabled and MetalLB not installed, or no IPAddressPool	Apply the MetalLB manifest and the pool plus L2Advertisement
Both ServiceLB svclb-* pods and MetalLB fight for a Service	servicelb not disabled on every server	Add servicelb to disable on all servers and restart k3s one at a time
Pods on different nodes cannot talk	Firewall drops VXLAN (UDP 8472) or pod CIDR traffic	Allow UDP 8472 and 10.42.0.0/16, 10.43.0.0/16 between nodes
kubectl from workstation: x509 certificate is valid for 127.0.0.1, not k3s.home.arpa	Name missing from tls-san	Add it to tls-san on every server and restart k3s
Longhorn volume stuck attaching	iscsid not running, or open-iscsi 2.1.12	systemctl enable --now iscsid; use 2.1.11 or 2.1.13 and later
Upgrade Plan creates no Jobs	Plan in the wrong namespace or selector matches no node	Create Plans in system-upgrade; check kubectl get nodes --show-labels

FAQ

How many nodes does a k3s HA cluster need?

Three servers is the minimum for embedded etcd that survives losing one node. The k3s docs require an odd number of servers so etcd keeps quorum; adding a fourth raises the quorum without adding fault tolerance. Agents can be any number.

Can I use k3s with SQLite instead of etcd?

Yes. A single server without cluster-init uses SQLite, which is lighter and fine for one node. You can convert later by restarting that server with --cluster-init, then join more servers.

Should I keep ServiceLB or use MetalLB?

ServiceLB is fine on a single node or if host ports are all you need. MetalLB gives each Service its own LAN IP that fails over between nodes, which is closer to how cloud load balancers behave and therefore better practice. You cannot sensibly run both on the same cluster.

How do I upgrade k3s safely?

Snapshot etcd, bump the pinned version in the system-upgrade-controller Plans one minor release at a time, and watch the Jobs drain and upgrade one node at a time. Read the release notes first for breaking changes in bundled components such as Traefik.

When something breaks in my own clusters, I check journalctl -u k3s on the affected node before anything in kubectl, because most k3s failures (tokens, ports, disks, certificates) happen below the API server. In a lab, I would put every file in this guide into a Git repository, hand it to Argo CD once the cluster is up, and rebuild from scratch monthly to prove the Git repo is the truth. The official references are the k3s documentation, the MetalLB release notes, the Longhorn 1.13 install docs and the system-upgrade-controller releases. For hardware choices and what to practise next, see the companion DevOps homelab post.

Related: How to Set Up kube-prometheus-stack, How to Back Up and Restore Kubernetes with Velero and How to Fix Kubernetes Pods Stuck in Pending.