How to Set Up the OpenTelemetry Collector on Kubernetes (2026)

This guide is for platform and DevOps engineers who want a production-shaped OpenTelemetry Collector setup on Kubernetes rather than the single-pod demo. At the end you will have an agent Collector on every node that enriches telemetry with Kubernetes metadata, a horizontally scaled gateway that batches, queues and exports to your backend, optional zero-code auto-instrumentation through the OpenTelemetry Operator, and the alerts that tell you when spans are being dropped. Everything here was checked against the current releases: the opentelemetry-collector Helm chart 0.176.0 (Collector 0.162.0), the opentelemetry-operator chart 0.126.1 (Operator v0.161.0, which supports Kubernetes 1.25 to 1.37 per its compatibility matrix), and cert-manager v1.21.2. I rendered both values files with helm template and ran the resulting Collector configs through otelcol-k8s validate (the agent config needs a service-account CA file to pass, since kubeletstats reads it at startup). You need kubectl, Helm 3, cluster-admin on a test cluster, and an OTLP-compatible backend endpoint.

Agent vs gateway: which deployment pattern do you need?

The Collector can run in three shapes on Kubernetes, and most production setups use two of them together.

• Agent (DaemonSet). One Collector per node. Applications send to the Collector on their own node, so traffic stays local. This is where you read container logs from /var/log/pods, scrape the kubelet, and attach pod metadata, because the agent can see which pod a connection came from.
• Gateway (Deployment). A few replicas behind a Service. Agents forward to it. This is where you do cluster-wide work: tail sampling, heavy filtering, credentials for the backend, and the sending queue that absorbs backend slowness.
• Sidecar. One Collector per pod, injected by the Operator. Useful for strict tenant isolation, wasteful otherwise.

If you run a small cluster and one backend, you can start with agents only and add the gateway later. I would still add it before production, because it is the single place to change backends and credentials without touching every node.

Step 1: Choose the Helm chart, the Operator, or both

There are two official ways to deploy the Collector, and they are not mutually exclusive.

The opentelemetry-collector Helm chart deploys one Collector per release (DaemonSet, Deployment or StatefulSet) with presets that wire up the RBAC, volumes and environment variables for common Kubernetes components. It is the simplest path and what this guide uses for the Collectors.

The OpenTelemetry Operator adds two CRDs: OpenTelemetryCollector (opentelemetry.io/v1beta1) to manage Collectors declaratively, and Instrumentation (opentelemetry.io/v1alpha1) to inject auto-instrumentation into pods. You need the Operator for auto-instrumentation regardless of how you deploy Collectors. There is also an opentelemetry-kube-stack chart (0.25.0 at the time of writing) that bundles the Operator with preconfigured collectors, if you prefer one opinionated install.

Add the chart repository:

helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update
helm search repo open-telemetry --versions | head -n 5

One thing has changed in 2026 that trips up old blog posts and ChatGPT answers: several components were renamed. The chart's upgrade notes document otlp and otlphttp exporters becoming otlp_grpc and otlp_http (Collector v0.144.0 or newer), filelog becoming file_log (v0.149.0), and the k8sattributes processor becoming k8s_attributes. The chart rewrites old names automatically while rewriteDeprecatedComponentNames: true (the default), but write the new names in new config.

Step 2: Create the namespace and the backend secret

Keep the backend credential in a Secret and reference it from the gateway only. Agents never need it.

kubectl create namespace observability

# Replace with your backend's token or API key
kubectl -n observability create secret generic otel-backend \
  --from-literal=token='REPLACE_ME'

Step 3: Deploy the gateway Collector as a Deployment

The chart requires you to choose an image. Use the otelcol-k8s distribution, which contains the Kubernetes components (k8s_attributes, kubeletstats, file_log, k8s_cluster) without the hundreds of extras in contrib. Save this as gateway-values.yaml:

# gateway-values.yaml for opentelemetry-collector chart 0.176.0
mode: deployment
replicaCount: 2

image:
  repository: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s
command:
  name: otelcol-k8s

# memory limit drives both GOMEMLIMIT (80%) and memory_limiter (80% / 25%)
resources:
  requests:
    cpu: 500m
    memory: 1Gi
  limits:
    memory: 2Gi

ports:
  jaeger-compact: { enabled: false }
  jaeger-thrift: { enabled: false }
  jaeger-grpc: { enabled: false }
  zipkin: { enabled: false }
  metrics: { enabled: true }   # expose :8888 internal metrics

extraEnvs:
  - name: BACKEND_TOKEN
    valueFrom:
      secretKeyRef:
        name: otel-backend
        key: token

config:
  receivers:
    jaeger: null     # remove chart defaults we do not use
    zipkin: null
  exporters:
    otlp_http:
      endpoint: https://otlp.example-backend.com   # your backend
      headers:
        Authorization: "Bearer ${env:BACKEND_TOKEN}"
      sending_queue:
        enabled: true
        queue_size: 5000        # batches, default sizer is requests
      retry_on_failure:
        enabled: true
        max_elapsed_time: 300s
  service:
    pipelines:
      traces:
        receivers: [otlp]
        exporters: [otlp_http, debug]   # drop debug after Step 5
      metrics:
        exporters: [otlp_http]
      logs:
        exporters: [otlp_http]

Install it:

helm install otel-gateway open-telemetry/opentelemetry-collector \
  --version 0.176.0 -n observability -f gateway-values.yaml

kubectl -n observability rollout status deploy/otel-gateway-opentelemetry-collector

The chart already puts memory_limiter and batch in every pipeline and, because useGOMEMLIMIT defaults to true, sets GOMEMLIMIT to 80 percent of the memory limit. With a 2Gi limit the rendered Deployment gets GOMEMLIMIT=1638MiB. The memory_limiter README recommends exactly that pairing. If you remove resources.limits.memory, both protections silently stop working, so do not.

Step 4: Deploy the agent Collector as a DaemonSet with k8s_attributes

The agent receives OTLP from pods on the same node through a hostPort, reads container logs, scrapes kubelet stats, adds pod metadata, and forwards everything to the gateway. Save as agent-values.yaml:

# agent-values.yaml for opentelemetry-collector chart 0.176.0
mode: daemonset

image:
  repository: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s
command:
  name: otelcol-k8s

presets:
  kubernetesAttributes:
    enabled: true     # adds k8s_attributes + RBAC, filtered to this node
  kubeletMetrics:
    enabled: true     # kubeletstats receiver + nodes/stats RBAC
  logsCollection:
    enabled: true     # file_log receiver on /var/log/pods

resources:
  requests:
    cpu: 100m
    memory: 256Mi
  limits:
    memory: 512Mi

ports:
  jaeger-compact: { enabled: false }
  jaeger-thrift: { enabled: false }
  jaeger-grpc: { enabled: false }
  zipkin: { enabled: false }

config:
  receivers:
    jaeger: null
    zipkin: null
  exporters:
    otlp_grpc:
      endpoint: otel-gateway-opentelemetry-collector.observability.svc.cluster.local:4317
      tls:
        insecure: true   # in-cluster hop; use mTLS or a mesh if required
  service:
    pipelines:
      traces:
        receivers: [otlp]
        exporters: [otlp_grpc]
      metrics:
        exporters: [otlp_grpc]
      logs:
        exporters: [otlp_grpc]

helm install otel-agent open-telemetry/opentelemetry-collector \
  --version 0.176.0 -n observability -f agent-values.yaml

kubectl -n observability rollout status ds/otel-agent-opentelemetry-collector-agent

It is worth seeing what the kubernetesAttributes preset generates, because this is the part people get wrong by hand. Rendered with helm template, the processor is filtered to the agent's own node (via a K8S_NODE_NAME env var from the downward API), extracts pod, namespace, workload, container and service.* attributes, and associates incoming data with a pod by k8s.pod.ip, then k8s.pod.uid, then the connection's source IP:

# excerpt of the rendered ConfigMap (helm template, chart 0.176.0)
k8s_attributes:
  filter:
    node_from_env_var: K8S_NODE_NAME
  passthrough: false
  extract:
    metadata: [k8s.namespace.name, k8s.pod.name, k8s.pod.uid, k8s.node.name,
               k8s.deployment.name, k8s.statefulset.name, k8s.daemonset.name,
               k8s.container.name, container.image.name, container.image.tag,
               service.name, service.version, service.instance.id]  # trimmed
    otel_annotations: true
  pod_association:
    - sources: [{ from: resource_attribute, name: k8s.pod.ip }]
    - sources: [{ from: resource_attribute, name: k8s.pod.uid }]
    - sources: [{ from: connection }]

The node filter matters: the k8s_attributes README warns that without it every agent watches every pod in the cluster, which costs memory on large clusters. Two other details from the render. The preset prepends k8s_attributes before memory_limiter in every pipeline, even if you list processors yourself, which is fine in practice but differs from the README's advice to put the limiter first. And if you ever enrich at the gateway instead, set the agent to passthrough: true so it only stamps the pod IP and the gateway does the lookups.

Step 5: Verify the pipeline end to end

Send synthetic traces with telemetrygen from inside the cluster, aimed at the gateway Service:

kubectl -n observability run telemetrygen --rm -it --restart=Never \
  --image=ghcr.io/open-telemetry/opentelemetry-collector-contrib/telemetrygen:v0.163.0 -- \
  traces --otlp-endpoint otel-gateway-opentelemetry-collector:4317 \
  --otlp-insecure --traces 5

kubectl -n observability logs deploy/otel-gateway-opentelemetry-collector --tail=20 | grep Traces

With the debug exporter in the traces pipeline you should see one line per batch. This is the real output from running the same Collector image (otelcol-k8s 0.162.0) locally with the same telemetrygen command:

info  Traces  {"otelcol.component.id": "debug", "otelcol.component.kind": "exporter", "otelcol.signal": "traces", "resource spans": 1, "spans": 2}
info  Traces  {"otelcol.component.id": "debug", "otelcol.component.kind": "exporter", "otelcol.signal": "traces", "resource spans": 1, "spans": 2}

Then check the agents are healthy and the backend is receiving data. Once it is, remove debug from the gateway pipeline and helm upgrade so you are not logging every batch forever.

Step 6: Add auto-instrumentation with the Operator and the Instrumentation CR

The Operator's admission webhook needs a TLS certificate; cert-manager is the default way to get one. Install cert-manager with the command from its Helm install docs, then the Operator:

helm install cert-manager oci://quay.io/jetstack/charts/cert-manager \
  --version v1.21.2 --namespace cert-manager --create-namespace \
  --set crds.enabled=true

helm install opentelemetry-operator open-telemetry/opentelemetry-operator \
  --version 0.126.1 -n opentelemetry-operator-system --create-namespace \
  --set "manager.collectorImage.repository=ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s"

If you cannot run cert-manager, the operator chart README documents --set admissionWebhooks.certManager.enabled=false --set admissionWebhooks.autoGenerateCert.enabled=true to have Helm generate a self-signed certificate.

Now create an Instrumentation resource in the application namespace. Point it at the agent on the pod's own node: the Operator injects OTEL_NODE_IP into instrumented containers for exactly this, according to the auto-instrumentation docs. Use port 4318: the Operator docs note that the Python, .NET and Go agents default to OTLP over HTTP, and the Java agent has done the same since v2.0.0.

# instrumentation.yaml
apiVersion: opentelemetry.io/v1alpha1
kind: Instrumentation
metadata:
  name: default
  namespace: shop
spec:
  exporter:
    endpoint: http://$(OTEL_NODE_IP):4318
  propagators:
    - tracecontext
    - baggage
  sampler:
    type: parentbased_traceidratio
    argument: "0.25"     # keep 25% of new traces; parents decide for children

kubectl apply -f instrumentation.yaml
kubectl -n shop get otelinst

# opt a Java deployment in; the annotation goes on the pod template
kubectl -n shop patch deployment checkout --type merge -p \
  '{"spec":{"template":{"metadata":{"annotations":{"instrumentation.opentelemetry.io/inject-java":"true"}}}}}'

The annotation value can be "true" (use the namespace's Instrumentation), a name, or namespace/name. Other languages use inject-python, inject-nodejs, inject-dotnet and inject-go; Python on Alpine images also needs instrumentation.opentelemetry.io/otel-python-platform: "musl". Injection happens at pod creation, so the patch triggers a rollout; existing pods are never modified in place.

If you would rather manage Collectors through the Operator too, the gateway above translates directly into an OpenTelemetryCollector with apiVersion: opentelemetry.io/v1beta1, spec.mode: deployment and the same config block. To let the Operator create the ClusterRole that k8s_attributes needs in that case, install it with --set manager.createRbacPermissions=true.

Step 7: Monitor the Collector so you know when it drops data

A Collector that silently drops spans is worse than none, because you trust the gaps. The internal telemetry docs list the metrics to alert on. Through the default Prometheus endpoint on port 8888, counters get a _total suffix:

# PrometheusRule snippet (prometheus-operator); thresholds are starting points
groups:
  - name: otel-collector
    rules:
      - alert: OtelCollectorRefusingSpans        # memory_limiter is pushing back
        expr: sum by (pod) (rate(otelcol_receiver_refused_spans_total[5m])) > 0
        for: 10m
      - alert: OtelCollectorQueueFull            # data dropped before export
        expr: sum by (pod) (rate(otelcol_exporter_enqueue_failed_spans_total[5m])) > 0
        for: 5m
      - alert: OtelCollectorExportFailing        # backend errors, may still retry
        expr: sum by (pod) (rate(otelcol_exporter_send_failed_spans_total[5m])) > 0
        for: 15m
      - alert: OtelCollectorQueueNearCapacity
        expr: max by (pod) (otelcol_exporter_queue_size / otelcol_exporter_queue_capacity) > 0.8
        for: 10m

Add the same rules for _metric_points and _log_records variants. refused means data was rejected at the door (usually memory_limiter), enqueue_failed means it was accepted then dropped because the sending queue was full, and send_failed means the backend rejected or timed out, which is only data loss once retries run out after max_elapsed_time.

Troubleshooting

Symptom	Likely cause	Fix
Collector pod CrashLoopBackOff right after an upgrade	Config uses a removed or renamed component, or invalid YAML	kubectl logs --previous; validate locally with otelcol-k8s validate --config=...; use otlp_grpc, otlp_http, k8s_attributes, file_log
helm install fails: additional properties 'rewriteDeprecatedProcessorNames' not allowed	Value renamed in chart 0.162.0	Rename to rewriteDeprecatedComponentNames
otelcol_receiver_refused_spans_total rising, apps log export errors	memory_limiter soft limit exceeded	Raise resources.limits.memory, add gateway replicas, reduce check_interval; keep GOMEMLIMIT at about 80% of the limit
Log line Dropping data because sending_queue is full	Backend slower than ingest, or queue too small	Increase sending_queue.queue_size or num_consumers, scale the gateway, check backend rate limits
OOMKilled despite memory_limiter	No memory limit set, so limiter and GOMEMLIMIT are inactive, or large non-OTLP payloads	Set resources.limits.memory; the README warns data can consume memory before the limiter rejects it
Spans arrive without k8s.pod.name	Pod association failed: source IP is the node IP (SNAT, hostNetwork) or traffic came via the gateway	Keep enrichment on the agent; set k8s.pod.ip or k8s.pod.uid as a resource attribute in the SDK; use passthrough: true on agents if enriching at the gateway
k8s_attributes errors: pods is forbidden	Missing RBAC when not using the preset	Grant get/list/watch on pods, namespaces and replicasets per the processor README
kubeletstats: x509: certificate signed by unknown authority	Kubelet serving cert not signed by the cluster CA (common on managed clusters)	Set config.receivers.kubeletstats.insecure_skip_verify: true, or enable kubelet serving cert rotation
Apps cannot reach NODE_IP:4318	hostPort not supported by the CNI, or a NetworkPolicy blocks it	Enable the CNI's portmap/hostPort support, or send to an internal-traffic-policy Local Service instead
Pods not instrumented after adding the annotation	Annotation on the Deployment, not the pod template, or pods not recreated, or webhook unreachable	Put it under spec.template.metadata.annotations, roll the pods, check Operator logs; on private GKE open port 9443 to the webhook

FAQ

Should I run the OpenTelemetry Collector as a DaemonSet or a Deployment?

Both, in most clusters. Run a DaemonSet agent for anything node-local (container logs, kubelet metrics, pod metadata, receiving from local apps) and a Deployment gateway for cluster-wide processing and export. A single Deployment is acceptable for small clusters that only need traces.

Which Collector image should I use on Kubernetes?

Use the otelcol-k8s distribution from ghcr.io/open-telemetry/opentelemetry-collector-releases, which includes the Kubernetes receivers and processors and is what the Operator chart uses by default. Use otelcol-contrib only if you need a component k8s lacks, or build your own with the OpenTelemetry Collector Builder to keep the attack surface small.

Is the batch processor deprecated?

No. In Collector v0.163 the batch processor is still beta for traces, metrics and logs, and the Helm chart includes it by default. Exporters can also batch inside sending_queue with a batch block, which some teams now prefer because it batches after the queue, closer to the network call.

Do I need the Operator if I use the Helm chart?

Only for auto-instrumentation or if you want to manage Collectors as custom resources. The Helm chart alone is enough to run agents and gateways; the Instrumentation CR and its injection webhook come from the Operator.

When I set this up for a new cluster, the first thing I check is not the pipeline but the Collector's own metrics: refused, enqueue-failed and send-failed counters at zero, queue well below capacity, and memory limits actually set. Then I pick one noisy service and confirm its spans carry k8s.pod.name and service.version, because missing attributes are the failure nobody notices until an incident. For the why behind all this, read the companion explainer, Observability vs Monitoring: What's the Difference in 2026?, and for keeping the bill sane, Observability in 2026. The official references are the Collector on Kubernetes docs, the collector chart and the Operator repository.

Related: Observability in 2026: OpenTelemetry Graduated, the Collector Is Still v0.162, eBPF Instrumentation Is v0.14, and Datadog Bills $1.12B a Quarter, Kubernetes as the Agent Control Plane in 2026: Agent Sandbox Hit v1.0, 300 Claims a Second, and DRA in Every Supported Release and Agent Observability in 2026: The Spans Are Standard, the Standard Isn't Stable, and Nobody Reads the Logs.