Observability vs Monitoring: What's the Difference in 2026?
Every few months someone asks me whether they need "observability" or whether their Prometheus and Grafana setup, which pages them when the error rate climbs, is already that. It is a fair question, because vendors have spent a decade using the two words interchangeably, mostly to sell the more expensive one. The short answer is that monitoring vs observability is not a choice between two products. Monitoring tells you that something you predicted might break has broken. Observability is the property of a system that lets you work out why it broke when you did not predict it. This post goes through the definitions from the people who coined them, the "three pillars vs wide events" argument, where OpenTelemetry fits, what Grafana's 2026 survey says teams actually run, and why the bill grows faster than the traffic.
What is monitoring?
The cleanest definition is still the one in Google's Site Reliability Engineering book, in the chapter by Rob Ewaschuk: monitoring is collecting, processing, aggregating and displaying real-time numbers about a system, such as request counts, error counts, latencies and server lifetimes. The same chapter splits it into white-box monitoring, which reads metrics the system exposes about its own internals, and black-box monitoring, which tests externally visible behaviour the way a user would.
It also gave us the four golden signals that most dashboards still copy: latency, traffic, errors and saturation. If you only measure four things about a user-facing service, the book says, measure those.
The important word in all of this is aggregating. A monitoring system decides in advance what to count, rolls it up at write time into a time series, and compares it to a threshold. That is cheap, fast and very good at answering questions you already knew to ask: is the error rate above 1 percent, is the disk 90 percent full, is p99 latency over 500 ms. It is bad at questions you did not think of, because the detail you would need was thrown away during aggregation.
None of that is a criticism. Alerting on symptoms with well-chosen thresholds and SLOs is the backbone of on-call, and nothing in the observability movement replaces it.
What is observability, and where did the word come from?
The term comes from control theory, where a system is observable if you can infer its internal state from its outputs. Software borrowed it around 2016, which Charity Majors of Honeycomb, who did more than anyone to popularise it, dates explicitly.
The OpenTelemetry project's observability primer puts it this way: observability lets you understand a system from the outside by asking questions about it without knowing its inner workings, and it helps you troubleshoot novel problems, the "unknown unknowns". Telemetry, in the same primer, is simply the data a system emits about its behaviour.
So the distinction is about questions, not tools:
- Monitoring answers "is it broken?" for failure modes you anticipated.
- Observability answers "why is it broken, and for whom?" for failure modes you did not.
A concrete example. Your latency SLO alert fires. Monitoring got you that far. Now you need to know that the slow requests are all from one tenant, on one API version, hitting one shard, after a deploy 40 minutes ago, and only when a particular feature flag is on. If your telemetry kept those attributes on each request, you can slice by them and find it in minutes. If it only kept an aggregated http_request_duration_seconds histogram with three labels, you cannot, and you are back to guessing and grepping.
That ability to slice by arbitrary, high-cardinality fields (user ID, tenant, build SHA, request ID) is the practical heart of observability. Everything else is packaging.
Observability vs monitoring: the differences that matter
| Monitoring | Observability | Source | |
|---|---|---|---|
| Core question | Is a known thing broken? | Why is it broken, for whom? | OTel primer |
| When data is shaped | At write time (pre-aggregated) | Often at read time (raw events kept) | Majors, 2024 |
| Typical signals | Metrics, health checks, synthetic probes | Traces, structured events, logs, metrics, profiles | SRE book |
| Cardinality | Kept low on purpose | High-cardinality fields are the point | Majors, 2024 |
| Main output | Alerts and dashboards | Ad hoc queries, trace views, correlations | |
| Cost driver | Number of time series | Volume of events, retention, duplication | Honeycomb |
| Fails when | The failure was not predicted | Instrumentation is missing or sampled away |
The table makes it look like two camps. In practice it is a stack: you alert with monitoring, then investigate with observability, and the same telemetry pipeline often feeds both.
Three pillars vs wide events: is "observability 2.0" real?
The phrase "three pillars of observability" (metrics, logs and traces) was popularised by Peter Bourgon around 2018, and most commercial products were built around it: one store for metrics, one for logs, one for traces, a UI that tries to stitch them together. Ben Sigelman and others have argued for years that these are just three kinds of telemetry, not three pillars of anything.
In an August 2024 post, Majors proposed calling the pillar model "observability 1.0" and the alternative "2.0". Her version 2.0 stores each unit of work as one wide, structured event with potentially hundreds of fields, keeps it raw, and computes metrics, traces and log views from that single source at query time. Her argument is that the 1.0 model forces you to choose your questions before the incident and makes you pay to store the same request several times in different formats.
Is it real? Partly. The direction is visible: columnar stores built on ClickHouse and similar engines have made "store raw events, aggregate later" affordable for many teams, and OpenTelemetry traces are, structurally, wide events with parent links. But it is not either/or. Prometheus-style metrics remain the cheapest way to alert on a known SLO, and nobody sensible is going to compute a 30-day error budget by scanning every raw event. My own position is boring: keep metrics for alerting, make your spans and logs wide and structured, and correlate them by trace ID. If you want the mechanics of how spans link up, the companion piece on distributed tracing goes through context propagation and sampling.
Here is what "wide" means in practice: one event per request, with the context you will want during an incident attached at the source.
{
"timestamp": "2026-10-20T09:14:03.512Z",
"service.name": "checkout-api",
"service.version": "2026.10.19-3f2c1a9",
"trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
"http.request.method": "POST",
"http.route": "/v2/orders",
"http.response.status_code": 502,
"duration_ms": 1840,
"tenant.id": "acme-corp",
"feature_flag.new_pricing": true,
"db.shard": "orders-07",
"k8s.pod.name": "checkout-api-7d9f8b6c4-x2kqp",
"k8s.node.name": "ip-10-0-42-17",
"error.type": "upstream_timeout"
}
Every field after duration_ms is something a pre-aggregated metric would usually drop because of cardinality. Every one of them is something I would want at 2 a.m.
Where does OpenTelemetry fit?
OpenTelemetry (OTel) is not an observability backend and does not store anything. It is a vendor-neutral standard and toolkit for producing and moving telemetry: APIs and SDKs for instrumenting code, a wire protocol (OTLP), semantic conventions so that http.route means the same thing everywhere, and the Collector, a pipeline process that receives, processes and exports data to whatever backend you choose.
It is now the default. The CNCF graduated OpenTelemetry on 21 May 2026, citing more than 12,000 contributors from over 2,800 companies and second place in project velocity among 240+ CNCF projects, behind Kubernetes. Graduation does not mean finished: the Collector is still versioned 0.x, with v0.163.1 released on 16 October 2026, and component names are still changing: the Helm chart's upgrade notes record the otlp exporter becoming otlp_grpc (Collector v0.144.0, January 2026), filelog becoming file_log, and k8sattributes becoming k8s_attributes. I wrote about what is and isn't stable, and what eBPF instrumentation adds, in Observability in 2026.
Why it matters for the monitoring vs observability question: OTel lets you instrument once and decide later where the data goes and how much of it you keep. That decoupling is what makes it possible to send cheap aggregated metrics to Prometheus for alerting and rich traces to a tracing backend for debugging, from the same instrumentation. If you are on Kubernetes, the practical starting point is a Collector agent on every node plus a gateway, which I walk through step by step in How to Set Up the OpenTelemetry Collector on Kubernetes.
What the 2026 Grafana survey says teams actually run
Grafana Labs' fourth annual Observability Survey, published 18 March 2026, collected 1,363 responses between 1 October 2025 and 6 January 2026. Grafana sells observability tooling, so read it as a large vendor survey, not a census. A few numbers are useful for this question.
Three things stand out. First, Prometheus and OpenTelemetry are not rivals in practice: 77 percent invest in Prometheus, 76 percent in OTel, and 65 percent in both. Monitoring-style metrics and observability-style traces live side by side. Second, OTel is used about as much for metrics (57 percent) as for traces (50 percent) and logs (48 percent), which tells you it has become plumbing, not a tracing niche. Third, the top complaints are not about missing data: 38 percent named complexity and overhead as their biggest concern, 34 percent signal-to-noise, 31 percent cost, and 30 percent said alert fatigue is the biggest obstacle to faster incident response.
The previous year's survey (1,255 respondents) adds context the 2026 release does not repeat: respondents used eight observability tools on average, 95 percent used metrics but only 57 percent used traces, and LLM observability was in production at just 7 percent of organisations.
Why is observability so expensive?
Because of duplication and cardinality, mostly. Majors' cost-crisis essay argues that storing the same request as a log line, a metric, a span, an APM record and a RUM event means paying for it five times, a multiplier she calls modest, and that bills can grow three to ten times faster than traffic. That is an argument from a vendor with a competing architecture, but the arithmetic is easy to check against your own bill.
The surveys back up the pain. In Grafana's 2025 survey, the 294 people who answered an optional question put observability at an average of 17 percent of infrastructure spend, with a median of 10 percent. In 2026, 65 percent said cost is the most important criterion when choosing tools, the third year running at the top, and 50 percent expect to spend more next year.
The levers are the same whichever side of the debate you are on:
- Stop storing things twice. If a request produces a span, you probably do not also need an INFO log line saying the same thing.
- Sample traces deliberately. Keep every error and slow request, sample the boring successes.
- Control metric cardinality. Never put user IDs or request IDs in Prometheus labels. Put them on spans and events instead.
- Filter at the Collector. Drop health-check spans and debug logs before they leave the cluster, not after you have paid to ingest them.
- Shorten retention. Most debugging happens within 72 hours. Thirty days of full-fidelity traces is rarely worth it.
Do you need observability, or is monitoring enough?
For a small team running a handful of services with predictable traffic, good monitoring gets you most of the way: RED or golden-signal dashboards, SLO-based alerts, and structured logs you can search. You will feel the gap the first time an incident involves a dependency chain you cannot see, or a problem affecting one customer out of thousands. That is usually when tracing pays for itself.
The case strengthens as systems get less predictable, and the AI era has made them a lot less predictable. An agent that decides at runtime which tools to call, or an LLM feature whose output varies with the prompt, is close to the definition of unknown unknowns: you cannot write threshold alerts for failure modes that depend on what a model decided. The OpenTelemetry GenAI conventions and what they do and don't capture are covered in my piece on agent observability, and the SLO side in SRE for LLM applications. The discipline does not change: instrument at the source, propagate context, alert on symptoms users feel, and investigate with data that kept its detail.
FAQ
What is the main difference between monitoring and observability?
Monitoring tracks predefined metrics and alerts when they cross thresholds, so it answers whether a known failure is happening. Observability is the ability to ask new questions of your telemetry to explain failures you did not anticipate. You need monitoring to know something is wrong and observability to find out why.
What are the three pillars of observability?
Metrics, logs and traces. The framing is useful for describing telemetry types but criticised for encouraging separate, siloed tools. Many practitioners now prefer wide structured events correlated by trace ID, with profiles increasingly treated as a fourth signal.
Is Prometheus monitoring or observability?
Prometheus is primarily a metrics monitoring system: it scrapes and stores time series and evaluates alerting rules. It is a core part of most observability stacks, but on its own it deliberately avoids high-cardinality data, so you pair it with tracing and logs for deep investigation.
Does OpenTelemetry replace monitoring tools?
No. OpenTelemetry standardises how telemetry is generated and transported; it does not store, visualise or alert. You still need backends such as Prometheus, Grafana, Jaeger, or a commercial vendor. Its value is that you can switch or combine those backends without re-instrumenting your code.
This quarter, if I were starting from a monitoring-only setup, I would not buy a new platform. I would put an OpenTelemetry Collector in front of whatever we already run, add tracing to the two or three services that cause the most incidents, make sure every log line carries a trace ID, and set a hard budget for retention before the first invoice arrives. Then I would delete one dashboard nobody has opened in six months, which is the most reliable observability improvement I know.
Related: Observability in 2026: OpenTelemetry Graduated, the Collector Is Still v0.162, eBPF Instrumentation Is v0.14, and Datadog Bills $1.12B a Quarter, Agent Observability in 2026: The Spans Are Standard, the Standard Isn't Stable, and Nobody Reads the Logs and DevOps Still Matters in 2026: AI Cut Delivery Stability 7.2%, Then Doubled Merged PRs and Added 91% to Review Time.
I'm Ajay Kumar — I build and operate PandaStack, an open-source Firecracker microVM cloud for AI agents, and write about the infrastructure and tooling around it.
Need this kind of infrastructure work? See what I do or email hello@ajayk.sh.
Related
How to Run Chaos Experiments on Kubernetes with Chaos Mesh (2026)
Step-by-step Chaos Mesh 2.8.5 guide: install with Helm on containerd, opt namespaces in, write PodChaos, NetworkChaos and StressChaos YAML, gate experiments on a Prometheus SLO, schedule them, and run the same test with a LitmusChaos promProbe, plus a troubleshooting table.
16 minOct 20, 2026How to Fix Kubernetes CrashLoopBackOff: Causes and Commands (2026)
Fix Kubernetes CrashLoopBackOff step by step: what the state means, the 10s to 5 minute backoff and the new per-node max in 1.35+, kubectl describe and logs --previous, exit codes 1, 127, 137 and 143, probe and OOM causes, and kubectl debug for crashing containers.
13 minOct 20, 2026How to Set Up Grafana Loki on Kubernetes with Alloy (2026)
Step-by-step Grafana Loki 3.7 setup on Kubernetes: the new grafana-community Helm chart, deployment modes, S3 storage, Grafana Alloy replacing end-of-life Promtail, labels vs structured metadata, LogQL, retention, and fixes for 429 and too many outstanding requests.
12 min