Continuous Profiling in 2026: OTel Profiles Alpha, eBPF Agents, Real Costs

Every infrastructure team I have worked with has a story about a service that was "just expensive" until someone looked at a flame graph and found the cost sitting in serialization, logging or a regex nobody remembered writing. That is the pitch for continuous profiling in one sentence: an always-on, low-rate sampling profiler across production that tells you which functions burn your CPU and memory, so you stop optimising by folklore. This post explains where continuous profiling stands in 2026: what the OpenTelemetry Profiles signal is and why "alpha" still matters, what Elastic's donated eBPF profiler does, how Grafana Pyroscope, Parca, Polar Signals and Datadog differ, what the overhead really is according to the people who publish numbers, and how a profile turns into a smaller cloud bill.

What is continuous profiling, and how is it different from tracing?

Metrics tell you a service is slow or expensive. Traces tell you which request and which span. Profiles tell you which line of code. A profiler samples stack traces at a fixed frequency (Parca Agent and Pyroscope's eBPF collector both default to 19 samples per second), aggregates them, and renders the result as a flame graph where width is cost. Run it continuously and you can compare last Tuesday's CPU profile with today's without reproducing anything.

The idea is not new. OpenTelemetry's own announcement points out the technique has been used for decades, and Google described its fleet-wide system in the 2010 IEEE Micro paper Google-Wide Profiling, claiming "negligible overhead" while producing stable profiles at datacenter scale. What changed is that the tooling stopped being something only hyperscalers built in-house. Two things did that: eBPF, which lets one agent per node profile every process without touching the application, and a vendor-neutral data format so that agent can talk to any backend.

For profiling, the key eBPF property (background in eBPF explained) is that one agent walks stacks from the kernel through native libraries into Python, Java or Node frames without loading anything into the process.

What is the status of the OpenTelemetry Profiles signal in 2026?

The Profiling SIG announced that Profiles had entered public alpha in March 2026. As of October it is still alpha: a ClickHouse write-up dated 29 July 2026 still lists profiles as alpha next to stable traces, metrics and logs, and I found no beta announcement. What the alpha consists of, per the SIG's post:

• A data model, profiles.proto, shipped in opentelemetry-proto v1.10.0 under the v1development package, with deduplicated stacks, dictionary tables, timestamps and links to trace and span IDs. (The proto has since moved to v1.11.1, released 29 September 2026.)
• Collector support from v0.148.0, including a pprof receiver, a lossless pprof translator, and the k8sattributes processor and OTTL working on profiles.
• A conformance checker, profcheck.
• The eBPF profiler running as a Collector receiver and shipped as an official distribution.

The same post is blunt about the caveats. The signal "should not be used for critical production workloads", no production-ready backends had emerged at the time, and symbolization (turning addresses into function names, especially for stripped native binaries) is still being standardised.

My reading: alpha here means the wire format can break between Collector releases. It does not mean the profiler itself is a toy. The eBPF agent has years of production use behind it under a different name, which brings us to Elastic.

What did Elastic donate, and why does it matter?

On 7 June 2024 OpenTelemetry accepted Elastic's donation of the eBPF agent behind its Universal Profiling product, with maintainers from Grafana Labs, Datadog, Red Hat and Elastic. That is the code now living at opentelemetry-ebpf-profiler. Its README sets the overhead budget plainly: "1% CPU and 250MB memory are our upper limits in testing", with the agent typically well below that. It unwinds native code using .eh_frame data, so it works on binaries without frame pointers or debug symbols on the host, and it supports HotSpot JVM, Python, Ruby, PHP, Node.js, V8, Perl, Erlang and .NET, on x86_64 and ARM64.

The strategic effect is bigger than the code. Once the best open eBPF unwinder became shared infrastructure, vendors stopped competing on the agent and started building on it. Grafana's pyroscope.ebpf component in Alloy embeds a fork of it. Datadog's Full Host Profiler, in preview, is built on the OTel eBPF profiler. Parca Agent depends on it too, through a Parca-maintained fork listed in its go.mod, and Polar Signals engineers are on the alpha's contributor list. The agent is becoming a commodity; the backend is the real decision.

Pyroscope vs Parca vs Datadog: which continuous profiler should you use?

Here is the field as of 20 October 2026, with versions taken from each project's release page.

Tool	What it is	Latest release	How it collects	License / model	Source
OTel eBPF profiler	Whole-system profiler, runs as a Collector receiver	otelcol-ebpf-profiler v0.163.1 (16 Oct 2026)	eBPF, Linux only, privileged	Apache-2.0	releases, README
Grafana Pyroscope	Profile database and UI	v2.4.0 (9 Oct 2026)	Language SDKs, Alloy pyroscope.ebpf, OTLP from the eBPF profiler	AGPL-3.0, or Grafana Cloud	releases, OTLP docs
Parca + Parca Agent	Profile database and eBPF agent	Parca v0.29.1 (28 Sep 2026), Agent v0.50.0 (21 Sep 2026)	eBPF at 19 Hz, kernel 5.3+ with BTF	Apache-2.0	parca, agent
Polar Signals Cloud	Hosted service from the Parca team	SaaS	Parca Agent	Commercial	blog
Datadog Continuous Profiler	In-process profilers plus Full Host Profiler (preview)	SaaS	Libraries for Go, Java, Node.js, PHP, Python, Ruby, .NET, Rust, C, C++; eBPF host profiler	Commercial	docs, OTel post
Elastic Universal Profiling	Origin of the donated agent	SaaS or self-managed	eBPF	Commercial	OTel blog

Pyroscope 2.0, announced on 21 April 2026, is the most significant backend change of the year. It drops the Cortex-style ingesters and writes profiles straight to object storage through new segment-writer, metastore, query-backend and compaction-worker components, the same direction Mimir and Loki took. The v2.0 release notes list breaking changes worth reading before you upgrade: the Go module path moved to /v2, the default write path is now the segment writer, and the default storage backend is filesystem. Grafana says Grafana Cloud Profiles ran v2 for a year before the open-source release and processed 19.5 PB of profiling data.

How I would choose. If you already run Grafana, Loki and Mimir, Pyroscope is the obvious fit, and Alloy can collect logs and profiles from the same DaemonSet (the logs half is in my Loki on Kubernetes guide). If you want a permissive licence and a small, focused project, Parca. If you already pay Datadog and your services use its tracers, the in-process profiler gives you code hotspots linked to traces with no new infrastructure, and the bill is the trade-off.

How much overhead does continuous profiling add?

This is where vendor pages get vague, so here is only what is published with a number attached:

• OTel eBPF profiler: 1 percent CPU and 250 MB memory as upper limits in testing, per its README.
• Grafana: a February 2026 post puts CPU and latency overhead in "the low single-digit percentages" for eBPF collection via Alloy, and SDK profilers at usually under 50 MB per pod (Grafana blog). It gives no exact percentage.
• Datadog: the profiler docs promise "minimal impact" on CPU and memory with no figure (docs).
• Meta: the Strobelight post describes budgeting by samples, for example 40,000 CPU-cycle samples per hour across 1,000 hosts for one service (Meta Engineering), which is a useful way to think about cost: you choose a sample budget, not a percentage.

Two practical notes. eBPF agents cost per node, not per pod, so on dense nodes the overhead per workload is small; on a fleet of tiny nodes it is proportionally larger. And the bigger cost is usually storage and egress for the profiles, not CPU on the host. Pyroscope 2.0's object-storage design exists because profile data is bulky.

Here is the shape of a minimal OTel-native pipeline, adapted from the example in Grafana's Pyroscope repository. Treat it as alpha configuration that may change between Collector releases.

# otelcol-ebpf-profiler 0.163.1, run as a privileged DaemonSet with hostPID: true
# Start with: --feature-gates=+service.profilesSupport (profiles are alpha)
receivers:
  profiling:
    samples_per_second: 19          # default is 20; lower = cheaper

processors:
  k8sattributes/profiles:            # needs a ServiceAccount that can read pods
    auth_type: serviceAccount
    extract:
      metadata: [k8s.namespace.name, k8s.pod.name, k8s.deployment.name, k8s.node.name]
    pod_association:
      - sources:
          - from: resource_attribute
            name: container.id
  resource/cluster:
    attributes:
      - action: upsert
        key: k8s.cluster.name
        value: prod-eu-1

exporters:
  otlp_grpc:                         # 'otlp' is the deprecated alias
    endpoint: pyroscope.observability.svc:4040
    tls:
      insecure: true                 # in-cluster only; use TLS across networks

service:
  pipelines:
    profiles:
      receivers: [profiling]
      processors: [k8sattributes/profiles, resource/cluster]
      exporters: [otlp_grpc]

The Pyroscope docs add one wrinkle: the eBPF profiler does not set service_name, so you need an ingestion relabel rule on the Pyroscope side or every flame graph is labelled by executable name.

How does continuous profiling reduce cloud costs?

Profiles find waste that metrics cannot attribute. The published wins all look alike: one hot function, one small change, a large drop in CPU.

Case	What the profile showed	Change	Result	Source
Meta, ads service	Unintended std::vector copy from C++ auto	Add & (one character)	Estimated 15,000 servers of capacity per year	Meta Engineering
S2 (streaming API)	SHA-256 checksums at 68.37% of CPU	Enable hardware acceleration on Graviton	31.82% of CPU, at least 2x throughput	Polar Signals
Coralogix pipeline service	CPU spikes in the io.circe JSON library	Swap JSON library	About 50% less CPU and cloud spend for that service	Coralogix

Be honest about what these are: single services, chosen because they were dramatic, reported by the company that benefited or the vendor that sold the tool. I found no independent fleet-wide study of average savings. The pattern is still real: in a codebase that has never been profiled, the hot spots tend to be in boring places like serialization, logging, regex and compression, and they are cheap to fix once someone can see them.

There is also a compounding benefit that needs no human at all. Go's profile-guided optimisation takes a production CPU profile as default.pgo and, per the Go docs, improved a representative set of programs by around 2 to 14 percent as of Go 1.22. Continuous profiling gives you a fresh, representative profile on demand. That is the cheapest performance win in the Go toolchain and most teams never wire it up.

For FinOps, the useful move is attributing CPU to functions and teams, then putting that next to the bill. I wrote about how AI spend broke cost attribution in FinOps for AI; profiles are the same discipline one layer down, and they matter more when agents are generating code nobody has read closely. Code that was cheap to write is not necessarily cheap to run.

Should you adopt OpenTelemetry Profiles now?

Adopt continuous profiling now. Adopt the OTel wire format carefully. Concretely:

• If you need profiling in production this quarter, use a mature path: Pyroscope SDKs or Alloy's pyroscope.ebpf, Parca Agent to Parca, or your APM vendor's profiler.
• Run the OTel eBPF profiler in staging or a non-critical cluster, pin the Collector version, and expect to re-test config on upgrades. The otlp exporter's rename to otlp_grpc is the kind of churn to expect.
• Keep profile labels aligned with your trace and log labels (k8s.namespace.name, service.name, cluster). Correlation is the point of a fourth signal, and it only works if the labels match. My earlier piece on OpenTelemetry and eBPF observability costs covers the same labelling discipline for the other three signals.

FAQ

Is OpenTelemetry Profiles production ready?

Not officially. The signal entered public alpha in March 2026 and the Profiling SIG says it should not be used for critical production workloads yet. The eBPF profiler underneath is mature code from Elastic, but the protocol and Collector configuration can still change between releases.

What is the overhead of eBPF continuous profiling?

The OpenTelemetry eBPF profiler targets at most 1 percent CPU and 250 MB of memory per host in testing, and usually stays below that. Grafana reports low single-digit percentage CPU and latency impact for eBPF collection. Overhead scales with sample rate, so lowering from 19 or 20 Hz reduces it further.

Pyroscope or Parca: what is the difference?

Both are open-source profile databases with eBPF collection. Pyroscope is AGPL-3.0, integrates tightly with Grafana and Alloy, and moved to an object-storage-first architecture in 2.0. Parca is Apache-2.0, smaller in scope, and backed commercially by Polar Signals.

Does continuous profiling replace distributed tracing?

No. Tracing shows where time goes across services for a request; profiling shows which functions consume CPU and memory inside a process. They work best linked, which is why the OTel profiles format carries trace and span IDs.

This quarter I would put a profiler on the three most expensive services by compute spend, using whichever backend matches the existing stack, and spend one afternoon per service reading flame graphs with the owning team. I would also run the OTel eBPF profiler on a staging cluster, pinned, so that when the signal reaches beta the migration is a config change rather than a project. For PandaStack (my company, a Firecracker microVM cloud), whole-host eBPF profiling is the only kind that makes sense, because the interesting costs sit in the VMM and the kernel, not in any one language runtime. Profiling is unglamorous, but it is the only observability signal that regularly pays for the rest.

Related: Observability in 2026: OpenTelemetry Graduated, the Collector Is Still v0.162, eBPF Instrumentation Is v0.14, and Datadog Bills $1.12B a Quarter, FinOps for AI in 2026 and Agent Observability in 2026.