How to Trace an LLM App with OpenTelemetry and Langfuse (2026)

This guide is for engineers with a Python LLM app or agent in front of users who want to see every model call, tool call, token and dollar without sending prompts to a SaaS. By the end you will have Langfuse v4 running on Docker Compose, a small tool-calling agent producing nested traces with cost, sessions grouped by conversation, user-feedback scores, and an OpenTelemetry Collector path for services that are not Python. Versions are current as of 24 October 2026: Langfuse server 4.69.0, Python SDK langfuse 5.0.4, openai 3.37.0, OpenTelemetry Python 1.45.1 and Collector 0.163.2. I checked every SDK call against the langfuse-python v5.0.4 source and ran the Compose configuration through docker compose config locally. You need Docker with Compose v2, Python 3.11 or newer, an OpenAI API key, and for anything beyond a laptop test a machine with 4 cores and 16 GiB of RAM. If you are still choosing a backend, read the companion LLM observability tools comparison first.

Step 1: Clone Langfuse and pin the version

Langfuse's official Compose file tracks the floating :4 image tag. That is convenient and also means docker compose pull can move you to a new minor version on a Tuesday afternoon. Clone the release tag and pin the two Langfuse images in an override file, which Compose merges automatically and which leaves the upstream file untouched for later upgrades.

git clone --depth 1 --branch v4.69.0 https://github.com/langfuse/langfuse.git
cd langfuse

# docker-compose.override.yml is merged into docker-compose.yml automatically
cat > docker-compose.override.yml <<'EOF'
services:
  langfuse-web:
    image: docker.io/langfuse/langfuse:4.69.0
  langfuse-worker:
    image: docker.io/langfuse/langfuse-worker:4.69.0
EOF

The stack is six containers: langfuse-web (UI and API on port 3000), langfuse-worker, Postgres 17, ClickHouse 25.12, Redis 7 and MinIO. Only ports 3000 and 9090 (MinIO, which browsers need for media uploads) listen on all interfaces; the rest bind to 127.0.0.1.

Step 2: Generate secrets and a headless project

Every secret in the file has an insecure default marked # CHANGEME, and each one is read as ${VAR:-default}, so a .env file next to the Compose file overrides them all. Some values must match each other: the Postgres password also appears inside DATABASE_URL, and the MinIO root password is the S3 secret for all three buckets. The script below handles both, and also uses Langfuse's headless initialisation variables so the organisation, project, API keys and first user exist at first boot, with no clicking.

cat > gen-env.sh <<'EOF'
#!/usr/bin/env bash
# Writes .env with random secrets for the Langfuse v4 Compose stack.
set -euo pipefail
PG_PASS=$(openssl rand -hex 16)
MINIO_PASS=$(openssl rand -hex 16)
cat > .env <<ENV
POSTGRES_PASSWORD=${PG_PASS}
DATABASE_URL=postgresql://postgres:${PG_PASS}@postgres:5432/postgres
SALT=$(openssl rand -hex 16)
ENCRYPTION_KEY=$(openssl rand -hex 32)
NEXTAUTH_SECRET=$(openssl rand -hex 32)
NEXTAUTH_URL=http://localhost:3000
CLICKHOUSE_PASSWORD=$(openssl rand -hex 16)
REDIS_AUTH=$(openssl rand -hex 16)
MINIO_ROOT_PASSWORD=${MINIO_PASS}
LANGFUSE_S3_EVENT_UPLOAD_SECRET_ACCESS_KEY=${MINIO_PASS}
LANGFUSE_S3_MEDIA_UPLOAD_SECRET_ACCESS_KEY=${MINIO_PASS}
LANGFUSE_S3_BATCH_EXPORT_SECRET_ACCESS_KEY=${MINIO_PASS}
TELEMETRY_ENABLED=false
# Headless init: values must NOT be wrapped in double quotes
LANGFUSE_INIT_ORG_ID=acme
LANGFUSE_INIT_ORG_NAME=Acme
LANGFUSE_INIT_PROJECT_ID=support-agent
LANGFUSE_INIT_PROJECT_NAME=support-agent
LANGFUSE_INIT_PROJECT_PUBLIC_KEY=pk-lf-$(openssl rand -hex 16)
LANGFUSE_INIT_PROJECT_SECRET_KEY=sk-lf-$(openssl rand -hex 16)
LANGFUSE_INIT_USER_EMAIL=admin@example.com
LANGFUSE_INIT_USER_NAME=admin
LANGFUSE_INIT_USER_PASSWORD=$(openssl rand -hex 12)
ENV
chmod 600 .env
echo "wrote .env"
EOF
chmod +x gen-env.sh && ./gen-env.sh

Before starting anything, confirm Compose sees the pins and the substitutions. This is real output from the v4 Compose file with the override and a generated .env (secrets replaced with <redacted>):

docker compose config | grep -E "image:|DATABASE_URL|TELEMETRY_ENABLED" | sort | uniq -c
#    2       DATABASE_URL: postgresql://postgres:<redacted>@postgres:5432/postgres
#    2       TELEMETRY_ENABLED: "false"
#    1     image: cgr.dev/chainguard/minio
#    1     image: docker.io/clickhouse/clickhouse-server:25.12
#    1     image: docker.io/langfuse/langfuse-worker:4.69.0
#    1     image: docker.io/langfuse/langfuse:4.69.0
#    1     image: docker.io/postgres:17
#    1     image: docker.io/redis:7

Step 3: Start the stack and check health

docker compose up -d
# The web container logs "Ready" after roughly 2 to 3 minutes on first boot
docker compose logs -f langfuse-web | grep -m1 Ready
curl -s http://localhost:3000/api/public/health
# {"status":"OK","version":"4.69.0"}

Log in at http://localhost:3000 with the LANGFUSE_INIT_USER_EMAIL and password from .env. The project support-agent is already there with your keys. The Compose docs are clear that this setup has no high availability, scaling or backups; for production, use the Helm chart or put Postgres, ClickHouse and S3 on managed services.

Step 4: Install the Langfuse Python SDK and connect

SDK v5 needs Python 3.11 or newer and, if self-hosted, a Langfuse server at v4.59.0 or later, per the v4 to v5 upgrade notes. On Python 3.10, or with an older server, stay on the v4 SDK.

python3.12 -m venv .venv && . .venv/bin/activate
pip install "langfuse==5.0.4" "openai==3.37.0"

# Keys from .env. LANGFUSE_BASE_URL defaults to Langfuse Cloud EU, so set it.
export LANGFUSE_PUBLIC_KEY=$(grep ^LANGFUSE_INIT_PROJECT_PUBLIC_KEY .env | cut -d= -f2)
export LANGFUSE_SECRET_KEY=$(grep ^LANGFUSE_INIT_PROJECT_SECRET_KEY .env | cut -d= -f2)
export LANGFUSE_BASE_URL=http://localhost:3000
export OPENAI_API_KEY=sk-...   # your OpenAI key

python -c "from langfuse import get_client; print(get_client().auth_check())"
# True

LANGFUSE_HOST still works but is deprecated in favour of LANGFUSE_BASE_URL in the SDK's environment variable definitions.

Step 5: Trace LLM calls and tool calls

The Langfuse SDK is an OpenTelemetry tracer underneath: creating the client registers a global TracerProvider if none exists, and every observation is an OTel span. Observations carry a type. Use agent for the loop, generation for model calls and tool for tool executions, so the UI can draw the agent graph and filter tool latency separately. The langfuse.openai drop-in records each OpenAI call as a generation with model, usage and cost.

# agent.py: a minimal tool-calling agent traced with Langfuse SDK v5
import json
import os

from langfuse import get_client, observe, propagate_attributes
from langfuse.openai import OpenAI  # drop-in: each completion becomes a generation

langfuse = get_client()  # reads LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_BASE_URL
client = OpenAI()        # reads OPENAI_API_KEY
MODEL = os.environ.get("MODEL", "gpt-5-mini")

TOOLS = [{
    "type": "function",
    "function": {
        "name": "get_order_status",
        "description": "Look up the shipping status of an order by its ID.",
        "parameters": {
            "type": "object",
            "properties": {"order_id": {"type": "string"}},
            "required": ["order_id"],
        },
    },
}]


@observe(as_type="tool", name="get_order_status")
def get_order_status(order_id: str) -> dict:
    # Replace with a real API call. Arguments, return value, duration
    # and any exception are recorded on the tool observation.
    return {"order_id": order_id, "status": "shipped", "carrier": "DHL"}


@observe(as_type="agent", name="support-agent")
def run_agent(question: str, user_id: str, session_id: str) -> dict:
    # Set trace-level attributes first so every child span carries them.
    with propagate_attributes(user_id=user_id, session_id=session_id,
                              tags=["support"], version="2026-10-24"):
        messages = [
            {"role": "system", "content": "You are a support agent. Use tools when needed."},
            {"role": "user", "content": question},
        ]
        tool_calls = 0
        for step in range(5):  # hard step budget: a looping agent is an expensive agent
            resp = client.chat.completions.create(
                model=MODEL, messages=messages, tools=TOOLS,
                name=f"llm-step-{step}",  # Langfuse-only kwarg, removed before the API call
            )
            msg = resp.choices[0].message
            if not msg.tool_calls:
                break
            messages.append(msg.model_dump(exclude_none=True))
            for call in msg.tool_calls:
                result = get_order_status(**json.loads(call.function.arguments))
                tool_calls += 1
                messages.append({"role": "tool", "tool_call_id": call.id,
                                 "content": json.dumps(result)})
        # Automated score on the whole trace: did the agent use a tool?
        langfuse.score_current_trace(name="used_tool", value=1 if tool_calls else 0,
                                     data_type="BOOLEAN")
        return {"answer": msg.content, "trace_id": langfuse.get_current_trace_id()}


if __name__ == "__main__":
    out = run_agent("Where is order 8812?", user_id="user-42", session_id="chat-001")
    print(out["answer"])
    print("trace:", langfuse.get_trace_url(trace_id=out["trace_id"]))
    langfuse.flush()  # short-lived script: send buffered spans before exiting

Run python agent.py and open the printed URL. You should see one trace named support-agent with an agent observation, typically two generations (llm-step-0 requesting the tool, llm-step-1 answering) with a get_order_status tool observation between them, and a used_tool score. The exact number of steps depends on what the model decides. Decorated functions capture their arguments and return values by default; set LANGFUSE_OBSERVE_DECORATOR_IO_CAPTURE_ENABLED=false if those contain data you must not store, or pass capture_input=False on a single decorator.

Step 6: Track cost, including models Langfuse does not know

For OpenAI, Anthropic and Google models, Langfuse ships price definitions and infers cost from the usage the wrapper sends. Two rules from the cost tracking docs save hours later. Cost is computed at ingestion and never recalculated, so a price you fix today only applies to new traces. And usage buckets must not overlap: if you report cached tokens separately, input must exclude them, or you pay for them twice in the dashboard.

For a self-hosted model, record the generation yourself with the usage your server returns:

from langfuse import get_client

langfuse = get_client()

def summarise(text: str) -> str:
    with langfuse.start_as_current_observation(
        as_type="generation", name="summarise",
        model="llama-3.3-70b-instruct", input=text,
    ) as gen:
        reply, usage = call_my_vllm(text)  # your client; returns OpenAI-style usage
        gen.update(output=reply, usage_details={
            "input": usage["prompt_tokens"],
            "output": usage["completion_tokens"],
        })
        return reply

Then give that model a price, either under Project Settings, Models in the UI or through the public API. Prices are USD per unit; the numbers below are an internal chargeback rate, not a quote from anyone:

curl -s -u "$LANGFUSE_PUBLIC_KEY:$LANGFUSE_SECRET_KEY" \
  -X POST "$LANGFUSE_BASE_URL/api/public/models" \
  -H "Content-Type: application/json" \
  -d '{
    "modelName": "llama-3.3-70b-instruct",
    "matchPattern": "(?i)^llama-3.3-70b-instruct$",
    "unit": "TOKENS",
    "pricingTiers": [{
      "name": "Standard", "isDefault": true, "priority": 0, "conditions": [],
      "prices": {"input": 0.0000006, "output": 0.0000008}
    }]
  }'

If you already know the exact cost (from a gateway, for example), pass cost_details={"input": ..., "output": ...} to gen.update() instead; ingested cost always beats inferred cost.

Step 7: Group traces into sessions and attach user feedback scores

propagate_attributes() in Step 5 already did the session work: every trace with the same session_id appears together under Sessions, in order, which is how you replay a whole conversation. Call it as early as possible, wrapping or immediately inside the root observation, because spans created before it do not get the attributes and drop out of per-user cost and session filters. IDs must be US-ASCII and at most 200 characters.

Scores attach quality to traces. The agent already writes an automated score; user feedback usually arrives later from your frontend, so return the trace ID with the answer and score it when the click comes in:

# feedback.py: called by your API when a user clicks thumbs up or down
from langfuse import get_client

langfuse = get_client()

def record_feedback(trace_id: str, thumbs_up: bool, comment: str | None = None) -> None:
    langfuse.create_score(
        trace_id=trace_id,
        name="user_feedback",
        value=1 if thumbs_up else 0,
        data_type="BOOLEAN",   # also NUMERIC, CATEGORICAL, TEXT, CORRECTION
        comment=comment,
    )
    langfuse.flush()

create_score also accepts session_id instead of trace_id for a conversation-level rating. In SDK v5 scores are sent in batches; rejected scores are logged and not retried, so watch your application logs after changing score names or types.

Step 8: Send OpenTelemetry from anything else

Two cases. Inside Python, you can swap the langfuse.openai wrapper for the official OpenTelemetry instrumentation; its spans carry gen_ai.* attributes, which pass the SDK's default export filter. Use one or the other, not both, or every call appears twice.

from langfuse import get_client
from openai import OpenAI
from opentelemetry.instrumentation.genai.openai import OpenAIInstrumentor  # pip install opentelemetry-instrumentation-genai-openai==1.3b0

langfuse = get_client()            # registers the global TracerProvider
OpenAIInstrumentor().instrument()  # plain OpenAI() clients now emit gen_ai.* spans
client = OpenAI()

For other languages, run an OpenTelemetry Collector next to Langfuse. Langfuse's OTLP endpoint accepts HTTP/protobuf and HTTP/JSON but not gRPC, so the Collector also earns its place as a protocol converter:

# otel-collector.yaml for otel/opentelemetry-collector-contrib:0.163.2
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
processors:
  batch: {}
exporters:
  otlphttp/langfuse:
    endpoint: http://langfuse-web:3000/api/public/otel  # exporter appends /v1/traces
    headers:
      Authorization: "Basic ${env:LANGFUSE_AUTH}"
      x-langfuse-ingestion-version: "4"  # real-time ingestion on the v4 data model
service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlphttp/langfuse]

export LANGFUSE_AUTH=$(printf '%s' "$LANGFUSE_PUBLIC_KEY:$LANGFUSE_SECRET_KEY" | base64 | tr -d '\n')
docker run -d --name otelcol --network langfuse_default \
  -e LANGFUSE_AUTH -p 4317:4317 -p 4318:4318 \
  -v "$PWD/otel-collector.yaml:/etc/otelcol-contrib/config.yaml:ro" \
  otel/opentelemetry-collector-contrib:0.163.2

langfuse_default is the network Compose created for the clone directory. Services then export to http://localhost:4318 or localhost:4317 with no Langfuse-specific code. If you run Collectors on Kubernetes already, the same exporter block drops into the setup from the OpenTelemetry Collector guide.

Troubleshooting

Symptom	Likely cause	Fix
Script runs, no trace appears	Process exited before the batch exporter sent anything	Call langfuse.flush() (or shutdown()) before exit in scripts, jobs and Lambdas
auth_check() is False or 401 errors	LANGFUSE_BASE_URL unset, so the SDK talks to Langfuse Cloud EU with self-hosted keys	Export LANGFUSE_BASE_URL=http://localhost:3000; confirm keys belong to this project
pip install langfuse gives a 4.x version	Python 3.10, or you pinned below 5	Use Python 3.11+, or stay on SDK v4 against servers older than v4.59.0
OTLP traces show up about 10 minutes late	Raw exporter missing x-langfuse-ingestion-version: 4	Add the header to OTEL_EXPORTER_OTLP_HEADERS or the Collector exporter
Collector or SDK fails on port 4317 to Langfuse	Langfuse does not accept OTLP gRPC	Export OTLP over HTTP to /api/public/otel; let the Collector receive gRPC
FastAPI, HTTP or DB spans missing	SDK v5 exports only Langfuse, gen_ai.* and known LLM instrumentation spans by default	Pass should_export_span= to Langfuse(), combining is_default_export_span from langfuse.span_filter with your scopes
Every LLM call appears twice	Both langfuse.openai and an OTel OpenAI instrumentor are active	Keep one of them
Cost is empty or $0	Model name matches no price definition, or usage was not sent	Add a model with a matchPattern, or send usage_details or cost_details
Cost looks doubled with prompt caching	input includes the cached tokens you also reported	Report exclusive buckets: input without input_cached_tokens
Session or user filters miss spans	propagate_attributes() called after child spans started	Wrap the root observation or call it first thing inside it

FAQ

Is the Langfuse Python SDK built on OpenTelemetry?

Yes. The SDK sets up an OpenTelemetry tracer when the client is created and exports spans over OTLP to /api/public/otel/v1/traces. That is why OTel instrumentations for OpenAI, LangChain and other libraries show up in the same trace tree.

Does Langfuse support OTLP over gRPC?

No. Langfuse accepts OTLP over HTTP with protobuf or JSON payloads. If your services export gRPC, put an OpenTelemetry Collector in front, receive gRPC on 4317 and export with otlphttp to Langfuse.

How much hardware does self-hosted Langfuse need?

Langfuse's Compose docs recommend at least 4 cores and 16 GiB of memory, with about 100 GiB of storage on a VM. Compose has no high availability or backups, so production deployments usually move to the Helm chart and managed Postgres, ClickHouse and object storage.

Why does cost not update after I change a model price?

Langfuse calculates inferred cost at ingestion time and does not recalculate old observations. New prices apply only to generations ingested after the change. Send cost_details yourself if you need exact historical figures.

When a team tells me their traces are missing, I check three things before anything clever: whether LANGFUSE_BASE_URL points where they think it does, whether the process calls flush() before it exits, and whether a raw OTLP exporter is sending the ingestion-version header. In production I keep observability vendor code out of the hot path, pin both the server images and the SDK, and give the Langfuse stack the same backup and retention policy as the application database, because it holds the same customer data. The references worth bookmarking are the Langfuse self-hosting docs, the Python SDK instrumentation docs, the observation types reference and the OpenTelemetry GenAI semantic conventions. For how those spans fit a wider tracing setup, see distributed tracing explained, and for gating prompt changes on scores, LLM evals in CI.

Related: Agent Observability in 2026: The Spans Are Standard, the Standard Isn't Stable, and Nobody Reads the Logs, How to Set Up the OpenTelemetry Collector on Kubernetes (2026) and How to Run LLM Evals in CI with GitHub Actions (promptfoo, 2026).