How to Triage Logs With Drain3 and a Local LLM (Ollama, 2026)

This guide is for the engineer who has been told to "add AI to the logs" and would like to do it without mailing a gigabyte of production output to someone else's model. By the end you will have a small Python pipeline that mines log templates with Drain3, compares a recent window against a baseline to find new and rare templates, redacts what is left, and hands a compact JSON summary to a local model running in Ollama for a plain-English explanation. Everything below was run on macOS with Python 3.12.8, drain3 0.9.11 (the latest release) and tiktoken 0.14.0 against public Loghub samples; the Ollama step targets Ollama v0.40.16 and qwen3:8b, and I show its code without captured output because I did not have Ollama on the test machine. You need Python 3.10 or later, curl, and about 6 GB of disk if you pull the model. The reasoning behind the design, and what the vendors ship instead, is in the companion post on AI log analysis in 2026.

Step 1: Create a virtualenv and install Drain3 0.9.11

Keep this out of your system Python. Drain3 has a handful of dependencies (jsonpickle, cachetools) and nothing heavy.

mkdir -p ~/log-triage/{code,data} && cd ~/log-triage
python3 -m venv venv
./venv/bin/pip install "drain3==0.9.11" "tiktoken==0.14.0"   # tiktoken only for counting tokens
./venv/bin/pip show drain3 | head -2

Expected output:

Name: drain3
Version: 0.9.11

Version 0.9.11 shipped in July 2022 and is still the newest on PyPI; the project moved from IBM to logpai/Drain3 on GitHub. That is fine for an algorithm this small, but pin it.

Step 2: Download real sample logs from Loghub

Loghub is the standard public collection of system logs used in log parsing research. Its README lists the full Linux dataset at 25,567 lines and HDFS_v1 at 11,175,629 lines, and the repository ships 2,000-line samples of each that are perfect for a first run.

cd ~/log-triage/data
curl -sSfLO https://raw.githubusercontent.com/logpai/loghub/master/HDFS/HDFS_2k.log
curl -sSfLO https://raw.githubusercontent.com/logpai/loghub/master/Linux/Linux_2k.log
wc -l HDFS_2k.log Linux_2k.log
head -2 HDFS_2k.log

    2000 HDFS_2k.log
    1999 Linux_2k.log
    3999 total
081109 203615 148 INFO dfs.DataNode$PacketResponder: PacketResponder 1 for block blk_38865049064139660 terminating
081109 203807 222 INFO dfs.DataNode$PacketResponder: PacketResponder 0 for block blk_-6952295868487656571 terminating

The Linux sample has no newline after its last line, hence 1,999 from wc; the script below reads 2,000 lines. Both files use CRLF line endings, which Python's text mode normalises to \n on read, so rstrip("\n") is enough. Loghub says the data is free for research and academic use, so check that before you put it in a commercial product's test fixtures.

Step 3: Configure Drain3 masking in drain3.ini

Masking is the step that decides whether your templates are readable. Drain3 applies the regexes in order before the Drain tree sees the line, replacing each match with <NAME>. Put the most specific patterns first. My first attempt masked long hex strings before HDFS block IDs, and every blk_-6952295868487656571 became blk_-<HEX>, which is useless.

Save this as code/drain3.ini:

[SNAPSHOT]
snapshot_interval_minutes = 10
compress_state = True

[DRAIN]
# similarity threshold: higher = more, narrower templates
sim_th = 0.4
depth = 4
max_children = 100
max_clusters = 1024

[MASKING]
masking = [
    {"regex_pattern": "blk_-?\\d+", "mask_with": "BLOCK"},
    {"regex_pattern": "((?<=[^A-Za-z0-9])|^)\\d{1,3}(\\.\\d{1,3}){3}(:\\d+)?((?=[^A-Za-z0-9])|$)", "mask_with": "IP"},
    {"regex_pattern": "[\\w.+-]+@[\\w-]+\\.[\\w.]+", "mask_with": "EMAIL"},
    {"regex_pattern": "((?<=[^A-Za-z0-9])|^)0x[0-9a-fA-F]+((?=[^A-Za-z0-9])|$)", "mask_with": "HEX"},
    {"regex_pattern": "(?<=\\[)\\d+(?=\\])", "mask_with": "PID"}
    ]
mask_prefix = <
mask_suffix = >

What the Drain settings do, per the defaults in Drain3's TemplateMinerConfig:

• sim_th (default 0.4): the share of tokens that must match for a line to join an existing cluster. Raise it toward 0.6 if unrelated messages are being merged; lower it if you get near-duplicate templates.
• depth (default 4): parse tree depth. Lines are routed by token count first, then by their leading tokens.
• max_children (default 100): branching limit per tree node.
• max_clusters: unset by default, which means unbounded. Set it in anything long-running so a noisy service cannot eat your memory; Drain3 evicts least recently used clusters beyond the cap.

Step 4: Mine templates and compare a window against a baseline

The script strips each line's header (timestamp, host, PID column), feeds the body to Drain3, and counts template IDs separately for the baseline (first 75% of lines) and the window (last 25%). A template is new if it never appeared in the baseline and rare if it appeared at most twice. It also keeps one masked example line per template and runs a second redaction pass for anything secret-shaped.

Save as code/triage.py:

#!/usr/bin/env python3
"""Mine log templates with Drain3, compare a recent window against a baseline,
and write a small, redacted JSON summary that is safe to hand to an LLM."""
import argparse, collections, json, re, sys
from drain3 import TemplateMiner
from drain3.template_miner_config import TemplateMinerConfig

# Strip the per-line header so timestamps and hostnames never reach Drain.
HEADER = re.compile(
    r"^(?:\w{3}\s+\d+\s[\d:]{8}\s\S+"          # syslog: "Jun 14 15:16:01 combo"
    r"|\d{6}\s\d{6}\s\d+)\s"                    # HDFS:   "081109 203615 148"
)
# Defence in depth: scrub anything secret-shaped that masking missed.
REDACT = [
    (re.compile(r"(?i)(password|passwd|secret|token|api[_-]?key)\s*[=:]\s*\S+"), r"\1=<REDACTED>"),
    (re.compile(r"(?i)bearer\s+[a-z0-9._~+/-]+=*"), "Bearer <REDACTED>"),
    (re.compile(r"\bAKIA[0-9A-Z]{16}\b"), "<AWS_KEY>"),
    (re.compile(r"[\w.+-]+@[\w-]+\.[\w.]+"), "<EMAIL>"),
    (re.compile(r"\b\d{1,3}(?:\.\d{1,3}){3}\b"), "<IP>"),
]
REPEAT = re.compile(r"((<\w+>|<\*>) ){3,}")    # "<BLOCK> <BLOCK> <BLOCK> ..." -> "<BLOCK> ..."

def redact(text: str) -> str:
    for rx, repl in REDACT:
        text = rx.sub(repl, text)
    return REPEAT.sub(lambda m: m.group(2) + " ... ", text)

def main() -> None:
    ap = argparse.ArgumentParser()
    ap.add_argument("logfile")
    ap.add_argument("--config", default="drain3.ini")
    ap.add_argument("--baseline-frac", type=float, default=0.75)
    ap.add_argument("--rare-max", type=int, default=2, help="baseline count at or below which a template is rare")
    ap.add_argument("--out", default="summary.json")
    args = ap.parse_args()

    cfg = TemplateMinerConfig()
    cfg.load(args.config)
    miner = TemplateMiner(config=cfg)            # add persistence_handler=... to keep state between runs

    with open(args.logfile, errors="replace") as fh:
        lines = [l.rstrip("\n") for l in fh if l.strip()]
    cut = int(len(lines) * args.baseline_frac)
    base, win = collections.Counter(), collections.Counter()
    example = {}
    for i, line in enumerate(lines):
        body = HEADER.sub("", line)
        res = miner.add_log_message(body)
        cid = res["cluster_id"]
        (base if i < cut else win)[cid] += 1
        if i >= cut:
            example.setdefault(cid, miner.masker.mask(body))

    tmpl = {c.cluster_id: c.get_template() for c in miner.drain.clusters}
    new = sorted((c for c in win if base[c] == 0), key=lambda c: -win[c])
    rare = sorted((c for c in win if 0 < base[c] <= args.rare_max), key=lambda c: -win[c])

    def row(c):
        return {"id": c, "template": redact(tmpl[c]), "window": win[c], "baseline": base[c],
                "example": redact(example[c])[:200]}

    summary = {
        "source": args.logfile.rsplit("/", 1)[-1],
        "baseline_lines": cut,
        "window_lines": len(lines) - cut,
        "templates_total": len(tmpl),
        "new_templates": [row(c) for c in new[:15]],
        "rare_templates": [row(c) for c in rare[:10]],
        "top_templates": [{"id": c, "template": redact(tmpl[c]), "window": n} for c, n in win.most_common(5)],
    }
    with open(args.out, "w") as fh:
        json.dump(summary, fh, indent=1)
    print(f"lines={len(lines)} templates={len(tmpl)} window_templates={len(win)} "
          f"new={len(new)} rare={len(rare)} summary_bytes={len(json.dumps(summary))}", file=sys.stderr)

if __name__ == "__main__":
    main()

Run it on both samples:

cd ~/log-triage
./venv/bin/python code/triage.py data/HDFS_2k.log  --config code/drain3.ini --out HDFS.summary.json
./venv/bin/python code/triage.py data/Linux_2k.log --config code/drain3.ini --out Linux.summary.json

lines=2000 templates=16 window_templates=12 new=3 rare=1 summary_bytes=1658
lines=2000 templates=108 window_templates=92 new=83 rare=0 summary_bytes=3283

The HDFS run finished in well under a second. As a rough throughput check, the HDFS sample repeated 50 times (100,000 lines) took about 2 seconds wall clock on an Apple M3 Pro (2.05 s in one run, 2.24 s in a rerun). Repeated lines are cache-friendly, so expect slower on real, more varied data.

Step 5: Read the results before any model does

Open HDFS.summary.json. The three new templates are all NameNode instructions:

{
  "id": 15,
  "template": "INFO dfs.FSNamesystem: BLOCK* ask <IP> to replicate <BLOCK> to datanode(s) <IP>",
  "window": 1,
  "baseline": 0,
  "example": "INFO dfs.FSNamesystem: BLOCK* ask <IP> to replicate <BLOCK> to datanode(s) <IP>"
}

Two of the three are the same event: "ask datanode to delete" followed by a list of block IDs. Drain clusters by token count, so a list of nine blocks and a list of a hundred become different templates. The REPEAT regex collapses <BLOCK> <BLOCK> <BLOCK> into <BLOCK> ... so the summary stays small, but the template IDs are still distinct. If a single event type keeps fragmenting, mask the whole list with one regex instead.

The Linux summary is more dramatic: 83 of the 92 templates in the window are new. The top ones are named[<PID>]: notify question section contains no SOA (16 times), BIOS memory map lines, kernel hash table sizes and syslog: <*> startup succeeded. Looking at the raw file confirms it: Linux version 2.6.5-1.358 appears at line 1,911, dated 27 July. The machine rebooted. A burst of new templates is an excellent reboot and deploy detector, and you should label it as such before an LLM gets a chance to call it an outage.

Notice also that the BIOS example line still shows 0000000000000000 - 00000000000a0000. My HEX mask only matches 0x-prefixed values. Whether that matters depends on your logs; that is why the summary carries one example per template, so a human can spot what masking missed.

Step 6: Redact and measure what you would send

Before choosing between a local model and an API, count the tokens. Logs are dense with numbers and punctuation, so they tokenise worse than prose.

# code/count_tokens.py: compare raw window vs summary (tiktoken 0.14.0, o200k_base)
import sys, tiktoken
enc = tiktoken.get_encoding("o200k_base")
log, summary = sys.argv[1], sys.argv[2]
lines = open(log).read().splitlines()
window = "\n".join(lines[int(len(lines) * 0.75):])
print("bytes/token (whole file):", round(len(open(log, "rb").read()) / len(enc.encode(open(log).read())), 2))
print("raw window tokens:", len(enc.encode(window)))
print("summary tokens:", len(enc.encode(open(summary).read())))

./venv/bin/python code/count_tokens.py data/HDFS_2k.log HDFS.summary.json
./venv/bin/python code/count_tokens.py data/Linux_2k.log Linux.summary.json

bytes/token (whole file): 2.97
raw window tokens: 25509
summary tokens: 629
bytes/token (whole file): 2.51
raw window tokens: 19603
summary tokens: 1303

That is roughly a 40x reduction for HDFS and 15x for Linux. At 2.97 bytes per token, one GiB of similar raw logs would be about 362 million tokens, around $723 of input at the $2 per million that Anthropic's pricing page lists for Sonnet 5.5. The page also notes that Claude 4.7 and later tokenizers produce about 30% more tokens for the same text, so tiktoken counts understate Claude costs. The summary approach costs a fraction of a cent per window wherever you send it.

Redaction here is layered: Drain3 masking catches IPs, PIDs, block IDs and emails at mining time, and redact() catches password=, token=, bearer tokens and AWS access key IDs in templates and examples. Add patterns for whatever your services leak. If you have not already, stop secrets reaching Git with Gitleaks; its rule set is a good source of regexes for logs too.

Step 7: Ask a local LLM via Ollama for an explanation

Install Ollama from its releases page or package manager and pull a model. qwen3:8b is a 5.2 GB download with a 40K context window per its library page, plenty for a 1,300-token summary.

ollama --version          # tested target: v0.40.16
ollama pull qwen3:8b
curl -s http://localhost:11434/api/tags | head -c 200   # server listens on 11434 by default

The script below sends only the summary JSON to POST /api/chat. It uses the chat API fields format (a JSON schema, so the reply is machine-checkable), think: false (Qwen3 reasons by default), options.temperature: 0 and keep_alive. Ollama's structured outputs docs recommend also putting the schema in the prompt, which the system message does. After the reply arrives, every finding is checked against the template IDs that were actually sent.

Save as code/explain.py:

#!/usr/bin/env python3
"""Send the redacted Drain3 summary (never raw logs) to a local Ollama model."""
import json, sys, urllib.request

OLLAMA = "http://localhost:11434/api/chat"
MODEL = "qwen3:8b"

SCHEMA = {
    "type": "object",
    "properties": {
        "verdict": {"type": "string", "enum": ["likely_incident", "needs_human", "probably_benign"]},
        "findings": {"type": "array", "items": {
            "type": "object",
            "properties": {
                "template_ids": {"type": "array", "items": {"type": "integer"}},
                "what_it_means": {"type": "string"},
                "next_check": {"type": "string"},
            },
            "required": ["template_ids", "what_it_means", "next_check"],
        }},
    },
    "required": ["verdict", "findings"],
}

SYSTEM = (
    "You are triaging log templates for an on-call engineer. You see templates, not raw logs. "
    "Only describe what the templates show. Cite template ids for every finding. "
    "Never state a root cause as fact; propose the next command or check instead. "
    "If the evidence is thin, return verdict needs_human. Reply as JSON matching this schema: "
    + json.dumps(SCHEMA)
)

def main(path: str) -> None:
    summary = json.load(open(path))
    body = {
        "model": MODEL,
        "stream": False,
        "think": False,                       # qwen3 thinks by default; we want the JSON only
        "format": SCHEMA,                     # structured output, enforced by Ollama
        "options": {"temperature": 0, "num_ctx": 8192},
        "keep_alive": "10m",
        "messages": [
            {"role": "system", "content": SYSTEM},
            {"role": "user", "content": json.dumps(summary)},
        ],
    }
    req = urllib.request.Request(OLLAMA, data=json.dumps(body).encode(),
                                 headers={"Content-Type": "application/json"})
    with urllib.request.urlopen(req, timeout=300) as r:
        resp = json.load(r)
    out = json.loads(resp["message"]["content"])
    known = {t["id"] for k in ("new_templates", "rare_templates", "top_templates") for t in summary[k]}
    for f in out["findings"]:                 # drop findings that cite templates we never sent
        f["grounded"] = all(i in known for i in f["template_ids"])
    print(json.dumps(out, indent=2))
    print(f"prompt_tokens={resp.get('prompt_eval_count')} output_tokens={resp.get('eval_count')}",
          file=sys.stderr)

if __name__ == "__main__":
    main(sys.argv[1])

./venv/bin/python code/explain.py Linux.summary.json

I have not shown output because I did not run this step. What to look for when you do: a verdict, findings that cite IDs like 27 (the named SOA template) and the BIOS templates, and next_check values you could actually run, such as checking last reboot or the host's uptime. Any finding with "grounded": false cited a template that was never in the summary and should be discarded. Response metadata includes prompt_eval_count and eval_count, which the script prints so you can track token use per run.

Treat the log text as untrusted. Usernames, URLs and user agents can carry instructions aimed at your model, which is the same problem covered in my prompt injection defence guide. Templating helps, because attacker-controlled values mostly become <*>, and the model in this pipeline has no tools to misuse.

Step 8: Keep Drain3 state between runs and schedule it

For a real service you want the baseline to persist, so "new" means new since last week rather than new since the script started. Drain3 ships FilePersistence, plus Redis and Kafka handlers.

# code/persist_demo.py: reuse templates across runs with FilePersistence
import sys
from drain3 import TemplateMiner
from drain3.file_persistence import FilePersistence
from drain3.template_miner_config import TemplateMinerConfig

cfg = TemplateMinerConfig()
cfg.load("code/drain3.ini")
miner = TemplateMiner(FilePersistence("drain3_state.bin"), cfg)   # loads state if the file exists
created = 0
for line in open(sys.argv[1], errors="replace"):
    r = miner.add_log_message(line.split(" ", 3)[-1].strip())
    created += r["change_type"] == "cluster_created"
miner.save_state("end of batch")
print(f"clusters={len(miner.drain.clusters)} created_this_run={created}")

$ ./venv/bin/python code/persist_demo.py data/HDFS_2k.log
clusters=16 created_this_run=16
$ ./venv/bin/python code/persist_demo.py data/HDFS_2k.log
clusters=16 created_this_run=0

The second run created nothing because every template was already known. In production I would run the mining step as a sidecar or a cron job over the last 15 minutes from Loki or a file, alert when new error templates appear, and call the LLM only for those windows. If you already run Loki, its pattern ingester does the mining for you; see my Grafana Loki setup guide.

Troubleshooting

Symptom	Likely cause	Fix
config file not found: drain3.ini warning	Relative path resolved from a different working directory	Pass --config with an absolute path, or run from the project root
Templates contain blk_-<HEX> or half-masked IDs	A broader regex runs before a specific one	Reorder [MASKING] so specific patterns (block IDs, UUIDs) come first
Hundreds of near-identical templates	Timestamps or headers not stripped, or sim_th too high	Fix the HEADER regex for your format; try sim_th = 0.3
Unrelated messages merged into one template	sim_th too low for short messages	Raise sim_th to 0.5 or 0.6
One event type appears as several new templates	Variable-length lists change the token count	Mask the whole list with one regex
Every template is "new" in the window	A reboot, deploy or log format change, or a baseline that is too short	Check for startup templates first; persist state across runs
Memory grows without bound in a long-running miner	max_clusters unset	Set max_clusters; Drain3 then evicts least recently used clusters
json.decoder.JSONDecodeError reading drain3.ini masking	Unescaped backslashes in the JSON array	Double every backslash inside the ini (\\d, \\.)
urllib.error.URLError: Connection refused	Ollama server not running	Start the Ollama app or ollama serve; check port 11434
Model reply is empty or not valid JSON	Thinking output or a model that ignores format	Keep think: false, temperature: 0, and the schema in the system prompt

FAQ

What does Drain3 sim_th do?

It is the similarity threshold for joining an existing cluster: the fraction of tokens in a new line that must match a template's tokens. The default is 0.4. Higher values produce more, narrower templates; lower values merge more aggressively.

Can I use OpenAI or Claude instead of Ollama?

Yes. The summary JSON is small enough to send anywhere, and that is the point of mining first. Swap the HTTP call for your provider's SDK, keep the JSON schema and the grounding check, and confirm your data agreement covers log content even after masking.

Is Drain3 accurate enough for production?

For alerting on new and rare templates, yes, as long as you tune masking. Research on the Loghub-2.0 benchmark found that accurately parsing rare-event logs remains a challenge for current parsers, so review the new templates a human sees rather than trusting counts alone.

Which local model should I use for log triage?

Any instruction-tuned model that supports Ollama's structured outputs and fits your hardware. An 8B model is enough to explain a 1,300-token summary; bigger models write better prose but the grounding check matters more than the model size.

When I wire this into a real system, the first thing I check is the masking: I read twenty new templates by eye before trusting any count, because one bad regex will fill the window with fake "new" events. Then I alert on new error templates, not on the LLM's verdict, and let the model write the first paragraph of the ticket. Official docs worth bookmarking: Drain3 README, Loghub, Ollama chat API and Ollama structured outputs.

Related: AI Log Analysis in 2026: What Works, What Hallucinates, What It Costs, How to Run a Local LLM With Ollama: Setup, GPU and API Guide (2026) and How to Set Up Grafana Loki on Kubernetes with Alloy (2026).