Prompt Caching Explained (2026): 0.05x Reads, 5-Minute TTLs, Real Math
How prompt caching works in 2026: KV-cache prefix matching, Anthropic vs OpenAI vs Gemini rules, TTLs from 5 minutes to 24 hours, write premiums of 1.25x to 2.5x, reads at 0.025x to 0.1x, batch stacking, and a worked bill that falls from $6,700 to $1,002.
12 minOct 31, 2026How to Install NVIDIA GPU Operator and Share GPUs: Time-Slicing, MIG (2026)
Step-by-step NVIDIA GPU Operator v26.7.2 install with Helm, verified with nvidia-smi and a CUDA test pod, then GPU sharing with a time-slicing ConfigMap or MIG profiles, DCGM Exporter metrics, and fixes for driver crash loops and nodes with no nvidia.com/gpu.
14 minOct 31, 2026Multi-Agent Systems in 2026: +90% on Research, -70% on Planning, 15x Tokens
When multi-agent systems beat a single agent in 2026: Anthropic's 90.2% research gain at 15x the tokens, Google's 260-configuration study (+80.8% to -70%), Cognition's single-writer rule, the MAST failure taxonomy, and how to evaluate before you split.
12 minOct 31, 2026LLM Inference Engines 2026: vLLM vs SGLang vs TensorRT-LLM vs Dynamo
vLLM 0.32, SGLang 0.5.22, TensorRT-LLM 1.2.1 stable (1.4 in RC), Dynamo 1.6 and llama.cpp compared as of October 2026, with TGI now archived. How batching, PagedAttention and speculative decoding differ, what published benchmarks show, and which engine to pick.
11 minOct 31, 2026GPUs on Kubernetes Explained (2026): Device Plugin, MIG, Time-Slicing, DRA
How GPUs on Kubernetes work in October 2026: the device plugin and nvidia.com/gpu, what GPU Operator v26.7.2 installs, MIG vs time-slicing vs MPS, DRA (GA since 1.34) and NVIDIA's DRA driver v0.5.1, Kueue quotas, EKS, GKE and AKS options, and why GPUs average 5% use.
14 minOct 31, 2026How to Cut LLM API Costs: Prompt Caching, Batching, Routing (2026)
Step-by-step guide to cutting LLM API costs in 2026: measure usage fields, stabilise prompt prefixes, add Claude cache_control and OpenAI cache breakpoints, use Gemini context caching, move offline work to 50%-off batch APIs, route by task, cap max_tokens, and fix cache misses.
12 minOct 31, 2026Context Engineering Explained (2026): Why 1M Tokens Still Isn't Enough
Context engineering in 2026, explained by an infrastructure engineer: why it replaced prompt engineering, what context rot research shows, and how retrieval, tool definitions, memory, compaction and sub-agents keep an agent's window small and useful.
11 minOct 31, 2026How to Build Agents With Claude Agent SDK Subagents (Python, 2026)
Step-by-step Claude Agent SDK guide pinned to claude-agent-sdk 0.2.176: install, the query loop, custom tools, subagent definitions, permissions and hooks, MCP servers, sessions, budget and depth caps, plus a troubleshooting table for the errors people hit.
11 minOct 31, 2026How to Build a Voice AI Agent with LiveKit Agents or Pipecat (2026)
Step-by-step: build a cascaded voice AI agent on LiveKit Agents 1.8.10 (or Pipecat 1.15.0) with STT, LLM and TTS, turn detection, tool calling, per-turn latency metrics, LiveKit Cloud deployment and a SIP phone number, plus fixes for common errors.
16 minOct 31, 2026How to Benchmark LLM Inference: TTFT, Throughput and Goodput (2026)
Benchmark an LLM inference server properly with vllm bench serve (vLLM 0.32) and GuideLLM 0.8: define TTFT, ITL, TPOT and goodput, run a concurrency sweep, use real prompts instead of random ones, read the knee in the results, and fix the mistakes that make benchmarks lie.
13 minOct 20, 2026What Is MCP (Model Context Protocol)? A Plain-English Guide for 2026
What the Model Context Protocol is and how it works in 2026: hosts, clients and servers, tools, resources and prompts, stdio and Streamable HTTP, every spec version up to the stateless 2026-07-28 release, Linux Foundation governance, adoption, and the real risks.
12 minOct 20, 2026What Are AI Agents? Workflows, ReAct and Real Success Rates (2026)
AI agents explained for engineers in 2026: Anthropic's workflows vs agents split, the ReAct loop, tool calling, planning, memory and multi-agent designs, measured success rates from OSWorld (66.3%) to TheAgentCompany (30%), and when not to build one.
13 minOct 20, 2026Vector Databases in 2026: Do You Need One, or Is pgvector Enough?
An engineer's guide to vector databases in 2026: pgvector 0.8.7, Pinecone, Qdrant, Milvus 3.0, Weaviate, turbopuffer, Amazon S3 Vectors and Elasticsearch compared on price and architecture, why the benchmarks mislead, and when Postgres is enough.
13 minOct 20, 2026SRE for LLM Apps in 2026: SLOs for TTFT, Provider Outages and Evals
How to apply Google SRE practice to LLM applications in 2026: SLIs for TTFT and time per output token, availability when your model provider had 15 major API incidents in 72 days, eval-based quality SLIs, error budgets that count fallbacks, and rate limits.
13 minOct 20, 2026Self-Hosting LLMs vs API Costs in 2026: The 62% Utilisation Break-Even
Self-hosting an open LLM vs paying per token in October 2026: H100 prices from $1.17 to $4.29 an hour, 8,973 tokens/s per GPU for gpt-oss-120b on vLLM, a $0.20 per million API price, the utilisation each price needs to break even, and the ops costs the spreadsheet leaves out.
13 min