Sep 7, 2026
LLM Inference in 2026: Prefill Is a Different Machine Now, and the KV Cache Has a Disk Tier
vLLM 0.28, SGLang on 400,000 GPUs, NVIDIA Dynamo 1.0 and llm-d in the CNCF all converged on the same architecture: split prefill from decode, route on cached prefixes, page the KV cache to CPU and SSD, and run MoE experts wide across a rack. What changed this year, with the papers, the flags, and the prices.
15 minSep 2, 2026Sandbox Creation Time Benchmarks Measure Different Things
Every AI sandbox matrix has a creation time column spanning 0.79ms to 2.7s. That spread is definitional, not performance. Here's how to make it comparable.
10 minSep 2, 2026I Measured Every Stage of My 179ms Firecracker Boot Path
Where every millisecond of a Firecracker create goes, measured on my own production fleet, plus why I refuse to stop the clock before the guest answers.
9 min