AI infrastructure

32 posts tagged ai-infrastructure.

Oct 20, 2026

What Is MCP (Model Context Protocol)? A Plain-English Guide for 2026

What the Model Context Protocol is and how it works in 2026: hosts, clients and servers, tools, resources and prompts, stdio and Streamable HTTP, every spec version up to the stateless 2026-07-28 release, Linux Foundation governance, adoption, and the real risks.

12 min
Oct 20, 2026

What Are AI Agents? Workflows, ReAct and Real Success Rates (2026)

AI agents explained for engineers in 2026: Anthropic's workflows vs agents split, the ReAct loop, tool calling, planning, memory and multi-agent designs, measured success rates from OSWorld (66.3%) to TheAgentCompany (30%), and when not to build one.

13 min
Oct 20, 2026

Vector Databases in 2026: Do You Need One, or Is pgvector Enough?

An engineer's guide to vector databases in 2026: pgvector 0.8.7, Pinecone, Qdrant, Milvus 3.0, Weaviate, turbopuffer, Amazon S3 Vectors and Elasticsearch compared on price and architecture, why the benchmarks mislead, and when Postgres is enough.

13 min
Oct 20, 2026

SRE for LLM Apps in 2026: SLOs for TTFT, Provider Outages and Evals

How to apply Google SRE practice to LLM applications in 2026: SLIs for TTFT and time per output token, availability when your model provider had 15 major API incidents in 72 days, eval-based quality SLIs, error budgets that count fallbacks, and rate limits.

13 min
Oct 20, 2026

Self-Hosting LLMs vs API Costs in 2026: The 62% Utilisation Break-Even

Self-hosting an open LLM vs paying per token in October 2026: H100 prices from $1.17 to $4.29 an hour, 8,973 tokens/s per GPU for gpt-oss-120b on vLLM, a $0.20 per million API price, the utilisation each price needs to break even, and the ops costs the spreadsheet leaves out.

13 min
Oct 20, 2026

How to Run a Local LLM With Ollama: Setup, GPU and API Guide (2026)

Step-by-step Ollama guide for 2026 (v0.40): install on macOS, Linux, Windows or Docker, confirm NVIDIA, AMD or Apple GPU use, pick a model that fits, set context length, write a Modelfile, call the OpenAI-compatible API, connect coding tools, and fix slow or out-of-memory runs.

14 min
Oct 20, 2026

RAG vs Long Context vs Fine-Tuning in 2026: 1M Tokens Isn't Free

RAG vs long context vs fine-tuning in 2026: current 1M-token context windows and prices, what context rot, NoLiMa and RULER show about long-context accuracy, where RAG fails, the per-query cost math, and a decision framework you can apply this quarter.

13 min
Oct 20, 2026

How to Build a RAG Pipeline With Postgres and pgvector (2026)

Step-by-step RAG pipeline on Postgres 18 and pgvector 0.8.7: install, schema, chunking, OpenAI embeddings, an HNSW index, hybrid search with full-text and reciprocal rank fusion, a Python example, retrieval evals, and fixes for slow queries and poor recall.

17 min
Oct 20, 2026

How to Tune pgvector Performance: HNSW, ef_search and Quantization (2026)

A step-by-step pgvector tuning guide for 0.8.x: HNSW vs IVFFlat, m, ef_construction and ef_search, maintenance_work_mem for fast builds, iterative scans for filtered queries, halfvec and binary quantization, EXPLAIN, and a troubleshooting table, with real timings.

14 min
Oct 20, 2026

Running LLMs Locally in 2026: Ollama vs llama.cpp vs LM Studio vs MLX

A practical 2026 guide to running LLMs locally: Ollama 0.40, llama.cpp 0.7, LM Studio 0.4, MLX and vLLM compared, which models fit in 8, 16, 32 and 64GB, how Q4_K_M quantization works, the bandwidth maths behind tokens per second, licences and privacy.

12 min
Oct 20, 2026

How to Set Up LiteLLM Proxy: Budgets, Fallbacks and Virtual Keys (2026)

Step-by-step LiteLLM proxy setup for v1.106.0: verify and pin the image, run it with Postgres on Docker Compose or Helm, write config.yaml, add fallbacks and retries, issue virtual keys with budgets and rate limits, log to Langfuse and OpenTelemetry, and troubleshoot the common errors.

13 min
Oct 20, 2026

Is Fine-Tuning LLMs Worth It in 2026? Prices, LoRA, RFT and OpenAI's Exit

When fine-tuning an LLM beats prompting and RAG in 2026: OpenAI is winding down its fine-tuning API, Google and AWS added RL tuning, LoRA matches full fine-tuning when configured right, and a 5,000-example run costs $5 to $300 depending on where you train it.

14 min
Oct 20, 2026

How to Fine-Tune an LLM with LoRA Using Unsloth or TRL (2026)

Step-by-step LoRA and QLoRA fine-tuning of an open-weight LLM with Unsloth 2026.10 or Hugging Face TRL 1.16: dataset format, hyperparameters, GPU memory, training script, evaluation against the base model, GGUF and vLLM export, and fixes for OOM and loss problems.

14 min
Oct 20, 2026

Docker in 2026: Engine 29, containerd 2.3 LTS, Podman 6 and the AI Push

The state of Docker and containers in October 2026: Docker Engine 29.9, Desktop 4.96 and its licence terms, Model Runner and the MCP Toolkit, free Hardened Images, Hub pull limits, containerd 2.3 LTS, Podman 6.0, BuildKit 0.34 and OCI artifacts.

12 min
Oct 20, 2026

DevOps Roadmap 2026: Skills, Certifications and What AI Changed

A DevOps roadmap for 2026 built on current data: Indeed's senior-heavy rebound in software postings, Stack Overflow 2025 and 2026 survey results on AI and pay, CKA and CNPA exam facts, and a learning path from Linux to platform engineering.

13 min