MLOps vs LLMOps vs AgentOps in 2026: What Carries Over, What's New

Every few months somebody asks me whether they need to hire an "LLMOps engineer", and a few months later the same person asks about AgentOps. I have been doing infrastructure for fourteen years and I am wary of any job title that ends in "Ops" and arrived with a vendor attached. But the question underneath is fair: if your team already does MLOps, or just DevOps, which parts of that discipline still apply when the thing you ship is a prompt wrapped around somebody else's model, or an agent that calls tools with real credentials? This post explains MLOps vs LLMOps vs AgentOps as they stand in 2026: how each lifecycle differs, what carries over untouched, what is genuinely new, and which tools cover which layer.

What is MLOps, and what problem did it solve?

MLOps is the set of practices for getting trained models into production and keeping them correct once they are there. The canonical framing is still Google's MLOps: Continuous delivery and automation pipelines in machine learning (last updated August 2024), which describes three maturity levels: level 0 where every step is manual, level 1 where the training pipeline is automated, and level 2 where the pipeline itself goes through CI/CD. The property it calls unique to ML is continuous training, retraining and redeploying a model when data or performance shifts.

The reason any of this exists is the 2015 NeurIPS paper Hidden Technical Debt in Machine Learning Systems, which argued that the model code is a small box in the middle of a much larger system of data pipelines, configuration, serving and monitoring, and that most of the debt lives in that surrounding system. A decade on, that is still the most useful idea in the field.

Classical MLOps gave us a toolkit that held up: data versioning (DVC, now at 3.67.1), experiment tracking, a model registry, feature stores (Feast 0.67.0 shipped on 13 October 2026), reproducible training pipelines, and drift monitoring. The unit of release is a trained artifact with weights you own, and the main production risk is that the world changes and the model silently gets worse.

What is LLMOps, and how is it different?

LLMOps is MLOps for applications built on large language models, and the biggest difference is that most teams no longer train anything. The model is an API or an open-weight checkpoint you serve. What you build, and therefore what you have to version, test and roll back, is everything around it: the system prompt, the retrieval configuration, the model choice and parameters, the output schema, and the evaluation set that tells you whether a change made things better.

That shifts three things. First, the prompt becomes code. A one-word change to a system prompt can move quality more than a retrain used to, so it needs a version, a reviewer and a way back. Second, testing becomes statistical. Outputs are non-deterministic, there is rarely a single correct string, and you end up grading with rubrics, reference answers and model judges. I wrote about why evals are the new unit tests and how far you can trust a judge; the short version is "less than the vendors imply". Third, cost becomes a per-request runtime metric. Tokens are billed on every call, so latency and spend sit next to quality on the dashboard.

The tooling followed. MLflow, the most widely installed open-source MLOps tool (about 19.9 million PyPI downloads in the last month), rebuilt itself around this with MLflow 3, released on 10 June 2025. It added a LoggedModel entity that is not tied to a training run, plus tracing, GenAI evaluation, and a prompt registry. The current release is 3.17.1, published on 13 October 2026. Langfuse, Arize Phoenix and others grew from the other direction, starting with traces and adding prompts and evals.

What is AgentOps?

AgentOps is the operational discipline for systems where the model does not just answer but acts: it plans, calls tools, reads the results, and loops until it decides it is done. The term has an academic anchor in CSIRO Data61's AgentOps: Enabling Observability of LLM Agents (November 2024), which proposes a taxonomy of what to trace across an agent's lifecycle, and a broader survey from 2025 that compares tools such as Langfuse, Arize Phoenix and OpenLLMetry by the metrics, logs and traces they capture.

Three things change when you go from an LLM app to an agent. The unit of work is now a trajectory, a tree of model calls and tool calls, not a single response, so a request log is useless and you need distributed traces. Quality has to be measured on outcomes over many runs, because the same task can succeed four times and fail the fifth. And the blast radius changes: an agent with a database tool and a token can do damage a chatbot cannot. The OWASP Top 10 for Agentic Applications for 2026, published in December 2025, leads with agent goal hijack and tool misuse for that reason.

So AgentOps overlaps with security and platform engineering far more than LLMOps did: tool permissions, credential scoping and isolated execution are operational controls, not model properties. (I build PandaStack, a Firecracker microVM sandbox service for agent code execution, so I have a commercial interest in this argument; the OWASP list makes it without me.)

MLOps vs LLMOps vs AgentOps: side by side

The table compares the three across the lifecycle. Each adds a layer; none replaces the last.

Lifecycle concern	MLOps	LLMOps	AgentOps	Source
Unit you release	Trained model artifact (weights)	Model choice + prompt + retrieval config	Prompt + tools + permissions + model	Google MLOps guide, MLflow 3 notes
What you version	Data, code, weights	Prompts, eval datasets, judge rubrics	Tool schemas, scopes, policies, plus all of LLMOps	MLflow prompt registry
Pre-release test	Holdout metrics (RMSE, AUC)	Eval suites with scorers and judges	Task success across repeated runs, tool-call correctness	MLflow evaluation docs
Production signal	Drift, latency, accuracy	Tokens, cost, quality scores, feedback	Traces of every model and tool call, actions taken	OTel GenAI conventions
Retrain or change trigger	Data or concept drift (continuous training)	Model deprecation, prompt regression	New tool, new permission, policy change	Google MLOps guide
Characteristic failure	Silent accuracy decay	Prompt regression, injection	Wrong action taken with real credentials	OWASP agentic Top 10

What carries over from MLOps (and DevOps) unchanged?

More than the new job titles suggest. Five practices transfer almost verbatim.

Versioning everything that changes behaviour. In MLOps that meant data and weights. In LLMOps it means prompts and eval sets; in AgentOps add tool definitions and permission policies. The rule is the same: if changing it can change what users see, it gets an immutable version and an audit trail.

A registry with movable pointers. The best idea in modern MLflow is the alias: a named, mutable pointer such as champion that points at an immutable version, documented in the model registry guide. The old Staging/Production stages have been deprecated since MLflow 2.9.0 (the @deprecated(since="2.9.0") decorators are still visible in client.py). The prompt registry uses exactly the same pattern, which is the clearest evidence that LLMOps is an extension rather than a new field. This is real code I ran against MLflow 3.17.1:

# pip install "mlflow==3.17.1"
import mlflow

mlflow.set_tracking_uri("http://127.0.0.1:5000")

# Register an immutable prompt version (same idea as a model version)
p = mlflow.genai.register_prompt(
    name="support-system",
    template="You are a support agent for {{product}}. Never issue refunds over {{limit}}.",
    commit_message="v1: add refund ceiling",
)

# Point a movable alias at it; rollback is moving the alias back
mlflow.genai.set_prompt_alias("support-system", alias="production", version=p.version)

# The application only ever loads the alias, never a hard-coded version
prompt = mlflow.genai.load_prompt("prompts:/support-system@production")
print(prompt.format(product="Acme", limit="$50"))
# -> You are a support agent for Acme. Never issue refunds over $50.

Swap prompts:/ for models:/ and you have the classical model-promotion workflow. My step-by-step MLflow 3 model registry and tracking guide walks through both with a Postgres and S3 backed server.

CI gates. A model needed to beat the current champion on a holdout set before promotion. A prompt needs to clear an eval threshold before its alias moves. Same pipeline, different test. If you already run tests in CI, you already have the machinery; see evals in CI with GitHub Actions for the LLM version.

Staged rollout and rollback. Canary and fast rollback matter more for prompts than binaries, because a prompt change ships in seconds without a build.

Monitoring with an owner. Unread drift dashboards became unread trace dashboards. An alert needs a runbook and a person.

What is genuinely new: prompts, evals, traces, tool permissions

Four things do not have a clean MLOps ancestor.

Prompts as first-class artifacts. They are code that non-engineers edit, which breaks the assumption that every behaviour change goes through a pull request. A prompt registry with aliases and approval is the minimum. More on this in prompt management for LLMOps.

Evals instead of metrics. You cannot compute RMSE on an essay. Evaluation now means a curated dataset, a set of scorers (some deterministic, some model judges), and a threshold. MLflow 3.17.1 ships built-in scorers including Correctness, RetrievalGroundedness, Safety, ToolCallCorrectness and ToolCallEfficiency (I listed them from the installed package's mlflow.genai.scorers module). The judges are useful and biased; treat their scores as estimates with error bars.

Traces as the primary signal. For agents, the trace is the log. The industry is converging on OpenTelemetry for this, but the conventions are not stable: the GenAI semantic conventions moved to their own repository in May 2026, and the agent span document (invoke_agent, execute_tool) is still marked Development. Instrument with OTel anyway, and pin the semconv version you emit. I went deeper on this in agent observability in 2026.

Tool permissions and identity. This is the part that makes AgentOps a platform and security problem. Every tool an agent can call is an API with credentials, and the question "what is this agent allowed to do, as whom" has no analogue in classical MLOps. The MCP specification, now at the 2026-07-28 revision, requires clients to send RFC 8707 resource indicators, requires servers to validate that tokens were issued for them, and forbids servers from passing tokens through to upstream APIs. Those are exactly the controls you would apply to any service-to-service call, written down for agents. Least-privilege scopes, short-lived tokens and isolated execution for anything that runs code are the AgentOps equivalent of not deploying a model that can write to production tables.

MLflow's own 3.17.0 release notes show where the operational weight is moving: fine-grained permissions on runs, traces and versions with explicit DENY, daily rollups because trace tables get large, and a schema upgrade that requires stopping all writers. That is database operations, not data science.

The 2026 tool map: which tools cover which layer?

You do not need a stack per acronym: one tracking platform, one tracing pipeline and your existing CI cover all three layers.

Layer	Open-source options (latest release checked 20 Oct 2026)	Notes
Data and pipeline versioning	DVC 3.67.1, Feast 0.67.0	Still matters for RAG corpora and fine-tuning data
Tracking, registry, prompts, evals	MLflow 3.17.1, Langfuse v4.64.0	MLflow covers classical and GenAI in one server
Tracing transport	OpenTelemetry Collector v0.163.1	GenAI semconv still Development
Model serving	KServe v0.21.0, MLflow models serve	See deploying with KServe
Agent controls	MCP authorization, policy engines, sandboxes	Mostly platform and security tooling, not ML tooling

Do you need a separate LLMOps or AgentOps team?

Usually not. What you need is for the existing owners to absorb the new artifacts. The ML or application team owns prompts and eval sets the way they own code. The platform team owns the tracking server, the trace pipeline and the CI gate, the way they own the build system. Security owns tool scopes and agent identity, the way they own IAM for services. The pattern to avoid is the opposite: a new "AI platform" group that builds a parallel deployment path with no change review, no rollback and its own credentials. That is a 2012 DevOps problem with a 2026 label.

FAQ

What is the difference between MLOps and LLMOps?

MLOps manages models you train yourself: data, training pipelines, weights, a registry and drift monitoring. LLMOps manages applications built on a model you mostly do not train, so the versioned artifacts become prompts, retrieval configuration and eval datasets, testing becomes statistical evaluation with scorers and judges, and token cost becomes a production metric.

Is AgentOps just LLMOps with a new name?

Partly, but agents add two things LLMOps did not have to handle: multi-step trajectories that need distributed tracing, and tools that act on real systems with real credentials. That makes permissions, identity and isolation part of the operational job, which pulls in platform and security teams.

Can MLflow handle LLMOps and AgentOps?

Since MLflow 3 (June 2025) it covers tracing, GenAI evaluation with built-in and custom scorers, a prompt registry with aliases, and LoggedModel versions for agents alongside the classical registry. It does not enforce tool permissions or sandbox execution; those belong to your platform and security layers.

Do I still need data versioning if I only call an LLM API?

Yes, if you use retrieval or fine-tuning, because the corpus and the training set change behaviour as much as the prompt does. You also need versioned eval datasets, otherwise you cannot tell whether a change improved anything.

This quarter I would do three things, in order: put every prompt behind a registry alias so rollback is one API call, wire an eval threshold into the same CI pipeline that already gates code, and write down, per agent, which tools it can call with which scopes and who approved them. None of it needs a new team or a new acronym. It needs the discipline we already had, applied to three new kinds of artifact.

Related: MLOps in 2026: Evals Are the New Unit Tests, Agent Observability in 2026 and Policy as Code for AI Agents in 2026.