Self-Hosting LLMs vs API Costs in 2026: The 62% Utilisation Break-Even

Every few weeks someone asks me whether they should stop paying per token and run an open model on their own GPUs. They usually arrive with a spreadsheet that multiplies a GPU's peak benchmark throughput by 24 hours, divides by an hourly price, and concludes that self-hosting is ten times cheaper. The arithmetic is right and the conclusion is usually wrong, because the only number that matters is how busy that GPU will actually be. This post works through the self-hosting LLM vs API cost question with numbers I could source in October 2026: current GPU hourly prices, measured throughput per GPU for an open model, the API price for the same model, the utilisation where the two cross, and the operational costs that never make it into the spreadsheet.

What does it cost to self-host an LLM in 2026?

Start with the hourly price, because it spans almost four times from cheapest to dearest for the same chip. I took list prices from provider pages on 20 October 2026 and a transacted index figure from my GPU cloud economics post.

Source	GPU	$/GPU-hour	Notes
SemiAnalysis TCO via InferenceX	H100	1.17	Cost of owning at hyperscaler volume, not a rental price
InferenceX retail tier	H100	2.00	The benchmark's own retail assumption
Ornn OCPI-H100 index	H100 SXM	2.59	Transacted neocloud rentals, 4 October 2026
RunPod	H100 SXM	2.69 / 3.99	Community cloud / secure cloud
Together GPU clusters	HGX H100	3.99	On demand; 3.19 reserved 91 to 180 days
Lambda	H100 SXM	3.99 / 4.29	Per GPU in an 8x node / single GPU
Together dedicated inference	HGX H100	5.49	Managed endpoint, you pay for the GPU-hour
Lambda	B200	6.69 / 6.99	8x node / single GPU

Two things stand out. First, the single-GPU on-demand price that a small team actually pays ($3.99 to $4.29) is two to three and a half times the "cost of ownership" figure that benchmark sites use to quote cost per token. Second, a single H100 running all month at Lambda's 1x price is $4.29 × 730 hours, about $3,130. Any comparison has to clear that monthly floor before it says anything about per-token economics.

That is the GPU. You also need the node it sits in (usually bundled), storage for weights, a load balancer, monitoring, and a second replica if you care about availability. I come back to those below.

How many tokens per second does one GPU actually serve?

This is where most self-hosting estimates go wrong, because "tokens per second" is not one number. It is a curve, and the point you pick on it depends on how fast each user needs to see tokens.

The most useful public data I know is SemiAnalysis's InferenceX, an open, continuously run benchmark. For OpenAI's gpt-oss-120b (117B parameters, 5.1B active, Apache 2.0, MXFP4 weights that fit a single 80 GB GPU) on one H100 running vLLM, with an 8k input and 1k output chat workload, runs dated March to May 2026 report:

• 8,973 tokens/s per GPU at 50 tokens/s per user
• 5,863 tokens/s per GPU at 100 tokens/s per user
• 2,558 tokens/s per GPU at 200 tokens/s per user

Those are total tokens, input plus output. InferenceX's methodology notes give the cost formula as GPU-hour price × 1,000,000 / (3,600 × throughput per GPU), per million total tokens, which is the formula I use below. The shape matters more than the exact values: asking for four times the per-user speed cut per-GPU throughput by about 3.5x, because the GPU can only batch so many requests before each one slows down. If you want the mechanics of why prefill and decode behave so differently, my post on LLM inference serving systems covers batching, KV cache and disaggregation.

The newer chips move the curve a long way. On InferenceX's B200 vs H100 comparison for the same model and workload, a B200 reaches about 36,090 tokens/s at 90 tokens/s per user against 6,362 for the H100. At Lambda's list prices the B200 costs 1.6 times as much per hour as an H100, for more than five times the throughput at that operating point. If you are going to self-host in late 2026, price Blackwell before you assume Hopper is the cheap option.

What does the API charge for the same model?

The fair comparison is the same open model served by someone else, not a frontier closed model. Together's serverless price for gpt-oss-120b is $0.15 per million input tokens and $0.60 per million output tokens. With the benchmark's 8:1 input to output mix, the blended price is (8 × 0.15 + 0.60) / 9 = $0.20 per million total tokens.

That is the bar. A provider serving thousands of customers keeps its GPUs busy around the clock, batches across tenants, and runs on hardware contracts you cannot get. Comparing your self-hosted gpt-oss against a frontier closed API at ten or fifty times the price is how the "ten times cheaper" spreadsheets get made: they are comparing two different products.

Self-hosting LLM vs API break-even: the utilisation math

Here is the whole calculation. Cost per million tokens at 100 percent busy is the hourly price divided by millions of tokens per GPU-hour. Break-even utilisation is that cost divided by the API price: the fraction of every hour your GPU must be doing useful work, at that operating point, for its cost to match paying per token.

# breakeven.py: self-hosting gpt-oss-120b vs a serverless API (Python 3.12)
# Throughput: InferenceX, 1x H100, vLLM, 8k in / 1k out, total tokens/s/GPU
# API: Together serverless gpt-oss-120b list price, October 2026
IN_PRICE, OUT_PRICE = 0.15, 0.60           # USD per 1M tokens
IN_TOK, OUT_TOK = 8, 1                     # same 8:1 mix as the benchmark
api_blended = (IN_TOK*IN_PRICE + OUT_TOK*OUT_PRICE) / (IN_TOK+OUT_TOK)

throughput = {50: 8973, 100: 5863, 200: 2558}   # tok/s/user -> tok/s/GPU
gpu_hour = {"SemiAnalysis owner TCO": 1.17, "InferenceX retail": 2.00,
            "Ornn H100 index": 2.59, "RunPod secure / Lambda 8x": 3.99,
            "Lambda 1x H100": 4.29}

print(f"API blended price: ${api_blended:.3f} per 1M tokens")
for speed, tps in throughput.items():
    m_tok_per_hour = tps * 3600 / 1e6
    print(f"\n{speed} tok/s per user -> {m_tok_per_hour:.1f}M tokens per GPU-hour")
    for name, price in gpu_hour.items():
        cost = price / m_tok_per_hour            # $ per 1M tokens at 100% busy
        be = cost / api_blended                   # utilisation needed to match API
        print(f"  {name:28s} ${price:.2f}/h  ${cost:.3f}/M  break-even {be:6.0%}")

Running it gives (trimmed):

API blended price: $0.200 per 1M tokens

50 tok/s per user -> 32.3M tokens per GPU-hour
  SemiAnalysis owner TCO       $1.17/h  $0.036/M  break-even    18%
  InferenceX retail            $2.00/h  $0.062/M  break-even    31%
  Ornn H100 index              $2.59/h  $0.080/M  break-even    40%
  RunPod secure / Lambda 8x    $3.99/h  $0.124/M  break-even    62%
  Lambda 1x H100               $4.29/h  $0.133/M  break-even    66%

100 tok/s per user -> 21.1M tokens per GPU-hour
  RunPod secure / Lambda 8x    $3.99/h  $0.189/M  break-even    95%
  Lambda 1x H100               $4.29/h  $0.203/M  break-even   102%

200 tok/s per user -> 9.2M tokens per GPU-hour
  SemiAnalysis owner TCO       $1.17/h  $0.127/M  break-even    64%
  InferenceX retail            $2.00/h  $0.217/M  break-even   109%

Read the chart as a question: can you keep a GPU that busy, at that latency, every hour of the month? At the rental price a small team pays, a single H100 needs to run at 62 to 66 percent of benchmark-saturated throughput around the clock just to tie with the API at 50 tokens/s per user. At 100 tokens/s per user, which is what a chat product or a coding agent feels like, it needs essentially 100 percent. At 200 tokens/s per user even the owner-TCO price needs 64 percent.

Put another way, $3,130 a month on a single H100 buys about 15.7 billion tokens at $0.20 per million. To beat the API on one GPU you need to push more than that through it, which is about 6,000 tokens per second averaged over every second of the month, nights and weekends included. Most internal workloads I see are bursty and diurnal. Their average is a small fraction of their peak, and you have to provision for the peak.

What are the hidden costs of self-hosting an LLM?

The break-even above is the optimistic case. It assumes one GPU, no redundancy and no people. In practice:

Redundancy doubles the floor. One replica means a node drain, a driver update or a bad rollout takes you down. Two replicas for availability means your break-even utilisation is now measured against twice the GPU-hours, and the second replica mostly idles.

Peak provisioning. If your traffic peaks at three times its average, you either provision for the peak and run at a third of capacity, or you autoscale. Autoscaling GPU pods is possible (I walk through it in the vLLM on Kubernetes guide) but a new replica has to pull a large image, load tens of gigabytes of weights and warm up before it serves anything, so you scale minutes behind demand, not seconds.

Software churn. vLLM ships fast. The v0.32.0 release on 18 October 2026 had 814 commits from 363 contributors, three security advisory fixes, a CORS default change and a list of breaking changes including dropping Python 3.10. Each of those is an upgrade you test, roll out and occasionally roll back. The model itself also changes: open model families now turn over every few months, and every swap means re-tuning memory, context length and parallelism.

People. Somebody owns the GPU Operator, the driver versions, the dashboards, the capacity plan and the 3 a.m. page. I am not going to give you a number for that, because loaded engineering rates vary too much, but put your own in. If an engineer spends ten hours a month on the stack, multiply by your hourly cost and add it to the GPU line. For a single-GPU deployment it is often bigger than the GPU bill. This is the same lesson as in FinOps for AI: the token line is the visible cost, not the whole one.

Benchmark gap. InferenceX tunes each configuration. Your deployment will run a different prompt mix, longer or shorter contexts, maybe a different quantisation, and probably less tuning. Expect to land below the published curve until you have done your own load test.

When does self-hosting an LLM make sense?

None of this means self-hosting is never right. It is right when one of these is true:

1. Sustained, predictable volume. Batch jobs, document processing, embeddings or evals that can run flat out overnight are where high utilisation is actually achievable. Queue the work and the GPU stays busy.
2. Hard data constraints. If prompts cannot leave your network or jurisdiction, the API price is irrelevant. Do the cost maths to size the cluster, not to decide whether to have one.
3. A model nobody serves. Fine-tuned weights, a niche architecture, or LoRA adapters per customer. Some providers host custom models, but often at dedicated-endpoint prices, which brings you back to paying for idle GPU-hours anyway.
4. Latency or control that serverless will not give you. Guaranteed throughput, no shared rate limits, pinned model versions.

In between sits the dedicated endpoint: Together lists a managed H100 at $5.49 an hour. You still pay for the GPU whether it is busy or not, so the utilisation maths is the same, but someone else carries the upgrade and on-call burden. For a team without a platform function, that premium over a raw $3.99 GPU is often the cheapest line in the comparison.

For laptops and single developer workstations the trade-off is different again, closer to a hardware purchase than a rental; the local LLMs post covers that.

FAQ

Is self-hosting an LLM cheaper than using an API?

Only when you keep the GPU busy. For gpt-oss-120b on a rented H100 at about $4 an hour, you need roughly 62 percent sustained utilisation at 50 tokens/s per user, and close to 100 percent at 100 tokens/s per user, to match a $0.20 per million token API price. Most interactive workloads do not reach that.

How many tokens per second can one H100 serve?

It depends on model, sequence lengths and the per-user speed you target. For gpt-oss-120b on vLLM with 8k input and 1k output, InferenceX measured about 8,973 total tokens/s per H100 at 50 tokens/s per user, falling to 2,558 at 200 tokens/s per user. Smaller models serve far more; larger dense models far less.

How do I calculate cost per million tokens for a self-hosted model?

Divide the GPU's hourly price by the millions of tokens it processes per hour at your target latency, then divide by your real average utilisation. A $4.29 GPU doing 32.3 million tokens an hour costs $0.133 per million at 100 percent busy and $0.266 at 50 percent.

What GPU price should I use in the calculation?

The one you will actually pay for the commitment you will actually make. Benchmark sites often use ownership costs around $1.17 an hour for an H100; single-GPU on-demand rentals in October 2026 were $3.99 to $4.29. The difference alone moves break-even from under 20 percent to over 60 percent.

This quarter, if I were the one deciding, I would start on a serverless API for the open model I wanted, log real token counts per hour for a month, and only then run the calculator against my own traffic shape. If the average is a large, steady fraction of the peak, or the data cannot leave, I would stand up two replicas on Kubernetes with vLLM and load-test them against my prompt mix before committing to anything longer than on-demand. If it is not, I would keep paying per token and spend the engineering time somewhere it compounds.

Related: LLM Inference in 2026: Prefill Is a Different Machine Now, and the KV Cache Has a Disk Tier, GPU Clouds in 2026: H100s at $2.59 an Hour, $104B of CoreWeave Backlog on $35B of Debt, and a Six-Year Bet on Silicon and FinOps for AI in 2026: 98% Manage Token Spend, 3 in 4 Can't Prove the Value, GPUs Run at 5%, and FOCUS Gets a Token Column in 1.5.