How to Fix LLM API 429 Rate Limit and 529 Overloaded Errors (2026)

This guide is for engineers whose LLM calls have started failing with 429 Too Many Requests, Anthropic's 529 overloaded_error or OpenAI's 503 server_is_overloaded, usually right after traffic grew or an agent started looping. By the end you will know which limit you hit, which errors are worth retrying and which never will be, and you will have a tested Python retry policy that honours retry-after, plus the three changes that actually make 429s rarer: fewer tokens per minute, batch for bulk work, and a fallback model behind a gateway. Everything was checked against the Claude and OpenAI docs on 10 November 2026, and the code was tested offline with tenacity 9.2.1, anthropic 2.7.0 and openai 3.48.0 on Python 3.12. You need Python 3.10 or later and an API key for at least one provider; nothing here needs a cluster.

What 429, 529 and 503 mean on each provider

The status codes look generic, but each provider uses them for several distinct conditions, and only some of them go away if you wait.

Status and type	Provider	Meaning	Retry?	Source
429 rate_limit_error	Anthropic	RPM, ITPM or OTPM limit exceeded, or an acceleration limit after a sharp traffic increase	Yes, after retry-after	Claude rate limits
429 rate_limit_error, error_code: enforced_spend_limit_reached	Anthropic	Tier monthly spend cap reached; no retry-after header	No, until next month or a tier change	Claude rate limits
529 overloaded_error	Anthropic	API overloaded across all users	Yes, with backoff	Claude errors
429, rate limit reached	OpenAI	RPM or TPM exceeded	Yes, after Retry-After	OpenAI error codes
429 rate_limit_error, code slow_down	OpenAI	Traffic ramped too fast, even if under RPM and TPM	Yes, and ramp more slowly	OpenAI rate limits
429, codes credit_balance_exhausted, *_spend_limit_exceeded, organization_usage_limit_exceeded	OpenAI	Billing, spend or usage cap	No	OpenAI error codes
503 service_unavailable_error, code server_is_overloaded	OpenAI	Model temporarily overloaded	Yes, after Retry-After	OpenAI rate limits

Two details matter. A user-set spend limit on Claude returns a 400 invalid_request_error, not a 429, so it will not trip your rate-limit handler at all. And OpenAI changed its error handling: rapid ramps that used to return 503 with slow_down now return 429 with slow_down, while true overload stays 503 with server_is_overloaded. Older handlers may be catching the wrong one.

Step 1: Find which limit you hit from the rate-limit headers

Both providers return the current state of your limits on every response, not just on errors. Anthropic measures requests per minute (RPM), input tokens per minute (ITPM) and output tokens per minute (OTPM) per model, and enforces them with a token bucket that refills continuously rather than resetting each minute. OpenAI measures RPM and TPM (plus daily variants on some models) at the organization and project level.

Dump the headers from one cheap request:

# Claude: anthropic-ratelimit-* headers (anthropic-version as in the current docs)
curl -sS -D - -o /dev/null https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2026-10-21" \
  -H "content-type: application/json" \
  -d '{"model":"claude-haiku-5-5","max_tokens":16,"messages":[{"role":"user","content":"ping"}]}' \
  | grep -iE '^(retry-after|anthropic-ratelimit|request-id)'

# OpenAI: x-ratelimit-* headers
curl -sS -D - -o /dev/null https://api.openai.com/v1/responses \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-6-luna","input":"ping","max_output_tokens":16}' \
  | grep -iE '^(retry-after|x-ratelimit|x-request-id)'

On Claude, look at anthropic-ratelimit-requests-remaining, anthropic-ratelimit-input-tokens-remaining and anthropic-ratelimit-output-tokens-remaining; the matching -reset headers are RFC 3339 timestamps. Token "remaining" values are rounded to the nearest thousand. On OpenAI, look at x-ratelimit-remaining-requests and x-ratelimit-remaining-tokens, whose -reset values are durations such as 1s or 6m0s, plus x-ratelimit-*-project-tokens when a project cap applies. Whichever remaining value reaches zero first is your bottleneck.

In application code, read the same headers from the SDK's raw response so you can log them next to your own request IDs:

# headers.py: log rate-limit headroom (anthropic 2.7.0)
import anthropic

client = anthropic.Anthropic()
raw = client.messages.with_raw_response.create(
    model="claude-haiku-5-5",
    max_tokens=16,
    messages=[{"role": "user", "content": "ping"}],
)
for h in ("anthropic-ratelimit-requests-remaining",
          "anthropic-ratelimit-input-tokens-remaining",
          "anthropic-ratelimit-output-tokens-remaining"):
    print(h, raw.headers.get(h))
message = raw.parse()  # the normal Message object
print("request id:", message._request_id)

For reference, the standard Claude limits for Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5 on the Start tier are 4,000 RPM, 2,000,000 ITPM and 400,000 OTPM each, rising to 13,000 RPM and 10,000,000 ITPM on Scale. OpenAI's Build tier gives Astra, Sol and Terra 5,000 RPM and 1,000,000 TPM, and Luna 2,000,000 TPM. Your console's limits page is the authority for your organization; new accounts can start in a lower evaluation tier.

Step 2: Separate retryable errors from spend caps

A retry loop that treats every 429 the same will spend its whole budget hammering an endpoint that has already told you it is closed for the month. Anthropic's spend-cap 429 carries error.details.error_code: enforced_spend_limit_reached and no retry-after, and the anthropic 2.7.0 SDK raises it as anthropic.EnforcedSpendLimitReachedError, a subclass of RateLimitError. OpenAI's billing 429s carry an error.code such as project_spend_limit_exceeded, which the SDK exposes as exc.code. Check those before anything else, alert a human, and stop.

Overload errors are different: Anthropic's 529 surfaces as anthropic.OverloadedError, a subclass of InternalServerError, and OpenAI's 503 as openai.InternalServerError. Both are worth retrying.

Step 3: Retry with exponential backoff, jitter and retry-after

Both official SDKs already retry 429 and 5xx responses twice by default, starting at 0.5 seconds and capping at 8 seconds, and both honour retry-after. For a chat UI that is often enough. For batch jobs and agents, you want more attempts, a total time budget and logging, and you want exactly one retry loop, so set max_retries=0 on the client and own the policy yourself. OpenAI's rate limit guide says the same about not nesting retry loops.

python3 -m venv .venv && . .venv/bin/activate
pip install tenacity==9.2.1 anthropic==2.7.0 openai==3.48.0

# llm_retry.py: one retry policy for Claude and OpenAI calls.
# Tested with tenacity 9.2.1, anthropic 2.7.0, openai 3.48.0 (Python 3.12).
import logging
import random

import anthropic
import openai
from tenacity import (
    before_sleep_log,
    retry,
    retry_if_exception,
    stop_after_attempt,
    stop_after_delay,
    wait_random_exponential,
)

log = logging.getLogger("llm")

# 429 codes that no amount of waiting fixes (OpenAI error-codes guide).
OPENAI_NO_RETRY_CODES = {
    "credit_balance_exhausted",
    "organization_spend_limit_exceeded",
    "project_spend_limit_exceeded",
    "organization_usage_limit_exceeded",
}


def is_retryable(exc: BaseException) -> bool:
    # Anthropic: the monthly spend cap is a 429 with no retry-after.
    if isinstance(exc, anthropic.EnforcedSpendLimitReachedError):
        return False
    if isinstance(exc, (anthropic.RateLimitError,        # 429
                        anthropic.InternalServerError,   # 5xx, includes 529 OverloadedError
                        anthropic.APIConnectionError)):  # includes APITimeoutError
        return True
    if isinstance(exc, openai.RateLimitError):           # 429
        return getattr(exc, "code", None) not in OPENAI_NO_RETRY_CODES
    return isinstance(exc, (openai.InternalServerError,  # 5xx, includes 503 server_is_overloaded
                            openai.APIConnectionError))


_backoff = wait_random_exponential(multiplier=1, max=60)  # full jitter, capped at 60 s


def wait_retry_after_or_backoff(retry_state) -> float:
    exc = retry_state.outcome.exception()
    response = getattr(exc, "response", None)
    if response is not None:
        value = response.headers.get("retry-after")
        try:
            # Treat retry-after as a floor and add up to 1 s of jitter.
            return float(value) + random.uniform(0, 1)
        except (TypeError, ValueError):
            pass
    return _backoff(retry_state)


llm_retry = retry(
    retry=retry_if_exception(is_retryable),
    wait=wait_retry_after_or_backoff,
    stop=stop_after_attempt(6) | stop_after_delay(120),
    before_sleep=before_sleep_log(log, logging.WARNING),
    reraise=True,
)

# Turn off SDK retries so the two loops do not multiply attempts.
claude = anthropic.Anthropic(max_retries=0)
oai = openai.OpenAI(max_retries=0)


@llm_retry
def ask_claude(prompt: str) -> str:
    msg = claude.messages.create(
        model="claude-sonnet-5-5",
        max_tokens=1024,
        messages=[{"role": "user", "content": prompt}],
    )
    return "".join(b.text for b in msg.content if b.type == "text")


@llm_retry
def ask_openai(prompt: str) -> str:
    resp = oai.responses.create(model="gpt-6.1-sol", input=prompt, max_output_tokens=1024)
    return resp.output_text

I tested the policy offline by raising fake SDK exceptions (a Claude 429 with retry-after: 3, a Claude 529, an OpenAI 429 slow_down) and recording the sleeps instead of sleeping. No network, no keys:

WARNING Retrying __main__.flaky in 3.84 seconds as it raised RateLimitError: rate limited.
WARNING Retrying __main__.flaky in 0.64 seconds as it raised OverloadedError: overloaded.
WARNING Retrying __main__.flaky in 3.77 seconds as it raised RateLimitError: slow down.
result: ok sleeps: [3.84, 0.64, 3.77]
project_spend_limit_exceeded retryable: False code attr: project_spend_limit_exceeded

The first sleep respected retry-after: 3 plus jitter; the next two used random exponential backoff, whose ceiling doubles each attempt up to 60 seconds. Jitter matters more than the exact numbers: without it, every worker that hit the limit at the same moment retries at the same moment.

One caveat for streaming. On both APIs an error can arrive as an event after the HTTP 200 has started (Anthropic's streaming docs show an overloaded_error event mid-stream, and OpenAI's guide warns against replaying a request after consuming output). Decide per endpoint whether a mid-stream failure should be retried from scratch, because a retry resends the full input and you pay for it again.

Step 4: Cap concurrency instead of waiting for the 429

Retries are recovery, not capacity. If you fan out 200 concurrent agent calls against a 4,000 RPM limit, a short burst can trip the limiter even though the average is fine; Anthropic notes that 60 RPM may be enforced as one request per second. A semaphore turns the burst into a queue:

# fanout.py: bounded concurrency with the async client (anthropic 2.7.0)
import asyncio
import anthropic

client = anthropic.AsyncAnthropic(max_retries=3)  # SDK retries are fine here
limit = asyncio.Semaphore(16)                     # tune to RPM and average latency

async def summarize(doc: str) -> str:
    async with limit:
        msg = await client.messages.create(
            model="claude-haiku-5-5",
            max_tokens=512,
            messages=[{"role": "user", "content": f"Summarize:\n\n{doc}"}],
        )
        # Haiku 5.5 thinks by default, so skip thinking blocks and join the text
        return "".join(b.text for b in msg.content if b.type == "text")

async def main(docs: list[str]) -> list[str]:
    return await asyncio.gather(*(summarize(d) for d in docs))

A useful starting value is RPM divided by 60, times the average request latency in seconds. Raise traffic gradually too: OpenAI's rule of thumb is that above 1 million TPM you should grow by no more than 50 percent every 15 minutes, and Anthropic warns about acceleration limits on sharp increases.

Step 5: Spend fewer tokens per minute

Most persistent 429s on modern models are token limits, not request limits, and the two providers count tokens differently enough to change the fix.

On Claude, ITPM counts only uncached input: input_tokens plus cache_creation_input_tokens, but not cache_read_input_tokens. Anthropic's own example is an 80 percent cache hit rate turning a 2,000,000 ITPM limit into 10,000,000 total input tokens a minute. Putting cache_control on the stable system prompt and tool definitions is the single biggest rate-limit fix for agents. OTPM counts only tokens actually generated, so a generous max_tokens costs nothing against the limit.

On OpenAI, the opposite holds for output: the rate limiter counts the larger of max_tokens and the estimated tokens of your request, so an oversized max_output_tokens on every call burns TPM you never use. Set it close to the real response size. The token and caching side of this is in the companion explainer on LLM tokens and pricing.

Step 6: Move bulk work to the Batch API

Anything that does not need an answer in seconds (evals, backfills, nightly classification) should not compete with interactive traffic. Claude's Message Batches API has its own limits shared across models, 100,000 requests per batch and 200,000 queued on the Start tier, and OpenAI states that the Batch API runs without affecting synchronous rate limits. Both bill batch at half price, so this fixes cost and 429s at once.

Step 7: Fall back to another model through a gateway

When a provider is overloaded, waiting longer does not help your users. A gateway can route around it. With LiteLLM (latest release v1.108.4 as I write), a fallback chain looks like this:

# config.yaml for LiteLLM proxy: retry, cool down, then fall back
model_list:
  - model_name: chat-primary
    litellm_params:
      model: anthropic/claude-sonnet-5-5
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: chat-fallback
    litellm_params:
      model: openai/gpt-6.1-sol
      api_key: os.environ/OPENAI_API_KEY

litellm_settings:
  num_retries: 2                                   # per model group
  request_timeout: 60
  fallbacks: [{"chat-primary": ["chat-fallback"]}] # rate limit and other errors

router_settings:
  allowed_fails: 3     # failures before a deployment is cooled down
  cooldown_time: 30    # seconds out of rotation

Test the fallback model on your own evals first; the same prompt can behave differently across vendors. The full proxy setup, with budgets and virtual keys, is in How to Set Up LiteLLM Proxy.

Troubleshooting

Symptom	Likely cause	Fix
429 on Claude with enforced_spend_limit_reached, no retry-after	Tier monthly spend cap	Stop retrying; request a tier increase; access resumes 00:00 UTC on the 1st
Claude returns 400 You have reached your specified API usage limits	Your own workspace or org spend limit	Raise or remove the limit in Billing; this is not a rate limit
OpenAI 429 with slow_down while under RPM and TPM	Ramp-rate limit after a traffic jump	Honour Retry-After, then grow traffic no faster than 50% per 15 minutes
OpenAI 429 project_spend_limit_exceeded or credit_balance_exhausted	Billing or spend cap	Fix limits or credits; exclude these codes from retries
529 overloaded_error bursts across all your models	Anthropic-wide load	Backoff with jitter; fall back to another model or provider after the time budget
429s only during agent runs, ITPM header near zero	Full history resent uncached every turn	Add prompt caching on system prompt, tools and history
OpenAI TPM exhausted with short answers	max_output_tokens set far above real output	Lower it to the expected response size
Retries fire in lockstep and fail again	No jitter, or SDK retries nested inside your own	Use random exponential wait; set max_retries=0 when you own the loop
429 even though dashboard average is well under limit	Bursts; limits enforced over sub-minute windows	Semaphore or queue to smooth concurrency
Batch jobs starve interactive traffic	Bulk work on the synchronous API	Move it to the Batch API, which has separate limits

FAQ

What is the difference between a 429 and a 529 on the Claude API?

A 429 rate_limit_error means your organization exceeded one of its own limits (RPM, ITPM, OTPM, an acceleration limit or the monthly spend cap). A 529 overloaded_error means the API itself is overloaded across all users, so it is not about your quota. Retry both with backoff, except the spend-cap 429.

Does the OpenAI or Anthropic SDK retry 429 errors automatically?

Yes. Both Python SDKs retry connection errors, 408, 409, 429 and 5xx responses twice by default with exponential backoff between 0.5 and 8 seconds, and both read retry-after. Change it with max_retries on the client, and set it to 0 if you wrap calls in your own retry policy.

Do cached tokens count toward rate limits?

On Claude, no: cache reads are excluded from ITPM, while cache writes and uncached input count. OpenAI's rate limit guide counts tokens per minute against the larger of max_tokens and the estimated request size, and does not describe a similar cache exemption, so plan TPM on total tokens there.

How long should I wait before retrying a rate-limited request?

If the response has a retry-after header, wait at least that many seconds and add a little random jitter. If it does not, use random exponential backoff with a cap (60 seconds is reasonable) and a total time budget, then fall back or queue the work instead of retrying forever.

What I check first

When a 429 page goes off, I look at the error code before the graph: spend caps and billing codes are a phone call, not an engineering problem. Then I read the remaining headers to see which meter ran dry, and more often than not the fix is caching or a smaller max_output_tokens, not a higher tier. Retries with jitter are the seatbelt; concurrency limits, batch and a tested fallback are the brakes. The provider docs are the source of truth and change often: Claude rate limits, Claude errors, OpenAI rate limits, OpenAI error codes and tenacity. For SLOs around provider outages, see SRE for LLM Apps.

Related: How to Set Up LiteLLM Proxy: Budgets, Fallbacks and Virtual Keys (2026), AI Gateways in 2026 and SRE for LLM Apps in 2026.