How to Get Reliable JSON From LLMs With Pydantic (2026)

This guide is for Python developers who need an LLM to return data that code can trust: invoices, tickets, classifications, tool arguments. By the end you will have one Pydantic model driving OpenAI and Anthropic structured outputs, a validation-and-retry loop that handles refusals and truncation, streaming partial JSON for progress UIs, and the same model working against a local Ollama or vLLM server. Everything below was checked on 2026-10-31 against Python 3.12, pydantic 2.14.2, openai 3.42.0, anthropic 2.3.0 and instructor 1.17.0; the schema and validation output shown is real output from those versions. You need an API key for at least one provider and basic Pydantic familiarity. If you want the background on why constrained decoding works and where it fails, read the companion explainer, LLM Structured Outputs Explained.

Step 1: Install pinned packages

Pin versions. Structured output helpers are some of the fastest-moving parts of both SDKs, and a minor release can change how a schema is transformed.

python3.12 -m venv .venv && source .venv/bin/activate
pip install "pydantic==2.14.2" "openai==3.42.0" "anthropic==2.3.0"

# Instructor goes in its own environment for now (see Step 6):
# instructor 1.17.0 pins jiter<0.15, openai 3.42.0 needs jiter>=0.16
python3.12 -m venv .venv-instructor
.venv-instructor/bin/pip install "instructor==1.17.0" "pydantic==2.14.2"

That jiter conflict is real: when I asked uv to resolve instructor==1.17.0 alongside openai==3.42.0 it refused, and resolving Instructor alone pulled in openai==3.3.0. Check the Instructor releases before you merge the two environments.

Step 2: Define the Pydantic model once

The Pydantic model is your contract. It generates the JSON Schema you send to the provider and validates what comes back. Write descriptions; they are part of the prompt.

# models.py
from typing import Literal
from pydantic import BaseModel, Field, field_validator

class LineItem(BaseModel):
    sku: str = Field(description="Product SKU, e.g. AB-1234", pattern=r"^[A-Z]{2}-\d{4}$")
    quantity: int = Field(ge=1, le=1000)
    unit_price_cents: int = Field(ge=0)

class Invoice(BaseModel):
    invoice_id: str
    currency: Literal["USD", "EUR", "GBP"]
    # No default, so the key is required; None is an allowed value.
    due_date: str | None = Field(description="ISO 8601 date, or null if absent")
    items: list[LineItem]

    @field_validator("items")
    @classmethod
    def at_least_one(cls, v: list[LineItem]) -> list[LineItem]:
        if not v:
            raise ValueError("invoice must have at least one line item")
        return v

Three rules keep this portable across providers. Make every field required and express optionality as X | None, because OpenAI requires all properties in required (OpenAI supported schemas). Use Literal for enums. Keep the root an object; wrap lists in a parent model. Here is what Pydantic generates for due_date:

{"anyOf": [{"type": "string"}, {"type": "null"}], "description": "ISO 8601 date, or null if absent", "title": "Due Date"}

Step 3: Call OpenAI with responses.parse

The OpenAI SDK converts the model to a strict schema (adding additionalProperties: false everywhere), sends it as text.format, and parses the result.

# openai_extract.py
import os
from openai import OpenAI
from models import Invoice

client = OpenAI()  # reads OPENAI_API_KEY
MODEL = os.environ.get("OPENAI_MODEL", "gpt-6-luna")

def extract_openai(text: str) -> Invoice:
    resp = client.responses.parse(
        model=MODEL,
        input=[
            {"role": "system", "content": "Extract the invoice. Use null for missing dates."},
            {"role": "user", "content": text},
        ],
        text_format=Invoice,
        max_output_tokens=2000,
    )
    if resp.status == "incomplete":
        # e.g. incomplete_details.reason == "max_output_tokens": the JSON is cut off
        raise RuntimeError(f"incomplete response: {resp.incomplete_details}")
    for item in resp.output:
        if item.type != "message":
            continue
        for part in item.content:
            if part.type == "refusal":
                raise RuntimeError(f"model refused: {part.refusal}")
            if part.parsed is not None:
                return part.parsed
    raise RuntimeError("no parsed output")

resp.output_parsed is the shortcut when you do not care why parsing failed. In production you do care: a refusal and a truncation need different handling, and the docs say both can return output that does not follow your schema. The OpenAI SDK keeps minimum, maximum and pattern in the strict schema, since OpenAI supports them. I checked the generated schema for quantity:

{"maximum": 1000, "minimum": 1, "title": "Quantity", "type": "integer"}

Step 4: Call Anthropic with messages.parse

Claude's structured outputs are stable and use output_config.format; the older output_format request field only works with the structured-outputs-2025-11-13 beta header (Anthropic docs). The Python helper still takes a keyword called output_format and translates it for you.

# anthropic_extract.py
import os
from anthropic import Anthropic
from models import Invoice

client = Anthropic()  # reads ANTHROPIC_API_KEY
MODEL = os.environ.get("ANTHROPIC_MODEL", "claude-haiku-5-5")

def extract_anthropic(text: str) -> Invoice:
    resp = client.messages.parse(
        model=MODEL,
        max_tokens=2000,
        system="Extract the invoice. Use null for missing dates.",
        messages=[{"role": "user", "content": text}],
        output_format=Invoice,  # SDK sends output_config.format and validates the reply
    )
    if resp.stop_reason == "refusal":
        raise RuntimeError("model refused")
    if resp.stop_reason == "max_tokens":
        raise RuntimeError("truncated: raise max_tokens")
    return resp.parsed_output

The important detail is what the SDK does to your constraints. Anthropic's grammar does not support numeric or string-length constraints, so the SDK strips them and moves them into the description. In 2.3.0 it did the same to pattern, even though the API docs list simple regex as supported. This is the output of anthropic.transform_schema(Invoice) for two fields in SDK 2.3.0:

{"type": "integer", "title": "Quantity", "description": "{maximum: 1000, minimum: 1}"}
{"type": "string", "description": "Product SKU, e.g. AB-1234\n\n{pattern: ^[A-Z]{2}-\\d{4}$}", "title": "Sku"}

So the model is asked nicely, not forced. messages.parse then validates the text against your original model with a Pydantic TypeAdapter, which means a reply with quantity: 0 raises pydantic.ValidationError from inside the call. Catch it, which is what Step 5 is for. Also note that JSON outputs cannot be combined with citations or assistant prefill.

Step 5: Validate and retry with the error message

Constrained decoding fixes syntax. Your own constraints, cross-field rules and business logic still need a validator, and when it fails, the cheapest fix is to show the model the error and ask again.

Here is a provider-neutral version using Anthropic's raw output_config so you can see the full loop without the helper:

# retry_loop.py
import json
from anthropic import Anthropic, transform_schema
from pydantic import ValidationError
from models import Invoice

client = Anthropic()
SCHEMA = transform_schema(Invoice)  # Anthropic dialect: constraints moved to descriptions

def extract_with_retry(text: str, model: str = "claude-haiku-5-5", attempts: int = 3) -> Invoice:
    messages = [{"role": "user", "content": f"Extract the invoice:\n\n{text}"}]
    last_error = None
    for _ in range(attempts):
        resp = client.messages.create(
            model=model,
            max_tokens=2000,
            messages=messages,
            output_config={"format": {"type": "json_schema", "schema": SCHEMA}},
        )
        if resp.stop_reason in ("refusal", "max_tokens"):
            raise RuntimeError(f"stop_reason={resp.stop_reason}")  # retrying will not help
        raw = next(b.text for b in resp.content if b.type == "text")
        try:
            return Invoice.model_validate_json(raw)
        except ValidationError as e:
            last_error = e
            messages += [
                {"role": "assistant", "content": raw},
                {"role": "user", "content": f"That JSON failed validation:\n{e}\nReturn corrected JSON."},
            ]
    raise RuntimeError(f"validation failed after {attempts} attempts: {last_error}")

What the model sees on a retry is Pydantic's own error text, which is specific enough to act on. This is the real output for a bad response:

3 validation errors for Invoice
currency
  Input should be 'USD', 'EUR' or 'GBP' [type=literal_error, input_value='usd', input_type=str]
items.0.sku
  String should match pattern '^[A-Z]{2}-\d{4}$' [type=string_pattern_mismatch, input_value='ab-12', input_type=str]
items.0.quantity
  Input should be greater than or equal to 1 [type=greater_than_equal, input_value=0, input_type=int]

Two or three attempts is enough. If a field fails three times, the problem is the prompt, the schema, or the input, and more retries just spend money.

Step 6: Use Instructor for many providers

If you call several providers, or models without native structured outputs, Instructor wraps all of this: it builds the schema, picks a mode (tools, JSON schema, or plain JSON parsing), validates with Pydantic, and re-asks with the validation error up to max_retries times.

# instructor_extract.py  (run in .venv-instructor)
import instructor
from models import Invoice

client = instructor.from_provider("anthropic/claude-haiku-5-5")

invoice = client.chat.completions.create(
    response_model=Invoice,
    max_retries=2,  # re-asks with the ValidationError text on failure
    max_tokens=2000,
    messages=[{"role": "user", "content": "Invoice INV-9, 2 x AB-1234 at $5.00, USD, due 2026-12-01"}],
)
print(invoice.model_dump())

The trade-off is another layer between you and the provider. Instructor 1.17.0 exposes modes such as TOOLS, TOOLS_STRICT, JSON_SCHEMA, ANTHROPIC_TOOLS and GENAI_STRUCTURED_OUTPUTS (I listed instructor.Mode in the installed package). Pick the strict or native-structured mode explicitly when the provider supports it, rather than relying on prompt-only JSON parsing.

Step 7: Stream partial JSON

For long extractions you can show progress before the object completes. pydantic_core.from_json accepts allow_partial, which parses a truncated buffer into whatever is complete so far.

# stream_partial.py
from anthropic import Anthropic, transform_schema
from pydantic_core import from_json
from models import Invoice

client = Anthropic()
buf = ""
with client.messages.stream(
    model="claude-haiku-5-5",
    max_tokens=4000,
    messages=[{"role": "user", "content": "Extract the invoice:\n\n..."}],
    output_config={"format": {"type": "json_schema", "schema": transform_schema(Invoice)}},
) as stream:
    for chunk in stream.text_stream:
        buf += chunk
        try:
            partial = from_json(buf, allow_partial=True)  # dict of completed values only
        except ValueError:
            continue  # empty buffer
        print(f"items so far: {len(partial.get('items', []))}", end="\r")

invoice = Invoice.model_validate_json(buf)  # validate the final object, never the partial

I checked the edge cases with pydantic-core 2.50.2. An empty string raises ValueError. With allow_partial=True, a half-written string is dropped ('{"a":"xy' gives {}), while allow_partial="trailing-strings" keeps it ({'a': 'xy'}). For a buffer cut mid-object, the completed fields come through:

{'invoice_id': 'INV-9', 'currency': 'USD', 'due_date': None, 'items': [{'sku': 'AB-1234'}]}

Never act on a partial: items[0] above has no quantity yet. Instructor offers the same idea as Partial[Invoice] with stream=True.

Step 8: Constrained decoding with local models

Local servers use the same Pydantic model. Ollama takes a JSON Schema in the format field (version 0.40.13 at the time of writing):

# pip install "ollama==0.6.3"
from ollama import chat
from models import Invoice

resp = chat(
    model="qwen3:8b",
    messages=[{"role": "user", "content": "Extract the invoice:\n\n..."}],
    format=Invoice.model_json_schema(),
    options={"temperature": 0},  # Ollama's docs recommend 0 for structured output
)
invoice = Invoice.model_validate_json(resp.message.content)

Ollama's docs note that its cloud does not support this yet, and recommend also describing the schema in the prompt. For vLLM 0.32.0, send response_format={"type": "json_schema", "json_schema": {...}} through the OpenAI client, or extra_body={"structured_outputs": {"json": schema}}; the old guided_json field was removed in 0.12.0 (vLLM docs). Choose a backend with vllm serve ... --structured-outputs-config.backend xgrammar or guidance. Small local models follow grammars perfectly and still produce silly values, so the validation loop from Step 5 matters more here, not less. My Ollama setup guide covers getting the server running.

Troubleshooting

Symptom	Likely cause	Fix
OpenAI 400: schema invalid with strict	A property is missing from required, or additionalProperties is not false	Generate the schema with responses.parse(text_format=...); model optional fields as X | None
OpenAI: endless whitespace or error with JSON mode	json_object used without the word JSON in the prompt	Switch to json_schema structured outputs, or mention JSON in the system prompt
Anthropic 400 when using output_format	Deprecated top-level field sent without the beta header	Send output_config={"format": ...}, or use messages.parse(output_format=Model)
Anthropic 400: Schema is too complex for compilation	Compiled grammar too large, even within the explicit caps (24 optional params, 16 union-typed params, 20 strict tools)	Make fields required with null unions, split the schema, or reduce tools
ValidationError raised inside messages.parse	A stripped constraint (ge, le, pattern) was not respected	Catch it and run the Step 5 retry loop
JSON cut off mid-string	Hit max_tokens or max_output_tokens	Check stop_reason or status; raise the budget or shrink the schema
Response has no parsed object	Model refused	Check for refusal content (OpenAI) or stop_reason == "refusal" (Anthropic); log the input
Enum value has the wrong case	Claude may recapitalise enum strings	Normalise with a field_validator(mode="before"); avoid values differing only by case
Gemini ignores minLength or uniqueItems	Gemini strips those keywords silently	Enforce them in Pydantic validators
pip cannot install instructor with openai 3.x	jiter version pins conflict (instructor 1.17.0)	Use a separate environment or wait for a release that widens the pin

FAQ

Do I still need Pydantic validation if the provider guarantees the schema?

Yes. The provider guarantees the subset of your schema it supports, and SDKs strip constraints they cannot enforce. Pydantic also catches refusals and truncation that arrive as non-conforming text, and it is where cross-field business rules live.

Should I use Instructor or the native SDK helpers?

Use the native helpers (responses.parse, messages.parse) when you target one or two providers; they have fewer moving parts and expose stop reasons directly. Use Instructor when you switch providers often or need retries and partial streaming behind one interface.

How do I make a field optional in OpenAI structured outputs?

List it as required and allow null, which in Pydantic means field: str | None. OpenAI requires every property to appear in required. The SDK's strict converter handles this for you: with openai 3.42.0, a field declared as a: str | None = None came out as required with an anyOf string-or-null type and the default dropped. Hand-written schemas have to do the same.

Why does the first request with a new schema take longer?

Anthropic compiles the schema into a grammar on first use and caches it for 24 hours from last use, so expect a latency spike after any deploy that changes the model class, and warm the schema with one request if p99 matters. OpenAI's guide says only fine-tuned models pay this first-request cost there.

In production, the first thing I check when "the JSON broke" is the stop reason distribution, because most incidents turn out to be truncation or refusals rather than bad syntax. The second is whether a schema change shipped without an eval. I keep the Pydantic model, the retry count and the token budget in the same module, version it like an API contract, and alert on the validation-failure rate. Official references: OpenAI structured outputs, OpenAI function calling, Anthropic structured outputs, Gemini structured output, Pydantic docs and Instructor. For the agent side of the same problem, see building an agent with tool calling, and for catching regressions, LLM evals in CI.

Related: How to Run LLM Evals in CI with GitHub Actions (promptfoo, 2026), How to Defend LLM Apps Against Prompt Injection: 8 Steps (2026) and Running LLMs Locally in 2026: Ollama vs llama.cpp vs LM Studio vs MLX.