Output Parsers & Structured Output¶

1. Why this matters¶

LLM raw output is a string. Your app needs typed data — a Python dict, a Pydantic object, a list. Without a parser, every chain ends in fragile json.loads() calls that crash on the slightest hallucination.

Output parsers:

Inject format instructions into the prompt (so the model knows what shape to emit).
Parse and validate the response (so downstream code gets typed data).
Retry / repair when parsing fails (some parsers can re-prompt to fix invalid output).

2. Mental model¶

Two strategies, pick based on the model:

flowchart TD
    Q{Does the model support<br/>tool/function calling?<br/>(GPT-4, Claude, Gemini)} -->|Yes| A[model.with_structured_output Schema<br/>Native, reliable, no parsing needed]
    Q -->|No / open-source| B[prompt with format_instructions<br/>+ PydanticOutputParser]
    style A fill:#e8f5e9
    style B fill:#fff4e5

Modern path (preferred): with_structured_output — the model returns valid structured data natively via tool-calling. Zero parsing logic in your code.
Classic path: Append format instructions to the prompt, parse the response. Works with any model, but more brittle.

3. Architecture / Flow¶

flowchart LR
    P[PromptTemplate<br/>with format_instructions] --> M[Chat Model]
    M --> S[Raw string]
    S --> PA[Parser<br/>.parse]
    PA --> O[Typed object<br/>Pydantic / dict / list]

    F[Parser.get_format_instructions] -.injected into.-> P

4. Core concepts¶

Parser	Output type	Use case
`StrOutputParser`	`str`	Default — strip out the `.content` field
`JsonOutputParser`	`dict` / `list`	Free-form JSON, no schema
`PydanticOutputParser`	a Pydantic model	Strict schema, validation, IDE autocomplete
`StructuredOutputParser`	`dict` with named fields	Lighter than Pydantic, good for prototyping
`CommaSeparatedListOutputParser`	`list[str]`	Quick lists
`OutputFixingParser`	wraps another parser	Auto-retries with the LLM if parsing fails
`RetryOutputParser`	wraps another parser	Same idea, slightly different retry strategy

The modern shortcut: ChatOpenAI(...).with_structured_output(MyPydanticModel) does it all internally — no separate parser needed.

5. Code — minimal working example¶

Just strip the .content:

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

prompt = ChatPromptTemplate.from_template("Tell me a joke about {topic}")
chain = prompt | ChatOpenAI(model="gpt-4o-mini") | StrOutputParser()
print(chain.invoke({"topic": "compilers"}))   # plain string

JSON parsing:

from langchain_core.output_parsers import JsonOutputParser

prompt = ChatPromptTemplate.from_template(
    "Return JSON with keys 'title' and 'year' for the movie {movie}."
)
chain = prompt | ChatOpenAI(model="gpt-4o-mini") | JsonOutputParser()
result = chain.invoke({"movie": "Inception"})
print(result["title"], result["year"])

6. Code — real-world pattern¶

The modern way — with_structured_output + Pydantic:

from pydantic import BaseModel, Field
from typing import Literal
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

# 1. Define your schema
class Review(BaseModel):
    sentiment: Literal["positive", "negative", "neutral"] = Field(
        description="Overall sentiment of the review"
    )
    key_themes: list[str] = Field(description="Main topics mentioned")
    summary: str = Field(description="One-sentence summary")

# 2. Bind the schema to the model — model now ALWAYS returns valid Review
model = ChatOpenAI(model="gpt-4o-mini").with_structured_output(Review)

prompt = ChatPromptTemplate.from_template(
    "Analyze this product review:\n\n{review_text}"
)

chain = prompt | model    # no parser needed!

result: Review = chain.invoke({"review_text": "Battery died in 2 days. Avoid."})
print(result.sentiment)   # → "negative"
print(result.key_themes)  # → ["battery life", "product quality"]

The classic way — PydanticOutputParser (works with any model, no native tool-calling needed):

from langchain_core.output_parsers import PydanticOutputParser
from langchain_core.prompts import PromptTemplate

parser = PydanticOutputParser(pydantic_object=Review)

prompt = PromptTemplate(
    template="Analyze this review.\n{format_instructions}\n\nReview: {review_text}",
    input_variables=["review_text"],
    partial_variables={"format_instructions": parser.get_format_instructions()},
)

chain = prompt | ChatOpenAI(model="gpt-4o-mini") | parser
result: Review = chain.invoke({"review_text": "Battery died in 2 days. Avoid."})

Auto-fixing parser (re-prompts the model if it returns invalid JSON):

from langchain.output_parsers import OutputFixingParser

robust = OutputFixingParser.from_llm(
    parser=parser,
    llm=ChatOpenAI(model="gpt-4o-mini"),
)
# Use `robust` in place of `parser` — on parse failure it asks the LLM to fix the output.

7. Common pitfalls¶

❗ Forgetting to inject format_instructions into the prompt. Without them, the model doesn't know what shape you expect — parsing fails.
❗ Using JsonOutputParser when you have a strict schema. Prefer PydanticOutputParser (or with_structured_output) — you get validation for free.
❗ Reaching for with_structured_output on a model that doesn't support tool-calling. Falls back to JSON-mode prompts on some providers, fails outright on others. Check the docs per provider.
❗ Putting Optional fields with no default in Pydantic. Use Optional[str] = None or the LLM may hallucinate values. Same for lists — default to [].
❗ Not setting temperature=0 for extraction tasks. Temperature 0.7 will give you a different schema every call. For structured output, always temperature=0.

8. When to use vs not use¶

Use	When
`StrOutputParser`	You want the response as plain text (chatbots, summaries)
`with_structured_output(Schema)`	Default for structured data with modern models
`PydanticOutputParser`	Open-source models, or providers without tool-calling
`JsonOutputParser`	Quick prototyping, schema not yet stable
`OutputFixingParser`	Flaky open-source model where you need retries

9. Cheatsheet¶

# StrOutputParser
from langchain_core.output_parsers import StrOutputParser

# JsonOutputParser
from langchain_core.output_parsers import JsonOutputParser
parser = JsonOutputParser()

# PydanticOutputParser
from langchain_core.output_parsers import PydanticOutputParser
parser = PydanticOutputParser(pydantic_object=MyModel)
# inject into prompt:
prompt = PromptTemplate(
    template="...{format_instructions}...",
    input_variables=[...],
    partial_variables={"format_instructions": parser.get_format_instructions()},
)

# StructuredOutputParser (lighter than Pydantic)
from langchain.output_parsers import StructuredOutputParser, ResponseSchema
schemas = [
    ResponseSchema(name="title", description="movie title"),
    ResponseSchema(name="year",  description="release year"),
]
parser = StructuredOutputParser.from_response_schemas(schemas)

# Modern shortcut — recommended
model = ChatOpenAI(...).with_structured_output(MyPydanticSchema)

# Auto-fixing wrapper
from langchain.output_parsers import OutputFixingParser
robust = OutputFixingParser.from_llm(parser=parser, llm=model)

10. Q&A — recall test¶

Q: What's the difference between with_structured_output and PydanticOutputParser? A: with_structured_output uses the provider's native tool/function calling — reliable, no extra prompt instructions. PydanticOutputParser injects schema instructions into the prompt text and parses the string response — works anywhere, less reliable.
Q: Why is StrOutputParser even useful — isn't the response already a string? A: Chat models return an AIMessage, not a string. StrOutputParser extracts .content so the next step in the chain gets a plain str.
Q: What does parser.get_format_instructions() return? A: A string with explicit JSON schema + examples, designed to be injected into the prompt so the model knows the exact output shape.
Q: How would you handle a model that occasionally returns invalid JSON? A: Wrap the parser in OutputFixingParser.from_llm(parser=..., llm=...). On parse failure, it asks the LLM to repair the output.
Q: Why set temperature=0 for parsers? A: Structured output is an extraction task, not a creative one. Temperature 0 gives the same output for the same input, which is what you want.

Practice¶

What does this print?

Expected: 42

import json
raw_llm_output = '{"answer": 42, "confidence": 0.9}'
data = json.loads(raw_llm_output)
print(data["answer"])

Parse the JSON safely (the LLM sometimes adds markdown code fences)

Expected: True

import json
llm_output = "```json\n{\"x\": 1}\n```"           # bug: can't json.loads directly with code fences
try:
    data = json.loads(llm_output)
    ok = True
except json.JSONDecodeError:
    ok = False
print(not ok)        # we expect json.loads to FAIL on the raw output

Quiz — Quick check¶

What you remember

Q1. What's the most reliable way to get JSON from an LLM?

Just ask for JSON in the prompt
Use with_structured_output(MySchema) — uses the provider's native function-calling under the hood
Regex extraction
Manual parsing

Why: Modern LLMs support structured output APIs — they're forced to return JSON matching your schema. Much more reliable than "please return JSON" and hoping for the best.

Q2. Why use Pydantic models for output parsing?

Required by LangChain
Provides validation, type hints, and clear schema definition
It's faster
Makes the prompt shorter

Why: Pydantic gives you a typed Python object back instead of a dict, with automatic validation. If the LLM returns wrong types, it raises a clear error.

Q3. What happens if the LLM returns malformed JSON?

Returns None
Parser raises an error — you catch it and either retry or use OutputFixingParser
Returns empty
LangChain auto-corrects

Why: Three strategies: (1) use native structured output (best), (2) wrap in OutputFixingParser which sends the error back to the LLM to fix, (3) catch and retry with a clearer prompt.

Common doubts¶

When to use PydanticOutputParser vs with_structured_output?

with_structured_output(MySchema) is the modern method — uses the provider's native structured-output APIs (function calling). PydanticOutputParser is older — instructs the LLM via prompt and parses the response. The native method is more reliable.

What's the difference between JsonOutputParser and StructuredOutputParser?

JsonOutputParser accepts any valid JSON. StructuredOutputParser validates against a schema (Pydantic). For production, always use schema-validated parsers — saves you from silent format drift.

How do I parse partial/streaming output as it arrives?

JsonOutputParser can parse incomplete JSON if the LLM is streaming. For Pydantic-based structured output, you typically wait for the complete response. Streaming structured output is still rough; most apps wait for the final result.