Models¶

1. Why this matters¶

Without LangChain, switching from OpenAI to Anthropic means rewriting API call code, error handling, and message formatting. With LangChain, you swap one line:

# OpenAI
model = ChatOpenAI(model="gpt-4o-mini")
# Anthropic
model = ChatAnthropic(model="claude-haiku-4-5-20251001")
# Google
model = ChatGoogleGenerativeAI(model="gemini-2.0-flash")

The rest of your chain doesn't change. That's the entire value proposition of the model layer.

2. Mental model¶

A LangChain model is a stateless function with retries and streaming built in:

Input: a list of messages (or a string, which gets auto-wrapped in a HumanMessage).
Output: an AIMessage (with .content, .usage_metadata, and possibly .tool_calls).
It's a Runnable, so it has .invoke, .stream, .batch, .ainvoke.

Three categories:

Category	Input	Output	Use when
Chat Model	`list[BaseMessage]`	`AIMessage`	Always, in 2025+
LLM (legacy)	`str`	`str`	Old code only — deprecated
Embedding Model	`str` or `list[str]`	`list[float]`	RAG, semantic search

3. Architecture / Flow¶

flowchart LR
    subgraph SG1 [Chat Model]
      A1[SystemMessage<br/>HumanMessage<br/>AIMessage list] --> B1[Provider API<br/>OpenAI / Anthropic / ...]
      B1 --> C1[AIMessage]
    end

    subgraph SG2 [Embedding Model]
      A2[Text strings] --> B2[Embedding API]
      B2 --> C2[Vectors<br/>e.g. 1536 floats]
      C2 --> D2[Vector DB]
    end

4. Core concepts¶

Provider package — every provider has its own pip package: langchain_openai, langchain_anthropic, langchain_google_genai, langchain_huggingface, langchain_ollama.
ChatOpenAI(model=..., temperature=..., max_tokens=...) — instantiate the model with config.
SystemMessage / HumanMessage / AIMessage / ToolMessage — the four message types.
AIMessage.content — the text response. .usage_metadata has token counts. .tool_calls has any tools the model wants to call.
Embeddings: .embed_query(text) for queries, .embed_documents([texts]) for bulk.
Open-source via HuggingFace / Ollama — same interface, just point at a local model.

5. Code — minimal working example¶

from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, SystemMessage
from dotenv import load_dotenv

load_dotenv()

model = ChatOpenAI(model="gpt-4o-mini", temperature=0.7)

response = model.invoke([
    SystemMessage(content="You explain things to a 10-year-old."),
    HumanMessage(content="What is gravity?"),
])

print(response.content)
print(response.usage_metadata)  # {'input_tokens': 25, 'output_tokens': 42, ...}

Embeddings — same shape:

from langchain_openai import OpenAIEmbeddings

emb = OpenAIEmbeddings(model="text-embedding-3-small")  # 1536 dims
vec = emb.embed_query("how big is Mars?")
print(len(vec))  # 1536

6. Code — real-world pattern¶

Swappable providers + a local fallback via Ollama:

from langchain_openai import ChatOpenAI
from langchain_anthropic import ChatAnthropic
from langchain_ollama import ChatOllama
from langchain_core.messages import HumanMessage
import os

def get_model():
    if os.getenv("OPENAI_API_KEY"):
        return ChatOpenAI(model="gpt-4o-mini", temperature=0)
    if os.getenv("ANTHROPIC_API_KEY"):
        return ChatAnthropic(model="claude-haiku-4-5-20251001", temperature=0)
    # local fallback — requires `ollama serve` running
    return ChatOllama(model="llama3.2")

model = get_model().with_retry(stop_after_attempt=3)

reply = model.invoke([HumanMessage(content="Summarize LangChain in 2 lines.")])
print(reply.content)

Streaming for chat UIs:

for chunk in model.stream("Write a haiku about retrieval."):
    print(chunk.content, end="", flush=True)

7. Common pitfalls¶

❗ Using OpenAI() instead of ChatOpenAI(). OpenAI is the legacy text-completion model (deprecated). For all modern GPT models use ChatOpenAI.
❗ Hard-coding API keys. Always use .env + python-dotenv (or your secret manager). Never commit keys.
❗ Setting temperature=0 everywhere by reflex. For classification/extraction yes; for creative writing or brainstorming, leave it at 0.7+.
❗ Forgetting that embedding models have a max token limit (~8k for OpenAI). Always chunk long documents before embedding.
❗ Mismatching embedding model between indexing and querying. If you indexed with text-embedding-3-small, you must query with the same model — different models live in different vector spaces.

8. When to use vs not use¶

Use the LangChain model layer when	Use the raw SDK when
You want provider portability	You're locked to one provider anyway and want zero deps
You're composing with other LangChain components (LCEL)	You need an obscure provider feature LangChain doesn't expose
You want built-in retry / streaming / batch	You're tracing/debugging at the bytes level

9. Cheatsheet¶

Provider packages and class names:

Provider	Package	Chat class	Embedding class
OpenAI	`langchain_openai`	`ChatOpenAI`	`OpenAIEmbeddings`
Anthropic	`langchain_anthropic`	`ChatAnthropic`	— (use a different provider)
Google	`langchain_google_genai`	`ChatGoogleGenerativeAI`	`GoogleGenerativeAIEmbeddings`
HuggingFace	`langchain_huggingface`	`ChatHuggingFace`	`HuggingFaceEmbeddings`
Ollama (local)	`langchain_ollama`	`ChatOllama`	`OllamaEmbeddings`
Cohere	`langchain_cohere`	`ChatCohere`	`CohereEmbeddings`

Common constructor args:

ChatOpenAI(
    model="gpt-4o-mini",       # or "gpt-4o", "o1-mini", etc.
    temperature=0.0,           # 0 = deterministic, 1+ = creative
    max_tokens=512,            # cap output length
    top_p=1.0,
    streaming=True,            # enable .stream()
    timeout=60,
    max_retries=2,
)

Common Runnable shortcuts:

model.with_config(tags=["prod"])           # tag for tracing
model.with_retry(stop_after_attempt=3)     # retry wrapper
model.with_fallbacks([ChatAnthropic(...)]) # fail over to another model
model.bind(stop=["\n\n"])                  # bind partial args

10. Q&A — recall test¶

Q: Difference between OpenAI and ChatOpenAI? A: OpenAI wraps the deprecated /completions endpoint (string → string). ChatOpenAI wraps /chat/completions (messages → message). Use ChatOpenAI for any modern model.
Q: What does .invoke() return for a chat model? A: An AIMessage object. Get the text via .content, the tokens via .usage_metadata, any tool calls via .tool_calls.
Q: Why must indexing-time embedding model = query-time embedding model? A: Each model has its own vector space. Cosine similarity between vectors from different models is meaningless — they're not comparable.
Q: How do you switch from OpenAI to Anthropic in a chain? A: Change one line: model = ChatOpenAI(...) → model = ChatAnthropic(...). The chain (prompt | model | parser) is unchanged.
Q: How do you stream tokens? A: for chunk in model.stream(input): print(chunk.content, end=""). Each chunk is a partial AIMessageChunk.

Practice¶

What does this print?

Expected: True

# Temperature 0 = deterministic; higher = more random
deterministic_temp = 0.0
creative_temp = 1.5
print(deterministic_temp < creative_temp)

Use a low temperature for structured extraction tasks (not 1.5)

Expected: True

config_for_extraction = {"temperature": 1.5}   # bug: too high; use 0 for deterministic extraction
print(config_for_extraction["temperature"] <= 0.3)

Quiz — Quick check¶

What you remember

Q1. What does temperature=0 do?

Makes the model deterministic — same input gives same output
Crashes the model
Uses no GPU
Skips reasoning

Why: Temperature controls sampling randomness. 0 always picks the most likely next token; higher values introduce diversity (good for creative writing, bad for extraction).

Q2. Which is best for structured-output extraction?

temperature=1.5
temperature=0
temperature=0.7
Doesn't matter

Why: Extraction is "find the right answer", not "be creative". Low temperature reduces hallucination and makes the output deterministic.

Q3. What's the difference between .invoke() and .stream()?

invoke returns the whole response at once; stream yields chunks as they arrive
No difference
stream is deprecated
invoke is faster

Why: Use stream for UIs where you want to show tokens as they're generated. Use invoke for backend tasks where you only care about the final result.

Common doubts¶

Should I use ChatModels or LLMs?

Use ChatModels (ChatOpenAI, ChatAnthropic). The text-completion LLM interface is legacy; all modern providers are chat-based. Even single-turn prompts work via chat models with a single message.

How do I keep cost down?

(1) Choose smaller models for routine tasks (gpt-4o-mini, claude-haiku). (2) Set max_tokens to bound response length. (3) Cache identical prompts. (4) Use prompt compression for retrieval contexts. (5) Track usage via LangSmith.

How do I switch from OpenAI to Anthropic?

Change one import: from langchain_anthropic import ChatAnthropic, then model = ChatAnthropic(model="claude-..."). The rest of your chain (prompts, parsers, chains) keeps working. That's the whole point of the abstraction.