How do I use a local Ollama model in LangChain?

Install langchain-ollama , import ChatOllama , and create it with a local model name: ChatOllama(model="llama3.1") . It behaves like any other LangChain chat model, so it fits existing chains.

What package do I need for ChatOllama?

Install the dedicated langchain-ollama package. Import the model with from langchain_ollama import ChatOllama , and OllamaEmbeddings from the same package when you need embeddings.

Can local Ollama models call tools in LangChain?

Yes, if the model supports tool calling. Attach tools with llm.bind_tools([...]) and use a capable model like llama3.1 . The reply then exposes tool_calls describing what to run.

How do I use Ollama with LangGraph?

LangGraph uses the same ChatOllama object as the model inside a node. Bind tools with bind_tools , wire the node into your graph, and swap models by changing a single line.

Build / AI / Ollama

Ollama With LangChain

Use the ChatOllama wrapper to drop local models into LangChain and LangGraph agent pipelines.

TL;DR

Wrap a local model with LangChain's ChatOllama class.
Install the langchain-ollama package before importing the ChatOllama class.
Add tools to a model with bind_tools for agents.

Jump to section

Install And Import
Basic Usage
Tool Calling
LangGraph Nodes
Tips
Warnings
In Practice
FAQ

Install And Import

Install

Add the dedicated LangChain Ollama package.

pip install langchain-ollama

Import ChatOllama

Bring in the chat model wrapper.

from langchain_ollama import ChatOllama

Embeddings

Import the embeddings class when needed.

from langchain_ollama import OllamaEmbeddings

Basic Usage

Create The Model

Instantiate ChatOllama with a local model name.

llm = ChatOllama(model="llama3.1")

Invoke

Send a prompt and get a message back.

resp = llm.invoke("Explain RAG briefly")

Read Content

Access the reply text on the content field.

print(resp.content)

Tool Calling

bind_tools

Attach tools so the model can call them.

model = llm.bind_tools([get_weather])

JSON Mode

Force JSON output for structured parsing.

ChatOllama(model="llama3.1", format="json")

Temperature

Set sampling options on the constructor.

ChatOllama(model="llama3.1", temperature=0)

LangGraph Nodes

Same Wrapper

Use ChatOllama as the model inside a graph.

llm = ChatOllama(model="llama3.1")

Bind In A Node

Give a graph node tool access via bind_tools.

node_llm = llm.bind_tools(tools)

Swap Models

Change one line to switch local models.

ChatOllama(model="qwen2.5-coder")

Tips

Use ChatOllama anywhere LangChain expects a chat model, so a local model swaps into existing chains and graphs unchanged.
Enable structured output with format="json" or bind_tools, which helps smaller local models return parseable, reliable results.

Warnings

Tool calling depends on the model; pick a tool-capable model like llama3.1, or bind_tools may be ignored silently.
Point ChatOllama at a running server; without ollama serve, calls fail with a connection error to localhost:11434.

In Practice

Wrap a local model, bind a tool, and let the model decide to call it in a LangChain flow.

The @tool decorator turns a Python function into a callable tool.
ChatOllama runs the model locally with no API key.
bind_tools gives the model the tool's schema to call.
The reply's tool_calls lists what the model chose to run.

from langchain_ollama import ChatOllama
from langchain_core.tools import tool

@tool
def add(a: int, b: int) -> int:
    """Add two numbers."""
    return a + b

llm = ChatOllama(model="llama3.1")
agent = llm.bind_tools([add])

resp = agent.invoke("What is 12 plus 30?")
print(resp.tool_calls)