Ollama has become one of the most popular ways to run large language models locally, and its official Python library makes it simple to integrate local LLMs directly into your Python applications. In this guide, you’ll learn how to install, configure, and use the Ollama Python library to generate text, build chat applications, stream responses, work with embeddings, and create custom workflows — all running on your own machine.
What Is the Ollama Python Library?
The Ollama Python library is the official client that lets you interact with a running Ollama server using simple Python function calls instead of raw HTTP requests. It wraps Ollama’s REST API in a clean, Pythonic interface, supporting both synchronous and asynchronous usage, streaming output, chat-style conversations, tool calling, and embeddings generation.
Because Ollama runs models locally, using the Python library means your prompts and data never leave your machine unless you explicitly configure it otherwise — an appealing option for privacy-sensitive or offline applications.
Prerequisites
Before you start, you’ll need Ollama installed and running on your system, along with Python 3.8 or later. You’ll also want at least one model pulled locally, such as llama3.2 or mistral, since the Python library talks to models that Ollama has already downloaded.
Installing Ollama and the Python Library
First, install Ollama itself from the official site for your operating system (macOS, Linux, or Windows), then start the Ollama service, which typically runs automatically after installation and listens on localhost port 11434. If you want a broader walkthrough that also covers the REST API and Docker-based setups, see our Getting Started with Ollama guide.
Next, install the Python package using pip:
pip install ollamaThen pull a model through the Ollama CLI so it’s available locally:
ollama pull llama3.2Your First Request: Generating Text
The simplest way to use the library is the generate function, which sends a prompt to a model and returns a single completion.
import ollama
response = ollama.generate(
model='llama3.2',
prompt='Explain what a Python decorator is in one paragraph.'
)
print(response['response'])The response object contains the generated text along with metadata such as the model used, total duration, and token counts, which can be useful for performance monitoring.
Building Chat Applications with the Chat Function
For conversational use cases, the chat function is more appropriate than generate, since it accepts a list of messages with roles (system, user, assistant) and maintains conversational context.
import ollama
messages = [
{'role': 'system', 'content': 'You are a helpful coding assistant.'},
{'role': 'user', 'content': 'How do I read a CSV file in Python?'}
]
response = ollama.chat(model='llama3.2', messages=messages)
print(response['message']['content'])You can extend the conversation by appending the assistant’s reply and the next user message to the same list, allowing the model to retain context across multiple turns. For more hands-on Python examples, check out our step-by-step Ollama Python integration guide.
Streaming Responses
For longer generations, streaming lets you display output incrementally instead of waiting for the entire response to finish, which greatly improves perceived responsiveness in interactive applications.
import ollama
stream = ollama.chat(
model='llama3.2',
messages=[{'role': 'user', 'content': 'Write a short poem about the ocean.'}],
stream=True
)
for chunk in stream:
print(chunk['message']['content'], end='', flush=True)Each chunk contains a small piece of the generated text, and iterating over the stream lets you print or process tokens as they arrive.
Working with the Ollama Client Class
Instead of calling module-level functions, you can instantiate a Client object, which is useful when you need to connect to a remote Ollama server or customize connection settings such as timeouts and headers.
from ollama import Client
client = Client(host='http://localhost:11434')
response = client.chat(
model='llama3.2',
messages=[{'role': 'user', 'content': 'What is retrieval-augmented generation?'}]
)
print(response['message']['content'])Asynchronous Usage with AsyncClient
For applications built on asyncio, such as web servers or concurrent pipelines, the library provides an AsyncClient that mirrors the synchronous API but supports await and concurrent requests.
import asyncio
from ollama import AsyncClient
async def main():
client = AsyncClient()
response = await client.chat(
model='llama3.2',
messages=[{'role': 'user', 'content': 'Summarize the plot of a mystery novel in two sentences.'}]
)
print(response['message']['content'])
asyncio.run(main())Async streaming works similarly, using async for to iterate over chunks as they arrive.
Generating Embeddings
Beyond text generation, Ollama supports embedding models, which convert text into numerical vectors useful for search, clustering, and retrieval-augmented generation systems.
import ollama
response = ollama.embeddings(
model='nomic-embed-text',
prompt='The quick brown fox jumps over the lazy dog.'
)
vector = response['embedding']
print(len(vector))These embeddings can be stored in a vector database and compared using cosine similarity to build semantic search or RAG pipelines entirely offline. To see embeddings used in a full pipeline, read our guide to building RAG applications with Ollama and Python.
Tool Calling and Structured Output
Newer versions of Ollama support tool calling, allowing a model to request that your code execute a function and return the result, which the model then incorporates into its response. This is defined by passing a tools list describing available functions, similar to function calling in other LLM APIs, and is useful for building agents that can perform calculations, look up data, or interact with external systems. For a deeper dive into building tool-using agents, see our guide to function calling with Ollama.
Managing Models Programmatically
The library also exposes functions for model management, so you can list installed models, pull new ones, or delete models you no longer need directly from Python rather than the command line.
import ollama
models = ollama.list()
for model in models['models']:
print(model['name'])This is helpful when building applications that need to check model availability or automate environment setup. If you’d rather manage models declaratively, check out our Ollama models setup guide with Docker Compose.
Common Issues and Troubleshooting
If you get a connection error, confirm the Ollama service is actually running and listening on the expected host and port. If a model response is unexpectedly slow, remember that performance depends heavily on your hardware, particularly available RAM and whether you have a compatible GPU. If you see a “model not found” error, make sure you’ve pulled the model with the Ollama CLI before referencing it in Python. For quick command references, bookmark our Ollama cheatsheet.
Frequently Asked Questions
Is the Ollama Python library free to use? Yes, it’s open source and free, since it simply wraps calls to your local Ollama installation.
Does it require an internet connection? No, once a model is downloaded, generation happens entirely offline on your machine.
Can I use it with FastAPI or Flask? Yes, both the synchronous and async clients integrate well into typical Python web frameworks for building local LLM-powered APIs.
What’s the difference between generate and chat? Generate is for single-turn prompt completion, while chat is designed for multi-turn conversations with role-based messages.
Conclusion
The Ollama Python library gives developers a straightforward way to run powerful language models locally, whether for simple text generation, multi-turn chat, embeddings-based search, or more advanced agentic workflows with tool calling. Because everything runs on your own hardware, it’s a strong choice for privacy-conscious projects, offline applications, and rapid local prototyping before moving to production infrastructure.
