With over 100,000 Docker Pulls, Ollama is hihgly popular, lightweight, extensible framework for building and running language models on the local machine. With Ollama, all your interactions with large language models happen locally without sending private data to third-party services. It provides a simple API for creating, running, and managing models, as well as a library of pre-built models that can be easily used in a variety of applications.
Some of the key features of Ollama include:
- Ease of use: Ollama is easy to install and use, even for users with no prior experience with language models.
- Extensible: Ollama is highly extensible, allowing users to create their own custom models or import models from other sources.
- Powerful: Ollama models can generate text, translate languages, write different kinds of creative content, and answer your questions in an informative way.
Ollama is a valuable tool for researchers, developers, and anyone who wants to experiment with language models. It is also a great platform for building language-based applications, such as chatbots, summarization tools, and creative writing assistants.
Benefits of using Ollama
Here are some of the benefits of using Ollama:
- Reduced development time: Ollama’s simple API and library of pre-built models can save developers a significant amount of time and effort.
- Improved model performance: Ollama’s extensible architecture allows developers to fine-tune models for specific tasks or applications.
- Increased flexibility: Ollama can be used to build a wide variety of language-based applications, limited only by the developer’s imagination.
If you are looking for a powerful, easy-to-use, and extensible framework for building language models, Ollama is a great option. With its wide range of features and benefits, Ollama can help you create innovative and effective language-based applications.
Ollama is now available as an official Docker image
Ollama is now available as an official Docker sponsored open-source image, making it simpler to get up and running with large language models using Docker containers.
Ollama handles running the model with GPU acceleration. It provides both a simple CLI as well as a REST API for interacting with your applications.
To get started, simply download from this link and install Ollama.
Note: Ollama Team recommend running Ollama alongside Docker Desktop for macOS in order for Ollama to enable GPU acceleration for models.
Ollama can run with GPU acceleration inside Docker containers for Nvidia GPUs.
To get started using the Docker image, please use the commands below.
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
Install the Nvidia container toolkit.
Run Ollama inside a Docker container
docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
Run a model
Now you can run a model like Llama 2 inside the container.
docker exec -it ollama ollama run llama2
curl https://ollama.ai/install.sh | sh
Ollama has active Discord community with over 9000+ community members
As published in their GitHub repo, Ollama supports a list of open-source models available on ollama.ai/library
Here are some example open-source models that can be downloaded:
ollama run mistral
ollama run llama2
ollama run codellama
|Llama 2 Uncensored
ollama run llama2-uncensored
|Llama 2 13B
ollama run llama2:13b
|Llama 2 70B
ollama run llama2:70b
ollama run orca-mini
ollama run vicuna
Note: You should have at least 8 GB of RAM to run the 3B models, 16 GB to run the 7B models, and 32 GB to run the 13B models.
Web & Desktop
- HTML UI
- Chatbot UI
- Typescript UI
- Minimalistic React UI for Ollama Models
- Web UI
- Cheshire Cat assistant framework
- LangChain and LangChain.js with example
- LangChainGo with example
- OllamaSharp for .NET
- Ollama-rs for Rust
- Ollama4j for Java
- ModelFusion Typescript Library
- OllamaKit for Swift
- Ollama for Dart
- Ollama for Laravel
- Maid (Mobile Artificial Intelligence Distribution)