Ollama: Getting Started and Model Management
You'll learn how to install Ollama, download models from its library, and run basic inference commands to serve LLMs locally on your machine.
What is Ollama?
Ollama is a user-friendly tool that simplifies running large language models on your local hardware. Think of it as a package manager for LLMs — similar to how Docker manages containers, Ollama manages local model installations and serving. It handles the complexity of model weights, configuration, and runtime, letting you focus on inference.
Installing Ollama
Installation is straightforward across platforms:
- macOS/Linux: Download from
ollama.aior use package managers - Windows: Native Windows support available
After installation, verify with:
ollama --version
Pulling Models from the Library
Ollama maintains a curated library of optimized models. To download a model:
ollama pull llama2
ollama pull mistral
ollama pull codellama
This downloads the model weights and prepares them for local inference. Models are stored efficiently and can be listed anytime:
ollama list
Running Basic Inference
Once a model is pulled, inference is simple:
ollama run llama2
This starts an interactive chat session. For single prompts:
ollama run llama2 "Explain quantum computing in one sentence"
You can also serve models as an API endpoint:
ollama serve
Then make HTTP requests to http://localhost:11434 for programmatic access.
Key Takeaway: Ollama transforms complex LLM deployment into three simple steps: install the tool, pull your desired model, and run inference commands — all managed locally without external API dependencies.