Course contentsShow
← AI Engineering
Lesson 1050 of 1,88626. Self-Hosted LLM DeploymentFree lesson

Ollama: Getting Started and Model Management

Installing Ollama, pulling models from the library, and basic inference commands for local LLM serving.

Ollama: Getting Started and Model Management

You'll learn how to install Ollama, download models from its library, and run basic inference commands to serve LLMs locally on your machine.

What is Ollama?

Ollama is a user-friendly tool that simplifies running large language models on your local hardware. Think of it as a package manager for LLMs — similar to how Docker manages containers, Ollama manages local model installations and serving. It handles the complexity of model weights, configuration, and runtime, letting you focus on inference.

Installing Ollama

Installation is straightforward across platforms:

  • macOS/Linux: Download from ollama.ai or use package managers
  • Windows: Native Windows support available

After installation, verify with:

ollama --version

Pulling Models from the Library

Ollama maintains a curated library of optimized models. To download a model:

ollama pull llama2
ollama pull mistral
ollama pull codellama

This downloads the model weights and prepares them for local inference. Models are stored efficiently and can be listed anytime:

ollama list

Running Basic Inference

Once a model is pulled, inference is simple:

ollama run llama2

This starts an interactive chat session. For single prompts:

ollama run llama2 "Explain quantum computing in one sentence"

You can also serve models as an API endpoint:

ollama serve

Then make HTTP requests to http://localhost:11434 for programmatic access.

Key Takeaway: Ollama transforms complex LLM deployment into three simple steps: install the tool, pull your desired model, and run inference commands — all managed locally without external API dependencies.