Running Models with Ollama

Build the mental model

Before tools like Ollama existed, running a local language model meant finding the right model file, matching it to a compatible runtime, converting formats, and hand-tuning settings just to get a single reply out of it. Ollama collapses that whole process into one command. Ollama is a command-line tool and background service that packages a model runtime, a model file downloader, and a local HTTP API into a single install.

Its core idea is convenience through sane defaults: instead of you choosing quantization levels, context length, and prompt formatting by hand, Ollama ships a “Modelfile” recipe for each model in its library that bakes in reasonable settings, so a beginner and an expert can both run the same model and get a similarly usable result on the first try.

ollama pull

Downloads a model's weights and Modelfile from Ollama's library to your machine, storing them locally so later runs don't re-download anything.

ollama run

Starts an interactive chat session with that model directly in your terminal, pulling it first automatically if it isn't already present.

ollama list

Shows every model you currently have stored locally, along with its size, so you can see what is taking up disk space.

ollama rm

Deletes a stored model to free that space back up.

Just as important as the CLI is what runs quietly behind it: whenever Ollama is running, it exposes a local HTTP API, by default at http://localhost:11434, that other programs on your machine can call directly, which is what turns Ollama from “a chat app” into “a local AI platform” other software can build on.

text

OLLAMA PULL TO CHAT LOOP
------------------------
------------------------------------------------------------
  $ ollama pull llama3.2
        |
        v
  LOCAL MODEL STORE  (weights + Modelfile saved on disk)
        |
        |  $ ollama run llama3.2
        v
  CHAT LOOP
    you type a prompt  ------------------>  model generates reply
        ^                                          |
        |                                          |
        +------------------- prints back <---------+

  meanwhile, in the background:
  Ollama service listening on http://localhost:11434
    (other programs can call this API at any time)

Connect it to a real scenario

Say you want a private coding assistant on a laptop with no internet dependency once set up.

Install Ollama

Install Ollama for your OS from ollama.com, which starts a background service automatically.

Pull the model

Open a terminal and run `ollama pull llama3.2`, which downloads that model's weights and Modelfile into Ollama's local storage; larger models simply take longer and use more disk space, but the command is identical regardless of size.

Run the model

Once the pull finishes, run `ollama run llama3.2` and you land directly in an interactive chat prompt in your terminal, ready to type questions and get replies without any additional setup.

List and remove models

If you want to see everything you've downloaded so far, perhaps because you tried a couple of different models while comparing quality, run `ollama list`, which prints each model's name, size on disk, and when it was last modified. When you decide a model isn't worth the disk space anymore, `ollama rm llama3.2` removes it cleanly, and a later `ollama pull` would fetch it fresh again if needed.

Throughout all of this, Ollama's local API at localhost:11434 is quietly available the whole time, which is what the next few lessons build on directly.

Ollama bundles a runtime, model downloader, and local API into one command-line tool with sane defaults.
pull, run, list, and remove cover the everyday model-management workflow.
Ollama exposes a local HTTP API by default, turning it into a platform other software can call, not just a chat app.

Try the working example

bash

# Download a model's weights and Modelfile into local storage
ollama pull llama3.2

# Start an interactive chat session with that model
ollama run llama3.2

# List every model currently stored locally, with size and date
ollama list

# Remove a stored model and free its disk space
ollama rm llama3.2

You should see

`ollama pull llama3.2` downloads the model's weights and Modelfile into Ollama's local storage on disk; it does not start a chat session by itself. `ollama run llama3.2` starts an interactive terminal chat with that model, pulling it first automatically if it isn't already present locally. `ollama list` prints a table of every model currently stored locally, showing each one's name, size on disk, and last-modified time. `ollama rm llama3.2` deletes the stored model and frees the disk space it was using; a later `ollama pull llama3.2` would re-download it from scratch.

5-minute try-it

Run `ollama pull` for a second, smaller model (for example `phi3` or `qwen2.5:0.5b`), then run `ollama list` and compare the two models' sizes on disk. Pick the smaller model with `ollama run`, ask it the same question you'd ask the first model, and note any difference in reply quality or speed. Finally, remove whichever model you don't plan to keep using `ollama rm`.

One important caution

Assuming `ollama run` blocks the local API from working — the API is available the whole time Ollama is running, chat session or not.

Forgetting that `ollama rm` only frees disk space and does not stop or uninstall the Ollama service itself.

Ollama README (CLI reference) — Local AI / Local LLM