<aside> ℹ️

Difficulty: Intermediate • Time: 30 minutes • Tools: Ollama

</aside>

API costs add up. Every classification, every summary, every boilerplate draft — each one burns tokens on a cloud model that's overqualified for the job. It's like hiring a senior architect to sort your mail.

The fix is straightforward: run a smaller model on your own machine for the mechanical stuff, and save the cloud API budget for work that actually requires reasoning. A 7-billion parameter model running locally handles summarization, classification, data extraction, and first drafts surprisingly well — and costs nothing per token after you've downloaded it.

By the end of this guide, you'll have a local model running on your machine, know which tasks to route to it, and have a clear mental model for when local is the right call versus when you need the cloud.

The starter setup

Run these in order. Takes about 10 minutes including the download.

# 1. Install Ollama (macOS)
brew install ollama

# 2. Start the service
ollama serve

# 3. Verify it's running
curl <http://localhost:11434/api/tags>

# 4. Pull a solid starter model
ollama pull qwen2.5-coder:7b

# 5. Test it interactively
ollama run qwen2.5-coder:7b
# Type a question, then /bye to exit.

A drop-in bash helper:

local_llm() {
  curl -s <http://localhost:11434/v1/chat/completions> \
    -H "Content-Type: application/json" \
    -d "{
      \"model\": \"qwen2.5-coder:7b\",
      \"messages\": [{\"role\": \"user\", \"content\": \"$1\"}]
    }" | jq -r '.choices[0].message.content'
}

# Usage:
local_llm "Summarize this in one sentence: $(cat meeting-notes.txt)"

What you'll need

• A machine with at least 8GB of RAM — 16GB is better if you want to run larger models. Apple Silicon Macs are particularly good at this because the unified memory architecture lets the GPU access all available RAM.
• Basic terminal comfort — you'll be running a few commands, nothing complex.
• Ollama — the tool that makes all of this painless.

Install Ollama

Ollama wraps the complexity of running local models into a few simple commands. No Python environments, no dependency management, no CUDA configuration.

macOS (Homebrew):

brew install ollama

Linux: