How to Run an AI Model on Your Own Computer With Ollama

Install Ollama, chat with a small open model in your terminal, call it from the local API, and give it a custom personality, all on your own computer.

Reviewed by Mindloop Editorial on . Every link in the Sources section at the end was re-checked that day.

Flat illustration of a laptop running a local AI chat, with a small chip icon and chat bubbles

Local AI models are having a moment. New open models land every few weeks, and “local vs cloud” is a running debate in hobbyist forums. This guide shows you how to install Ollama, chat with a small model in your terminal, call it from code, and give it a custom personality. Every command below comes from Ollama’s official docs.

What you need

Ollama is a free tool that downloads open models and runs them on your own machine. It gives you a command line and a local web API. Once a model is downloaded, your chats are processed on your computer.

Here is the checklist:

  • A supported operating system. The official docs list macOS Sonoma (v14) or newer, and Windows 10 22H2 or newer. Linux is supported through an install script.
  • Disk space. The Windows docs say the app needs at least 4GB for the binary install, and that models “can be tens to hundreds of GB in size”. Small models are far smaller than that.
  • Enough memory for the model you pick. A model has to fit in your GPU’s memory (VRAM) or your regular RAM. Each model page on the Ollama library lists its download size, which is a good first guide. Pick a model that is clearly smaller than the memory you have free.
  • A GPU helps, but isn’t required. On Macs, Apple M-series chips get CPU and GPU support, while Intel Macs run on the CPU only. On Windows and Linux, Ollama supports many NVIDIA and AMD cards. Check the GPU docs for your card.

Running on the CPU works. It’s just slower.

Step 1: Install Ollama

The download page offers a one-line installer for each system.

On macOS or Linux, open a terminal and run:

Terminal
curl -fsSL https://ollama.com/install.sh | sh

On Windows, open PowerShell and run:

Code
irm https://ollama.com/install.ps1 | iex

If you’d rather not pipe a script into your shell, you can download the app instead: Ollama.dmg for Mac or OllamaSetup.exe for Windows, both linked from the download page. On a Mac, drag the app into Applications. The first time it starts, it checks that the ollama command is in your PATH. If it isn’t, it asks permission to create a link in /usr/local/bin.

Check that it worked:

Code
ollama -v

You should see a version number. On Windows, Ollama runs in the background after install. The ollama command works in cmd, PowerShell or any terminal.

Step 2: Pick a small model

The library has hundreds of models, and many are big. For a first try, use Llama 3.2. Its tags page lists llama3.2:3b at 2.0GB and llama3.2:1b at 1.3GB. The plain llama3.2 name points to the 3B version. It’s also the model Ollama uses in many of its own API examples.

If your machine has little memory, start with llama3.2:1b. It’s smaller and lighter, with simpler answers.

Step 3: Run your first chat

One command downloads the model (if needed) and opens a chat:

Code
ollama run llama3.2

The first run shows a download progress bar. Then you get a prompt. The output looks roughly like this (abbreviated):

Code
pulling manifest
pulling ... 100%
success
>>> Explain what a CPU does in one sentence.
A CPU is the part of a computer that carries out instructions ...
>>> /bye

Type /bye to leave the chat. To paste text across several lines, wrap it in triple quotes (""").

Step 4: Manage your models

These four commands cover almost everything you’ll do day to day.

Download a model without starting a chat:

Code
ollama pull llama3.2:1b

See which models are on your disk (ollama ls does the same thing):

Code
ollama list

See which models are loaded in memory right now:

Code
ollama ps

The PROCESSOR column is worth a look. According to the FAQ, 100% GPU means the model is fully on the GPU. 100% CPU means it’s in system memory. A split like 48%/52% CPU/GPU means it’s partly in each. By default, models stay loaded for 5 minutes after you last use them. ollama stop llama3.2 unloads a model right away.

Delete a model you no longer need:

Code
ollama rm llama3.2:1b

Step 5: Talk to it through the local API

While Ollama is running, it serves an API at http://localhost:11434. Any app on your computer can use it. Here is a chat request, following the API reference:

Terminal
curl http://localhost:11434/api/chat -d '{
  "model": "llama3.2",
  "messages": [
    { "role": "user", "content": "Why is the sky blue?" }
  ],
  "stream": false
}'

Without "stream": false, you get a stream of small JSON chunks instead of one reply. With it, you get a single object shaped like this (trimmed):

Code
{
  "model": "llama3.2",
  "created_at": "...",
  "message": {
    "role": "assistant",
    "content": "The sky looks blue because ..."
  },
  "done": true
}

There is also /api/generate, which takes a single "prompt" string instead of a messages list. It returns the text in a "response" field.

Many tools already speak the OpenAI API format. Ollama’s OpenAI compatibility docs describe an endpoint at http://localhost:11434/v1/. You can call it like this:

Terminal
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.2",
    "messages": [{ "role": "user", "content": "Hello!" }]
  }'

OpenAI client libraries need an API key. The docs set it to ollama and note that it is “required but ignored”.

Step 6: Give the model a personality with a Modelfile

A Modelfile is a short text file that builds a custom model on top of an existing one. The Modelfile reference lists the instructions. FROM (required) sets the base model, PARAMETER changes settings, and SYSTEM sets the system message.

Create a file called Modelfile with this content:

Code
FROM llama3.2
PARAMETER temperature 0.7
SYSTEM """You are a patient tutor. Explain things in plain English, in short paragraphs, and give one example."""

Build it and run it:

Code
ollama create tutor -f ./Modelfile
ollama run tutor

Your new tutor model now shows up in ollama list.

Step 7: Change where models are stored, or who can reach the server

According to the FAQ, models are stored here by default:

  • macOS: ~/.ollama/models
  • Linux: /usr/share/ollama/.ollama/models
  • Windows: C:\Users\%username%\.ollama\models

To use another drive, set the OLLAMA_MODELS environment variable to the folder you want. How you set it depends on your system:

  • macOS app: run launchctl setenv OLLAMA_MODELS "/path/to/folder", then restart Ollama.
  • Linux (systemd): run systemctl edit ollama.service and add Environment="OLLAMA_MODELS=/path/to/folder" under [Service]. Then run systemctl daemon-reload and systemctl restart ollama. The ollama user also needs read and write access to the folder.
  • Windows: quit Ollama, then search Settings for “environment variables”. Add OLLAMA_MODELS to your user account and start Ollama again.

OLLAMA_HOST works the same way. By default, Ollama only listens on 127.0.0.1 port 11434, so other devices can’t reach it. Setting OLLAMA_HOST to 0.0.0.0:11434 opens it to your network. Do that only on a network you trust. Anything on your network could then send it requests.

A quick word on privacy

The Ollama FAQ says it plainly: “Ollama runs locally. We don’t see your prompts or data when you run locally.” Downloading models still needs the internet. Ollama also offers optional cloud models, and those run on Ollama’s servers. If you want to stay fully offline, the FAQ explains how to turn the cloud features off. You can set OLLAMA_NO_CLOUD=1, or put "disable_ollama_cloud": true in ~/.ollama/server.json.

That makes local models a good fit for private notes and drafts.

Common errors and fixes

  • “command not found” right after installing. Open a new terminal window so it picks up the updated PATH. On a Mac, make sure you accepted the prompt to create the link in /usr/local/bin.
  • “connection refused” on localhost:11434, or the server isn’t running. The desktop app has to be open (look for it in the menu bar or taskbar). On Linux, run systemctl status ollama, or start the server by hand with ollama serve.
  • Port 11434 already in use. This usually means Ollama is already running, for example the app plus a manual ollama serve. Quit one of them. If another program really owns that port, use OLLAMA_HOST to make Ollama listen on a different one.
  • Model too big or out of memory. Switch to a smaller tag like llama3.2:1b, close other heavy apps, and check ollama ps to see where the model loaded.
  • Very slow replies. If ollama ps shows 100% CPU, the model is running without your GPU. Check that your card and driver are on the supported list in the GPU docs, or use a smaller model.
  • Need more detail? The troubleshooting page explains where the logs are. On a Mac, check ~/.ollama/logs/server.log. On Linux, run journalctl -u ollama. On Windows, open %LOCALAPPDATA%\Ollama.

Wrapping up

You now have a working local AI setup. It can chat in the terminal, answer API calls on localhost, and run a custom tutor you built yourself. Start small, watch ollama ps, and try larger models from the library only once you know how much memory your machine has spare.

Sources and official docs

Links last checked on 15 November 2026.

Written by Mindloop Editorial

The Mindloop Experience editorial team writes clear, carefully sourced explainers and how-tos on AI and technology, software and the business of tech, for anyone who likes to know how things actually work.

Meet the editorial team

Responses

Write a response

More from Mindloop Experience

See all from Mindloop Experience

You can clap up to 50 times.