Run Local AI Models on Your Mac

Run chat, vision, and reasoning AI models locally on your Mac using Ollama.

Introduction

30 Second Summary

AI tools like ChatGPT and Claude run on massive servers in the cloud. But modern laptops are powerful enough to run AI models locally, giving you complete privacy and zero ongoing costs.

In this project, you will set up a local AI playground on your Mac where you run and compare three fundamentally different types of AI models entirely on your own machine.

What You'll Build

A personal AI playground powered by Ollama where you run chat, vision, and reasoning models locally on your Mac with zero internet required after download.

By the end of this project, you'll have:

A working Ollama installation with multiple downloaded LLMs running on Apple Silicon.
Hands-on experience with three model types: a chat model for general tasks, a vision model for understanding images, and a reasoning model that thinks out loud.
A custom AI personality you built yourself using a Modelfile.
Secret Mission: Talk directly to your local AI through its REST API and build a one-liner that summarizes any file on your Mac.

Are there any prerequisites?

None. This project includes setup for all required tools. You just need a Mac with an Apple M-series chip (M1 or later) and about 15GB of free disk space for the AI models.

Install Ollama

Before you can run AI models locally, you need a runtime that handles downloading, loading, and running them.

In this step, you'll install Ollama, a lightweight tool that does all of this on your Mac. One command gets you from zero to ready.

In this step, get ready to:

Download and install Ollama using the official install script.
Verify that Ollama is running and ready for models.
Understand what Ollama does under the hood.

Check if Ollama is already installed

Check if Ollama is already on your Mac by running this command:

ollama --version

✔️ I see a version number

Ollama is already installed.

You may see a Warning: could not connect to a running Ollama instance message followed by a version like client version is 0.14.1. That's completely normal. The warning just means Ollama's background service isn't running yet.

Skip ahead to the Verify the installation section below.

ⓧ Command not found

Ollama isn't installed yet. Continue with the next section to install it.

Download and install Ollama

Install Ollama by running this command in your Terminal:

curl -fsSL https://ollama.com/install.sh | sh

What does this command do?

This downloads Ollama's official install script and immediately runs it.

The script installs the ollama command-line tool and starts Ollama as a background service. You only need to run this once.

🙋‍♀️ Seeing a permission error?

If the script asks for your password, type your Mac login password and press Enter. The characters won't appear as you type. This is normal.

If you see a message about curl not being found, open a fresh Terminal window and try again. macOS includes curl by default.

help me troubleshoot the Ollama install script on macOS

Verify the installation

Confirm Ollama is installed by running:

ollama ls

✔️ I see a table header or a list of models

Ollama is installed and running. You might see an empty table with just column headers, or a list of models you've downloaded before (like qwen2.5, tinyllama, etc.). Both are fine.

ⓧ Command not found

The install script may not have finished successfully. Try these steps:

Close your Terminal window completely and open a new one.
Run the install command again:

curl -fsSL https://ollama.com/install.sh | sh

After it finishes, run ollama ls again.

If it still doesn't work, check that your Mac is running macOS Sonoma or newer by clicking the Apple menu and selecting About This Mac.

Check your models anytime

You can run ollama ls anytime to see which models are on your machine, their sizes, and when they were last modified. It's a quick way to check what's taking up disk space.

Understand what you just installed

What is Ollama?

Ollama is a lightweight runtime that lets you run open AI models locally. It handles downloading models, loading them into memory, running inference, and exposing a local API that other apps can talk to.

On your Mac's Apple Silicon chip, Ollama uses the Metal GPU framework for hardware acceleration. This means models run fast without any extra configuration.

Models are stored at ~/.ollama/models and can range from a few GB to tens of GB in size.

Your local AI runtime is ready to go. Next up, you'll download your first model and have a real conversation with it.

Chat With Your First Local AI

Ollama is installed and running, but without any models it's like having a music player with no songs loaded.

Time to download your first AI model and have a real conversation with it. You'll use Gemma 3, a lightweight model from Google that runs comfortably on your Mac and responds fast enough to feel like a real chat.

In this step, get ready to:

Download and run your first local AI model.
Have an interactive chat conversation entirely on your machine.
Learn the essential Ollama commands for managing models.

Pull and run Gemma 3

Gemma 3 is a Large Language Model (LLM) from Google. The 4B parameter version is about 3.3GB and downloads in a minute or two depending on your internet speed.

What does 4B parameters mean?

Parameters are the numbers (weights) inside a neural network that store everything the model has learned. More parameters generally means a more capable model, but also more memory and disk space to run it.

Here's how 4B fits into the bigger picture:

1-4B (like Gemma 3 4B): lightweight, fast, runs on any M-series Mac. Great for everyday chat and simple tasks.
7-14B (like Llama 3 8B, DeepSeek-R1 14B): mid-range. Noticeably smarter, needs 8-16GB of RAM.
30-70B (like Llama 3 70B): very capable, but needs 32-64GB of RAM. Only runs on higher-end Macs.
200B+ (like GPT-4, Claude): too large to run locally. These need data center hardware, which is why they run in the cloud.

The file is only 3.3GB (not 4GB) because Ollama uses quantization — a compression technique that stores each parameter in fewer bits while keeping most of the quality.

Download and start chatting with Gemma 3 by running this command:

ollama run gemma3

What does this command do?

This single command does two things. First, it downloads the Gemma 3 model (about 3.3GB) and stores it at ~/.ollama/models.

Second, it drops you straight into an interactive chat session. The first run takes a minute or two for the download, but every run after that starts instantly because the model is already on your machine.

After the download finishes, you'll see a >>> prompt appear. This means the model is loaded into memory and ready for your messages.

Seeing a connection error or slow download?

Make sure you have a stable internet connection. The model is about 3.3GB, so a slow connection may take several minutes.
If you see a timeout error, try running the command again. Ollama resumes downloads where they left off.
Make sure you have at least 5GB of free disk space for the model files.

Help me troubleshoot why ollama run gemma3 isn't working

Have a conversation

You're now chatting with an AI running entirely on your Mac. Nothing you type leaves your machine.

Try a few different types of prompts to feel out what the model can do:
Ask it to explain something: Explain quantum computing to a 10 year old
Ask it to write something creative: Write a haiku about debugging code
Ask it to help with a task: Give me 5 unique date night ideas in my city

Notice the speed. You're getting about 40+ tokens per second, which is faster than typing speed.

Why is this so fast?

Your M-series Mac has unified memory that both the CPU and GPU share. Ollama uses Apple's Metal GPU framework to accelerate inference, so the model runs on your GPU cores without any extra setup.

Learn the essential commands

Before exploring more models, you need a few commands to manage what's running on your machine.

Exit the chat session by pressing Ctrl+D.

You're back at your normal terminal prompt. Now try the management commands.

See which models are downloaded on your machine by running:

ollama ls

You should see gemma3 listed with its size and when it was last modified.

Check which models are currently loaded in memory by running:

ollama ps

Why does the model stay in memory?

Ollama keeps the last model in memory for 5 minutes by default. This means if you send another prompt quickly, the response is instant because the model doesn't need to reload.

Free up the memory by stopping the model:

ollama stop gemma3

Confirm the model is no longer running by running ollama ps again.

The output should now be empty, meaning no models are using memory. The model is still downloaded and ready to use anytime. It just isn't loaded into RAM right now.

ollama stop not working?

Make sure you spell the model name exactly as it appears in ollama ls output.
If ollama ps already shows no running models, the model was automatically unloaded. That's fine.

Help me understand why ollama stop isn't freeing my memory

You just had your first conversation with a local AI. Next up, you'll discover that this same model can also understand images.

Explore Vision Capabilities

In the previous step, you had a full text conversation with Gemma 3. But this model can do more than just process words.

Gemma 3 is a multimodal model. This means it can understand both text and images. In this step, you'll show it a picture and watch it describe what it sees.

In this step, get ready to:

Download a test image to your Desktop.
Ask Gemma 3 to analyze the image using the command line.
Explore advanced vision tasks like UI feedback and text transcription.

Find an image to analyze

The same Gemma 3 model you chatted with can also understand images. You don't need to download a separate model for this.

Download a sample image to your Desktop by running this command:

curl -L -o ~/Desktop/test-image.jpg "https://upload.wikimedia.org/wikipedia/commons/3/3a/Cat03.jpg"

What does this command do?

curl is a tool for downloading files from the internet. The -L flag follows any redirects, and -o ~/Desktop/test-image.jpg saves the file to your Desktop with that name.

You're downloading a photo of a cat from Wikimedia Commons. You can use any .jpg or .png image on your Mac instead if you prefer.

Confirm the image downloaded by running:

ls ~/Desktop/test-image.jpg

You should see the file path printed back to you with no error message.

Don't see the file?

Make sure you're connected to the internet. The curl command needs to download the image from Wikimedia Commons.

If you see a "permission denied" error, try running the command again. If the problem persists, you can use any image already on your Desktop instead.

Why can't I download the test image?

Ask the model about your image

Now you'll point Gemma 3 at your image and ask it to describe what it sees. The model processes the image pixels entirely on your machine.

Run Gemma 3 with your image by running this command:

ollama run gemma3 "What's in this image? Describe everything you see." ~/Desktop/test-image.jpg

You should see a detailed description of the cat photo appear in your terminal. The model identifies objects, colors, and spatial relationships in the image.

What's happening here?

Ollama sends both your text prompt and the image data to Gemma 3. The model's vision capabilities let it "see" the image and generate a text description of its contents.

This is the same model you chatted with earlier. Multimodal means one model handles multiple types of input (text and images) without needing a separate download.

Try asking follow-up questions about the image while still in the chat session.
Ask something like "What breed might this cat be?" or "Describe the background in more detail."

Seeing an error about the image path?

Make sure the file path matches exactly. If you used a different image, replace ~/Desktop/test-image.jpg with the full path to your image file.

Only .jpg and .png files are supported for vision input.

Why is my image not being recognized by Ollama?

Push the vision capabilities

A cat photo is a good starting point, but vision models can handle much more complex visual tasks. Try pointing Gemma 3 at different types of images to see what it can do.

Try these advanced visual tasks while still in the chat (or start a new session with a different image):
Point it at a screenshot of a website and ask "What could be improved about this UI?"
Show it a handwritten note and ask it to transcribe the text.
Give it a chart or graph and ask it to explain the data.

Why is local vision useful?

Cloud vision APIs require uploading your images to a third-party server. With local vision, sensitive images (medical documents, private photos, internal company screenshots) never leave your machine.

This is especially valuable for analyzing confidential UI designs, scanning private documents, or processing images that contain personal information.

When you're done experimenting, type /bye to exit the chat.

You've just used a single local model for both text chat and image analysis. Next up, you'll try a completely different type of AI that shows you its thinking process.

Try a Reasoning Model

In the previous step, you explored how Gemma 3 can understand images alongside text. Now you'll try something fundamentally different: a model that shows its thinking process before answering.

DeepSeek-R1 is a reasoning model. Instead of jumping straight to an answer, it works through problems step by step in visible <think> blocks. You'll run it, feed it logic puzzles, and compare its approach to Gemma 3.

In this step, get ready to:

Download and run DeepSeek-R1, a 14B parameter reasoning model.
Observe visible reasoning blocks as the model thinks through problems.
Compare how reasoning models differ from chat models like Gemma 3.

Pull and run the reasoning model

Gemma 3 is optimized for fast, general-purpose chat. DeepSeek-R1 takes a different approach. It's built specifically to reason through complex problems, and it shows you its thinking along the way.

Download and start DeepSeek-R1 by running this command:

ollama run deepseek-r1:14b

What does this command do?

This downloads the DeepSeek-R1 14B parameter model (about 9GB) and drops you into an interactive chat. The 14B size is a distilled version of DeepSeek's full reasoning technology, optimized to run on your Mac.

The first run takes several minutes to download depending on your internet speed. Future runs start instantly since the model is cached locally.

Wait for the download to finish. You'll see progress bars as the 9GB model downloads.
Once complete, an interactive chat prompt appears. You're now talking to the reasoning model.

Download taking too long or failing?

A 9GB download can take 5-10 minutes on a typical connection. If the download fails partway through, run the same command again. Ollama resumes from where it left off.

If you see a connection error, check that you have an active internet connection and try again.

help me troubleshoot my DeepSeek-R1 download in Ollama

Watch it reason through problems

The magic of a reasoning model is that you can literally watch it think. Give it problems that require step-by-step logic and watch what happens in the output.

Try this logic puzzle first. Type it into the chat:

I have 3 boxes. Box A has a cat. I swap Box A and Box B, then swap Box B and Box C. Where is the cat?

Now try a coding problem:

Write a Python function that finds all prime numbers up to n. Explain your approach.

Try one more classic puzzle:

If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets?

What are those <think> blocks?

Notice the <think>...</think> sections in the output. This is the model's internal reasoning process, made visible to you.

The model works through the problem before giving its final answer. For the box puzzle, you'll see it tracking each swap step by step. For the coding problem, it plans its algorithm before writing code.

Compare the experience

You've now used two fundamentally different types of AI models. The difference isn't just about speed or accuracy. It's about what each model is optimized for.

When to use which model type

DeepSeek-R1 takes longer to start responding because it thinks first. Its answers to logic puzzles and math problems are often more accurate because of that thinking step.

Gemma 3 responds almost instantly and works great for simple questions, creative writing, and image understanding. It's overkill-free for tasks that don't need chain-of-thought reasoning.

This is the key insight: different model types are optimized for different jobs. A reasoning model for logic puzzles, a multimodal model for everyday chat and vision tasks.

Type /bye to exit the DeepSeek-R1 chat.

Confirm both models are stored locally by running:

ollama ls

You should see both gemma3 and deepseek-r1:14b listed with their sizes.

Don't see both models listed?

Make sure you exited the chat with /bye before running ollama ls. The ls command works from the regular terminal prompt, not inside a model chat.

If DeepSeek-R1 isn't listed, the download may not have completed. Run ollama run deepseek-r1:14b again to resume the download.

help me check why my Ollama models aren't showing up

You've now experienced three types of AI: general chat, vision, and reasoning. Next up, you'll create your own custom AI personality from scratch using a Modelfile.

Build Your Own AI Personality

You've now explored reasoning models that show their thinking process. That's three fundamentally different AI experiences on one machine.

Now for the most creative part of this project: building your own custom AI personality. Using a Modelfile, you can define exactly how an AI speaks, what it knows, and what rules it follows. No training required.

In this step, get ready to:

Create a Modelfile that defines a custom AI persona.
Build and run your custom model with Ollama.
Chat with your custom AI and iterate on the personality.

Create a Modelfile

A Modelfile is a recipe that tells Ollama how to build a custom model. It specifies which base model to use and what personality rules to apply.

Create your Modelfile by running this command in your terminal:

cat << 'EOF' > ~/Modelfile
FROM gemma3
SYSTEM """You are Captain Nova, a retired space explorer who has visited 47 star systems. You speak with wonder about the cosmos and always relate everyday questions back to your space adventures. You're warm, slightly eccentric, and occasionally make up constellations. Keep responses under 3 sentences unless asked for a longer story."""
EOF

What does this code do?

The FROM gemma3 line tells Ollama to use Gemma 3 as the foundation for your custom model. All of Gemma 3's capabilities carry over.

The SYSTEM line defines the personality and rules your AI follows for every conversation. This is what makes Captain Nova respond like a retired space explorer instead of a generic assistant.

Confirm the Modelfile was created by running:

cat ~/Modelfile

You should see the full contents of your Modelfile printed in the terminal, starting with FROM gemma3 and including the Captain Nova system prompt.

Seeing "No such file or directory"?

Make sure you copied the entire cat command including the EOF line at the end. The heredoc syntax requires both the opening << 'EOF' and the closing EOF on its own line.

help me create my Modelfile for Ollama

✔️ Awesome, I've got everything!

Great. Double check you see the full Modelfile contents in your terminal output.

ⓧ I'd like to double check the full code

Your ~/Modelfile should contain exactly this:

FROM gemma3
SYSTEM """You are Captain Nova, a retired space explorer who has visited 47 star systems. You speak with wonder about the cosmos and always relate everyday questions back to your space adventures. You're warm, slightly eccentric, and occasionally make up constellations. Keep responses under 3 sentences unless asked for a longer story."""

Build and run your custom model

Now that the Modelfile exists, you can turn it into a runnable model that Ollama manages just like any other downloaded model.

Create the model from your Modelfile by running:

ollama create captain-nova -f ~/Modelfile

What does this command do?

This tells Ollama to register a new model called captain-nova using the instructions in your Modelfile. It doesn't re-download Gemma 3. It creates a lightweight layer on top of the base model with your system prompt baked in.

Verify the model was created by running:

ollama ls

You should see captain-nova in the list alongside gemma3 and deepseek-r1:14b.

Don't see captain-nova in the list?

Check that the -f flag points to the correct path. The file must be at ~/Modelfile (capital M, no file extension).

If you see an error about the base model not being found, make sure Gemma 3 is still downloaded by running ollama ls and confirming gemma3 appears.

help me troubleshoot my ollama create command

Start chatting with your custom model by running:

ollama run captain-nova

You're now in an interactive chat with Captain Nova. The model responds using the personality and rules you defined in the Modelfile.

Test and iterate on the personality

The best way to see if your custom personality works is to ask it everyday questions and notice how it weaves in the space explorer persona.

Try asking Captain Nova a few questions:
"What should I have for dinner?"
"How do I fix a leaky faucet?"
"Tell me a bedtime story"

Why does this work without training?

You didn't train a new model. You gave an existing model a permanent set of instructions it follows for every response. This is the power of system prompts. The base model's full capabilities remain intact, but now filtered through your persona.

If you want to adjust the personality, edit the Modelfile and run ollama create captain-nova -f ~/Modelfile again. The model updates instantly.

When you're done chatting, type /bye to exit the conversation.

Captain Nova not staying in character?

If the responses feel too generic, check that your SYSTEM prompt uses the triple quotes ("""). Single quotes can cause the prompt to be truncated.

Also try being more specific in your system prompt. The more constraints you give ("always", "never", "keep responses under 3 sentences"), the more consistently the model follows them.

help me fix my custom Ollama model personality

Secret mission

Talk to Your AI Through the API

Ollama runs a REST API at http://localhost:11434. Every app that uses local AI talks to this same endpoint. In this secret mission, you'll use curl to talk directly to the API and build a one-liner script that can summarize any file on your Mac.

Clean Up Your Resources

This project runs entirely on your Mac with no cloud services, so there are no ongoing costs. Decide whether to keep your models on disk for future use, free up some space, or remove everything entirely.

Resources you used:

Ollama application (installed at /usr/local/bin/ollama and /Applications/Ollama.app).
Downloaded models stored at ~/.ollama/models (Gemma 3 ~3.3GB, DeepSeek-R1 14B ~9GB, captain-nova). Total: roughly 12-15GB.
Custom Modelfile at ~/Modelfile.
Test image at ~/Desktop/test-image.jpg.
Secret Mission script summarize.sh.

Keep everything running

No action needed. Choose this if you want to keep experimenting with local AI models.

Ollama uses no CPU or memory when idle. Models stay on disk and are ready to run anytime with ollama run.
Your custom captain-nova model and Modelfile stay available for editing and re-creating.
The only cost is disk space (~12-15GB for all downloaded models).

Pause - I'll come back to this later

Free up disk space by removing the larger models you don't need right now. Keep Ollama installed so you can re-download models later.

Stop any running models by running:

ollama stop gemma3

Remove the larger models to free up ~9GB by running:

ollama rm deepseek-r1:14b
ollama rm captain-nova

Keep gemma3 (~3.3GB) if you want a general-purpose model ready to go. Remove it too with ollama rm gemma3 if you want all disk space back.
When you're ready to come back, just run ollama run gemma3 and it will re-download in about a minute.

Delete - I don't want to use this again

Remove all project resources and Ollama itself to start completely fresh.

First, stop all running models:

ollama stop gemma3

Remove all downloaded models:

ollama rm gemma3
ollama rm deepseek-r1:14b
ollama rm captain-nova

Uninstall Ollama completely by running:

sudo rm -rf /Applications/Ollama.app
sudo rm /usr/local/bin/ollama
rm -rf ~/.ollama

Remove the Modelfile and test image:

rm ~/Modelfile
rm ~/Desktop/test-image.jpg

Remove the Secret Mission script. If you saved it to your Desktop, run:

rm ~/Desktop/summarize.sh

After these commands, Ollama and all models are fully removed from your Mac.

That's a Wrap!

You just built your own local AI playground. You ran three fundamentally different types of AI models entirely on your Mac, with zero internet required after download and zero cost.

Here's what you accomplished:

Ran Gemma 3, a multimodal model that handles both text conversations and image analysis locally on your Mac.
Explored DeepSeek-R1, a reasoning model with visible <think> chains that shows its step-by-step problem-solving process.
Created a custom AI personality (Captain Nova) using a Modelfile to define personality and behaviour rules.
Talked directly to the Ollama REST API and built a file summarizer script that pipes any text file into your local AI.

Ready to quiz yourself?