Running LLMs locally on my Mac

My machine is not that powerful. It has an M1 and mere 8GB of RAM and, of course, the insulting 256GB SSD, which is ridiculous. Now, let’s get started. First things first, install Homebrew if you don’t have it.

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Then, install Ollama. This is what makes it possible for us to easily grab LLMs and run them locally with almost no effort.

brew install ollama

Now, we need to start it.

brew services start ollama

Now that the service is running, you can use the command line to grab LLMs and run them. Here you can easily check available LLMs: https://ollama.com/search, just search for an LLM then run it. For me, since my machine is not that impressive, I’ll get a lightweight model such as Gemma3.

You can see a select box on the left. You select the version you want to run. In my case, I chose 1b, which means 1 billion parameters. Then, the command you need to run on the terminal is on the right side. In my case, it is

ollama run gemma3:1b

This command downloads the model. If you don’t have it, then run it immediately.

The smaller the number of parameters, the faster it runs and the dumber it is. You can see it just by the answer it gave me.

You can keep chatting with it in your terminal as much as you like, but honestly, this is no ideal, you probably want a nice chat interface. In this case, let’s grab one. In my case, I always aim to use native apps, therefore, I’ll use https://github.com/kevinhermawan/Ollamac. If you want to use it as well, just go to the releases page, then download the last release, unpack it, and move it to your applications folder. 

It automatically detects Llama models, and you can chat with it pretty nicely, but you have to select the model on the right panel before chatting.

That’s much better, right? That’s it!