5 Steps to Your Own Private AI Assistant (No Cloud Required)

Every message you send to ChatGPT or Claude leaves your machine. For most questions that's fine. For the ones involving your finances, your health records, or a client's confidential document, it isn't — and the fix isn't "be more careful what you type," it's running the model somewhere the question never leaves.

I resisted local AI for a long time because I assumed it meant a worse model, a painful setup, and a laptop fan that never stopped spinning. None of that turned out to be true. Modern open-weight models running on ordinary consumer hardware are genuinely good at summarising, drafting, and answering questions — not GPT-5 good, but good enough for a real slice of what I used to send to the cloud, with the added benefit that it works with the wifi off.

This is the exact five-step path from a blank machine to a private AI assistant you control completely: what to install, which model to pick, how to actually use it day to day, and how to keep it running smoothly once the novelty wears off.

Why bother, when ChatGPT already exists

Three reasons this earns a place next to your cloud tools rather than replacing them entirely: privacy — nothing you type ever leaves your computer, which matters for anything sensitive; cost — no subscription, no per-token bill, ever; and reliability — it works on a train, on a flight, or the moment your broadband drops. The trade-off is honest too: a model small enough to run on your laptop won't match a frontier cloud model on hard reasoning or huge context windows. Use it for what it's good at, and keep ChatGPT or Claude open in another tab for the rest.

Step 1: Install Ollama

Ollama is the tool that makes this realistic for a non-engineer. It downloads models, runs them efficiently on your CPU or GPU, and gives you a clean way to chat with them — all through one small application, free and open source.

• Mac: download the app from Ollama's website and drag it into Applications, or run brew install ollama if you already use Homebrew.
• Windows: download and run the installer — it sets up a background service and a small system-tray icon.
• Linux: run the one-line install script from Ollama's site, which handles the service setup for you.

Once installed, open Terminal (or Command Prompt on Windows) and type ollama --version. If you see a version number back, you're done with the hardest part.

Step 2: Pull a model that fits your machine

This is the step people overthink. You don't need to research benchmark leaderboards — you need a model sized to your RAM. As a rough rule, a model needs roughly as much free RAM as its parameter count in gigabytes.

• 8–16GB of RAM: start with llama3.2 (the 3B version) or phi3 — small, fast, surprisingly capable for everyday writing and Q&A.
• 16–32GB of RAM: llama3.1:8b or mistral give you noticeably better reasoning while still running comfortably.
• 32GB+ or a dedicated GPU: llama3.1:70b or mixtral get you closer to cloud-model quality, at the cost of a much heavier download and slower replies.

Pull your chosen model with one command, for example: ollama pull llama3.2. It downloads in the background — a few gigabytes for the small models, considerably more for the large ones — and only needs to happen once per model.

Step 3: Have your first conversation, entirely offline

Turn off your wifi, then run ollama run llama3.2 in the terminal. You'll drop straight into a chat prompt. Ask it something real — draft an email, explain a concept, summarise a paragraph you paste in — and watch it respond with no internet connection at all. That moment is the whole point of this guide: the same kind of assistant you're used to, running entirely on hardware you own.

Type /bye to end the session. The model stays downloaded, so the next time you run that same command it starts instantly — no re-downloading, no login, no usage counter.

Step 4: Add a proper chat interface

The terminal is fine for quick tests but not where you want to live day to day. Install Open WebUI, a free browser-based interface that talks to Ollama and looks and feels like ChatGPT: conversation history, multiple chats side by side, file uploads, and model switching from a dropdown. The simplest route is Docker — one command pulls and runs it, and it opens at localhost:3000 in your normal browser.

If you'd rather skip Docker entirely, lightweight native apps like Msty or Chatbox connect to your local Ollama install with zero configuration and install like any normal application. Either way, you end up with a bookmarkable tab that feels exactly like the cloud tools you're used to — except every conversation in it stays on your machine.

Step 5: Put it to work on something real

A local model earns its place once it's doing a specific job, not just answering trivia. Three uses that have genuinely stuck for me:

1. First-pass drafting for sensitive documents. Client proposals, HR-adjacent emails, anything with names and numbers I'd rather not paste into a cloud chatbot gets drafted locally first, then polished wherever I like.
2. Summarising long PDFs and contracts. Open WebUI lets you attach a document directly to the chat — useful for a first-pass summary of a lease or terms-of-service document before you read the whole thing yourself.
3. A always-available second opinion. Because there's no per-message cost, I use it far more casually than I use paid cloud tools — quick rewrites, brainstorming a subject line, checking a sentence reads naturally. Zero hesitation, because it's free and instant.

"The best local model isn't the biggest one you can technically run — it's the smaller one that answers in two seconds instead of two minutes, so you actually keep using it."

Keeping it running smoothly

A few habits keep this from becoming a chore:

• Check installed models with ollama list and remove ones you've stopped using with ollama rm <model> — they take up several gigabytes each and are easy to forget about.
• Update models occasionally with ollama pull <model> again — this fetches the latest version if the maintainers have released an improved one under the same name.
• Close it when you're not using it. Ollama only loads a model into memory while it's actively answering, but quitting the background app fully frees up RAM for everything else.

Common mistakes worth avoiding

1. Grabbing the biggest model first. A 70B model on 16GB of RAM will either fail to load or run so slowly it's unusable — start small, confirm it works, then size up.
2. Expecting cloud-model reasoning from a laptop-sized model. Local models are excellent at drafting, summarising, and conversation. They're weaker at multi-step logic and up-to-the-minute facts — know which job you're asking it to do.
3. Treating it as all-or-nothing. This isn't about abandoning ChatGPT or Claude. It's about having a private, free option for the slice of your work that shouldn't leave your machine in the first place.

The bottom line

Running AI locally used to be a hobbyist project. Now it's a genuinely practical Tuesday-afternoon setup — Ollama installs in minutes, a sensibly sized model downloads once, and Open WebUI gives you an interface that feels like the tools you already use. What you get back is real: no subscription, no data leaving your machine, and an assistant that works exactly as well on a plane as it does at your desk.

Install Ollama today, pull one small model, and have a single offline conversation before you close this tab. That's the whole barrier to entry — everything after it is just deciding what you'd rather keep private.

Questions people actually ask

Can I run an AI assistant on my own computer without the internet?

Yes. Ollama runs open-weight models like Llama 3 entirely on your own machine, so once a model has downloaded it works with the wifi off — on a train, on a flight, or when your broadband drops. Nothing you type ever leaves your computer.

Is a local AI model as good as ChatGPT?

Honestly, no — a model small enough to run on a laptop won't match a frontier cloud model on hard reasoning or very long documents. But for drafting, summarising and everyday questions it's genuinely good, and the sensible setup is both: local for anything sensitive, cloud for the heavy lifting.

How much RAM do I need to run a local AI model?

As a rough rule, a model needs about as much free RAM as its parameter count in gigabytes. With 8–16GB start with a small model like Llama 3.2 or Phi-3; 16–32GB runs an 8B model like Llama 3.1 or Mistral comfortably; only go bigger if you have 32GB-plus or a dedicated GPU.

Is Ollama free to use?

Yes — Ollama is free and open source, and the models it runs cost nothing to download or use. There's no subscription and no per-message charge; the only real cost is a few gigabytes of disk space per model.