How to Run Local LLMs on a Laptop
What fits in 8-64GB RAM, what Q4/Q8 quantization really means, CPU vs GPU vs Apple Silicon speed reality, and the exact commands for your first local LLM.
Note
Before you start
- A laptop with at least 8GB of RAM (16GB makes a visible difference)
- About 10GB of free disk for your first model
- A terminal and a few minutes - none of this needs a GPU
- If you're brand new to Ollama, run through its install guide first (linked below)
Jump to section
- Before you start
- Step 1: The two memories — RAM vs VRAM
- Step 2: What Q4/Q8 quantization actually means
- Step 3: CPU vs GPU vs Apple Silicon — what each can actually do
- Step 4: Pick your hardware tier (honest table)
- Step 5: First run — the exact commands
- How to verify it worked
- Troubleshooting
- You did it
- 1
Learn the two memories
RAM is system memory, VRAM is GPU memory. Local LLMs live entirely in one of them - knowing which one your model landed in explains 80% of laptop performance puzzles.
- 2
Understand Q4 vs Q8 quantization
Quantization trades precision for size. A 7B model drops from 12.6GiB to 3.6GiB at Q4 with a small-medium quality loss - the single most useful number in local AI.
- 3
Check what your laptop can actually compute
CPU always works, NVIDIA/AMD GPUs accelerate when the driver is supported, Apple Silicon uses shared memory via Metal. Verify with ollama ps, never with vibes.
- 4
Pick your hardware tier honestly
8GB RAM → 3-4B models at Q4. 16GB → 7B Q4 is the sweet spot. 32GB → 14B class. 64GB → 30B class, 70B only if you accept slow CPU rates.
- 5
Run your first local LLM
Install Ollama, pull llama3.2, chat, and confirm with ollama ps that the model and your expectations landed in the same memory.
- 6
Fix the classic failure modes
Out-of-memory, 1 token/second, and swap thrash all have the same root cause and the same three fixes: smaller model, smaller quant, smaller context.
A laptop can run local LLMs right now - the trick is matching the model to the memory your laptop actually has. This guide makes “local LLMs on a laptop” concrete: what fits in 8/16/32/64GB of RAM, what Q4 vs Q8 quantization means in plain numbers, what CPU vs GPU vs Apple Silicon can actually deliver, and the exact commands to run your first model. Realistic expectation up front: a 7B model at Q4 (3.6GiB) on a MacBook is the everyday sweet spot - comfortable and genuinely useful. A 70B on a workstation-class machine is a different world, and this guide shows you why.
This is the planning guide that sits next to our Ollama install guide (the runtime), the DeepSeek VRAM deep-dive (what server-class actually costs), and the V4-Flash local run guide (when laptops stop being enough).
Before you start
- Know your laptop’s RAM and whether it has a discrete GPU. Nothing else matters as much.
- Have ~10GB free disk for the first model (models live on disk, not just in memory).
- Assumptions in this guide are marked: official = verified against Ollama/llama.cpp docs this session; field experience = widespread practical consensus, no official table exists.
Step 1: The two memories — RAM vs VRAM
This is the #1 confusion in local AI, and it’s cheap to fix.
- RAM (system memory) - the laptop’s main memory, e.g. 8/16/32/64GB. Every model can run here.
- VRAM (GPU memory) - fast memory on the graphics card, e.g. 4-24GB on laptop GPUs. Models prefer to run here because it’s 10-50x faster to access.
One memory is fast and scarce (VRAM); one is slower and ample (RAM). A model runs in one or the other - or split across both when it only half-fits. Ollama’s own FAQ shows exactly this with ollama ps, which prints a PROCESSOR column: 100% GPU, 100% CPU, or a split like 48%/52% CPU/GPU.
On Apple Silicon, RAM and VRAM are the same physical chips (unified memory, accessed via Metal) - which is why 16GB MacBooks run 7B models that 16GB Windows laptops with weak GPUs run slower. Laptop GPUs with 8GB VRAM can hold a 7B Q4 (3.6GiB) entirely.
Step 2: What Q4/Q8 quantization actually means
LLM files come in precisions. The official llama.cpp benchmarks list a 7B model at three sizes:
| Precision | Bits/weight | 7B model size (official) | Quality vs F16 |
|---|---|---|---|
| F16 | 16 | 12.55 GiB | full precision (baseline) |
| Q8_0 | ~8 | 6.67 GiB | ”very small loss … usually no noticeable impact” |
| Q4_0 | ~4 | 3.56 GiB | ”small-medium loss” |
Rule of thumb: file size ≈ parameters × bits-per-weight ÷ 8. That’s official arithmetic made simple - 7B × 4 bits ÷ 8 = ~3.5GB, matching the official 3.56GiB. Scale it: a 14B at Q4 ≈ 7GB, a 30B at Q4 ≈ 15GB, a 70B at Q4 = 36.2GiB (official figure from the same benchmark thread).
Official memory maths for quantized 8-bit vs 4-bit
Ollama’s docs explain the KV-cache side (the part of memory that grows with conversation length): 8-bit uses “approximately 1/2 the memory of f16”, 4-bit “approximately 1/4”. Smaller is also less precise - their words: 8-bit has “a very small loss in precision” and 4-bit “a small-medium loss that may be more noticeable at higher context sizes”. Exactly why Q4 is the everyday choice for laptops and Q8 for quality-first setups.
What it means for you: for chat on a laptop, Q4 is the default. Q8 is for when you have headroom and want near-lossless. F16 stays on server GPUs.
Step 3: CPU vs GPU vs Apple Silicon — what each can actually do
CPU (any laptop): always works, slow at generation. Official expectation: Ollama runs everywhere; ollama ps will show 100% CPU. Field experience: 7B Q4 on pure CPU generates roughly 1-3 tokens/sec - fine for short answers, frustrating for documents.
NVIDIA GPU: the official support bar is compute capability 5.0+ and driver 550+ (older 5.0-6.2 cards need driver 570+). That covers most GTX 10-series onward. An RTX 4060-class laptop GPU (8GB VRAM) comfortably holds 7B Q4 whole.
AMD GPU: official support via ROCm (Linux, plus a Windows list) with a Vulkan fallback for extra cards; the docs include workaround guidance (HSA_OVERRIDE_GFX_VERSION) for GPUs ROCm doesn’t name.
Apple Silicon: official - “Ollama supports GPU acceleration on Apple devices via the Metal API”. Unified memory means a 16GB M-series Mac runs the same model a 16GB-RAM Intel laptop runs - but at GPU speed instead of CPU speed.
Speed reality check (official llama.cpp benchmark thread, community-reported, slightly dated build - treat as indicative):
| Machine | 7B Q4_0 generation speed |
|---|---|
| M1 (base, 8 GPU cores) | ~14 tokens/s |
| M2 (10 cores) / M3 (10 cores) | ~22 / ~21 tokens/s |
| M4 (10 cores) | ~24 tokens/s |
| M4 Pro (16 cores) | ~50 tokens/s |
| M2 Ultra (76 cores) | ~88-94 tokens/s |
| 2× RTX 3090 (desktop, for scale) | ~87 tokens/s |
Reading the table: even a base M1 MacBook generates chat-speed tokens on a 7B Q4 (~14 t/s is comfortable reading speed). “Text generation ≈ memory bandwidth” is the takeaway - faster chips don’t multiply this, more memory bandwidth does.
Step 4: Pick your hardware tier (honest table)
Model sizes are official; the per-RAM recommendations are field experience:
| RAM | What fits comfortably | Sweet spot | Ceiling (loveable) |
|---|---|---|---|
| 8GB | 3-4B at Q4 (~2GB) | 3-4B Q4, small context | 7B Q4 (3.6GiB) with tight context |
| 16GB | 7B Q4 + context | 7B Q4 | 14B Q4 (~7GB) |
| 32GB | 14B Q4 + room | 14B Q4 | 30B Q4 (~15GB) |
| 64GB | 30B Q4 + room | 30B Q4 | 70B Q4 (36.2GiB) only at CPU-class speeds |
- RAM number = total system RAM. Your OS eats a few GB first; assume 6-7GB available in an 8GB machine.
- Context eats extra: Ollama’s default is 4096 tokens (official FAQ), each token costs memory - roughly doubling the model’s cache with every large-context jump. Smaller
num_ctx= more headroom. - Apple Silicon note: unified memory means the GPU shares RAM, so a 16GB Mac can hold a 7B Q4 and run it fast - the Windows-laptop-with-8GB-GPU equivalent holds the same model but slower.
Don't chase bench numbers from model threads
“Runs on a laptop” in model announcements usually means “runs with massive quantization on a server”. V4-Flash: official download 166.9GB - the smallest community build is 82.5GB. That is not laptop territory, and our VRAM guide shows the math. Your laptop’s league is 3-30B at Q4 - and that league is genuinely useful.
Step 5: First run — the exact commands
Install Ollama (official script; Windows/macOS download from ollama.com):
curl -fsSL https://ollama.com/install.sh | shPull and start chatting (first pull downloads ~4-9GB depending on the model):
ollama run llama3.2Type a prompt and hit Enter. Keep the context honest for your RAM:
/set parameter num_ctx 4096Then, in a second terminal, verify where the model actually landed - the official ollama ps check:
ollama psExpected output shape (official examples):
NAME ID SIZE PROCESSOR UNTIL
llama3.2 <hash> 4.9 GB 100% CPU 5 minutes from now100% GPU = the whole model sits in VRAM (Apple Silicon always shows GPU-class here thanks to Metal). 100% CPU = system RAM (your case on most laptops - and it’s fine). A split like 48%/52% CPU/GPU = partial spill, an early sign you’re pushing the memory envelope.
Memory hygiene (official FAQ mechanics)
Models stay loaded 5 minutes after last use, then unload - ollama stop llama3.2 forces it immediately. If you’re juggling models, set OLLAMA_MAX_LOADED_MODELS (default: 3× GPU count, or 3 for CPU) and note that for GPU inference a model must fit in VRAM entirely to load concurrently - that’s verbatim official behavior.
How to verify it worked
-
ollama pslists your model with a PROCESSOR value you understand - You read the model’s answers at a speed you can stand (field bar: above 10 tokens/s feels fine; under 5 is still workable)
- Your laptop’s fans don’t pin and your OS doesn’t start paging hard
- You know which of the three knobs (model size, quant level, context) to turn when it doesn’t fit
Troubleshooting
”Out of memory” / model fails to load
The model doesn’t fit available RAM (or VRAM for GPU loads). Exact fixes, easiest first: pick a smaller quant (Q8 → Q4), shrink context with /set parameter num_ctx or num_ctx, or switch to a smaller model (14B → 7B → 3-4B). Close browser tabs - field experience: 40+ tabs eats more RAM than a 7B model does.
1 token per second — unusably slow
You’re on CPU (or a split load). Check ollama ps - 100% CPU on a big model is the usual story. Options, in order: use Q4 instead of Q8 (official docs: ~4x less weight-memory, faster memory-bound generation), use a smaller model, or accept it for idle chat. On macOS make sure you’re actually using a Metal build; on NVIDIA/AMD verify your GPU is on the official supported list (compute capability 5.0+ / ROCm).
Disk thrashing / everything froze (swap)
The model + context exceeded RAM and the OS started paging to disk. Fix: ollama stop, then re-run with a smaller model, and while testing keep context at 4096. If you must run close to your limit, leave a full GB of headroom - swap thrash is 100x more expensive than any model quality gain.
My GPU isn’t used at all (NVIDIA/AMD)
Ollama lists supported cards explicitly in its GPU docs - NVIDIA needs compute capability 5.0+ (that’s ~GTX 10-series and newer) and driver 550+; AMD needs ROCm with a documented card list. If your card is on the list but not used, the docs’ first checks: driver version, and on Linux the CUDA_VISIBLE_DEVICES/ROCR_VISIBLE_DEVICES selection variables. On macOS Docker, note GPU acceleration isn’t available for Docker Desktop at all (official FAQ) - run Ollama natively there.
Model quality is noticeably worse than the API
Expected. An API model is a frontier model at full precision on a server; your laptop runs quantized small models. If quality disappoints, the ladder is: bigger context first, then Q8, then the next model size up - but each rung costs memory, so re-read Step 4 before climbing.
You did it
- You can explain RAM vs VRAM and why it determines what fits
- You can explain Q4/Q8 and their official size/quality tradeoffs
-
ollama psshows your model and you read the PROCESSOR column - You know your hardware tier and the model size that fits it
- You can fix oom / slow / swap with the three-knob method
- You’re ready for a chat UI: Open WebUI on top of Ollama
Official sources
- Ollama FAQ -
ollama ps, PROCESSOR column, context, keep_alive, concurrency: docs.ollama.com/faq - Ollama GPU support - NVIDIA/AMD/Metal/Vulkan lists and drivers: docs.ollama.com/gpu
- llama.cpp Apple Silicon benchmarks - official 7B sizes (12.55/6.67/3.56GiB) and tokens/sec: ggml-org/llama.cpp discussion #4167
- Our own server-class reality check: DeepSeek V4-Flash VRAM guide
All fetched 13 August 2026. RAM-tier recommendations are field experience, labeled as such.
Next steps: put a proper interface on top with Open WebUI (the discoverable directory entry is here), and when your laptop outgrows itself, our V4-Flash local run guide shows the whole ladder from Ollama to llama.cpp with full control.
Questions, answered first
Do I need a GPU to run local LLMs on a laptop?
No. LLMs run on CPU alone - every laptop with enough RAM can do it - you just trade speed. The official Ollama GPU docs list supported NVIDIA cards (compute capability 5.0+), AMD Radeon (ROCm), and Apple Silicon via Metal; on machines with none of those, Ollama runs on CPU and `ollama ps` shows `100% CPU` (field experience: 7B Q4 on CPU lands around 1-3 tokens/sec, usable for casual chat but slow for long documents).
What's the best model for an 8GB or 16GB laptop?
For 8GB of RAM, a 3-4B model at Q4 (about 2GB) is the comfortable daily driver and a 7B Q4 (3.6GiB) works if you keep context small. For 16GB, 7B Q4 is the sweet spot with room for context, and 14B Q4 becomes feasible. These sizes come from official model files (the llama.cpp repo lists 7B Q4 at 3.56GiB); the per-tier picks are field experience rather than an official table.
Is a bigger model always better?
No - bigger only helps if it fits. A 70B model at Q4 is 36.2GiB (official llama.cpp benchmark figure), which on a laptop means running from system RAM at CPU-class speeds, often 10-30x slower than a 7B. The practical quality ceiling on a laptop is usually 14B-30B at Q4; past that, RAM size and token speed punish you more than the quality gain rewards you.
Why is my computer swapping and crawling?
Because the model doesn't fit in memory. When a model exceeds available VRAM, Ollama spills into system RAM (its own docs show mixed load like `48%/52% CPU/GPU`), and when it exceeds RAM entirely, the OS pages to disk - that's swap thrash. Fixes, in order: reduce context (smaller `num_ctx`), pick a smaller quant, then a smaller model. `ollama stop` frees memory when you're done.
What do the Q numbers in model files mean?
They're quantization levels: how many bits per weight the model uses. F16 = 16-bit float (the big file), Q8_0 ≈ 8-bit, Q4_0 ≈ 4-bit. Ollama's official docs say 8-bit uses roughly half the memory of F16 with 'a very small loss in precision' and 4-bit uses roughly a quarter with a 'small-medium loss' - that trade is exactly why Q4 is the default for local laptops.
I keep seeing VRAM numbers for models like DeepSeek V4-Flash. Isn't that a laptop model?
No, and that confusion is worth killing now. V4-Flash's official download is 166.9GB - it needs server-class memory, not a laptop, as our VRAM guide breaks down. 'Runs on a laptop' claims in model threads usually mean 'runs there if you accept heavy quantization plus CPU,' which for 160GB+ models means not really. Laptop-class local AI means 7B-30B, Q4, smallish context - and that's genuinely useful.
You did it
- You can explain RAM vs VRAM and why the confusion matters
- You can state what Q4/Q8 mean and roughly how much smaller Q4 is than F16
- You ran a model and confirmd via `ollama ps` which memory it used
- You know your laptop's tier and what size model fits it
- You can fix a model that won't load (smaller quant, smaller context, or smaller model)
- You know your laptop is ready for Open WebUI on top of Ollama