Local LLM on Raspberry Pi 5: 2026 UK Practical Guide

The case for hosting your own language model on a Raspberry Pi comes down to three things: privacy, cost, and control. Your prompts never touch a third-party server, you avoid pay-per-token bills, and you can wire the model into local tools - Home Assistant, a personal note system, a coding helper - without worrying about an API key being rotated or a free tier disappearing.

A Pi 5 is not the fastest machine you could pick. It is, however, the cheapest box capable of running modern 3-billion-parameter models at a usable speed, with idle power low enough that running it 24/7 costs less than a pint each month. The trade-off is patient generation: a Pi answers in measured paragraphs, not real-time streams.

Skip the 4GB model. Modern quantised LLMs need 3–6GB of RAM for the model weights plus headroom for the context window and the operating system. The Pi 5 8GB is the entry point that runs everything from TinyLlama through to Llama 3.2 3B comfortably. The Pi 5 16GB (launched January 2025) is the choice if you want to occasionally load an 8B-parameter model - slow but feasible - or if you want lots of context window for document Q&A.

Pi 4 still works but is around 30–40% slower per token because of the older Cortex-A72 cores and missing NEON-FP16 instructions used by recent llama.cpp builds. If you already own one, fine; if you're buying, the £15 saving is not worth it.

Storage matters more than people expect

Model files are large - Llama 3.2 3B in Q4_K_M is about 2.0GB, Phi-3 Mini 3.8B is around 2.3GB - and SD card random-read speeds throttle the first load of every session. An NVMe SSD, a high-speed solid-state drive using the NVMe protocol over a PCIe interface, on the official Pi M.2 HAT cuts cold-start time from 8–15 seconds down to 1–2 seconds. For a chat endpoint that idles between queries, it's the single biggest quality-of-life upgrade after the active cooler.

Typical retail at Pimoroni, The Pi Hut, and RS Components as of early 2026. Prices include VAT but exclude shipping. Stock is reliable for the 8GB SKU; 16GB occasionally sells through.

The good models for Pi-class hardware all share one trait: they were designed or fine-tuned with edge inference in mind. Pure parameter count is misleading - a well-trained 3B model often beats a clumsier 7B at the same quantisation level.

The shortlist

Llama 3.2 3B (Meta) - the default recommendation. Strong general reasoning, good UK English handling, runs at a usable speed on Pi 5 8GB. Pull with ollama pull llama3.2:3b.

Phi-3 Mini 3.8B (Microsoft) - punches above its weight on factual recall and coding. Slightly slower than Llama 3.2 3B but answers tend to be more precise. ollama pull phi3:mini.

Qwen 2.5 1.5B (Alibaba) - surprisingly capable for its size, much faster than the 3B options. Good for tool-use and structured output. ollama pull qwen2.5:1.5b.

TinyLlama 1.1B - the speed champion. Use it when latency matters more than quality, like a voice assistant that needs to respond quickly. ollama pull tinyllama.

Gemma 2 2B (Google) - strong instruction-following, MIT-friendly licence, decent middle ground between TinyLlama and Llama 3.2. ollama pull gemma2:2b.

LFM3-1.7B (Liquid AI) - the newest name on the list, released in October 2026 and built specifically for on-device agents. It is small enough to sit alongside Qwen 2.5 1.5B on a Pi 5 8GB; see the section below for what makes it different and why we have not added it to our speed table yet.

What to avoid on a Pi 5 8GB: Llama 3 8B, Mistral 7B, anything labelled "70B". They will technically load on the 16GB SKU using heavy quantisation but generation drops below one token per second and the experience is painful.

Probably, if your Pi is going to run an agent rather than a chat window. Liquid AI announced LFM3 on 27 October 2026: a family of open-weight models covering text, vision, audio, a combined "omni" model and two decision models, all branched from one shared base of around 2 billion parameters. Liquid AI (a company founded in 2023 that builds compact foundation models for on-device use) names the Raspberry Pi 5 as one of 14 device and runtime combinations it used to shape the architecture, so the Pi was a design target rather than an afterthought.

What does "tri-hybrid" mean in plain English?

Most chat models are stacks of identical attention layers, where every word looks back at every earlier word. That is accurate but gets expensive as a conversation grows. LFM3 mixes three kinds of layer instead: cheap short-range convolution layers that only look at nearby words, attention layers that look back over a limited recent window, and a smaller number of full attention layers that can see the whole conversation. Liquid AI says 13 of the 24 layers in the text model are the cheap convolution type. The point of the mix is that the model spends its expensive look-back budget only where it is needed, which is what keeps a long context affordable on a small CPU.

What does 4-bit quantisation actually do?

A model's weights are normally stored as 16-bit numbers. Quantisation rounds them to fewer bits - 4-bit means each weight takes a quarter of the space - at the cost of a small loss in precision. It is the same trick behind the Q4 models in the speed table below. According to Liquid AI, the 1.7-billion-parameter LFM3 text model needs about 1 GB at 4-bit precision, which puts it in the same size bracket as Qwen 2.5 1.5B rather than the 2 GB-plus Llama and Phi options.

Why a 1 GB model with a 256K context window matters on a Pi

The context window is how much text the model can hold in view at once. Liquid AI quotes 256,000 tokens for LFM3-1.7B, and says a full 128K-token context needs about 1.6 GB of working memory (the KV cache) on top of the weights. On a Pi 5 8GB, a 1 GB model plus a few gigabytes of cache leaves room for the operating system, Home Assistant add-ons and anything else you run. That combination is what an always-on agent needs: a smart-home assistant that keeps a day of sensor events, a calendar and your house rules in view, or a background job that sorts notes and email without a cloud subscription.

Keep expectations grounded. A Pi 5 still has to read every token you feed it, and the long-context limits described in "Where a Pi 5 falls short" below still apply - filling a huge context on a CPU takes a long time. The practical win is a capable model that leaves memory free, not instant answers over a whole book.

What does Liquid AI claim for it?

These are the vendor's own figures from its launch announcement, not our measurements:

• Smaller and better than its predecessor. LFM3-1.7B outperforms the earlier LFM2.5-2.6B while using 35% fewer parameters.
• Vision with video input. LFM3-VL-2B is the first Liquid vision model to accept video, and Liquid AI reports it matching Google's Gemma 4 12B on average across 62 vision benchmarks.
• Built for agents. The text model is post-trained for tool calling and agent frameworks, and Liquid AI's own tables show it leading much larger models on instruction-following and function-calling tests, while larger models still win on coding and general knowledge.
• Cheap to customise. Every model in the family can be fine-tuned on a single GPU - not on the Pi itself, but on a desktop, before you copy the result across.

We have not benchmarked LFM3 on a Pi 5, so it is not in our tokens-per-second table below. Liquid AI lists Ollama, LM Studio and llama.cpp among its supported local runtimes; check the model card on Hugging Face for the current download and licence terms before you pull it.

Independent benchmarks across the community converge on the numbers below for a Pi 5 8GB running Pi OS 64-bit with a stock active cooler and the standard Ollama build (Q4_K_M quantisation, default context size). These are CPU-only - VideoCore VII does not accelerate matmuls in current llama.cpp builds, so adding a GPU isn't an option.

For context: ChatGPT-3.5 streams responses at roughly 30–60 tokens per second. A Pi 5 running Llama 3.2 3B is two to ten times slower. That sounds bad until you remember it's running on 7 watts at the end of your hallway, with no monthly bill.

A Pi 5 draws around 3 watts at idle, climbs to 7–9 watts during active inference, and peaks briefly around 12 watts when all four cores are flat out. Average it across a setup that handles a handful of voice queries an hour plus the odd LAN chat, and you land at roughly 5 watts averaged across the day.

At a UK Energy Price Cap unit rate of around 25p per kWh (the exact figure shifts every three months - check Ofgem's current cap), that works out at:

5W × 24h × 30 days ÷ 1000 = 3.6 kWh/month × £0.25 = around 90p per month.

That's the entire running cost of a private inference endpoint. By comparison, leaving an old desktop running idle 24/7 costs £6–10 per month before you've even invoked the model. The Pi's electricity-cost advantage is the second-biggest argument for using one, after the privacy story.

If you outgrow a Pi 5, the realistic next steps are an Apple Mac mini M4, a dedicated mini-PC with a discrete GPU, or repurposing a desktop tower. Each makes different trade-offs.

*Mixed usage, 24/7 uptime, 25p/kWh.

For most home setups the Pi wins on total cost of ownership unless you specifically need the speed for code completion, large-context document Q&A, or a multi-user setup. Even there, an M4 Mac mini is the sensible upgrade - silent, fast, and only £1–3/month to run. The gaming-GPU path makes sense only if you already own the box.

Without remote access, a local LLM is only useful when you're sat on the home network. Tailscale solves this in about ten minutes: it gives every device in your account a stable private IP and DNS name on an overlay network, with no port-forwarding, no dynamic DNS, and no public exposure of port 11434.

Setup

On the Pi: curl -fsSL https://tailscale.com/install.sh | sh followed by sudo tailscale up. The command prints a one-time login URL - open it on a laptop, sign in, and the Pi joins your Tailnet.

On your phone or laptop, install the Tailscale app and sign in to the same account. The Pi is now reachable at http://pi-llm:11434 - or its 100.x.y.z Tailscale IP - from any other device on the Tailnet, anywhere in the world. The free tier covers up to 100 devices and 3 users, which is plenty for a household.

Don't expose Ollama directly to the public internet via port-forwarding. It has no built-in auth and no rate limiting. Tailscale's WireGuard tunnel is the simple, safe answer.

Since the Home Assistant 2025.1 release, Ollama is a first-class conversation agent. You can route Voice Assistant queries to your Pi-hosted model instead of OpenAI - keeping the entire voice pipeline (wake word, speech-to-text, LLM, text-to-speech) on your own hardware.

Connecting Ollama to HA

In Home Assistant, open Settings → Devices & Services → Add Integration and search for Ollama. For the URL, enter http://pi-llm:11434 (or the Tailscale name if HA is on a different host). Pick the model you pulled earlier from the dropdown - Llama 3.2 3B is a good starting point for voice. Tweak the system prompt to constrain the assistant to your house.

Routing voice queries

Open Settings → Voice Assistants, edit your assistant, and set the conversation agent to Ollama. The default HA conversation engine handles structured intents ("turn off the lounge lamp") while Ollama fields the open-ended questions ("what should I have for dinner that uses what's in the fridge?"). The handoff is built into HA - you don't need to script it.

For more pattern examples and ready-made automations, see our companion guide on Home Assistant automation examples for beginners.

Setting realistic expectations matters. A Pi 5 is not a workstation. It will struggle with the following:

• Long-context reasoning. Stuffing a 20,000-token document into Llama 3.2 3B for summarisation works but takes 90+ seconds to first token. For RAG-style document Q&A, prefer to retrieve short relevant chunks instead of feeding the whole file.
• Coding assistance. 3B models are passable for one-liners and explaining code but make obvious errors on anything non-trivial. If coding is the use case, the budget gap to a Mac mini M4 is worth closing.
• Multiple concurrent users. Ollama processes requests serially per model. A household where two people might ask simultaneously will see one query queue. For multi-user setups, you want a faster machine, not just more cores.
• Image generation. Diffusion models are out of scope on a Pi 5 - even tiny ones take minutes per image. For local image generation, see our beginner's guide to local AI image generators.

Within its lane - short conversational answers, lightweight tool use, voice-assistant fallback, draft text generation - the Pi 5 is genuinely usable.