Local LLM hardware shopping has rarely been as confusing as it is in August 2026. The open-weight models you would want to run at home have never been stronger — Gemma 4, Qwen3.5, gpt-oss and Meta’s freshly released Muse Glimmer are all capable, permissively licensed and free to download. The machines you would normally buy to run them, however, have drifted a very long way from their launch prices, and the old advice about which GPU to pick no longer survives contact with a price tracker.
The culprit is memory. GDDR7 supply pressure pushed median RTX 50-series prices up by as much as 41% in August 2026, with TechPowerUp calling MSRP “a distant dream”. Memory reportedly accounts for more than 80% of the bill of materials on some high-end cards. An RTX 5090 that launched at $1,999 now sells for roughly $4,600 on the street, and even mainstream cards carry heavy premiums.
That inflation has flipped the local LLM hardware value calculus towards unified-memory machines — a $1,999 Framework Desktop with 128GB, a 128GB M5 Max MacBook Pro, or a 512GB Mac Studio — for anything beyond the mid-size model tiers. This guide, part of our AI models and tools hub, stays strictly on the inference-shopping side of artificial intelligence at home. If you later want to train LLMs on your own data, fine-tuning is a separate discipline with its own, much heavier requirements.
Below, we map every model tier to real local LLM hardware with real August 2026 street prices, explain how quantisation changes the arithmetic, compare NVIDIA, AMD and Apple Silicon head to head, and finish with the software stack and a budget playbook. Every price, benchmark and date in this guide comes from the sources listed in the References section at the end.
Table of contents
- Why Local LLM Hardware Looks Different in 2026
- Local LLM Hardware Requirements by Model Tier
- Quantisation: How to Shrink Local LLM Hardware Needs
- NVIDIA GPUs: The Default Local LLM Hardware Choice
- AMD Radeon: The Price-Sane Alternative
- Apple Silicon as Local LLM Hardware
- Unified-Memory PCs: Strix Halo and DGX Spark
- CPU-Only Inference: The Honest Floor
- Matching 2026’s Open Models to Your Machine
- Software That Unlocks Your Local LLM Hardware
- Budget Playbook: Four Price Points That Make Sense
- Mistakes That Waste Money in 2026
- Frequently Asked Questions
- References
Why Local LLM Hardware Looks Different in 2026
Three forces reshaped local LLM hardware buying this year: a memory-price crisis on discrete GPUs, a maturing generation of unified-memory machines, and open models engineered for specific memory budgets. Understanding all three is what separates a confident purchase from an expensive mistake, so we start with the market before we touch a single spec sheet.
The 2026 memory-price crisis in numbers
Price tracking tells the story bluntly. The RTX 5090 was listed at $4,598.90 on 1 August 2026 — roughly 130% above its $1,999 MSRP — having already opened its tracking period on 10 June 2026 at $4,269.95. TechPowerUp put the card’s median price about 135% above MSRP in the same month, in the same report that measured median RTX 50-series prices jumping up to 41% in August alone.
The pain reaches the mainstream too. The RTX 5060 Ti 16GB hit a median of $804.99 in August 2026, 88% above its $429 launch price, while Newegg medians showed the RTX 5070 up about 36% and the RTX 5060 up about 27%. No tier of the local LLM hardware market escaped the squeeze, which is why this guide leans on street prices rather than the fantasy of MSRP.
What local inference demands from a machine
An LLM generates text one token at a time, and for every single token the runtime must read the model’s active weights out of memory. That makes two numbers decisive for local LLM hardware: how much fast memory you have (capacity) and how quickly the processor can read it (bandwidth). Capacity decides which models load at all; bandwidth largely decides how many tokens per second you get once they do.
Raw compute — TFLOPs, core counts, shiny AI-TOPS figures — matters far less than GPU marketing implies. A card that cannot hold your model produces zero tokens per second regardless of how fast its cores are, and that framing will recur throughout this local LLM hardware guide.
Bandwidth rules local LLM hardware performance
The gap between platforms is dramatic. A dual-channel DDR5-6000 desktop moves about 96 GB/s, while an RTX 5090’s GDDR7 moves 1,792 GB/s — nearly nineteen times more. That single ratio explains why 13B-plus models are considered impractical for CPU-only inference on consumer desktops, and why the same model can feel instant on one machine and glacial on another.
Keep bandwidth in mind throughout this guide: it is the quiet specification that separates good local LLM hardware from expensive disappointment, and we chart the full bandwidth ladder in the unified-memory section below.
Local LLM Hardware Requirements by Model Tier
The table below maps each model-size tier to the memory it actually needs. The figures follow 2026 guidance that includes KV-cache and runtime overhead, not just raw weights — treat them as working floors for local LLM hardware planning rather than theoretical minimums.
| Model tier | Memory at 4-bit | Memory at FP16 | Hardware that fits |
|---|---|---|---|
| 7B | 6–8GB | 16–20GB | Any 8GB+ GPU, base M5 Mac |
| 13B–22B | 12–16GB | — | 16GB cards: RTX 5060 Ti, 5070 Ti, 5080, RX 9070 XT |
| 27B–33B | 20–28GB | — | RTX 5090 (32GB), 32GB+ Macs, unified memory |
| 70B | 40–48GB (35–40GB Q4_K_M) | 140–160GB | Two 24GB cards, one 48GB workstation card, 64GB+ unified memory |
| 100B+ MoE | 60–70GB (120B class) | — | DGX Spark, Framework Desktop 128GB, Mac Studio |
The 7B tier: entry-level local LLM hardware
A 7B model needs roughly 6–8GB of VRAM at 4-bit quantisation, against 16–20GB at FP16. Current 2026 guidance puts about 8GB of VRAM as the working floor for the tier once KV-cache and runtime overhead are counted. Practically any modern discrete GPU with 8GB or more, or a base Apple M5 configuration, clears that bar without drama.
This is where most people should start. Small models are fast, cheap to serve and surprisingly capable in 2026 — Qwen3.5 alone ships 9B, 4B, 2B and 0.8B open-weight variants — and the lessons you learn at 7B transfer directly to bigger local LLM hardware decisions later.
The 13B–22B tier: the 16GB sweet spot
Roughly 12–16GB of memory covers the 13B–22B tier at sensible quantisation. It is no accident that OpenAI’s gpt-oss-20b was explicitly designed to run within 16GB of memory using MXFP4 4-bit quantisation: 16GB is the most common ceiling on affordable local LLM hardware in 2026, and model builders now target it deliberately.
Every current 16GB card serves this tier well — the RTX 5060 Ti 16GB, RTX 5070 Ti and RTX 5080 on the NVIDIA side, and the Radeon RX 9070 XT on AMD’s. Gemma 4’s 12B, with an official BF16 footprint of 26.7GB, also drops comfortably into 16GB once quantised to 4-bit-class formats.
The 27B–33B tier: where 16GB runs out
A 33B-class model needs roughly 20–28GB at 4-bit, and the current crop sits exactly there. Meta’s Muse Glimmer, at ~29.6B parameters, requires at least 26GB of RAM at its ~4-bit compression according to LM Studio’s model page. Qwen3.5’s 27B dense model and Gemma 4’s larger variants keep the same company.
That pushes you past every 16GB card to a 32GB RTX 5090, a 32GB-plus Mac or a unified-memory PC. This tier is 2026’s quality sweet spot, which makes the missing 24GB consumer card — covered below — the single most frustrating gap in the local LLM hardware market.
The 70B tier: where local LLM hardware gets serious
A 70B model needs roughly 40–48GB of VRAM at 4-bit — about 35–40GB with a Q4_K_M file — versus a staggering 140–160GB at FP16. No single consumer GPU holds that. The realistic options are two 24GB cards working together, one 48GB workstation card, or a unified-memory machine with 64GB or more.
For most buyers this is the point where Apple Silicon or Strix Halo becomes cheaper than stacking discrete GPUs, especially at 2026 street prices. The local LLM hardware arithmetic simply favours pooled memory once requirements pass the 32GB consumer ceiling.
100B-plus and MoE models: the unified-memory tier
Mixture-of-experts flagships stretch further still. gpt-oss-120b targets a single 80GB GPU; on a 128GB DGX Spark, a 120B model at 4-bit occupies roughly 60–70GB, leaving comfortable space for context. At the extreme end, a 512GB Mac Studio runs the 671B-parameter DeepSeek R1 in 4-bit — a ~404GB model — at roughly 17–18 tokens per second.
Local LLM hardware for this tier is a specialist purchase measured in thousands of dollars, and we compare the three realistic candidates in the unified-memory section below.
Quantisation: How to Shrink Local LLM Hardware Needs
Quantisation — storing model weights at lower numerical precision — is the single biggest lever you control as a buyer. It routinely turns an impossible model into a comfortable one without changing a single component in your local LLM hardware, which is why every tier table in this guide quotes 4-bit figures first.
What Q4_K_M means for local LLM hardware buyers
The most popular GGUF preset, Q4_K_M, averages about 4.5 bits per weight rather than a uniform 4 bits, because it keeps the most sensitive layers at higher precision. The effect on memory is dramatic all the same: Llama 2 70B drops from 140GB at FP16 to about 38GB at Q4_K_M — a 3.7x reduction from one download choice.
One quantisation step cuts a 70B model’s memory footprint by 3.7x, per WillItRunAI’s GGUF quantisation guide:
GGUF and CPU offloading
GGUF is the standard quantised-weights format used by both Ollama and llama.cpp, and its trump card is offloading: llama.cpp can split a model’s layers between GPU VRAM and system RAM, so a model larger than your VRAM still runs — just at reduced speed. The format has become the lingua franca of the whole local stack.
Offloading is the safety valve of local LLM hardware: it converts “will not load” into “loads, slowly”. Buy for the speed you want on your everyday models, but know the fallback exists for the occasional oversized experiment.
Picking a quantisation level
llama.cpp supports quantisation all the way from 1.5 to 8 bits, so there is a genuine dial to turn rather than a single switch. The practical 2026 convention is simple: 4-bit-class files such as Q4_K_M are the default trade-off between quality and size, higher-bit files are worth it when you have memory to spare, and very low-bit files are a last resort when nothing else fits your local LLM hardware.
The tier table earlier already assumes this convention, which is why its 4-bit column is the one to read first when matching models to local LLM hardware budgets.
NVIDIA GPUs: The Default Local LLM Hardware Choice
CUDA support remains the widest in the ecosystem, and every serious runner treats NVIDIA as a first-class target. The problem in 2026 is not capability — it is what GDDR7 pricing has done to street values across the whole stack. Here is the line-up as it actually stands in August 2026.
| Card | VRAM | Launch MSRP | Aug 2026 street price | Notes |
|---|---|---|---|---|
| RTX 5090 | 32GB GDDR7 | $1,999 | $4,598.90 (1 Aug) | 1,792 GB/s bandwidth |
| RTX 5080 | 16GB GDDR7 | $999 | ≈$1,500 | 960 GB/s bandwidth |
| RTX 5070 Ti | 16GB GDDR7 | — | ≈$1,100 | 256-bit, 28 Gbps, 300W |
| RTX 5060 Ti 16GB | 16GB GDDR7 | $429 | $804.99 median | 88% above MSRP |
| Radeon RX 9070 XT | 16GB GDDR6 | $599 | $649–719 (mid-2026) | Cheapest new 16GB card |
RTX 5090: the consumer local LLM hardware ceiling
The RTX 5090 pairs 32GB of GDDR7 with 1,792 GB/s of memory bandwidth — the largest and fastest memory on any consumer card. As local LLM hardware it is superb: everything up to the 27B–33B tier runs on a single card with room to spare, at token speeds nothing else in this guide matches.
The price is the wound. Launched at $1,999 on 30 January 2025, it listed at $4,598.90 on 1 August 2026 after opening its tracking window in June at $4,269.95. You are paying roughly 130% over MSRP, essentially for that 32GB of GDDR7.
RTX 5080 and RTX 5070 Ti: fast but capped at 16GB
The RTX 5080 offers 16GB of GDDR7 at 960 GB/s with a $999 MSRP; in August 2026 tracking it sold at around $1,500, up 3% month-on-month from $1,462. The RTX 5070 Ti — 16GB on a 256-bit bus at 28 Gbps, with 8,960 CUDA cores and a 300W board — held around $1,100 in the same tracking.
Both are excellent engines for the 13B–22B tier, and the 5070 Ti in particular is the cheapest current NVIDIA card with 16GB above the 5060 Ti tier. Neither can touch the 27B-plus tier alone, however, which limits their long-term headroom as local LLM hardware.
RTX 5060 Ti 16GB: the squeezed budget pick
On paper the RTX 5060 Ti 16GB, launched on 16 April 2025 at $429, is the obvious entry card for local inference. In practice its median US price surged to $804.99 in August 2026 — 88% above MSRP — eroding exactly the value that made it the default recommendation for a year.
If you can find one near its launch price it remains perfectly good starter local LLM hardware for the 7B-to-20B range. At $805, the Radeon alternative below deserves your attention first.
The 24GB card missing from the local LLM hardware market
To be clear: no 24GB RTX 5080 Super exists to buy. The rumoured 24GB RTX 5080 Super and RTX 5070 Ti Super were reportedly finished and sitting with board partners by late July 2026, but NVIDIA held the launch back because the 3GB GDDR7 modules they require cost roughly three times more than standard chips.
The consumer NVIDIA VRAM ceiling therefore remains 32GB, and there is a gaping hole between 16GB and 32GB — exactly where the 27B–33B tier’s ideal card should sit. Street premiums over MSRP in August 2026 tracking show how unevenly that shortage bites:
AMD Radeon: The Price-Sane Alternative
AMD has quietly become the value answer for 16GB local LLM hardware, largely because GDDR6 has not inflated the way GDDR7 has. For buyers who refuse to pay double MSRP on principle, Radeon is where that principle survives 2026 intact.
Radeon RX 9070 XT: value local LLM hardware
The RX 9070 XT carries 16GB of GDDR6 on a 256-bit bus, launched at a $599 MSRP, and sold for roughly $649–719 in mid-2026 — one of the cheapest new 16GB cards on the market. Against an $805 RTX 5060 Ti, it is the clear price-per-gigabyte winner among new cards.
Sixteen gigabytes covers the whole 13B–22B tier, including gpt-oss-20b’s designed-for-16GB footprint, which makes this card the most sensible sub-$750 local LLM hardware purchase of the moment.
Software support: Vulkan and HIP have matured
The old objection to AMD — weak inference software — has faded. llama.cpp ships first-class Vulkan and HIP (AMD) backends alongside CUDA, Metal, SYCL and WebGPU, and Jan added dedicated AMD HIP support in its v0.8.2 release in June 2026. For mainstream GGUF inference, Radeon owners are no longer second-class citizens.
The practical consequence for local LLM hardware buyers: pick the memory you need first, and let the runner handle the vendor. All four major tools in this guide run on AMD today.
Who should choose AMD
Pick Radeon when your target is the 7B–22B tier and your budget sits under $750. The 16GB ceiling binds at 27B-plus exactly as it does on NVIDIA’s mid-range — no amount of driver polish changes the memory arithmetic of local LLM hardware, and AMD currently offers no consumer answer to the 5090’s 32GB.
Apple Silicon as Local LLM Hardware
Apple’s unified-memory design lets the GPU address the machine’s entire memory pool, which quietly turns Mac configuration menus into VRAM menus. In a year when discrete VRAM is savagely overpriced, that has made Macs genuinely competitive local LLM hardware rather than a curiosity for enthusiasts.
The base M5: a capable entry point
The base M5 chip, announced on 15 October 2025 in the 14-inch MacBook Pro, delivers 153 GB/s of unified memory bandwidth — nearly 30% more than M4 — and configures up to 32GB. That comfortably covers the 7B tier and reaches into 13B territory at 4-bit quantisation.
For anyone already buying a MacBook for other work, the base M5 is free local LLM hardware in practice: no extra purchase, a Neural-Accelerator-equipped GPU, and enough memory for genuinely useful small models.
M5 Pro and M5 Max: 64GB and 128GB laptops
The M5 Pro supports up to 64GB of unified memory at 307 GB/s; the M5 Max reaches 128GB at 614 GB/s. Both debuted in the 14- and 16-inch MacBook Pro with pre-orders from 4 March 2026, and Apple claims up to 4x the AI performance of the previous MacBook Pro generation, with a Neural Accelerator in every GPU core.
A 128GB M5 Max is remarkable local LLM hardware by any standard: it holds a 70B model at 4-bit — which needs 35–48GB — with enormous headroom left for context, in a machine that runs on battery.
Mac Studio M3 Ultra: the 512GB desktop ceiling
Apple’s desktop flagship configures up to 512GB of unified memory, at $9,499 with 1TB of storage. That is enough to run the 671B-parameter DeepSeek R1 locally in 4-bit — a ~404GB model — at roughly 17–18 tokens per second. Nothing else consumer-adjacent comes close to that capacity.
It remains the desktop ceiling of Apple’s line-up in August 2026, and the only way short of server hardware to hold frontier-scale open models in one box.
When a Mac beats discrete local LLM hardware
The crossover point is 32GB. Below it, a discrete GPU is usually the faster tool for the money. Above it, Apple’s capacity pricing wins by default: no consumer NVIDIA card exceeds 32GB at any price, while Mac configurators go on to 64GB, 128GB and 512GB.
For 70B-plus ambitions on local LLM hardware, Apple Silicon is now the pragmatic default rather than the contrarian pick — a sentence that would have sounded absurd before the 2026 memory crisis.
Unified-Memory PCs: Strix Halo and DGX Spark
Two PC-side answers copy Apple’s trick of pooling large memory behind an integrated processor — and one of them is 2026’s best-value machine for big models on local LLM hardware budgets.
| Machine | Unified memory | Bandwidth | Price | Headline capability |
|---|---|---|---|---|
| Framework Desktop (Ryzen AI Max+ 395) | 128GB LPDDR5X-8000 | — | $1,999 (mid-2026) | ~100 tok/s on Qwen3-30B-class MoE; loads 120B models |
| NVIDIA DGX Spark | 128GB LPDDR5x | 273 GB/s | $4,699 (Feb 2026; launched $3,999) | 120B at 4-bit in 60–70GB; ~200B with aggressive quantisation |
| MacBook Pro M5 Max | Up to 128GB | 614 GB/s | — | 70B at 4-bit in a laptop |
| Mac Studio M3 Ultra | Up to 512GB | — | $9,499 (512GB, 1TB storage) | DeepSeek R1 671B at ~17–18 tok/s |
Framework Desktop: unified-memory local LLM hardware
AMD’s Ryzen AI Max+ 395 “Strix Halo” pairs 16 Zen 5 cores with a Radeon 8060S iGPU (40 RDNA 3.5 compute units), a 50 TOPS XDNA 2 NPU and up to 128GB of LPDDR5X-8000 unified memory. In the Framework Desktop, the full 128GB configuration cost $1,999 in mid-2026.
The platform reaches about 100 tokens per second on Qwen3-30B-class MoE models and can load 120B-parameter models that no single consumer discrete GPU can hold. At the RTX 5090’s launch MSRP — and well under half its current street price — it is the local LLM hardware bargain of the year.
NVIDIA DGX Spark: CUDA in a 128GB box
NVIDIA’s DGX Spark builds on the GB10 Grace Blackwell superchip: 128GB of unified LPDDR5x at 273 GB/s and about 1 PFLOP of sparse FP4 compute. It launched at $3,999 in October 2025 and rose to $4,699 in February 2026 amid the same memory supply constraints squeezing the rest of the market.
Capacity plus ecosystem is the sell: a 120B model at 4-bit occupies roughly 60–70GB of its 128GB, ~200B-parameter models fit with aggressive quantisation, and the box speaks native CUDA for local LLM hardware workflows that depend on it.
Where unified memory wins — and where it loses
Unified memory buys capacity, not speed. Memory bandwidth decides tokens per second, and the spread across 2026 local LLM hardware is enormous:
A $4,600 RTX 5090 will generate tokens far faster than a $1,999 Framework Desktop on any model both can hold. The unified-memory machines win by holding models the GPU cannot load at all. Decide which failure mode you can live with — slow tokens, or no tokens — and your local LLM hardware shortlist writes itself.
CPU-Only Inference: The Honest Floor
You can start with no GPU at all — as long as you calibrate expectations with measured numbers rather than hope. This is the cheapest local LLM hardware tier because it is hardware you already own.
CPU-only local LLM hardware: what actually works
A dual-channel DDR5 desktop in the Ryzen 7 class manages about 14 tokens per second on a 7B model at Q4, and about 35 tokens per second on a 3B model. An older DDR4 laptop manages roughly 8 tokens per second on the same 7B class. Those speeds are perfectly usable for chat, drafting and light assistant work.
Why 13B-plus is impractical without a GPU
The mathematics is bandwidth again: about 96 GB/s from dual-channel DDR5-6000 versus 1,792 GB/s on an RTX 5090. Even a dual-Xeon server with eight memory channels only reaches around 3–5 tokens per second on a 70B model — which is why 13B-plus models are considered impractical CPU-only on consumer desktops.
If your ambitions stop at small models, though, this floor is higher than most people expect, and it costs nothing to test before committing to new local LLM hardware.
Sensible CPU-only use cases
CPU-only setups suit background summarisation, scripted batch jobs and evaluating the ecosystem before spending money. Treat the machine you own as a free trial of local LLM hardware: run a 3B–7B model this afternoon, note what frustrates you, and let that frustration write your upgrade shortlist.
Matching 2026's Open Models to Your Machine
Hardware only matters in relation to the models you plan to run, and the August 2026 open-weight generation is unusually explicit about its memory demands. Here is how the four headline families map onto real local LLM hardware.
Gemma 4 on consumer local LLM hardware
Google DeepMind released Gemma 4 on 2 April 2026 in five sizes — E2B, E4B, 12B, a 26B A4B MoE and a 31B dense — with official BF16 memory footprints of 11.4GB, 17.9GB, 26.7GB, 57.7GB and 69.9GB respectively, and 128K–256K context windows across the family.
Note the MoE catch: the 26B A4B activates only ~3.8B parameters per token, which is excellent for speed, but it still requires all ~25.2B parameters loaded in memory. Mixture-of-experts saves compute, not capacity — a distinction that trips up plenty of local LLM hardware planning.
Qwen3.5: a size for every machine
Alibaba shipped the Qwen3.5 open-weight family between 16 February and 2 March 2026, all Apache 2.0: a 397B-A17B MoE flagship, then 122B-A10B, 35B-A3B and a 27B dense model, then 9B, 4B, 2B and 0.8B small models. It is the widest size ladder in open weights today.
Whatever your local LLM hardware budget, there is a Qwen3.5 that fits it — from a 0.8B that runs on anything, to the 35B-A3B that flies on a Framework Desktop, to flagships that need the unified-memory tier.
gpt-oss: engineered for 16GB and 80GB
OpenAI’s gpt-oss pair is refreshingly explicit about hardware targets: gpt-oss-20b is designed to run within 16GB of memory using its MXFP4 4-bit quantisation, while gpt-oss-120b runs on a single 80GB GPU. The design targets read like a map of the current local LLM hardware landscape.
If you own any 16GB card from the tables above, gpt-oss-20b is the obvious first download; the 120B belongs on the DGX Spark class of machine.
Muse Glimmer: Meta’s agentic comeback
Meta returned to open source in August 2026 with Muse Glimmer — an Apache 2.0, ~29.6B-parameter agentic model, including a 1.8B vision encoder, with a 131,072-token context window, positioned to run on a single consumer GPU or a Mac.
Mind the floor: LM Studio’s model page puts the smallest Muse Glimmer build at 26GB of RAM minimum at its ~4-bit compression. That is beyond every 16GB card, and squarely in RTX 5090, 32GB-Mac or unified-memory territory for local LLM hardware owners planning around it.
Software That Unlocks Your Local LLM Hardware
The runner you choose decides how much of your machine’s potential you actually use. Four tools dominate in 2026, each strongest on a different slice of the local LLM hardware landscape, and all four are free.
| Tool | Latest release (Aug 2026) | Strongest on | Standout feature |
|---|---|---|---|
| Ollama | v0.32.9 (11 Aug 2026) | NVIDIA + Apple (MLX runtime) | Interactive agent CLI, day-one model support |
| llama.cpp | ~123.8k GitHub stars | Everything: CUDA, Metal, Vulkan, HIP, SYCL, WebGPU | Dependency-free C/C++, OpenAI-compatible server |
| LM Studio | 0.4.21 (12 Aug 2026) | Desktop GUI, multi-GPU rigs | Tensor parallelism; Locally iPhone/iPad companion |
| Jan | v0.8.4 (23 Jul 2026) | AMD HIP, open-source purists | Local OpenAI-compatible API |
Ollama: the default local LLM hardware companion
Ollama’s latest release was v0.32.9 on 11 August 2026, and its 2026 run has been aggressive: a dedicated MLX runtime for Apple Silicon, Gemma 4 support, day-one Muse Glimmer and Nemotron support, and a v0.32 CLI reworked into an interactive agent. It remains the easiest on-ramp on any platform in this guide.
llama.cpp: the engine underneath
llama.cpp — roughly 123.8k GitHub stars — is the dependency-free C/C++ engine most local tools build upon. It supports CUDA, Metal, Vulkan, HIP (AMD), SYCL (Intel) and WebGPU backends, 1.5- to 8-bit quantisation, a built-in web UI and an OpenAI-compatible server.
When you want maximum control over local LLM hardware — custom offloading splits, exotic quantisation levels, unusual backends — you skip the wrappers and drive it directly.
LM Studio: the polished desktop
LM Studio reached 0.4.21 on 12 August 2026. Its 0.4.15 release (May 2026) added tensor parallelism for multi-GPU loading — directly relevant to anyone running two cards for the 70B tier — and 0.4.16 (June 2026) raised the default context to 8k tokens and introduced the Locally companion app for iPhone and iPad streaming over LM Link.
If reaching your desktop’s models from a phone matters to you, our guide to local LLM mobile app integration covers that whole workflow.
Jan: open source and API-first
Jan reached v0.8.4 on 23 July 2026; its v0.8.2 release (June 2026) added AMD HIP support and context-size auto-adjustment, and the app serves an OpenAI-compatible API locally, so anything built for cloud endpoints can point at your own machine instead. Our Jan AI setup walkthrough covers installation end to end.
AirLLM: when the model will not fit
If a model exceeds your memory even at 4-bit, layer-offloading tools attack the problem by loading the network one slice at a time. We explain the technique in our AirLLM explainer — the short version is that patience substitutes for local LLM hardware, at a heavy cost in speed. It is a workaround worth knowing, not a strategy worth buying around.
Budget Playbook: Four Price Points That Make Sense
Combining the street prices and memory floors above, here is how we would actually spend the money in August 2026, bracket by bracket.
Local LLM hardware under $1,000
The Radeon RX 9070 XT at roughly $649–719 is the strongest new-card value: 16GB covers everything up to the 22B tier, including gpt-oss-20b’s designed-for-16GB footprint. The RTX 5060 Ti 16GB does the same job with CUDA attached, but its $804.99 median price makes it the second choice rather than the first.
Add nothing else. At this budget, every dollar should go to memory, and both cards drop into a PC you already own.
$1,000–$2,500: the value tier
This bracket holds 2026’s standout: the $1,999 Framework Desktop with 128GB of Strix Halo unified memory, which loads models a $4,600 RTX 5090 cannot. The alternative spend — an RTX 5070 Ti at around $1,100 inside an existing PC — buys much faster 16GB inference instead of capacity.
$2,500–$5,000: serious capability
Three very different machines compete here: a 128GB M5 Max MacBook Pro (fast, portable, 614 GB/s), the $4,699 DGX Spark (128GB, native CUDA) and the $4,598.90 RTX 5090 (fastest tokens of all, but a 32GB cap). Choose by whichever of capacity, CUDA or raw speed leads your local LLM hardware priorities — there is no single right answer in this bracket.
$5,000 and up: the frontier at home
The 512GB Mac Studio at $9,499 stands alone, running the 671B-parameter DeepSeek R1 at a usable 17–18 tokens per second. Buyers at this level are usually equipping a team rather than a hobby; our AI and machine learning services and AI strategy consulting exist for exactly this kind of deployment planning.
Mistakes That Waste Money in 2026
The most expensive errors in this market are strategic rather than technical, and three of them come up constantly in local LLM hardware conversations.
Buying compute when you need memory
A faster GPU with too little VRAM loses to a slower machine with enough of it. Memory capacity is binary — the model loads or it does not — so size your local LLM hardware for the models you actually want to run, then spend whatever budget remains on speed. Never the other way around.
Forgetting context and KV-cache overhead
Weights are not the whole bill. The 2026 VRAM guidance quoted throughout this guide already includes KV-cache and runtime overhead, and defaults keep growing — LM Studio raised its default context to 8k tokens in June 2026. Leave headroom beyond the raw file size, or your comfortable fit becomes an out-of-memory error mid-conversation.
Waiting for local LLM hardware that may never ship
The 24GB RTX 5080 Super is the cautionary tale: reportedly finished, sitting with board partners since late July 2026, and still unlaunched because its 3GB GDDR7 modules cost roughly three times more than standard chips. Buy the machine that exists today, and treat rumoured cards as a pleasant surprise if they ever arrive.
Frequently Asked Questions
What local LLM hardware do I need for a 70B model?
Roughly 40–48GB of memory at 4-bit, or about 35–40GB with Q4_K_M files. In practice that means two 24GB GPUs, one 48GB workstation card, or a unified-memory machine such as a 64GB-plus Mac, a 128GB Framework Desktop or a DGX Spark.
Is Apple Silicon good local LLM hardware in 2026?
Yes — above the 16GB tier it is often the value leader. The M5 Max offers 128GB of unified memory at 614 GB/s in a laptop, and the Mac Studio configures to 512GB, capacities no consumer GPU approaches at any price.
What is the cheapest local LLM hardware to start with?
The computer you already own. A dual-channel DDR5 desktop runs a 7B model at about 14 tokens per second at Q4, which is genuinely usable. The first worthwhile purchase after that is a 16GB card such as the Radeon RX 9070 XT at roughly $649–719.
How much VRAM does a 7B model need?
About 6–8GB at 4-bit quantisation, or 16–20GB at FP16. Guidance for 2026 treats 8GB as the practical floor once KV-cache and runtime overhead are counted, so an 8GB card clears the tier with sensible settings.
Do I need an RTX 5090?
Only if you must run the 27B–33B dense tier on one card. At $4,598.90 street, most buyers get more from a 16GB card plus offloading patience, or from unified-memory local LLM hardware at similar money with four times the capacity.
Can local LLM hardware run 100B-plus models?
Yes — on unified memory. A 120B model at 4-bit fits in roughly 60–70GB, comfortably inside a 128GB Framework Desktop or DGX Spark; gpt-oss-120b targets a single 80GB GPU; and a 512GB Mac Studio runs 671B-class models at usable speeds.
References
Gemma 4 model overview (Google AI for Developers)
Apple debuts M5 Pro and M5 Max (Apple Newsroom)
Apple unleashes M5 (Apple Newsroom)
RTX 50-series median prices jump up to 41% in August (TechPowerUp)
How much VRAM for 7B-70B LLMs (Database Mart)
Mac Studio with M3 Ultra runs DeepSeek R1 671B locally (MacRumors)
More AI coverage: explore Progressive Robot's AI Models, Tools & Releases hub — hands-on reviews, setup guides and benchmarks in one place.