Steven Kramp / SelfHostedSI

Steven Kramp / @SelfHostedSI

Data updated · Last measured

Your memory. My measurements.

I test local AI on my own hardware. Start with your memory budget and see which results I have, and which answers are still a calculation.

The latest published measurement is over 7 days old. These are dated results, not a test of today's newest models.

Mac Studio testing starts after delivery, expected 17–24 November 2026. No Mac results yet.

Some historical records lack exact commands, prompts, driver versions or model checksums. Each download lists what is missing.

Memory budgets

What can I show you for your memory?

Open a budget for measured evidence and separate file-based estimates. These decimal GB budgets do not verify a device or its usable memory. Runtime and file-format checks remain in the finder.

16 GB2 models measured hereFile estimates for 20 variants

Measured evidence

Calculated from recorded file sizes

Estimate: file size × 1.25. No measured speed or format compatibility is inferred. GB/GiB and rounded sizes can change a boundary result.

24 GBFile estimates for 20 variants

Measured evidence

No measured speed, power or loading result for this memory class.

Calculated from recorded file sizes

Estimate: file size × 1.25. No measured speed or format compatibility is inferred. GB/GiB and rounded sizes can change a boundary result.

32 GBFile estimates for 20 variants

Measured evidence

No measured speed, power or loading result for this memory class.

Calculated from recorded file sizes

Estimate: file size × 1.25. No measured speed or format compatibility is inferred. GB/GiB and rounded sizes can change a boundary result.

64 GBFile estimates for 20 variants

Measured evidence

No measured speed, power or loading result for this memory class.

Calculated from recorded file sizes

Estimate: file size × 1.25. No measured speed or format compatibility is inferred. GB/GiB and rounded sizes can change a boundary result.

128 GBFile estimates for 20 variants

Measured evidence

No measured speed, power or loading result for this memory class.

Calculated from recorded file sizes

Estimate: file size × 1.25. No measured speed or format compatibility is inferred. GB/GiB and rounded sizes can change a boundary result.

256 GBFile estimates for 20 variants

Measured evidence

No measured speed, power or loading result for this memory class.

Calculated from recorded file sizes

Estimate: file size × 1.25. No measured speed or format compatibility is inferred. GB/GiB and rounded sizes can change a boundary result.

512 GBFile estimates for 20 variants

Measured evidence

No measured speed, power or loading result for this memory class.

Calculated from recorded file sizes

Estimate: file size × 1.25. No measured speed or format compatibility is inferred. GB/GiB and rounded sizes can change a boundary result.

1 TBFile estimates for 20 variants

Measured evidence

No measured speed, power or loading result for this memory class.

1 TB is a budget across two planned 512 GB machines. It is not one shared memory pool; no cluster result is implied.

Calculated from recorded file sizes

Estimate: file size × 1.25. No measured speed or format compatibility is inferred. GB/GiB and rounded sizes can change a boundary result.

Published measurement history

The latest runs, with their dates

Each card is one model in an approved run. Text generation with a short prompt; recorded medians. Different inputs and methods prevent a controlled speed ranking.

One practical local-AI article a week, by email: subscribe free on Substack

SK / LAB · WILL IT RUNHow this works

What will your machine actually run?

From coding to your own files. Find a place to start, with the memory and evidence in view.

Two measured runs. Maker-backed uses. Memory estimates use file size × 1.25; format checks stay separate.

Change
Your machine & memory
Your values

1 GB = 1,000,000,000 bytes. 1 GiB = 1.073741824 GB. Imports convert binary units; device presets are comparison budgets.

Prefer a hardware readout?

Paste a local command's output below. Multiple GPUs stay separate. RAM is never added to VRAM.

Browser hints cannot confirm your memory. Hardware text stays in this tab. Nothing is sent. Chromium caps its RAM hint at 32 GB; Brave may hide the GPU.

Paste output from Windows, macOS or Linux

Read-only hardware information. No installation, downloads or administrator access. NVIDIA is read only if nvidia-smi is already installed; other GPU memory can be entered manually.

YOUR MEMORY, IN VIEWExample
16GB

RTX 4080

Slide to explore more memory. Arrow keys step between sizes. The scale is logarithmic.

Set your hardware

Your machine, job by job

Our task tests: not run yet

The model files

Open a file for the maker’s description, format check and sources.

Save this and post it

Links use gpu, optional name, mem, ram and job. Open a result link to restore the same device, label, memory and activity. Your label stays in this browser unless you share the link. Shared values remain an example until confirmed.

How we count

A file is the starting point. Qwen used 6.9 GB for a 6.55 GB file (+5%); Gemma used 8.9 GB for 7.15 GB (+24%). We estimate memory as file size × 1.25: an estimate from our two runs, not a guarantee. On our RTX 4080, its measured GPU memory takes the place of that estimate.

Measured fit: recorded memory fits on our measurement device. Estimated fit: estimated need fits and format is supported. File fits, estimate does not: only the file fits. Too big: the file exceeds your memory. Different format needed and unknown support are shown separately. Context length, software and other apps can change what runs.

VRAM, system RAM and shared memory are separate. We never add them. 1 GB = 1,000,000,000 bytes; imports convert MiB and GiB. The 2 TB ruler label means 2,000 GB.

Sources: maker descriptions and format support from the Grok card review (1 November 2026). File sizes and the two RTX 4080 runs come from the supplied prototype and lab catalogue. Five supplemental files use the dated maker format review. Bolt, the official Embedding GGUF and Mistral have exact API byte records checked on 1 November 2026. Bolt image input also needs its separate projector; its discrete recipe starts at 32 GB. The task uses describe model families, not tests of each quantized file.

More memory cannot change an unsupported file format. The ruler names the next supported coding, agents or chat file at a listed memory step, not a hardware recommendation. Other formats without a reviewed runtime stay unknown at any size. GPU runs: €0.35/kWh, excluding the rest of the PC. Prompts differ, so these are no speed ranking. Tokens include reasoning; the first-token wall-time proxy is derived from duration minus generation time (stream:false). No Mac runs yet.

LMArena numbers below are selected from LMArena’s leaderboard dataset, CC BY 4.0. Labels and selection adapted; scores unchanged. They describe a hosted model, not this file, not our run. Text uses Bradley-Terry; Agents uses IPS. We keep the scales separate.

Steven Kramp / SelfHostedSI · Measurements, model pages and data

Approved runs · RTX 4080

Measured so far

These are measured results from my hardware. Short prompts differ between models, so this is not a controlled ranking.

Qwen3.5 9B · Q4_K_M

Measured: 93.1 tok/s · short prompt, recorded medians.

First token (wall)
2.13 s · engine: 0.09 s
GPU memory used
6.9 GB
GPU power only
234.8 W
Configured context window
16,384 tokens
Input tokens (median)
35
Electricity per 1M tokens, short prompt
€0.25
Electricity per 1M tokens, long prompt
€0.26
Electricity per 1M tokens, whole PC at the wall
not measured yet

Electricity: GPU power only at €0.35/kWh, per 1M output tokens. Hardware cost not included; not an API price.

All generated tokens incl. reasoning.

Wall TTFT method: derived from request duration minus generation time (non-streaming).

Power sensor: power.draw.instant.

NVIDIA GeForce RTX 4080 · .

Conditions and limits · Source: MESSUNG_2026-10-27.json

Gemma 4 12B QAT · Q4_0

Measured: 73.9 tok/s · short prompt, recorded medians.

First token (wall)
2.12 s · engine: 0.13 s
GPU memory used
8.9 GB
GPU power only
235.8 W
Configured context window
16,384 tokens
Input tokens (median)
41
Electricity per 1M tokens, short prompt
€0.31
Electricity per 1M tokens, long prompt
€0.36
Electricity per 1M tokens, whole PC at the wall
not measured yet

Electricity: GPU power only at €0.35/kWh, per 1M output tokens. Hardware cost not included; not an API price.

All generated tokens incl. reasoning.

Wall TTFT method: derived from request duration minus generation time (non-streaming).

Power sensor: power.draw.instant.

NVIDIA GeForce RTX 4080 · .

Conditions and limits · Source: MESSUNG_2026-10-27.json

File size is a starting point; loading, context and runtime overhead need a real run.

How we count

Fits on paper assumes decimal comparison budgets from 16 GB to 1 TB (1 GB = 1,000,000,000 bytes). These are not verified usable hardware capacities; GB/GiB can change the answer. For example, Muse NVFP4 exceeds 512 decimal GB but is below 512 GiB. Today's RTX 4080 has 16 GB VRAM and is the only device I measure on. The other device names (RTX 3090/4090/5090, DGX Spark, Mac Studio) only label a budget size; I do not own a DGX Spark. Both Mac Studios are planned and have not been delivered. 2× DGX Spark (256 GB) and 2× Mac Studio (1 TB) are sums of two machines, not one shared memory pool. Devices count memory in GiB; where that changes the answer, the table says so.

Measurements come only from explicitly approved MESSUNG files. Missing values stay “not measured yet”. GPU-only power and whole-PC electricity at the wall are shown separately; each run states its electricity tariff. There are no Mac results yet.

Available raw values, commands and prompts are public in each dated run. Missing historical records are labelled. File sizes displayed in GB are rounded to two decimals; exact recorded bytes remain in the model details.

The static 16 / 24 / 512 GB / 1 TB columns work without JavaScript. Use the finder above for your own hardware and memory; runtime status stays separate.

Measured models: file budgets. File arithmetic only. Loading, context and runtime compatibility are separate. Exact byte sizes are in each model's details.
Model / variantFiles16 GB · RTX 408024 GB · RTX 3090/4090512 GB · Mac Studio M5 Ultra1 TB · 2× Mac StudioRuntime statusLatest measurement / size retrieval
Qwen3.5 9B
Q4_K_M		6.55 GB (rounded source)files fit on paper (rounded size)files fit on paper (rounded size)files fit on paper (rounded size)files fit on paper (rounded size)measured / 2026-10-26
Gemma 4 12B QAT
Q4_0		7.15 GB (rounded source)files fit on paper (rounded size)files fit on paper (rounded size)files fit on paper (rounded size)files fit on paper (rounded size)measured / 2026-10-26

Not run yet: file size only

These candidates have dated file-size records. No measured speed, power or loading result yet.

Not run yet: file budgets. File arithmetic only. Loading, context and runtime compatibility are separate. Exact byte sizes are in each model's details.
Model / variantFiles16 GB · RTX 408024 GB · RTX 3090/4090512 GB · Mac Studio M5 Ultra1 TB · 2× Mac StudioRuntime statusLatest measurement / size retrieval
Nemotron 3.5 Lightning 30B-A3B
NVFP4, NVIDIA safetensors		21.56 GB (display rounded to 0.01 GB)files too bigfiles fit on paperfiles fit on paperfiles fit on papernot run yet— / 2026-10-27
Qwen3.8 27B
NVFP4, NVIDIA safetensors		21.92 GB (display rounded to 0.01 GB)files too bigfiles fit on paperfiles fit on paperfiles fit on papernot run yet— / 2026-10-27
Qwen3.6 35B-A3B
NVFP4, NVIDIA safetensors		23.44 GB (display rounded to 0.01 GB)files too bigfiles fit on paperfiles fit on paperfiles fit on papernot run yet— / 2026-10-27
gpt-oss-120b
MXFP4, official safetensors		65.25 GB (display rounded to 0.01 GB)files too bigfiles too bigfiles fit on paperfiles fit on papernot run yet— / 2026-10-27
Qwen3.5 122B-A10B
NVFP4, safetensors incl. MTP file		79.05 GB (display rounded to 0.01 GB)files too bigfiles too bigfiles fit on paperfiles fit on papernot run yet— / 2026-10-27
Qwen3 235B-A22B
NVFP4, NVIDIA safetensors		134.13 GB (display rounded to 0.01 GB)files too bigfiles too bigfiles fit on paperfiles fit on papernot run yet— / 2026-10-27
DeepSeek V4.1 Flash 1017
Official safetensors; quantization not recorded		510.3 GB (display rounded to 0.01 GB)files too bigfiles too bigfiles fit on paperfiles fit on papernot run yet— / 2026-10-19
GLM-5.3
FP8 (e4m3), official safetensors		755.63 GB (display rounded to 0.01 GB)files too bigfiles too bigfiles too bigfiles fit on papernot run yet— / 2026-10-21
Kimi K3
MXFP4, official safetensors		1,560.94 GB (display rounded to 0.01 GB)files too bigfiles too bigfiles too bigfiles too bignot run yet— / 2026-10-17
GLM-5.3 Flash
EXL3 4bpw TensorFold		175.64 GB (display rounded to 0.01 GB)files too bigfiles too bigfiles fit on paperfiles fit on papernot run yet— / 2026-10-19
MiniMax M2.7
UD-Q4_K_M		140.17 GB (display rounded to 0.01 GB)files too bigfiles too bigfiles fit on paperfiles fit on papernot run yet— / 2026-10-19
Beam 1.0 501B A23B
UD-Q4_K_XL		306.53 GB (display rounded to 0.01 GB)files too bigfiles too bigfiles fit on paperfiles fit on papernot run yet— / 2026-10-16
Muse Spark Open 816B A42B
NVFP4, safetensors		533.01 GB (display rounded to 0.01 GB)files too bigfiles too bigfiles too big; files fit at 512 GiBfiles fit on papernot run yet— / 2026-10-16
Muse Spark Open 816B A42B
UD-Q3_K_XL		401.5 GB (rounded source)files too big (rounded size)files too big (rounded size)files fit on paper (rounded size)files fit on paper (rounded size)not run yet— / 2026-10-15
Muse Spark Open 816B A42B
UD-Q4_K_XL		505.5 GB (display rounded to 0.01 GB)files too bigfiles too bigfiles fit on paperfiles fit on papernot run yet— / 2026-10-17
DeepSeek V4.1 Flash 1017
UD-IQ1_S including IQ4_NL engram shard		232.98 GB (display rounded to 0.01 GB)files too bigfiles too bigfiles fit on paperfiles fit on papernot run yet— / 2026-10-19
Muse Spark Open 816B A42B
UD-IQ1_S		175.64 GB (display rounded to 0.01 GB)files too bigfiles too bigfiles fit on paperfiles fit on papernot run yet— / 2026-10-17
DeepSeek V4.1 Flash 1017
UD-Q4_K_XL		500.71 GB (display rounded to 0.01 GB)files too bigfiles too bigfiles fit on paperfiles fit on papernot run yet— / 2026-10-22

What the numbers mean

Measurements you can inspect

I measure output tokens per second, time to the first token, GPU memory and GPU power. Whole-PC power at the wall is a separate measurement and stays missing when it was not recorded. This is a run log, not a speed ranking or a quality score.

Conditions travel with each result

Three input lengths, fresh identifier and topic per request, no reused prompt cache. Medians and recorded individual speed samples are shown. Two repetitions describe a small sample. The input text can differ between models; the speeds do not establish a controlled ranking.

  • 2026-10-27 · Gemma 4 12B QAT · Q4_0: 2 repetitions per input length, maximum output 1,500 tokens, temperature not measured yet, context 16,384 tokens, software 2.61.0, electricity tariff €0.35/kWh.
  • 2026-10-27 · Qwen3.5 9B · Q4_K_M: 2 repetitions per input length, maximum output 1,500 tokens, temperature not measured yet, context 16,384 tokens, software 2.61.0, electricity tariff €0.35/kWh.
  • 2026-10-13 · Gemma 4 12B QAT · Q4_0: 2 repetitions per input length, maximum output 300 tokens, temperature not measured yet, context not measured yet, software version not recorded, electricity tariff €0.35/kWh.
  • 2026-10-13 · Qwen3.5 9B · Q4_K_M: 2 repetitions per input length, maximum output 300 tokens, temperature not measured yet, context not measured yet, software version not recorded, electricity tariff €0.35/kWh.

The 13 October data used the older smoothed GPU-power sensor. Later runs identify the power field and distinguish engine time from request-to-first-token time. Non-streaming wall time is derived from request duration minus generation time. Generated-token speed includes reasoning when the record says so. Those method changes limit comparisons. Old timestamps without a timezone are ordered by their recorded clock; their absolute order against other timezones is unknown.

How the memory labels work

The estimate is recorded model-file size × 1.25. “Estimated fit” means that allowance fits the entered budget. “Tight estimate” means the file fits but that allowance does not. “Does not fit on paper” means no space remains for context, or the file exceeds the budget. Missing or boundary-rounded sizes remain unknown. These labels do not prove that a runtime supports a format. The finder checks format support separately and can show a measured fit only for the measured hardware.

Budget tables show file-size subtraction only, with the remainder for context, buffers and the operating system. 1 GB means 1,000,000,000 bytes; device GiB values are different. Two machines do not create one shared memory pool.

Inspect before you compare

Each dated run links to a public JSON download with its available raw values, exact command, prompts, software and driver versions and model-file SHA256. Older records lack parts of that recipe; each missing part is labelled. I do not backfill measurements from guesses.

The lab letter keeps its 48-hour head start. The first Mac measurement on delivery day is the one-off exception and can be public immediately.

Build diary

Road to 1TB

The plan: two Mac Studio M5 Ultra machines, 512 GB each, linked over Thunderbolt 5. 1 TB of memory across two machines, doing daily work in a real business. Until they arrive, it's a plan.

  1. The test plan

    Same tasks on the RTX 4080 first, then on one Mac, plus an API comparison at list price.

  2. Two machines

    The series changed from Road to 512GB to Road to 1TB. Two planned 512 GB machines, connected over Thunderbolt 5.

  3. The first part is here

    A 0.5 m CalDigit Thunderbolt 5 Pro cable. The Macs are still planned; a cable is not a working cluster.

Follow the series on X: @SelfHostedSI

Get the weekly article by email on Substack

The lab letter

The measurement, 48 hours early.

Get the measurement row 48 hours before /lab, plus failed attempts, the exact command to reproduce the run and a memory worksheet. At most once a month, only when a new measurement is done.

Both checkboxes are required. After submitting, please confirm your email address using the link in your inbox.

Privacy policy, in German

The hardware behind the numbers

Setup photo pending. A real photo of my test bench will go here when it is available.

Current test device: my RTX 4080 with 16 GB VRAM. Both Mac Studios are awaiting delivery.

Steven Kramp

Tested by Steven

I use AI in my business and publish what I can verify on my own hardware. If a run fails or a record is missing, it stays visible as a limit.

Compare notes

Got a different result?

Send me the model, quantization, hardware, runtime version and prompt length, and I'll compare it with mine. Testing local AI for real work? Tell me what breaks and what holds up.

Email Steven

I also build websites for small businesses in Germany. Webdesign auf Deutsch.

For businesses in Germany, I can help turn incoming email enquiries into a task list with a follow-up reminder: describe your workflow in the free Büro-Check.

The legal information and privacy policy linked above apply to this lab and the German part of the site. Both documents are in German.