Steven Kramp / SelfHostedSI
← All models and the lab diary

Steven Kramp / @SelfHostedSI · Local AI lab

Gemma 4 12B QATQ4_0

Measured: 73.9 tok/s, First token (wall): 2.12 s, 8.9 GB GPU memory used, 235.8 W GPU only; short prompt, recorded medians, 2026-10-27.

Download this model's data (JSON)

Gemma 4 12B QAT · Q4_0

Measured: 73.9 tok/s · short prompt, recorded medians.

First token (wall)
2.12 s · engine: 0.13 s
GPU memory used
8.9 GB
GPU power only
235.8 W
Configured context window
16,384 tokens
Input tokens (median)
41
Electricity per 1M tokens, short prompt
€0.31
Electricity per 1M tokens, long prompt
€0.36
Electricity per 1M tokens, whole PC at the wall
not measured yet

Electricity: GPU power only at €0.35/kWh, per 1M output tokens. Hardware cost not included; not an API price.

All generated tokens incl. reasoning.

Wall TTFT method: derived from request duration minus generation time (non-streaming).

Power sensor: power.draw.instant.

NVIDIA GeForce RTX 4080 · .

Conditions and limits · Source: MESSUNG_2026-10-27.json

Files, not parameter estimates

7.15 GB (rounded source)

The exact Hugging Face repository was not recorded in the approved run.

Source: MESSUNG_2026-10-13.json, recorded model-file size. Retrieved . Local model-file size recorded on 13 October 2026, rounded to 0.01 GB; exact bytes and Hugging Face repository were not recorded.

Hugging Face listings can change. These are dated file-size records; a revision hash was not retained. Download size does not prove that a runtime can load the model.

Fits on paper

Subtract this variant's recorded file size from each memory budget; the remainder is the upper bound for context and other overhead.

Decimal: 1 GB = 1,000,000,000 bytes; 1 TB = 1,000 GB. These are arithmetic comparison budgets, not verified usable hardware capacities. The answer can change with GB/GiB: 512 GiB = 549,755,813,888 bytes. The measured RTX 4080 has 16 GB VRAM; every other device name only labels a comparison budget.

16 GB · RTX 4080

files fit on paper (rounded size)

about 8.85 GB left

My measured card; usable free memory is not verified.

24 GB · RTX 3090/4090

files fit on paper (rounded size)

about 16.85 GB left

Comparison budget; not measured.

32 GB · RTX 5090

files fit on paper (rounded size)

about 24.85 GB left

Comparison budget; not measured.

128 GB · DGX Spark

files fit on paper (rounded size)

about 120.85 GB left

Comparison budget; I do not own one. Usable free memory is lower and not verified.

256 GB · Mac Studio M5 Ultra or 2× DGX Spark

files fit on paper (rounded size)

about 248.85 GB left

One Mac Studio M5 Ultra, or the sum of two DGX Sparks (two memories, not one pool). Not measured.

512 GB · Mac Studio M5 Ultra

files fit on paper (rounded size)

about 504.85 GB left

Planned; not delivered.

1 TB · 2× Mac Studio

files fit on paper (rounded size)

about 992.85 GB left

Across two planned machines; not one shared memory pool.

GB values are rounded to 0.01 GB for display; “left” is what remains for context, runtime and the operating system. Exact bytes are in this model's data (JSON).

Budget minus model files leaves an upper bound for context, runtime buffers and the operating system; no context length or runtime compatibility is promised. Exactly filling a budget leaves no room for context, so the answer is “no room for context on paper”; files larger than the budget are “files too big”.

Two Mac Studio machines, 512 GB each; not delivered. Two DGX Sparks are likewise two memories, not one pool. A model split across two machines needs a compatible runtime and must fit each machine's allocation. This table is arithmetic, not a cluster test.

Append-only run history · oldest first

Approved measurements

NVIDIA GeForce RTX 4080 · LM Studio version: not recorded.

Configured context window: not measured yet. GPU memory used: 8.9 GB.

Recorded medians by prompt length; input tokens are not the configured context window.
PromptInput tokensOutput tokens/sFirst token (engine) (s)Power (W)Electricity per 1M tokens
Short37.5740.14161.2€0.21
Medium87072.80.42178€0.24
Long3,449.566.11.01185.2€0.27

Power on 13 October was measured with a smoothed sensor and is too low; see the 27 October measurement using instantaneous power.

Recorded model-level electricity cost per 1M output tokens (short prompt): €0.21. Power scope: GPU only, €0.35/kWh. GPU power only; not whole-system power. Excludes hardware cost; not an API price.

Method and limits

Temperature: not measured yet; maximum output: 300 tokens; repeats per prompt length: 2. Fresh identifier and topic per request; no prompt cache. The source reports medians and individual output-speed samples.

Available exact commands and prompts are public in the raw-data record below. Exact prompts were not recorded for this run. Timezone was not recorded; this entry uses its local clock for ordering. Its absolute order against other timezones is unknown. Missing software versions and context limits remain missing. A small sample describes this run, not a reproducible benchmark or a speed guarantee.

Source: MESSUNG_2026-10-13.json. Raw measurement extract and individual speed samples (JSON).

Raw data and reproduction details · incomplete historical record

Download raw data, command, prompts and versions (JSON)

Public extract of the approved MESSUNG file. It includes recorded individual values and available GPU sensor samples. Internal paths and operational notes are excluded. Missing records stay null.

Software
LM Studio · version not recorded
Driver version
not recorded
Model-file SHA256
not recorded
Exact recorded command

Not recorded. No command reconstructed from today's scripts.

Exact prompts by repetition

Not recorded. Task: text generation with fresh short, medium and long inputs; a new identifier and topic per request.

This record cannot yet reproduce the original run exactly. Missing: exact command not recorded; exact kurz prompts not recorded for every repetition; exact mittel prompts not recorded for every repetition; exact lang prompts not recorded for every repetition; software version not recorded; driver version not recorded; model-file SHA256 not recorded.

NVIDIA GeForce RTX 4080 · Engine version: 2.61.0.

Configured context window: 16,384 tokens. GPU memory used: 8.9 GB.

Recorded medians by prompt length; input tokens are not the configured context window.
PromptInput tokensOutput tokens/sFirst token (wall) (s)Power (W)Electricity per 1M tokens
Short4173.92.12 · engine: 0.13 s235.8€0.31
Medium863.5722.23 · engine: 0.47 s235€0.32
Long3,46065.32.35 · engine: 1.02 s239.6€0.36

All generated tokens incl. reasoning. Wall TTFT is shown first; engine time is separate. Wall TTFT was derived from request duration minus generation time (non-streaming). Power sensor: power.draw.instant.

Recorded model-level electricity cost per 1M output tokens (short prompt): €0.31. Power scope: GPU only, €0.35/kWh. GPU power only; not whole-system power. Excludes hardware cost; not an API price.

Method and limits

Temperature: not measured yet; maximum output: 1,500 tokens; repeats per prompt length: 2. Fresh identifier and topic per request; no prompt cache. The source reports medians and individual output-speed samples.

Available exact commands and prompts are public in the raw-data record below. Timezone is included in the timestamp. A small sample describes this run, not a reproducible benchmark or a speed guarantee.

Source: MESSUNG_2026-10-27.json. Raw measurement extract and individual speed samples (JSON).

Raw data and reproduction details · incomplete historical record

Download raw data, command, prompts and versions (JSON)

Public extract of the approved MESSUNG file. It includes recorded individual values and available GPU sensor samples. Internal paths and operational notes are excluded. Missing records stay null.

Software
LM Studio · 2.61.0 (engine version)
Driver version
not recorded
Model-file SHA256
not recorded
Exact recorded command

Not recorded. No command reconstructed from today's scripts.

Exact prompts by repetition

Not recorded. Task: text generation with fresh short, medium and long inputs; a new identifier and topic per request.

This record cannot yet reproduce the original run exactly. Missing: exact command not recorded; exact kurz prompts not recorded for every repetition; exact mittel prompts not recorded for every repetition; exact lang prompts not recorded for every repetition; driver version not recorded; model-file SHA256 not recorded.

The JSON is an extract of approved measurement fields. Internal paths and operational notes are excluded. New runs append here; the model URL stays the same.