One model, two RTX 4090s. You be the judge.
The same DeepSeek-R1-Distill-Qwen-7B on two identical RTX 4090s: one pressed to 4-bit by Trapetum (5.5 GB), one left at full fp16 (15.2 GB). Same prompt, both cards. This is a recording of a real run, replayed at each side's true measured speed.
PC A, Trapetum 4-bit
PC B, fp16 uncompressed
Why these numbers are real
Both consoles replay the actual token stream each RTX 4090 produced, at each side's measured decode speed. Trapetum runs the 4-bit model in its pure-Rust fused kernel; PC B runs the untouched fp16 weights the way a home user would. Same model, same card, a third of the memory, about 3.0x the speed. Live decode speed varies run to run (fp16 especially), and often reaches 4x.
See the full per-GPU benchmarks, watch the 671B on a single 4090, or follow the Road to x100. Numbers here are decode tokens per second, measured on the endpoints, prompt "Explain quantization to a 12-year-old.".
And the full 671B, inverted: 2.46 tok/s on one machine
Compression is not only about speed. The same 4-bit codebook idea lets the largest open-source model, DeepSeek-R1 671B, run on a single commodity node: the routed experts stay resident in host RAM and the CPU decodes them in place, while the GPU holds only attention. Replayed token by token at its measured steady-state speed of 407 ms/token.
The full run takes about 24 seconds at real speed. Decode is bound by host memory bandwidth now, not disk: the experts are decoded in place on the CPU at ~67 GB/s aggregate. See the benchmarks and the Road to x100.
671B at 2.46 tok/s: with vs without Trapetum
Same model (DeepSeek-R1 671B, 1.34 TB in bf16). Same peak speed. Only the hardware and the cost change.
Without the inversion a single GPU tops out at 0.33 tok/s (naive offload, measured), so the weights must go in VRAM: a cluster. The job needs 350 GB at only ~67 GB/s, which refurbished DDR4 delivers for ~$8K; all-new DDR5 (512 GB ~$15K) makes it ~$28K. Street prices, July 2026.