01 // Representative inference estimates
Single-stream decode · llama.cpp
Aggregated from published community sources. Test setups differ, so use close results as directional evidence.
Sources and normalization →01b // Performance across quantization
vs. nearest competitors
How tok/s scales from FP16 → Q8 → Q4 compared to GPUs in a similar price/VRAM range.
02 // Hardware specs
ArchitectureGB10 Grace Blackwell
Process nodeTSMC 4NP
Memory128 GB
Memory bandwidth273 GB/s
FP16 compute31 TFLOPS
INT8 compute62 TOPS
TDP170 W
PCIeUnified
Form factorMini-desktop
CoolingActive
03 // Model fit
Weight estimate plus 15% planning headroom for runtime buffers and a modest KV cache. Long context can require more.
+ STRENGTHS
- ✓128GB clears our Qwen2.5 72B Q4 planning target
- ✓273 GB/s memory bandwidth · top tier in its class
- ✓Indexed formats: FP16, FP8, Q8, Q4 · verify support in your runtime
− TRADE-OFFS
- −Draws 170W under load — plan PSU and thermals accordingly
- −Limited to mini-desktop chassis
- −Driver lock-in to vendor stack
related research
Research behind NVIDIA DGX Spark inference tradeoffs
These papers explain the quantization, cache, bandwidth, and runtime constraints that matter before buying this GPU for local AI.
LLM quantization research
GPTQ, AWQ, GGUF, FP4, NF4, and what low-bit formats mean for VRAM fit.
Open GPU inference optimization papers
Memory bandwidth, FlashAttention, dequant kernels, and backend maturity.
Open 2026 LLM inference papers
Fresh 2026 work on FP4, KV cache, kernels, AMD serving, and local controllers.
Open