01 // Representative inference estimates
Single-stream decode · llama.cpp
Aggregated from published community sources. Test setups differ, so use close results as directional evidence.
Sources and normalization →01b // Performance across quantization
vs. nearest competitors
How tok/s scales from FP16 → Q8 → Q4 compared to GPUs in a similar price/VRAM range.
02 // Hardware specs
ArchitectureRDNA 4
Process nodeTSMC 4nm
Memory16 GB
Memory bandwidth512 GB/s
FP16 compute48.7 TFLOPS
INT8 compute97 TOPS
TDP304 W
PCIeGen 4 x16
Form factorDual-slot
CoolingAxial
03 // Model fit
Weight estimate plus 15% planning headroom for runtime buffers and a modest KV cache. Long context can require more.
+ STRENGTHS
- ✓16GB clears our Codestral 22B Q4 planning target
- ✓512 GB/s memory bandwidth · top tier in its class
- ✓Indexed formats: FP16, Q8, Q4 · verify support in your runtime
− TRADE-OFFS
- −Draws 304W under load — plan PSU and thermals accordingly
- −Limited to dual-slot chassis
- −Driver lock-in to vendor stack
related research
Research behind Radeon RX 9070 XT inference tradeoffs
These papers explain the quantization, cache, bandwidth, and runtime constraints that matter before buying this GPU for local AI.
GPU inference optimization papers
Memory bandwidth, FlashAttention, dequant kernels, and backend maturity.
Open Local AI inference papers
llama.cpp, Apple Silicon, constrained GPUs, offload, and one-box inference.
Open LLM serving systems papers
vLLM, PagedAttention, speculative decoding, batching, and GPU servers.
Open