Diagnosing FP4 inference: a layer-wise and block-wise sensitivity analysis of NVFP4 and MXFP4

A component-wise sensitivity study of NVFP4 and MXFP4 quantization across Qwen2.5 model scales and transformer blocks. FP4 support is a major Blackwell and next-gen accelerator selling point, but not every layer tolerates the same low precision.

Blackwell buyers need this context because FP4 is a purchasing feature only when model quality and kernels hold up.

FP4 support is a major Blackwell and next-gen accelerator selling point, but not every layer tolerates the same low precision. The useful part for GPU Hunter readers is not the abstract result alone; it is the hardware implication: whether a model fits, whether a runtime can use the format, or whether throughput is limited by memory movement instead of arithmetic.

Treat FP4 as a hardware capability that still needs model-aware quantization policy, not a universal speed switch. For local inference, quantization is only valuable when it improves model fit without breaking the runtime path. A smaller checkpoint can still be slow if dequantization, packed-weight kernels, or activation handling are weak.