Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization

A study of MXFP4 and NVFP4 post-training quantization that introduces Micro-Rotated-GPTQ and reports GPU kernel-backed speedups. FP4 is marketed as the next inference leap, but this paper shows why format-specific quantization is required to unlock it.

This is a core source for evaluating RTX 5090 and Blackwell FP4 claims against deployable quantized inference.

FP4 is marketed as the next inference leap, but this paper shows why format-specific quantization is required to unlock it. The useful part for GPU Hunter readers is not the abstract result alone; it is the hardware implication: whether a model fits, whether a runtime can use the format, or whether throughput is limited by memory movement instead of arithmetic.

RTX 5090 and Blackwell FP4 claims should be judged against real FP4 kernels and accuracy, not just advertised tensor formats. For local inference, quantization is only valuable when it improves model fit without breaking the runtime path. A smaller checkpoint can still be slow if dequantization, packed-weight kernels, or activation handling are weak.