KV Cache Transform Coding for Compact Storage in LLM Inference

A lightweight transform coder that compresses reusable KV caches using PCA-style decorrelation, adaptive quantization, and entropy coding. Shared-prefix chat and coding workflows can accumulate stale caches that consume GPU memory or force recomputation.

This paper is relevant for local coding agents and document tools where repeated prompts make cache reuse a product feature.

Shared-prefix chat and coding workflows can accumulate stale caches that consume GPU memory or force recomputation. The useful part for GPU Hunter readers is not the abstract result alone; it is the hardware implication: whether a model fits, whether a runtime can use the format, or whether throughput is limited by memory movement instead of arithmetic.

Persistent KV cache storage can become a product feature for coding agents and local chat apps, not just a runtime optimization. For long-context work, KV cache behavior is often the constraint that shows up after the model weights already fit. Cache precision, eviction, reuse, and memory movement can change the practical value of the same GPU.