Profiling Large Language Model Inference on Apple Silicon: A Quantization Perspective

A profiling study of Apple Silicon's unified memory architecture for on-device LLM inference under different quantization choices. Apple Silicon competes on unified memory capacity rather than discrete-GPU VRAM, so it needs a different inference mental model.

This paper backs the Mac Studio and MacBook Pro coverage with a separate inference model instead of forcing CUDA assumptions onto Apple hardware.

Apple Silicon competes on unified memory capacity rather than discrete-GPU VRAM, so it needs a different inference mental model. The useful part for GPU Hunter readers is not the abstract result alone; it is the hardware implication: whether a model fits, whether a runtime can use the format, or whether throughput is limited by memory movement instead of arithmetic.

Mac recommendations should compare quantized throughput, memory pressure, and unified-memory behavior against CUDA GPUs. For one-box local AI, the practical issue is how model format, runtime, memory hierarchy, and offload policy interact. This is where a cheaper GPU can be a good choice or a frustrating compromise.