Model Compression2 articles

Model Compression

Articles

Quantization-Aware Training in Large Language Models: How Fake Quantization, STE, and Learned Step Sizes Preserve Low-Bit Accuracy
Quantization-Aware Training (QAT) is a model compression paradigm that integrates precision loss directly into the training or fine-tuning graph. While Post-Training Quantization (PTQ) calibrates fixed floating-point weights without updating underlying network parameters, QAT simulates the numerical discretization of weights, activations, and key-value (KV) caches during both forward and backward passes. This closed-loop optimization forces neural network weights to co-adapt to discrete integer
August 20, 20261 min
Weight Pruning in Large Language Models: How SparseGPT, Wanda, and Semi-Structured 2:4 Sparsity Compress Neural Networks Without Retraining
Modern large language models require tens to hundreds of gigabytes of high-bandwidth memory to store billions of parameters. While post-training quantization compresses model footprints by reducing numerical precision from 16-bit floating point to 8-bit or 4-bit integers, weight pruning attacks model size along an orthogonal axis: setting redundant parameter values exactly to zero. Historically, pruning deep neural networks required iterative cycles of magnitude thresholding and compute-heavy r
August 20, 20261 min