https://tianshihao.github.io/tags/quantization/Recent content in Quantization on 笔记本子Hugo -- gohugo.iocnCopyright © 2026Tue, 25 Aug 2026 13:36:10 +0800https://tianshihao.github.io/posts/quantization-in-depth/Tue, 25 Aug 2026 13:36:10 +0800https://tianshihao.github.io/posts/quantization-in-depth/<ul> <li><strong>Quantization</strong>: Store the parameters of the model in lower precision</li> <li><strong>Knowledge Distillation</strong>: Train a smaller model (student) using a original model (instructor)</li> <li><strong>Pruning</strong>: Remove connections (weights) from the model</li> </ul> <p><strong>FP32</strong></p> <ul> <li>Sign: 1 bit</li> <li>Exponent (range): 8 bits</li> <li>Fraction (precision): 23 bits</li> <li>Total: 32 bits</li> </ul> <h2 id="quantization">Quantization</h2> <p>Quantization refers to the process of mapping a large set to a smaller set of values. For example, mapping the continuous range of real numbers [-234.1, 251.51] to a discrete set of integers [-128, 127].</p> <h2 id="neural-network-quantization">Neural Network Quantization</h2> <p>You can quantize:</p> <ul> <li>The weights: Neural network parameters</li> <li>The Activations: Values that propagate through the layers of the neural network</li> </ul> <p>If you quantize the NN after it has been quantized, you are doing <strong>post training quantization</strong>(PTQ)</p> <h2 id="advantages-of-quantization">Advantages of Quantization</h2> <ul> <li>Smaller model</li> <li>Speed gains <ul> <li>Memory bandwidth: Less data to transfer</li> <li>Faster operations <ul> <li>GEMM: General Matrix Multiply</li> <li>GEMV: General Matrix Vector Multiplication</li> </ul> </li> </ul> </li> </ul> <h2 id="challenges-of-quantization">Challenges of Quantization</h2> <ul> <li>Quantization error</li> <li>Retraining (Quantization Aware Training, QAT)</li> <li>Limited hardware support</li> <li>Calibration dataset needed</li> <li>Packing/unpacking</li> </ul> <h2 id="linear-quantization">Linear Quantization</h2> <ul> <li>Idea: linear mapping</li> <li>Formula:</li> </ul> <p>$$ r=s(q-z) $$</p> <p>$s$ is the scale factor, $z$ is the zero point, $r$ is the real value (e.g. in FP32), $q$ is the quantized value (e.g. in INT8).</p> <p><img src="images/linear-quantization.png" alt="Linear Quantization"></p> <p>Example with $s=2$, and $z=0$:</p> <ul> <li>We got $r=2(q-0)=2q$</li> <li>For $q=10$, we have $r=20$</li> </ul>