Quantization in Depth on 笔记本子

Quantization in Depth 3

Wed, 02 Sep 2026 13:55:10 +0800

Quantization in Depth 3

Linear Quantization Mode

There are two modes in linear quantization:

Asymmetric: Mapping $[r_\text{min}, r_\text{max}]$ to $[q_\text{min}, q_\text{max}]$
Symmetric: Mapping $[-r_\text{max}, r_\text{max}]$ to $[-q_\text{max}, q_\text{max}]$

We don’t need to use zero point($z=0$) in symmetric mode.

This happens because the floating-point range and the quantized range are symmetric with respect to zero.

Hence, we can simplify the equations to:

$$ \begin{cases} q=int(round(r/s)) \\ s=r_\text{max}/q_\text{max} \end{cases} $$

Trade-off:

Utilization of quantized range:
- When using asymmetric quantization, the quantized range is fully utilized.
- When symmetric mode, if the float range is biased towards one side, this will result in a quantized range where a part of the range is dedicated to values that we’ll never use.
Simplicity: Symmetric mode is simpler compared to asymmetric mode.
Memory: We don’t store the zero-point for symmetric quantization.

import torch
def get_q_scale_symmetric(tensor, dtype=torch.int8):
r_max = tensor.abs().max().item()
q_max = torch.iinfo(dtype).max
return r_max / q_max
test_tensor = torch.randn((4, 4))
test_tensor

tensor([[-1.1284, 2.3800, -2.2940, 0.6971],
[-0.4221, -0.4675, -0.6180, -1.4234],
[ 0.0705, 1.3060, -1.2461, 0.2461],
[-2.6319, 1.3008, 0.9376, 0.7069]])

get_q_scale_symmetric(test_tensor)

0.020723763413316623

from helper import linear_q_with_scale_and_zero_point
def linear_q_symmetric(tensor, dtype=torch.int8):
scale = get_q_scale_symmetric(tensor, dtype=dtype)
quantized_tensor = linear_q_with_scale_and_zero_point(tensor, scale, zero_point=0, dtype=dtype)
return quantized_tensor, scale
quantized_tensor, scale = linear_q_symmetric(test_tensor)
from helper import linear_dequantization, plot_quantization_errors
from helper import quantization_error
dequantized_tensor = linear_dequantization(quantized_tensor, scale, 0)
plot_quantization_errors(test_tensor, quantized_tensor, dequantized_tensor)

这里补充图像

printf(f"""Quantization Error : \
{quantization_error(test_tensor, dequantized_tensor)}""")

这里应该输出一个Quantization Error的值

Quantization Error :

Quantization in Depth 2

Thu, 27 Aug 2026 14:12:40 +0800

Quantization in Depth 2

Get the scale and zero point

Linear quantization maps the floating point range $[r_{min}, r_{max}]$ to the quantized range $[q_{min}, q_{max}]$.

If we look the extreme values, we could get:

$$ \begin{cases} r_{min} = s(q_{min} - zero_ point) \\ r_{max} = s(q_{max} - zero_ point) \end{cases} $$

Substracting the first equation from the second, we could get the scale $s$:

$$ r_{max} - r_{min} = s (q_{max} - q_{min}) \\ $$ $$ s = (r_{max} - r_{min})/(q_{max} - q_{min}) $$

For the zero point $z$, we need to round the value since it is an integer:

$$ z = round(q_{min} - \frac{r_{min}}{s}) $$

Why make $z$ an integer?

$$ z = int(round(q_{min} - \frac{r_{min}}{s})) $$

Example

to be continued…

Zero point out of range

$z$ < $q_{min}$: set $z$ = $q_{min}$
$z$ > $q_{max}$: set $z$ = $q_{max}$

Linear quantization scale and zero point pytorch

import torch
from helper import linear_q_with_scale_and_zero_point, linear_dequantization
test_tensor = torch.tensor(
[[191.6, -13.5, 728.6],
[92.14, 295.5, -184],
[0, 684.6, 245.5]]
)
q_min = torch.iinfo(torch.int8).min
q_max = torch.iinfo(torch.int8).max

q_min

-128

q_max

r_min = test_tensor.min().item()
r_min

-184.0

r_max = test_tensor.max().item()
r_max

728.5999755859375

scale = (r_max - r_min) / (q_max - q_min)
scale

3.578823433670343

zero_point = q_min - (r_min / scale)
zero_point

-76.58645490333825

zero_point = int(round(q_min - (r_min / scale)))

-77

def get_q_scale_and_zero_point(tensor, dtype=torch.int8):
q_min, q_max = torch.iinfo(dtype).min, torch.iinfo(dtype).max
r_min, r_max = tensor.min().item(), tensor.max().item()
scale = (r_max - r_min) / (q_max - q_min)
zero_point = q_min - (r_min / scale)
if zero_point < q_min:
zero_point = q_min
elif zero_point > q_max:
zero_point = q_max
else:
zero_point = int(round(zero_point))
return scale, zero_point
new_scale, new_zero_point = get_q_scale_and_zero_point(test_tensor)

3.578823433670343

new_zero_point

-77

quantized_tensor = linear_q_with_scale_and_zero_point(test_tensor, new_scale, new_zero_point)
dequantized_tensor = linear_dequantization(quantized_tensor, new_scale, new_zero_point)
(dequantized_tensor - test_tensor).square().mean()

tensor(1.5730)

from helper import plot_quantization_errors
plot_quantization_errors(test_tensor, quantized_tensor, dequantized_tensor)

(dequantized_tensor - test_tensor).square().mean()

tensor(1.5730)

def linear_quantization(tensor, dtype=torch.int8):
scale, zero_point = get_q_scale_and_zero_point(tensor, dtype=dtype)
quantized_tensor = linear_q_with_scale_and_zero_point(tensor, scale, zero_point, dtype=dtype)
return quantized_tensor, scale, zero_point
r_tensor = torch.randn((4, 4))
r_tensor

tensor([[ 1.6714, -1.3308, 0.4337, 1.0956],
[-1.8622, 2.4572, 1.0099, -1.0139],
[ 0.2309, 1.3375, -0.8019, -0.6140],
[-0.4675, -1.4661, -1.0904, 0.4055]])

quantized_tensor, scale, zero_point = linear_quantization(r_tensor)
quantized_tensor

tensor([[ 81, -97, 8, 47],
[-128, 127, 42, -78],
[ -4, 61, -65, -54],
[ -46, -105, -82, 6]], dtype=torch.int8)

scale

0.01693895798103482

zero_point

-18

dequantized_tensor = linear_dequantization(quantized_tensor, scale, zero_point)
plot_quantization_errors(r_tensor, quantized_tensor, dequantized_tensor)

(dequantized_tensor - r_tensor).square().mean()

tensor(2.7701e-05)

Quantization in Depth 1

Tue, 25 Aug 2026 13:36:10 +0800

Quantization: Store the parameters of the model in lower precision
Knowledge Distillation: Train a smaller model (student) using a original model (instructor)
Pruning: Remove connections (weights) from the model

FP32

Sign: 1 bit
Exponent (range): 8 bits
Fraction (precision): 23 bits
Total: 32 bits

Quantization

Quantization refers to the process of mapping a large set to a smaller set of values. For example, mapping the continuous range of real numbers [-234.1, 251.51] to a discrete set of integers [-128, 127].

Neural Network Quantization

You can quantize:

The weights: Neural network parameters
The Activations: Values that propagate through the layers of the neural network

If you quantize the NN after it has been quantized, you are doing post training quantization(PTQ)

Advantages of Quantization

Smaller model
Speed gains
- Memory bandwidth: Less data to transfer
- Faster operations
  - GEMM: General Matrix Multiply
  - GEMV: General Matrix Vector Multiplication

Challenges of Quantization

Quantization error
Retraining (Quantization Aware Training, QAT)
Limited hardware support
Calibration dataset needed
Packing/unpacking

Linear Quantization

Idea: linear mapping
Formula:

$$ r=s(q-z) $$

$s$ is the scale factor, $z$ is the zero point, $r$ is the real value (e.g. in FP32), $q$ is the quantized value (e.g. in INT8).

Example with $s=2$, and $z=0$:

We got $r=2(q-0)=2q$
For $q=10$, we have $r=20$

Getting q

$$ r=s(q-z) \implies q=\frac{r}{s}+z \quad (7.4) $$

$$ q=round(\frac{r}{s})+z 7.0 $$

$$ q=int(round(\frac{r}{s})+z) 7 $$

PyTorch

import torch
def linear_q_with_scale_and_zero_point(
tensor, scale, zero_point, dtype=torch.int8):
scaled_and_shifted_tensor = tensor / scale + zero_point
rounded_tensor = torch.round(scaled_and_shifted_tensor)
q_min = torch.iinfo(dtype).min
q_max = torch.iinfo(dtype).max
q_tensor = rounded_tensor.clamp(q_min, q_max).to(dtype)
return q_tensor
def linear_dequantization(quantized_tensor, scale, zero_point):
return scale * (quantized_tensor.float() - zero_point)

Test case with random scale and zero point:

test_tensor = torch.tensor(
[[191.6, -13.5, 728.6],
[92.14, 295.5, -184],
[0, 684.6, 245.5]]
)
scale = 3.5
zero_point = -70
quantized_tensor = linear_q_with_scale_and_zero_point(
test_tensor, scale, zero_point)

tensor([[ -15, -74, 127],
[ -44, 14, -123],
[ -70, 126, 0]], dtype=torch.int8)

dequantized_tensor = linear_dequantization(quantized_tensor, scale, zero_point)

tensor([[ 192.5000, -14.0000, 689.5000],
[ 91.0000, 294.0000, -185.5000],
[ 0.0000, 686.0000, 245.0000]])

dequantized_tensor - test_tensor

tensor([[ 0.9000, -0.5000, -39.1000],
[ -1.1400, -1.5000, -1.5000],
[ 0.0000, 1.4000, -0.5000]])

(dequantized_tensor - test_tensor).square()

tensor([[8.0999e-01, 2.5000e-01, 1.5288e+03],
[1.2996e+00, 2.2500e+00, 2.2500e+00],
[0.0000e+00, 1.9601e+00, 2.5000e-01]])

(dequantized_tensor - test_tensor).square().mean()

tensor(170.8753)

from helper import plot_quantization_errors
plot_quantization_errors(test_tensor, quantized_tensor, dequantized_tensor)