Quantized models are smaller and faster than full-precision models. But what actually is quantization? What’s being done to the model to make it smaller? And how can we quantify the damage this process does to a model? In this talk I’ll cover the mathematical process applied to models to make them smaller, and introduce you to tools you can use to gauge how much a quantized model differs to its full-precision original.
