Carl Franzen / Venture::
Zurich Lab of Huawei reveals Sinq, an open source quantification method which, according to her, can reduce the use of LLM memory from 60 to 70% without significant quality loss– – Double axis scale: instead of using a single scale factor to quantify a matrix, SINQ uses separate scaling vectors for lines and columns.