Abstract:Quantizing the weights of large language models (LLMs) from 16-bit to lower bitwidth is the de facto approach to deploy massive transformers onto more affordable accelerators. While GPTQ emerged as one of the standard methods for one-shot post-training quantization at LLM scale, its inner workings are described as a sequence of algebraic updates that obscure geometric meaning or worst-case guarantees. In this work, we show that, when executed back-to-front (from the last to first dimension) for a linear layer, GPTQ is mathematically identical to Babai's nearest plane algorithm for the classical closest vector problem (CVP) on a lattice defined by the Hessian matrix of the layer's inputs. This equivalence is based on a sophisticated mathematical argument, and has two analytical consequences: first, the GPTQ error propagation step gains an intuitive geometric interpretation; second, GPTQ inherits the error upper bound of Babai's algorithm under the assumption that no weights are clipped. Leveraging this bound, we design post-training quantization methods that avoid clipping, and outperform the original GPTQ. In addition, we provide efficient GPU inference kernels for the resulting representation. Taken together, these results place GPTQ on a firm theoretical footing and open the door to importing decades of progress in lattice algorithms towards the design of future quantization algorithms for billion-parameter models. Source code is available at this https URL.
| Comments: | Published as a conference paper at the Fourteenth International Conference on Learning Representations (ICLR 2026): this https URL |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2507.18553 [cs.LG] |
| (or arXiv:2507.18553v4 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2507.18553 arXiv-issued DOI via DataCite |
Submission history
From: Jiale Chen [view email]
[v1]
Thu, 24 Jul 2025 16:22:18 UTC (81 KB)
[v2]
Wed, 1 Oct 2025 13:25:12 UTC (293 KB)
[v3]
Mon, 2 Mar 2026 10:25:35 UTC (308 KB)
[v4]
Wed, 13 May 2026 13:18:37 UTC (308 KB)