NeUQI: Near-Optimal Uniform Quantization Parameter Initialization Article Swipe

MAKE IMPACT

Lin Li , Xinyu Hu , Xiaojun Wan · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2505.17595

Large language models (LLMs) achieve impressive performance across domains but face significant challenges when deployed on consumer-grade GPUs or personal devices such as laptops, due to high memory consumption and inference costs. Post-training quantization (PTQ) of LLMs offers a promising solution that reduces their memory footprint and decoding latency. In practice, PTQ with uniform quantization representation is favored for its efficiency and ease of deployment since uniform quantization is widely supported by mainstream hardware and software libraries. Recent studies on $\geq 2$-bit uniform quantization have led to noticeable improvements in post-quantization model performance; however, they primarily focus on quantization methodologies, while the initialization of quantization parameters is underexplored and still relies on the suboptimal Min-Max strategies. In this work, we propose NeUQI, a method devoted to efficiently determining near-optimal initial parameters for uniform quantization. NeUQI is orthogonal to prior quantization methodologies and can seamlessly integrate with them. The experiments with the LLaMA and Qwen families on various tasks demonstrate that our NeUQI consistently outperforms existing methods. Furthermore, when combined with a lightweight distillation strategy, NeUQI can achieve superior performance to PV-tuning, a much more resource-intensive approach.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2505.17595

PDF

https://arxiv.org/pdf/2505.17595

OA Status

green

OpenAlex ID

https://openalex.org/W4415035381

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4415035381

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2505.17595

Digital Object Identifier

Title

NeUQI: Near-Optimal Uniform Quantization Parameter Initialization

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2025

Year of publication

Publication date

2025-05-23

Full publication date if available

Authors

Lin Li, Xinyu Hu, Xiaojun Wan

List of authors in order

Landing page

https://arxiv.org/abs/2505.17595

Publisher landing page

PDF URL

https://arxiv.org/pdf/2505.17595

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2505.17595

Direct OA link when available

Cited by

0

Total citation count in OpenAlex

Full payload