KurTail : Kurtosis-based LLM Quantization Article Swipe

PDF

Mohammad Sadegh Akhondzadeh , Aleksandar Bojchevski , Evangelos Eleftheriou , Martino Dazzi ·

YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2503.01483

One of the challenges of quantizing a large language model (LLM) is the presence of outliers. Outliers often make uniform quantization schemes less effective, particularly in extreme cases such as 4-bit quantization. We introduce KurTail, a new post-training quantization (PTQ) scheme that leverages Kurtosis-based rotation to mitigate outliers in the activations of LLMs. Our method optimizes Kurtosis as a measure of tailedness. This approach enables the quantization of weights, activations, and the KV cache in 4 bits. We utilize layer-wise optimization, ensuring memory efficiency. KurTail outperforms existing quantization methods, offering a 13.3\% boost in MMLU accuracy and a 15.5\% drop in Wiki perplexity compared to QuaRot. It also outperforms SpinQuant with a 2.6\% MMLU gain and reduces perplexity by 2.9\%, all while reducing the training cost. For comparison, learning the rotation using SpinQuant for Llama3-70B requires at least four NVIDIA H100 80GB GPUs, whereas our method requires only a single GPU, making it a more accessible solution for consumer GPU.

Related Topics

Truth And Reconciliation Commission Of Canada

Concepts

No concepts available.

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2503.01483
PDF: https://arxiv.org/pdf/2503.01483
OA Status: green
OpenAlex ID: https://openalex.org/W4415084894

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4415084894

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2503.01483

Digital Object Identifier
Title: KurTail : Kurtosis-based LLM Quantization

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2025

Year of publication
Publication date: 2025-03-03

Full publication date if available
Authors: Mohammad Sadegh Akhondzadeh, Aleksandar Bojchevski, Evangelos Eleftheriou, Martino Dazzi

List of authors in order
Landing page: https://arxiv.org/abs/2503.01483

Publisher landing page
PDF URL: https://arxiv.org/pdf/2503.01483

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2503.01483

Direct OA link when available
Cited by: 0

Total citation count in OpenAlex

Full payload

id	https://openalex.org/W4415084894
doi	https://doi.org/10.48550/arxiv.2503.01483
ids.doi	https://doi.org/10.48550/arxiv.2503.01483
ids.openalex	https://openalex.org/W4415084894
fwci
type	preprint
title	KurTail : Kurtosis-based LLM Quantization
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T10181
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9854000210762024
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1702
topics[0].subfield.display_name	Artificial Intelligence
topics[0].display_name	Natural Language Processing Techniques
is_xpac	False
apc_list
apc_paid
language	en
locations[0].id	pmh:oai:arXiv.org:2503.01483
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2503.01483
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2503.01483
locations[1].id	doi:10.48550/arxiv.2503.01483
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2503.01483
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5047455660
authorships[0].author.orcid
authorships[0].author.display_name	Mohammad Sadegh Akhondzadeh
authorships[0].author_position	first
authorships[0].raw_author_name	Akhondzadeh, Mohammad Sadegh
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5058887708
authorships[1].author.orcid
authorships[1].author.display_name	Aleksandar Bojchevski
authorships[1].author_position	middle
authorships[1].raw_author_name	Bojchevski, Aleksandar
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5014533256
authorships[2].author.orcid
authorships[2].author.display_name	Evangelos Eleftheriou
authorships[2].author_position	middle
authorships[2].raw_author_name	Eleftheriou, Evangelos
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5051779027
authorships[3].author.orcid	https://orcid.org/0000-0002-4184-2170
authorships[3].author.display_name	Martino Dazzi
authorships[3].author_position	last
authorships[3].raw_author_name	Dazzi, Martino
authorships[3].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2503.01483
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-12T00:00:00
display_name	KurTail : Kurtosis-based LLM Quantization
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T10181
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9854000210762024
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1702
primary_topic.subfield.display_name	Artificial Intelligence
primary_topic.display_name	Natural Language Processing Techniques
cited_by_count	0
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2503.01483
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2503.01483
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2503.01483
primary_location.id	pmh:oai:arXiv.org:2503.01483
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2503.01483
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2503.01483
publication_date	2025-03-03
publication_year	2025
referenced_works_count	0
abstract_inverted_index.4	75
abstract_inverted_index.a	6, 35, 58, 90, 97, 111, 148, 153
abstract_inverted_index.It	106
abstract_inverted_index.KV	72
abstract_inverted_index.We	32, 77
abstract_inverted_index.as	29, 57
abstract_inverted_index.at	136
abstract_inverted_index.by	118
abstract_inverted_index.in	25, 48, 74, 93, 100
abstract_inverted_index.is	11
abstract_inverted_index.it	152
abstract_inverted_index.of	1, 4, 14, 51, 60, 67
abstract_inverted_index.to	45, 104
abstract_inverted_index.For	126
abstract_inverted_index.One	0
abstract_inverted_index.Our	53
abstract_inverted_index.all	120
abstract_inverted_index.and	70, 96, 115
abstract_inverted_index.for	133, 157
abstract_inverted_index.new	36
abstract_inverted_index.our	144
abstract_inverted_index.the	2, 12, 49, 65, 71, 123, 129
abstract_inverted_index.80GB	141
abstract_inverted_index.GPU,	150
abstract_inverted_index.GPU.	159
abstract_inverted_index.H100	140
abstract_inverted_index.MMLU	94, 113
abstract_inverted_index.This	62
abstract_inverted_index.Wiki	101
abstract_inverted_index.also	107
abstract_inverted_index.drop	99
abstract_inverted_index.four	138
abstract_inverted_index.gain	114
abstract_inverted_index.less	22
abstract_inverted_index.make	18
abstract_inverted_index.more	154
abstract_inverted_index.only	147
abstract_inverted_index.such	28
abstract_inverted_index.that	41
abstract_inverted_index.with	110
abstract_inverted_index.(LLM)	10
abstract_inverted_index.(PTQ)	39
abstract_inverted_index.2.6\%	112
abstract_inverted_index.4-bit	30
abstract_inverted_index.GPUs,	142
abstract_inverted_index.LLMs.	52
abstract_inverted_index.bits.	76
abstract_inverted_index.boost	92
abstract_inverted_index.cache	73
abstract_inverted_index.cases	27
abstract_inverted_index.cost.	125
abstract_inverted_index.large	7
abstract_inverted_index.least	137
abstract_inverted_index.model	9
abstract_inverted_index.often	17
abstract_inverted_index.using	131
abstract_inverted_index.while	121
abstract_inverted_index.13.3\%	91
abstract_inverted_index.15.5\%	98
abstract_inverted_index.2.9\%,	119
abstract_inverted_index.NVIDIA	139
abstract_inverted_index.making	151
abstract_inverted_index.memory	82
abstract_inverted_index.method	54, 145
abstract_inverted_index.scheme	40
abstract_inverted_index.single	149
abstract_inverted_index.KurTail	84
abstract_inverted_index.QuaRot.	105
abstract_inverted_index.enables	64
abstract_inverted_index.extreme	26
abstract_inverted_index.measure	59
abstract_inverted_index.reduces	116
abstract_inverted_index.schemes	21
abstract_inverted_index.uniform	19
abstract_inverted_index.utilize	78
abstract_inverted_index.whereas	143
abstract_inverted_index.KurTail,	34
abstract_inverted_index.Kurtosis	56
abstract_inverted_index.Outliers	16
abstract_inverted_index.accuracy	95
abstract_inverted_index.approach	63
abstract_inverted_index.compared	103
abstract_inverted_index.consumer	158
abstract_inverted_index.ensuring	81
abstract_inverted_index.existing	86
abstract_inverted_index.language	8
abstract_inverted_index.learning	128
abstract_inverted_index.methods,	88
abstract_inverted_index.mitigate	46
abstract_inverted_index.offering	89
abstract_inverted_index.outliers	47
abstract_inverted_index.presence	13
abstract_inverted_index.reducing	122
abstract_inverted_index.requires	135, 146
abstract_inverted_index.rotation	44, 130
abstract_inverted_index.solution	156
abstract_inverted_index.training	124
abstract_inverted_index.weights,	68
abstract_inverted_index.SpinQuant	109, 132
abstract_inverted_index.introduce	33
abstract_inverted_index.leverages	42
abstract_inverted_index.optimizes	55
abstract_inverted_index.outliers.	15
abstract_inverted_index.Llama3-70B	134
abstract_inverted_index.accessible	155
abstract_inverted_index.challenges	3
abstract_inverted_index.effective,	23
abstract_inverted_index.layer-wise	79
abstract_inverted_index.perplexity	102, 117
abstract_inverted_index.quantizing	5
abstract_inverted_index.activations	50
abstract_inverted_index.comparison,	127
abstract_inverted_index.efficiency.	83
abstract_inverted_index.outperforms	85, 108
abstract_inverted_index.tailedness.	61
abstract_inverted_index.activations,	69
abstract_inverted_index.particularly	24
abstract_inverted_index.quantization	20, 38, 66, 87
abstract_inverted_index.optimization,	80
abstract_inverted_index.post-training	37
abstract_inverted_index.quantization.	31
abstract_inverted_index.Kurtosis-based	43
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	4
citation_normalized_percentile