Sigma: Differential Rescaling of Query, Key and Value for Efficient Language Models Article Swipe
Zhimin Lin
,
Zhenjie Tang
,
Xiao Liu
,
Yeyun Gong
,
Yi‐Bing Cheng
,
Qi Chen
,
Hang Li
,
Ying Xin
,
Ziyue Yang
,
Kailai Yang
,
Yu Yan
,
Jianxin Li
,
S. L. Lu
,
Yiming Huang
,
Zheheng Luo
,
Lei Qu
,
Xuan Feng
,
Yaoxiang Wang
,
Yuqing Xia
,
Feiyang Chen
,
Yuting Jiang
,
Yan Hu
,
Hao Ni
,
Binyang Li
,
Guoshuai Zhao
,
Jui-Hao Chiang
,
Zhongxin Guo
,
Lin Chen
,
Kun Kuang
,
Li Wan
,
Yelong Shen
,
Jian Jiao
,
Peng Cheng
,
Mao Yang
·
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2501.13629
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2501.13629
We introduce Sigma, an efficient large language model specialized for the system domain, empowered by a novel architecture including DiffQKV attention, and pre-trained on our meticulously collected system domain data. DiffQKV attention significantly enhances the inference efficiency of Sigma by optimizing the Query (Q), Key (K), and Value (V) components in the attention mechanism differentially, based on their varying impacts on the model performance and efficiency indicators. Specifically, we (1) conduct extensive experiments that demonstrate the model's varying sensitivity to the compression of K and V components, leading to the development of differentially compressed KV, and (2) propose augmented Q to expand the Q head dimension, which enhances the model's representation capacity with minimal impacts on the inference speed. Rigorous theoretical and empirical analyses reveal that DiffQKV attention significantly enhances efficiency, achieving up to a 33.36% improvement in inference speed over the conventional grouped-query attention (GQA) in long-context scenarios. We pre-train Sigma on 6T tokens from various sources, including 19.5B system domain data that we carefully collect and 1T tokens of synthesized and rewritten data. In general domains, Sigma achieves comparable performance to other state-of-arts models. In the system domain, we introduce the first comprehensive benchmark AIMicius, where Sigma demonstrates remarkable performance across all tasks, significantly outperforming GPT-4 with an absolute improvement up to 52.5%.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2501.13629
- https://arxiv.org/pdf/2501.13629
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4406793209
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4406793209Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2501.13629Digital Object Identifier
- Title
-
Sigma: Differential Rescaling of Query, Key and Value for Efficient Language ModelsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2025Year of publication
- Publication date
-
2025-01-23Full publication date if available
- Authors
-
Zhimin Lin, Zhenjie Tang, Xiao Liu, Yeyun Gong, Yi‐Bing Cheng, Qi Chen, Hang Li, Ying Xin, Ziyue Yang, Kailai Yang, Yu Yan, Jianxin Li, S. L. Lu, Yiming Huang, Zheheng Luo, Lei Qu, Xuan Feng, Yaoxiang Wang, Yuqing Xia, Feiyang Chen, Yuting Jiang, Yan Hu, Hao Ni, Binyang Li, Guoshuai Zhao, Jui-Hao Chiang, Zhongxin Guo, Lin Chen, Kun Kuang, Li Wan, Yelong Shen, Jian Jiao, Peng Cheng, Mao YangList of authors in order
- Landing page
-
https://arxiv.org/abs/2501.13629Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2501.13629Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2501.13629Direct OA link when available
- Concepts
-
Sigma, Differential (mechanical device), Key (lock), Value (mathematics), Computer science, Six Sigma, Physics, Chemistry, Machine learning, Astronomy, Chromatography, Cascade, Computer security, ThermodynamicsTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4406793209 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2501.13629 |
| ids.doi | https://doi.org/10.48550/arxiv.2501.13629 |
| ids.openalex | https://openalex.org/W4406793209 |
| fwci | |
| type | preprint |
| title | Sigma: Differential Rescaling of Query, Key and Value for Efficient Language Models |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10215 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9294000267982483 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Semantic Web and Ontologies |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C2778049214 |
| concepts[0].level | 2 |
| concepts[0].score | 0.7430445551872253 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q7512234 |
| concepts[0].display_name | Sigma |
| concepts[1].id | https://openalex.org/C93226319 |
| concepts[1].level | 2 |
| concepts[1].score | 0.6579930782318115 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q193137 |
| concepts[1].display_name | Differential (mechanical device) |
| concepts[2].id | https://openalex.org/C26517878 |
| concepts[2].level | 2 |
| concepts[2].score | 0.6521903276443481 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q228039 |
| concepts[2].display_name | Key (lock) |
| concepts[3].id | https://openalex.org/C2776291640 |
| concepts[3].level | 2 |
| concepts[3].score | 0.5962281227111816 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q2912517 |
| concepts[3].display_name | Value (mathematics) |
| concepts[4].id | https://openalex.org/C41008148 |
| concepts[4].level | 0 |
| concepts[4].score | 0.45757055282592773 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[4].display_name | Computer science |
| concepts[5].id | https://openalex.org/C23119410 |
| concepts[5].level | 3 |
| concepts[5].score | 0.41582798957824707 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q236908 |
| concepts[5].display_name | Six Sigma |
| concepts[6].id | https://openalex.org/C121332964 |
| concepts[6].level | 0 |
| concepts[6].score | 0.2818443775177002 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q413 |
| concepts[6].display_name | Physics |
| concepts[7].id | https://openalex.org/C185592680 |
| concepts[7].level | 0 |
| concepts[7].score | 0.11096897721290588 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q2329 |
| concepts[7].display_name | Chemistry |
| concepts[8].id | https://openalex.org/C119857082 |
| concepts[8].level | 1 |
| concepts[8].score | 0.08925268054008484 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q2539 |
| concepts[8].display_name | Machine learning |
| concepts[9].id | https://openalex.org/C1276947 |
| concepts[9].level | 1 |
| concepts[9].score | 0.06422850489616394 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q333 |
| concepts[9].display_name | Astronomy |
| concepts[10].id | https://openalex.org/C43617362 |
| concepts[10].level | 1 |
| concepts[10].score | 0.0 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q170050 |
| concepts[10].display_name | Chromatography |
| concepts[11].id | https://openalex.org/C34146451 |
| concepts[11].level | 2 |
| concepts[11].score | 0.0 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q5048094 |
| concepts[11].display_name | Cascade |
| concepts[12].id | https://openalex.org/C38652104 |
| concepts[12].level | 1 |
| concepts[12].score | 0.0 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q3510521 |
| concepts[12].display_name | Computer security |
| concepts[13].id | https://openalex.org/C97355855 |
| concepts[13].level | 1 |
| concepts[13].score | 0.0 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q11473 |
| concepts[13].display_name | Thermodynamics |
| keywords[0].id | https://openalex.org/keywords/sigma |
| keywords[0].score | 0.7430445551872253 |
| keywords[0].display_name | Sigma |
| keywords[1].id | https://openalex.org/keywords/differential |
| keywords[1].score | 0.6579930782318115 |
| keywords[1].display_name | Differential (mechanical device) |
| keywords[2].id | https://openalex.org/keywords/key |
| keywords[2].score | 0.6521903276443481 |
| keywords[2].display_name | Key (lock) |
| keywords[3].id | https://openalex.org/keywords/value |
| keywords[3].score | 0.5962281227111816 |
| keywords[3].display_name | Value (mathematics) |
| keywords[4].id | https://openalex.org/keywords/computer-science |
| keywords[4].score | 0.45757055282592773 |
| keywords[4].display_name | Computer science |
| keywords[5].id | https://openalex.org/keywords/six-sigma |
| keywords[5].score | 0.41582798957824707 |
| keywords[5].display_name | Six Sigma |
| keywords[6].id | https://openalex.org/keywords/physics |
| keywords[6].score | 0.2818443775177002 |
| keywords[6].display_name | Physics |
| keywords[7].id | https://openalex.org/keywords/chemistry |
| keywords[7].score | 0.11096897721290588 |
| keywords[7].display_name | Chemistry |
| keywords[8].id | https://openalex.org/keywords/machine-learning |
| keywords[8].score | 0.08925268054008484 |
| keywords[8].display_name | Machine learning |
| keywords[9].id | https://openalex.org/keywords/astronomy |
| keywords[9].score | 0.06422850489616394 |
| keywords[9].display_name | Astronomy |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2501.13629 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2501.13629 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2501.13629 |
| locations[1].id | doi:10.48550/arxiv.2501.13629 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2501.13629 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5101736732 |
| authorships[0].author.orcid | https://orcid.org/0000-0003-4902-5244 |
| authorships[0].author.display_name | Zhimin Lin |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Lin, Zhenghao |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5101885891 |
| authorships[1].author.orcid | |
| authorships[1].author.display_name | Zhenjie Tang |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Tang, Zihao |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5049736356 |
| authorships[2].author.orcid | https://orcid.org/0009-0000-9541-0560 |
| authorships[2].author.display_name | Xiao Liu |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Liu, Xiao |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5111237866 |
| authorships[3].author.orcid | |
| authorships[3].author.display_name | Yeyun Gong |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Gong, Yeyun |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5017827110 |
| authorships[4].author.orcid | https://orcid.org/0000-0003-0604-1965 |
| authorships[4].author.display_name | Yi‐Bing Cheng |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Cheng, Yi |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5029488879 |
| authorships[5].author.orcid | https://orcid.org/0000-0002-6510-907X |
| authorships[5].author.display_name | Qi Chen |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Chen, Qi |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5100455145 |
| authorships[6].author.orcid | https://orcid.org/0000-0002-7987-213X |
| authorships[6].author.display_name | Hang Li |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Li, Hang |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5003270959 |
| authorships[7].author.orcid | |
| authorships[7].author.display_name | Ying Xin |
| authorships[7].author_position | middle |
| authorships[7].raw_author_name | Xin, Ying |
| authorships[7].is_corresponding | False |
| authorships[8].author.id | https://openalex.org/A5029661948 |
| authorships[8].author.orcid | https://orcid.org/0000-0002-1658-0260 |
| authorships[8].author.display_name | Ziyue Yang |
| authorships[8].author_position | middle |
| authorships[8].raw_author_name | Yang, Ziyue |
| authorships[8].is_corresponding | False |
| authorships[9].author.id | https://openalex.org/A5066983614 |
| authorships[9].author.orcid | https://orcid.org/0000-0003-3142-2516 |
| authorships[9].author.display_name | Kailai Yang |
| authorships[9].author_position | middle |
| authorships[9].raw_author_name | Yang, Kailai |
| authorships[9].is_corresponding | False |
| authorships[10].author.id | https://openalex.org/A5015264469 |
| authorships[10].author.orcid | https://orcid.org/0000-0003-1627-5335 |
| authorships[10].author.display_name | Yu Yan |
| authorships[10].author_position | middle |
| authorships[10].raw_author_name | Yan, Yu |
| authorships[10].is_corresponding | False |
| authorships[11].author.id | https://openalex.org/A5100380476 |
| authorships[11].author.orcid | https://orcid.org/0000-0003-2500-9143 |
| authorships[11].author.display_name | Jianxin Li |
| authorships[11].author_position | middle |
| authorships[11].raw_author_name | Liang, Xiao |
| authorships[11].is_corresponding | False |
| authorships[12].author.id | https://openalex.org/A5019255096 |
| authorships[12].author.orcid | https://orcid.org/0009-0004-9046-5094 |
| authorships[12].author.display_name | S. L. Lu |
| authorships[12].author_position | middle |
| authorships[12].raw_author_name | Lu, Shuai |
| authorships[12].is_corresponding | False |
| authorships[13].author.id | https://openalex.org/A5100772800 |
| authorships[13].author.orcid | https://orcid.org/0000-0001-5967-557X |
| authorships[13].author.display_name | Yiming Huang |
| authorships[13].author_position | middle |
| authorships[13].raw_author_name | Huang, Yiming |
| authorships[13].is_corresponding | False |
| authorships[14].author.id | https://openalex.org/A5070908607 |
| authorships[14].author.orcid | |
| authorships[14].author.display_name | Zheheng Luo |
| authorships[14].author_position | middle |
| authorships[14].raw_author_name | Luo, Zheheng |
| authorships[14].is_corresponding | False |
| authorships[15].author.id | https://openalex.org/A5101894557 |
| authorships[15].author.orcid | https://orcid.org/0000-0001-7167-879X |
| authorships[15].author.display_name | Lei Qu |
| authorships[15].author_position | middle |
| authorships[15].raw_author_name | Qu, Lei |
| authorships[15].is_corresponding | False |
| authorships[16].author.id | https://openalex.org/A5114204997 |
| authorships[16].author.orcid | https://orcid.org/0000-0002-0338-3145 |
| authorships[16].author.display_name | Xuan Feng |
| authorships[16].author_position | middle |
| authorships[16].raw_author_name | Feng, Xuan |
| authorships[16].is_corresponding | False |
| authorships[17].author.id | https://openalex.org/A5076537656 |
| authorships[17].author.orcid | |
| authorships[17].author.display_name | Yaoxiang Wang |
| authorships[17].author_position | middle |
| authorships[17].raw_author_name | Wang, Yaoxiang |
| authorships[17].is_corresponding | False |
| authorships[18].author.id | https://openalex.org/A5103023427 |
| authorships[18].author.orcid | https://orcid.org/0000-0002-6965-3267 |
| authorships[18].author.display_name | Yuqing Xia |
| authorships[18].author_position | middle |
| authorships[18].raw_author_name | Xia, Yuqing |
| authorships[18].is_corresponding | False |
| authorships[19].author.id | https://openalex.org/A5046010230 |
| authorships[19].author.orcid | https://orcid.org/0000-0003-0582-7333 |
| authorships[19].author.display_name | Feiyang Chen |
| authorships[19].author_position | middle |
| authorships[19].raw_author_name | Chen, Feiyang |
| authorships[19].is_corresponding | False |
| authorships[20].author.id | https://openalex.org/A5101642309 |
| authorships[20].author.orcid | https://orcid.org/0000-0002-0400-6042 |
| authorships[20].author.display_name | Yuting Jiang |
| authorships[20].author_position | middle |
| authorships[20].raw_author_name | Jiang, Yuting |
| authorships[20].is_corresponding | False |
| authorships[21].author.id | https://openalex.org/A5001864512 |
| authorships[21].author.orcid | https://orcid.org/0000-0003-1558-1541 |
| authorships[21].author.display_name | Yan Hu |
| authorships[21].author_position | middle |
| authorships[21].raw_author_name | Hu, Yasen |
| authorships[21].is_corresponding | False |
| authorships[22].author.id | https://openalex.org/A5103816886 |
| authorships[22].author.orcid | |
| authorships[22].author.display_name | Hao Ni |
| authorships[22].author_position | middle |
| authorships[22].raw_author_name | Ni, Hao |
| authorships[22].is_corresponding | False |
| authorships[23].author.id | https://openalex.org/A5045875310 |
| authorships[23].author.orcid | https://orcid.org/0000-0001-9013-1386 |
| authorships[23].author.display_name | Binyang Li |
| authorships[23].author_position | middle |
| authorships[23].raw_author_name | Li, Binyang |
| authorships[23].is_corresponding | False |
| authorships[24].author.id | https://openalex.org/A5079805574 |
| authorships[24].author.orcid | https://orcid.org/0000-0003-4392-8450 |
| authorships[24].author.display_name | Guoshuai Zhao |
| authorships[24].author_position | middle |
| authorships[24].raw_author_name | Zhao, Guoshuai |
| authorships[24].is_corresponding | False |
| authorships[25].author.id | https://openalex.org/A5105250571 |
| authorships[25].author.orcid | |
| authorships[25].author.display_name | Jui-Hao Chiang |
| authorships[25].author_position | middle |
| authorships[25].raw_author_name | Chiang, Jui-Hao |
| authorships[25].is_corresponding | False |
| authorships[26].author.id | https://openalex.org/A5003841379 |
| authorships[26].author.orcid | |
| authorships[26].author.display_name | Zhongxin Guo |
| authorships[26].author_position | middle |
| authorships[26].raw_author_name | Guo, Zhongxin |
| authorships[26].is_corresponding | False |
| authorships[27].author.id | https://openalex.org/A5100443753 |
| authorships[27].author.orcid | https://orcid.org/0000-0002-6352-4704 |
| authorships[27].author.display_name | Lin Chen |
| authorships[27].author_position | middle |
| authorships[27].raw_author_name | Lin, Chen |
| authorships[27].is_corresponding | False |
| authorships[28].author.id | https://openalex.org/A5041727387 |
| authorships[28].author.orcid | https://orcid.org/0000-0001-7024-9790 |
| authorships[28].author.display_name | Kun Kuang |
| authorships[28].author_position | middle |
| authorships[28].raw_author_name | Kuang, Kun |
| authorships[28].is_corresponding | False |
| authorships[29].author.id | https://openalex.org/A5100320147 |
| authorships[29].author.orcid | https://orcid.org/0000-0002-9797-0315 |
| authorships[29].author.display_name | Li Wan |
| authorships[29].author_position | middle |
| authorships[29].raw_author_name | Li, Wenjie |
| authorships[29].is_corresponding | False |
| authorships[30].author.id | https://openalex.org/A5101180037 |
| authorships[30].author.orcid | |
| authorships[30].author.display_name | Yelong Shen |
| authorships[30].author_position | middle |
| authorships[30].raw_author_name | Shen, Yelong |
| authorships[30].is_corresponding | False |
| authorships[31].author.id | https://openalex.org/A5039068226 |
| authorships[31].author.orcid | https://orcid.org/0000-0002-5206-8037 |
| authorships[31].author.display_name | Jian Jiao |
| authorships[31].author_position | middle |
| authorships[31].raw_author_name | Jiao, Jian |
| authorships[31].is_corresponding | False |
| authorships[32].author.id | https://openalex.org/A5021052588 |
| authorships[32].author.orcid | https://orcid.org/0000-0003-1091-7894 |
| authorships[32].author.display_name | Peng Cheng |
| authorships[32].author_position | middle |
| authorships[32].raw_author_name | Cheng, Peng |
| authorships[32].is_corresponding | False |
| authorships[33].author.id | https://openalex.org/A5100721449 |
| authorships[33].author.orcid | https://orcid.org/0000-0002-1535-0498 |
| authorships[33].author.display_name | Mao Yang |
| authorships[33].author_position | last |
| authorships[33].raw_author_name | Yang, Mao |
| authorships[33].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2501.13629 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Sigma: Differential Rescaling of Query, Key and Value for Efficient Language Models |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10215 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9294000267982483 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Semantic Web and Ontologies |
| related_works | https://openalex.org/W2010530423, https://openalex.org/W2376050517, https://openalex.org/W2526355781, https://openalex.org/W1601839798, https://openalex.org/W2012465829, https://openalex.org/W1819446, https://openalex.org/W2725982103, https://openalex.org/W594616480, https://openalex.org/W2018304657, https://openalex.org/W1964627399 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2501.13629 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2501.13629 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2501.13629 |
| primary_location.id | pmh:oai:arXiv.org:2501.13629 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2501.13629 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2501.13629 |
| publication_date | 2025-01-23 |
| publication_year | 2025 |
| referenced_works_count | 0 |
| abstract_inverted_index.K | 83 |
| abstract_inverted_index.Q | 99, 103 |
| abstract_inverted_index.V | 85 |
| abstract_inverted_index.a | 15, 134 |
| abstract_inverted_index.1T | 168 |
| abstract_inverted_index.6T | 153 |
| abstract_inverted_index.In | 175, 186 |
| abstract_inverted_index.We | 0, 149 |
| abstract_inverted_index.an | 3, 209 |
| abstract_inverted_index.by | 14, 39 |
| abstract_inverted_index.in | 50, 137, 146 |
| abstract_inverted_index.of | 37, 82, 91, 170 |
| abstract_inverted_index.on | 23, 56, 60, 115, 152 |
| abstract_inverted_index.to | 79, 88, 100, 133, 182, 213 |
| abstract_inverted_index.up | 132, 212 |
| abstract_inverted_index.we | 68, 164, 190 |
| abstract_inverted_index.(1) | 69 |
| abstract_inverted_index.(2) | 96 |
| abstract_inverted_index.(V) | 48 |
| abstract_inverted_index.KV, | 94 |
| abstract_inverted_index.Key | 44 |
| abstract_inverted_index.all | 203 |
| abstract_inverted_index.and | 21, 46, 64, 84, 95, 121, 167, 172 |
| abstract_inverted_index.for | 9 |
| abstract_inverted_index.our | 24 |
| abstract_inverted_index.the | 10, 34, 41, 51, 61, 75, 80, 89, 102, 108, 116, 141, 187, 192 |
| abstract_inverted_index.(K), | 45 |
| abstract_inverted_index.(Q), | 43 |
| abstract_inverted_index.data | 162 |
| abstract_inverted_index.from | 155 |
| abstract_inverted_index.head | 104 |
| abstract_inverted_index.over | 140 |
| abstract_inverted_index.that | 73, 125, 163 |
| abstract_inverted_index.with | 112, 208 |
| abstract_inverted_index.(GQA) | 145 |
| abstract_inverted_index.19.5B | 159 |
| abstract_inverted_index.GPT-4 | 207 |
| abstract_inverted_index.Query | 42 |
| abstract_inverted_index.Sigma | 38, 151, 178, 198 |
| abstract_inverted_index.Value | 47 |
| abstract_inverted_index.based | 55 |
| abstract_inverted_index.data. | 29, 174 |
| abstract_inverted_index.first | 193 |
| abstract_inverted_index.large | 5 |
| abstract_inverted_index.model | 7, 62 |
| abstract_inverted_index.novel | 16 |
| abstract_inverted_index.other | 183 |
| abstract_inverted_index.speed | 139 |
| abstract_inverted_index.their | 57 |
| abstract_inverted_index.where | 197 |
| abstract_inverted_index.which | 106 |
| abstract_inverted_index.33.36% | 135 |
| abstract_inverted_index.52.5%. | 214 |
| abstract_inverted_index.Sigma, | 2 |
| abstract_inverted_index.across | 202 |
| abstract_inverted_index.domain | 28, 161 |
| abstract_inverted_index.expand | 101 |
| abstract_inverted_index.reveal | 124 |
| abstract_inverted_index.speed. | 118 |
| abstract_inverted_index.system | 11, 27, 160, 188 |
| abstract_inverted_index.tasks, | 204 |
| abstract_inverted_index.tokens | 154, 169 |
| abstract_inverted_index.DiffQKV | 19, 30, 126 |
| abstract_inverted_index.collect | 166 |
| abstract_inverted_index.conduct | 70 |
| abstract_inverted_index.domain, | 12, 189 |
| abstract_inverted_index.general | 176 |
| abstract_inverted_index.impacts | 59, 114 |
| abstract_inverted_index.leading | 87 |
| abstract_inverted_index.minimal | 113 |
| abstract_inverted_index.model's | 76, 109 |
| abstract_inverted_index.models. | 185 |
| abstract_inverted_index.propose | 97 |
| abstract_inverted_index.various | 156 |
| abstract_inverted_index.varying | 58, 77 |
| abstract_inverted_index.Rigorous | 119 |
| abstract_inverted_index.absolute | 210 |
| abstract_inverted_index.achieves | 179 |
| abstract_inverted_index.analyses | 123 |
| abstract_inverted_index.capacity | 111 |
| abstract_inverted_index.domains, | 177 |
| abstract_inverted_index.enhances | 33, 107, 129 |
| abstract_inverted_index.language | 6 |
| abstract_inverted_index.sources, | 157 |
| abstract_inverted_index.AIMicius, | 196 |
| abstract_inverted_index.achieving | 131 |
| abstract_inverted_index.attention | 31, 52, 127, 144 |
| abstract_inverted_index.augmented | 98 |
| abstract_inverted_index.benchmark | 195 |
| abstract_inverted_index.carefully | 165 |
| abstract_inverted_index.collected | 26 |
| abstract_inverted_index.efficient | 4 |
| abstract_inverted_index.empirical | 122 |
| abstract_inverted_index.empowered | 13 |
| abstract_inverted_index.extensive | 71 |
| abstract_inverted_index.including | 18, 158 |
| abstract_inverted_index.inference | 35, 117, 138 |
| abstract_inverted_index.introduce | 1, 191 |
| abstract_inverted_index.mechanism | 53 |
| abstract_inverted_index.pre-train | 150 |
| abstract_inverted_index.rewritten | 173 |
| abstract_inverted_index.attention, | 20 |
| abstract_inverted_index.comparable | 180 |
| abstract_inverted_index.components | 49 |
| abstract_inverted_index.compressed | 93 |
| abstract_inverted_index.dimension, | 105 |
| abstract_inverted_index.efficiency | 36, 65 |
| abstract_inverted_index.optimizing | 40 |
| abstract_inverted_index.remarkable | 200 |
| abstract_inverted_index.scenarios. | 148 |
| abstract_inverted_index.components, | 86 |
| abstract_inverted_index.compression | 81 |
| abstract_inverted_index.demonstrate | 74 |
| abstract_inverted_index.development | 90 |
| abstract_inverted_index.efficiency, | 130 |
| abstract_inverted_index.experiments | 72 |
| abstract_inverted_index.improvement | 136, 211 |
| abstract_inverted_index.indicators. | 66 |
| abstract_inverted_index.performance | 63, 181, 201 |
| abstract_inverted_index.pre-trained | 22 |
| abstract_inverted_index.sensitivity | 78 |
| abstract_inverted_index.specialized | 8 |
| abstract_inverted_index.synthesized | 171 |
| abstract_inverted_index.theoretical | 120 |
| abstract_inverted_index.architecture | 17 |
| abstract_inverted_index.conventional | 142 |
| abstract_inverted_index.demonstrates | 199 |
| abstract_inverted_index.long-context | 147 |
| abstract_inverted_index.meticulously | 25 |
| abstract_inverted_index.Specifically, | 67 |
| abstract_inverted_index.comprehensive | 194 |
| abstract_inverted_index.grouped-query | 143 |
| abstract_inverted_index.outperforming | 206 |
| abstract_inverted_index.significantly | 32, 128, 205 |
| abstract_inverted_index.state-of-arts | 184 |
| abstract_inverted_index.differentially | 92 |
| abstract_inverted_index.representation | 110 |
| abstract_inverted_index.differentially, | 54 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 34 |
| citation_normalized_percentile |