Compression Hacking: A Supplementary Perspective on Informatics Properties of Language Models from Geometric Distortion Article Swipe
J. J. Zang
,
Menɡ Ninɡ
,
Yuguang Wei
,
Shihan Dou
,
Jiazheng Zhang
,
Nan Mo
,
Binghong Li
,
Tao Gui
,
Qi Zhang
,
Xuanjing Huang
·
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2505.17793
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2505.17793
Recently, the concept of ``compression as intelligence'' has provided a novel informatics metric perspective for language models (LMs), emphasizing that highly structured representations signify the intelligence level of LMs. However, from a geometric standpoint, the word representation space of highly compressed LMs tends to degenerate into a highly anisotropic state, which hinders the LM's ability to comprehend instructions and directly impacts its performance. We found this compression-anisotropy synchronicity is essentially the ``Compression Hacking'' in LM representations, where noise-dominated directions tend to create the illusion of high compression rates by sacrificing spatial uniformity. Based on this, we propose three refined compression metrics by incorporating geometric distortion analysis and integrate them into a self-evaluation pipeline. The refined metrics exhibit strong alignment with the LM's comprehensive capabilities, achieving Spearman correlation coefficients above 0.9, significantly outperforming both the original compression and other internal structure-based metrics. This confirms that compression hacking substantially enhances the informatics interpretation of LMs by incorporating geometric distortion of representations.
Related Topics
Concepts
No concepts available.
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2505.17793
- https://arxiv.org/pdf/2505.17793
- OA Status
- green
- OpenAlex ID
- https://openalex.org/W4414942271
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4414942271Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2505.17793Digital Object Identifier
- Title
-
Compression Hacking: A Supplementary Perspective on Informatics Properties of Language Models from Geometric DistortionWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2025Year of publication
- Publication date
-
2025-05-23Full publication date if available
- Authors
-
J. J. Zang, Menɡ Ninɡ, Yuguang Wei, Shihan Dou, Jiazheng Zhang, Nan Mo, Binghong Li, Tao Gui, Qi Zhang, Xuanjing HuangList of authors in order
- Landing page
-
https://arxiv.org/abs/2505.17793Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2505.17793Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2505.17793Direct OA link when available
- Cited by
-
0Total citation count in OpenAlex
Full payload
| id | https://openalex.org/W4414942271 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2505.17793 |
| ids.doi | https://doi.org/10.48550/arxiv.2505.17793 |
| ids.openalex | https://openalex.org/W4414942271 |
| fwci | |
| type | preprint |
| title | Compression Hacking: A Supplementary Perspective on Informatics Properties of Language Models from Geometric Distortion |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T12002 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.3382999897003174 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1703 |
| topics[0].subfield.display_name | Computational Theory and Mathematics |
| topics[0].display_name | Computability, Logic, AI Algorithms |
| topics[1].id | https://openalex.org/T10181 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.31700000166893005 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Natural Language Processing Techniques |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2505.17793 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | cc-by |
| locations[0].pdf_url | https://arxiv.org/pdf/2505.17793 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | https://openalex.org/licenses/cc-by |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2505.17793 |
| locations[1].id | doi:10.48550/arxiv.2505.17793 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2505.17793 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5079402607 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | J. J. Zang |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Zang, Jianxiang |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5088844549 |
| authorships[1].author.orcid | https://orcid.org/0000-0001-9265-3372 |
| authorships[1].author.display_name | Menɡ Ninɡ |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Ning, Meiling |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5077785879 |
| authorships[2].author.orcid | https://orcid.org/0000-0001-7563-150X |
| authorships[2].author.display_name | Yuguang Wei |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Wei, Yongda |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5083173421 |
| authorships[3].author.orcid | https://orcid.org/0009-0002-6013-3035 |
| authorships[3].author.display_name | Shihan Dou |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Dou, Shihan |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5043925137 |
| authorships[4].author.orcid | https://orcid.org/0000-0003-1963-791X |
| authorships[4].author.display_name | Jiazheng Zhang |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Zhang, Jiazheng |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5078307564 |
| authorships[5].author.orcid | https://orcid.org/0009-0002-8481-2972 |
| authorships[5].author.display_name | Nan Mo |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Mo, Nijia |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5035963460 |
| authorships[6].author.orcid | https://orcid.org/0000-0001-7534-5837 |
| authorships[6].author.display_name | Binghong Li |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Li, Binhong |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5108571703 |
| authorships[7].author.orcid | |
| authorships[7].author.display_name | Tao Gui |
| authorships[7].author_position | middle |
| authorships[7].raw_author_name | Gui, Tao |
| authorships[7].is_corresponding | False |
| authorships[8].author.id | https://openalex.org/A5100360242 |
| authorships[8].author.orcid | https://orcid.org/0000-0001-9611-6697 |
| authorships[8].author.display_name | Qi Zhang |
| authorships[8].author_position | middle |
| authorships[8].raw_author_name | Zhang, Qi |
| authorships[8].is_corresponding | False |
| authorships[9].author.id | https://openalex.org/A5088834359 |
| authorships[9].author.orcid | https://orcid.org/0000-0001-9197-9426 |
| authorships[9].author.display_name | Xuanjing Huang |
| authorships[9].author_position | last |
| authorships[9].raw_author_name | Huang, Xuanjing |
| authorships[9].is_corresponding | False |
| has_content.pdf | True |
| has_content.grobid_xml | True |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2505.17793 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Compression Hacking: A Supplementary Perspective on Informatics Properties of Language Models from Geometric Distortion |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T12002 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.3382999897003174 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1703 |
| primary_topic.subfield.display_name | Computational Theory and Mathematics |
| primary_topic.display_name | Computability, Logic, AI Algorithms |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2505.17793 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | cc-by |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2505.17793 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | https://openalex.org/licenses/cc-by |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2505.17793 |
| primary_location.id | pmh:oai:arXiv.org:2505.17793 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | cc-by |
| primary_location.pdf_url | https://arxiv.org/pdf/2505.17793 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | https://openalex.org/licenses/cc-by |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2505.17793 |
| publication_date | 2025-05-23 |
| publication_year | 2025 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 9, 31, 46, 110 |
| abstract_inverted_index.LM | 74 |
| abstract_inverted_index.We | 63 |
| abstract_inverted_index.as | 5 |
| abstract_inverted_index.by | 88, 101, 153 |
| abstract_inverted_index.in | 73 |
| abstract_inverted_index.is | 68 |
| abstract_inverted_index.of | 3, 27, 38, 84, 151, 157 |
| abstract_inverted_index.on | 93 |
| abstract_inverted_index.to | 43, 55, 80 |
| abstract_inverted_index.we | 95 |
| abstract_inverted_index.LMs | 41, 152 |
| abstract_inverted_index.The | 113 |
| abstract_inverted_index.and | 58, 106, 136 |
| abstract_inverted_index.for | 14 |
| abstract_inverted_index.has | 7 |
| abstract_inverted_index.its | 61 |
| abstract_inverted_index.the | 1, 24, 34, 52, 70, 82, 120, 133, 148 |
| abstract_inverted_index.0.9, | 129 |
| abstract_inverted_index.LM's | 53, 121 |
| abstract_inverted_index.LMs. | 28 |
| abstract_inverted_index.This | 141 |
| abstract_inverted_index.both | 132 |
| abstract_inverted_index.from | 30 |
| abstract_inverted_index.high | 85 |
| abstract_inverted_index.into | 45, 109 |
| abstract_inverted_index.tend | 79 |
| abstract_inverted_index.that | 19, 143 |
| abstract_inverted_index.them | 108 |
| abstract_inverted_index.this | 65 |
| abstract_inverted_index.with | 119 |
| abstract_inverted_index.word | 35 |
| abstract_inverted_index.Based | 92 |
| abstract_inverted_index.above | 128 |
| abstract_inverted_index.found | 64 |
| abstract_inverted_index.level | 26 |
| abstract_inverted_index.novel | 10 |
| abstract_inverted_index.other | 137 |
| abstract_inverted_index.rates | 87 |
| abstract_inverted_index.space | 37 |
| abstract_inverted_index.tends | 42 |
| abstract_inverted_index.this, | 94 |
| abstract_inverted_index.three | 97 |
| abstract_inverted_index.where | 76 |
| abstract_inverted_index.which | 50 |
| abstract_inverted_index.(LMs), | 17 |
| abstract_inverted_index.create | 81 |
| abstract_inverted_index.highly | 20, 39, 47 |
| abstract_inverted_index.metric | 12 |
| abstract_inverted_index.models | 16 |
| abstract_inverted_index.state, | 49 |
| abstract_inverted_index.strong | 117 |
| abstract_inverted_index.ability | 54 |
| abstract_inverted_index.concept | 2 |
| abstract_inverted_index.exhibit | 116 |
| abstract_inverted_index.hacking | 145 |
| abstract_inverted_index.hinders | 51 |
| abstract_inverted_index.impacts | 60 |
| abstract_inverted_index.metrics | 100, 115 |
| abstract_inverted_index.propose | 96 |
| abstract_inverted_index.refined | 98, 114 |
| abstract_inverted_index.signify | 23 |
| abstract_inverted_index.spatial | 90 |
| abstract_inverted_index.However, | 29 |
| abstract_inverted_index.Spearman | 125 |
| abstract_inverted_index.analysis | 105 |
| abstract_inverted_index.confirms | 142 |
| abstract_inverted_index.directly | 59 |
| abstract_inverted_index.enhances | 147 |
| abstract_inverted_index.illusion | 83 |
| abstract_inverted_index.internal | 138 |
| abstract_inverted_index.language | 15 |
| abstract_inverted_index.metrics. | 140 |
| abstract_inverted_index.original | 134 |
| abstract_inverted_index.provided | 8 |
| abstract_inverted_index.Hacking'' | 72 |
| abstract_inverted_index.Recently, | 0 |
| abstract_inverted_index.achieving | 124 |
| abstract_inverted_index.alignment | 118 |
| abstract_inverted_index.geometric | 32, 103, 155 |
| abstract_inverted_index.integrate | 107 |
| abstract_inverted_index.pipeline. | 112 |
| abstract_inverted_index.comprehend | 56 |
| abstract_inverted_index.compressed | 40 |
| abstract_inverted_index.degenerate | 44 |
| abstract_inverted_index.directions | 78 |
| abstract_inverted_index.distortion | 104, 156 |
| abstract_inverted_index.structured | 21 |
| abstract_inverted_index.anisotropic | 48 |
| abstract_inverted_index.compression | 86, 99, 135, 144 |
| abstract_inverted_index.correlation | 126 |
| abstract_inverted_index.emphasizing | 18 |
| abstract_inverted_index.essentially | 69 |
| abstract_inverted_index.informatics | 11, 149 |
| abstract_inverted_index.perspective | 13 |
| abstract_inverted_index.sacrificing | 89 |
| abstract_inverted_index.standpoint, | 33 |
| abstract_inverted_index.uniformity. | 91 |
| abstract_inverted_index.coefficients | 127 |
| abstract_inverted_index.instructions | 57 |
| abstract_inverted_index.intelligence | 25 |
| abstract_inverted_index.performance. | 62 |
| abstract_inverted_index.``Compression | 71 |
| abstract_inverted_index.``compression | 4 |
| abstract_inverted_index.capabilities, | 123 |
| abstract_inverted_index.comprehensive | 122 |
| abstract_inverted_index.incorporating | 102, 154 |
| abstract_inverted_index.outperforming | 131 |
| abstract_inverted_index.significantly | 130 |
| abstract_inverted_index.substantially | 146 |
| abstract_inverted_index.synchronicity | 67 |
| abstract_inverted_index.intelligence'' | 6 |
| abstract_inverted_index.interpretation | 150 |
| abstract_inverted_index.representation | 36 |
| abstract_inverted_index.noise-dominated | 77 |
| abstract_inverted_index.representations | 22 |
| abstract_inverted_index.self-evaluation | 111 |
| abstract_inverted_index.structure-based | 139 |
| abstract_inverted_index.representations, | 75 |
| abstract_inverted_index.representations. | 158 |
| abstract_inverted_index.compression-anisotropy | 66 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 10 |
| citation_normalized_percentile |