Inducing Neural Collapse to a Fixed Hierarchy-Aware Frame for Reducing Mistake Severity Article Swipe
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2303.05689
There is a recently discovered and intriguing phenomenon called Neural Collapse: at the terminal phase of training a deep neural network for classification, the within-class penultimate feature means and the associated classifier vectors of all flat classes collapse to the vertices of a simplex Equiangular Tight Frame (ETF). Recent work has tried to exploit this phenomenon by fixing the related classifier weights to a pre-computed ETF to induce neural collapse and maximize the separation of the learned features when training with imbalanced data. In this work, we propose to fix the linear classifier of a deep neural network to a Hierarchy-Aware Frame (HAFrame), instead of an ETF, and use a cosine similarity-based auxiliary loss to learn hierarchy-aware penultimate features that collapse to the HAFrame. We demonstrate that our approach reduces the mistake severity of the model's predictions while maintaining its top-1 accuracy on several datasets of varying scales with hierarchies of heights ranging from 3 to 12. Code: https://github.com/ltong1130ztr/HAFrame
Related Topics
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2303.05689
- https://arxiv.org/pdf/2303.05689
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4324107462
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4324107462Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2303.05689Digital Object Identifier
- Title
-
Inducing Neural Collapse to a Fixed Hierarchy-Aware Frame for Reducing Mistake SeverityWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2023Year of publication
- Publication date
-
2023-03-10Full publication date if available
- Authors
-
Liang Tong, Jim DavisList of authors in order
- Landing page
-
https://arxiv.org/abs/2303.05689Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2303.05689Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2303.05689Direct OA link when available
- Concepts
-
Artificial neural network, Computer science, Mistake, Classifier (UML), Artificial intelligence, Deep neural networks, Simplex, Hierarchy, Exploit, Pattern recognition (psychology), Machine learning, Algorithm, Mathematics, Combinatorics, Economics, Law, Market economy, Computer security, Political scienceTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4324107462 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2303.05689 |
| ids.doi | https://doi.org/10.48550/arxiv.2303.05689 |
| ids.openalex | https://openalex.org/W4324107462 |
| fwci | |
| type | preprint |
| title | Inducing Neural Collapse to a Fixed Hierarchy-Aware Frame for Reducing Mistake Severity |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T11652 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9918000102043152 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Imbalanced Data Classification Techniques |
| topics[1].id | https://openalex.org/T12535 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.975600004196167 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Machine Learning and Data Classification |
| topics[2].id | https://openalex.org/T11512 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9733999967575073 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Anomaly Detection Techniques and Applications |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C50644808 |
| concepts[0].level | 2 |
| concepts[0].score | 0.6927560567855835 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q192776 |
| concepts[0].display_name | Artificial neural network |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.652065098285675 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C2777179996 |
| concepts[2].level | 2 |
| concepts[2].score | 0.6012968420982361 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q911222 |
| concepts[2].display_name | Mistake |
| concepts[3].id | https://openalex.org/C95623464 |
| concepts[3].level | 2 |
| concepts[3].score | 0.5849228501319885 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q1096149 |
| concepts[3].display_name | Classifier (UML) |
| concepts[4].id | https://openalex.org/C154945302 |
| concepts[4].level | 1 |
| concepts[4].score | 0.5524932742118835 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[4].display_name | Artificial intelligence |
| concepts[5].id | https://openalex.org/C2984842247 |
| concepts[5].level | 3 |
| concepts[5].score | 0.5163496732711792 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q197536 |
| concepts[5].display_name | Deep neural networks |
| concepts[6].id | https://openalex.org/C62438384 |
| concepts[6].level | 2 |
| concepts[6].score | 0.5082142353057861 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q331350 |
| concepts[6].display_name | Simplex |
| concepts[7].id | https://openalex.org/C31170391 |
| concepts[7].level | 2 |
| concepts[7].score | 0.5062033534049988 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q188619 |
| concepts[7].display_name | Hierarchy |
| concepts[8].id | https://openalex.org/C165696696 |
| concepts[8].level | 2 |
| concepts[8].score | 0.46783486008644104 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q11287 |
| concepts[8].display_name | Exploit |
| concepts[9].id | https://openalex.org/C153180895 |
| concepts[9].level | 2 |
| concepts[9].score | 0.45844224095344543 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q7148389 |
| concepts[9].display_name | Pattern recognition (psychology) |
| concepts[10].id | https://openalex.org/C119857082 |
| concepts[10].level | 1 |
| concepts[10].score | 0.3428731858730316 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q2539 |
| concepts[10].display_name | Machine learning |
| concepts[11].id | https://openalex.org/C11413529 |
| concepts[11].level | 1 |
| concepts[11].score | 0.33068349957466125 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q8366 |
| concepts[11].display_name | Algorithm |
| concepts[12].id | https://openalex.org/C33923547 |
| concepts[12].level | 0 |
| concepts[12].score | 0.23680946230888367 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[12].display_name | Mathematics |
| concepts[13].id | https://openalex.org/C114614502 |
| concepts[13].level | 1 |
| concepts[13].score | 0.07901781797409058 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q76592 |
| concepts[13].display_name | Combinatorics |
| concepts[14].id | https://openalex.org/C162324750 |
| concepts[14].level | 0 |
| concepts[14].score | 0.0 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q8134 |
| concepts[14].display_name | Economics |
| concepts[15].id | https://openalex.org/C199539241 |
| concepts[15].level | 1 |
| concepts[15].score | 0.0 |
| concepts[15].wikidata | https://www.wikidata.org/wiki/Q7748 |
| concepts[15].display_name | Law |
| concepts[16].id | https://openalex.org/C34447519 |
| concepts[16].level | 1 |
| concepts[16].score | 0.0 |
| concepts[16].wikidata | https://www.wikidata.org/wiki/Q179522 |
| concepts[16].display_name | Market economy |
| concepts[17].id | https://openalex.org/C38652104 |
| concepts[17].level | 1 |
| concepts[17].score | 0.0 |
| concepts[17].wikidata | https://www.wikidata.org/wiki/Q3510521 |
| concepts[17].display_name | Computer security |
| concepts[18].id | https://openalex.org/C17744445 |
| concepts[18].level | 0 |
| concepts[18].score | 0.0 |
| concepts[18].wikidata | https://www.wikidata.org/wiki/Q36442 |
| concepts[18].display_name | Political science |
| keywords[0].id | https://openalex.org/keywords/artificial-neural-network |
| keywords[0].score | 0.6927560567855835 |
| keywords[0].display_name | Artificial neural network |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.652065098285675 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/mistake |
| keywords[2].score | 0.6012968420982361 |
| keywords[2].display_name | Mistake |
| keywords[3].id | https://openalex.org/keywords/classifier |
| keywords[3].score | 0.5849228501319885 |
| keywords[3].display_name | Classifier (UML) |
| keywords[4].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[4].score | 0.5524932742118835 |
| keywords[4].display_name | Artificial intelligence |
| keywords[5].id | https://openalex.org/keywords/deep-neural-networks |
| keywords[5].score | 0.5163496732711792 |
| keywords[5].display_name | Deep neural networks |
| keywords[6].id | https://openalex.org/keywords/simplex |
| keywords[6].score | 0.5082142353057861 |
| keywords[6].display_name | Simplex |
| keywords[7].id | https://openalex.org/keywords/hierarchy |
| keywords[7].score | 0.5062033534049988 |
| keywords[7].display_name | Hierarchy |
| keywords[8].id | https://openalex.org/keywords/exploit |
| keywords[8].score | 0.46783486008644104 |
| keywords[8].display_name | Exploit |
| keywords[9].id | https://openalex.org/keywords/pattern-recognition |
| keywords[9].score | 0.45844224095344543 |
| keywords[9].display_name | Pattern recognition (psychology) |
| keywords[10].id | https://openalex.org/keywords/machine-learning |
| keywords[10].score | 0.3428731858730316 |
| keywords[10].display_name | Machine learning |
| keywords[11].id | https://openalex.org/keywords/algorithm |
| keywords[11].score | 0.33068349957466125 |
| keywords[11].display_name | Algorithm |
| keywords[12].id | https://openalex.org/keywords/mathematics |
| keywords[12].score | 0.23680946230888367 |
| keywords[12].display_name | Mathematics |
| keywords[13].id | https://openalex.org/keywords/combinatorics |
| keywords[13].score | 0.07901781797409058 |
| keywords[13].display_name | Combinatorics |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2303.05689 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | cc-by-nc-sa |
| locations[0].pdf_url | https://arxiv.org/pdf/2303.05689 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | |
| locations[0].license_id | https://openalex.org/licenses/cc-by-nc-sa |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2303.05689 |
| locations[1].id | doi:10.48550/arxiv.2303.05689 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2303.05689 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5101430381 |
| authorships[0].author.orcid | https://orcid.org/0000-0003-3971-6949 |
| authorships[0].author.display_name | Liang Tong |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Liang, Tong |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5012185466 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-3364-027X |
| authorships[1].author.display_name | Jim Davis |
| authorships[1].author_position | last |
| authorships[1].raw_author_name | Davis, Jim |
| authorships[1].is_corresponding | False |
| has_content.pdf | True |
| has_content.grobid_xml | True |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2303.05689 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Inducing Neural Collapse to a Fixed Hierarchy-Aware Frame for Reducing Mistake Severity |
| has_fulltext | True |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T11652 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9918000102043152 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Imbalanced Data Classification Techniques |
| related_works | https://openalex.org/W1590719878, https://openalex.org/W4244271513, https://openalex.org/W4376622330, https://openalex.org/W2365974527, https://openalex.org/W4306382224, https://openalex.org/W4226517682, https://openalex.org/W3108263396, https://openalex.org/W2895872277, https://openalex.org/W17155033, https://openalex.org/W1561425952 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2303.05689 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | cc-by-nc-sa |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2303.05689 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | |
| best_oa_location.license_id | https://openalex.org/licenses/cc-by-nc-sa |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2303.05689 |
| primary_location.id | pmh:oai:arXiv.org:2303.05689 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | cc-by-nc-sa |
| primary_location.pdf_url | https://arxiv.org/pdf/2303.05689 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | |
| primary_location.license_id | https://openalex.org/licenses/cc-by-nc-sa |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2303.05689 |
| publication_date | 2023-03-10 |
| publication_year | 2023 |
| referenced_works_count | 0 |
| abstract_inverted_index.3 | 154 |
| abstract_inverted_index.a | 2, 17, 42, 63, 94, 99, 109 |
| abstract_inverted_index.In | 83 |
| abstract_inverted_index.We | 124 |
| abstract_inverted_index.an | 105 |
| abstract_inverted_index.at | 11 |
| abstract_inverted_index.by | 56 |
| abstract_inverted_index.is | 1 |
| abstract_inverted_index.of | 15, 33, 41, 74, 93, 104, 133, 145, 150 |
| abstract_inverted_index.on | 142 |
| abstract_inverted_index.to | 38, 52, 62, 66, 88, 98, 114, 121, 155 |
| abstract_inverted_index.we | 86 |
| abstract_inverted_index.12. | 156 |
| abstract_inverted_index.ETF | 65 |
| abstract_inverted_index.all | 34 |
| abstract_inverted_index.and | 5, 28, 70, 107 |
| abstract_inverted_index.fix | 89 |
| abstract_inverted_index.for | 21 |
| abstract_inverted_index.has | 50 |
| abstract_inverted_index.its | 139 |
| abstract_inverted_index.our | 127 |
| abstract_inverted_index.the | 12, 23, 29, 39, 58, 72, 75, 90, 122, 130, 134 |
| abstract_inverted_index.use | 108 |
| abstract_inverted_index.ETF, | 106 |
| abstract_inverted_index.deep | 18, 95 |
| abstract_inverted_index.flat | 35 |
| abstract_inverted_index.from | 153 |
| abstract_inverted_index.loss | 113 |
| abstract_inverted_index.that | 119, 126 |
| abstract_inverted_index.this | 54, 84 |
| abstract_inverted_index.when | 78 |
| abstract_inverted_index.with | 80, 148 |
| abstract_inverted_index.work | 49 |
| abstract_inverted_index.Code: | 157 |
| abstract_inverted_index.Frame | 46, 101 |
| abstract_inverted_index.There | 0 |
| abstract_inverted_index.Tight | 45 |
| abstract_inverted_index.data. | 82 |
| abstract_inverted_index.learn | 115 |
| abstract_inverted_index.means | 27 |
| abstract_inverted_index.phase | 14 |
| abstract_inverted_index.top-1 | 140 |
| abstract_inverted_index.tried | 51 |
| abstract_inverted_index.while | 137 |
| abstract_inverted_index.work, | 85 |
| abstract_inverted_index.(ETF). | 47 |
| abstract_inverted_index.Neural | 9 |
| abstract_inverted_index.Recent | 48 |
| abstract_inverted_index.called | 8 |
| abstract_inverted_index.cosine | 110 |
| abstract_inverted_index.fixing | 57 |
| abstract_inverted_index.induce | 67 |
| abstract_inverted_index.linear | 91 |
| abstract_inverted_index.neural | 19, 68, 96 |
| abstract_inverted_index.scales | 147 |
| abstract_inverted_index.classes | 36 |
| abstract_inverted_index.exploit | 53 |
| abstract_inverted_index.feature | 26 |
| abstract_inverted_index.heights | 151 |
| abstract_inverted_index.instead | 103 |
| abstract_inverted_index.learned | 76 |
| abstract_inverted_index.mistake | 131 |
| abstract_inverted_index.model's | 135 |
| abstract_inverted_index.network | 20, 97 |
| abstract_inverted_index.propose | 87 |
| abstract_inverted_index.ranging | 152 |
| abstract_inverted_index.reduces | 129 |
| abstract_inverted_index.related | 59 |
| abstract_inverted_index.several | 143 |
| abstract_inverted_index.simplex | 43 |
| abstract_inverted_index.varying | 146 |
| abstract_inverted_index.vectors | 32 |
| abstract_inverted_index.weights | 61 |
| abstract_inverted_index.HAFrame. | 123 |
| abstract_inverted_index.accuracy | 141 |
| abstract_inverted_index.approach | 128 |
| abstract_inverted_index.collapse | 37, 69, 120 |
| abstract_inverted_index.datasets | 144 |
| abstract_inverted_index.features | 77, 118 |
| abstract_inverted_index.maximize | 71 |
| abstract_inverted_index.recently | 3 |
| abstract_inverted_index.severity | 132 |
| abstract_inverted_index.terminal | 13 |
| abstract_inverted_index.training | 16, 79 |
| abstract_inverted_index.vertices | 40 |
| abstract_inverted_index.Collapse: | 10 |
| abstract_inverted_index.auxiliary | 112 |
| abstract_inverted_index.(HAFrame), | 102 |
| abstract_inverted_index.associated | 30 |
| abstract_inverted_index.classifier | 31, 60, 92 |
| abstract_inverted_index.discovered | 4 |
| abstract_inverted_index.imbalanced | 81 |
| abstract_inverted_index.intriguing | 6 |
| abstract_inverted_index.phenomenon | 7, 55 |
| abstract_inverted_index.separation | 73 |
| abstract_inverted_index.Equiangular | 44 |
| abstract_inverted_index.demonstrate | 125 |
| abstract_inverted_index.hierarchies | 149 |
| abstract_inverted_index.maintaining | 138 |
| abstract_inverted_index.penultimate | 25, 117 |
| abstract_inverted_index.predictions | 136 |
| abstract_inverted_index.pre-computed | 64 |
| abstract_inverted_index.within-class | 24 |
| abstract_inverted_index.Hierarchy-Aware | 100 |
| abstract_inverted_index.classification, | 22 |
| abstract_inverted_index.hierarchy-aware | 116 |
| abstract_inverted_index.similarity-based | 111 |
| abstract_inverted_index.https://github.com/ltong1130ztr/HAFrame | 158 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 2 |
| citation_normalized_percentile |