Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning Article Swipe
Huan Wang
,
Can Qin
,
Yue Bai
,
Yun Fu
·
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2301.05219
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2301.05219
The state of neural network pruning has been noticed to be unclear and even confusing for a while, largely due to "a lack of standardized benchmarks and metrics" [3]. To standardize benchmarks, first, we need to answer: what kind of comparison setup is considered fair? This basic yet crucial question has barely been clarified in the community, unfortunately. Meanwhile, we observe several papers have used (severely) sub-optimal hyper-parameters in pruning experiments, while the reason behind them is also elusive. These sub-optimal hyper-parameters further exacerbate the distorted benchmarks, rendering the state of neural network pruning even more obscure. Two mysteries in pruning represent such a confusing status: the performance-boosting effect of a larger finetuning learning rate, and the no-value argument of inheriting pretrained weights in filter pruning. In this work, we attempt to explain the confusing state of network pruning by demystifying the two mysteries. Specifically, (1) we first clarify the fairness principle in pruning experiments and summarize the widely-used comparison setups; (2) then we unveil the two pruning mysteries and point out the central role of network trainability, which has not been well recognized so far; (3) finally, we conclude the paper and give some concrete suggestions regarding how to calibrate the pruning benchmarks in the future. Code: https://github.com/mingsun-tse/why-the-state-of-pruning-so-confusing.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2301.05219
- https://arxiv.org/pdf/2301.05219
- OA Status
- green
- Cited By
- 14
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4316135746
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4316135746Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2301.05219Digital Object Identifier
- Title
-
Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network PruningWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2023Year of publication
- Publication date
-
2023-01-12Full publication date if available
- Authors
-
Huan Wang, Can Qin, Yue Bai, Yun FuList of authors in order
- Landing page
-
https://arxiv.org/abs/2301.05219Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2301.05219Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2301.05219Direct OA link when available
- Concepts
-
Pruning, Computer science, Artificial neural network, Machine learning, Artificial intelligence, Agronomy, BiologyTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
14Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 4, 2024: 5, 2023: 5Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4316135746 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2301.05219 |
| ids.doi | https://doi.org/10.48550/arxiv.2301.05219 |
| ids.openalex | https://openalex.org/W4316135746 |
| fwci | |
| type | preprint |
| title | Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T11612 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9936000108718872 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Stochastic Gradient Optimization Techniques |
| topics[1].id | https://openalex.org/T10036 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9815000295639038 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1707 |
| topics[1].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[1].display_name | Advanced Neural Network Applications |
| topics[2].id | https://openalex.org/T10320 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9672999978065491 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Neural Networks and Applications |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C108010975 |
| concepts[0].level | 2 |
| concepts[0].score | 0.8004745841026306 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q500094 |
| concepts[0].display_name | Pruning |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.7145835161209106 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C50644808 |
| concepts[2].level | 2 |
| concepts[2].score | 0.6202480792999268 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q192776 |
| concepts[2].display_name | Artificial neural network |
| concepts[3].id | https://openalex.org/C119857082 |
| concepts[3].level | 1 |
| concepts[3].score | 0.583986222743988 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q2539 |
| concepts[3].display_name | Machine learning |
| concepts[4].id | https://openalex.org/C154945302 |
| concepts[4].level | 1 |
| concepts[4].score | 0.5267823338508606 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[4].display_name | Artificial intelligence |
| concepts[5].id | https://openalex.org/C6557445 |
| concepts[5].level | 1 |
| concepts[5].score | 0.0 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q173113 |
| concepts[5].display_name | Agronomy |
| concepts[6].id | https://openalex.org/C86803240 |
| concepts[6].level | 0 |
| concepts[6].score | 0.0 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q420 |
| concepts[6].display_name | Biology |
| keywords[0].id | https://openalex.org/keywords/pruning |
| keywords[0].score | 0.8004745841026306 |
| keywords[0].display_name | Pruning |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.7145835161209106 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/artificial-neural-network |
| keywords[2].score | 0.6202480792999268 |
| keywords[2].display_name | Artificial neural network |
| keywords[3].id | https://openalex.org/keywords/machine-learning |
| keywords[3].score | 0.583986222743988 |
| keywords[3].display_name | Machine learning |
| keywords[4].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[4].score | 0.5267823338508606 |
| keywords[4].display_name | Artificial intelligence |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2301.05219 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2301.05219 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2301.05219 |
| locations[1].id | doi:10.48550/arxiv.2301.05219 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2301.05219 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5100332049 |
| authorships[0].author.orcid | https://orcid.org/0009-0004-0124-1081 |
| authorships[0].author.display_name | Huan Wang |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Wang, Huan |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5021042598 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-0712-5378 |
| authorships[1].author.display_name | Can Qin |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Qin, Can |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5102281154 |
| authorships[2].author.orcid | |
| authorships[2].author.display_name | Yue Bai |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Bai, Yue |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5005819096 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-5098-2853 |
| authorships[3].author.display_name | Yun Fu |
| authorships[3].author_position | last |
| authorships[3].raw_author_name | Fu, Yun |
| authorships[3].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2301.05219 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T11612 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9936000108718872 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Stochastic Gradient Optimization Techniques |
| related_works | https://openalex.org/W2961085424, https://openalex.org/W4306674287, https://openalex.org/W3046775127, https://openalex.org/W3107602296, https://openalex.org/W3170094116, https://openalex.org/W4386462264, https://openalex.org/W4364306694, https://openalex.org/W4312192474, https://openalex.org/W4283697347, https://openalex.org/W4210805261 |
| cited_by_count | 14 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 4 |
| counts_by_year[1].year | 2024 |
| counts_by_year[1].cited_by_count | 5 |
| counts_by_year[2].year | 2023 |
| counts_by_year[2].cited_by_count | 5 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2301.05219 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2301.05219 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2301.05219 |
| primary_location.id | pmh:oai:arXiv.org:2301.05219 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2301.05219 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2301.05219 |
| publication_date | 2023-01-12 |
| publication_year | 2023 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 16, 103, 110 |
| abstract_inverted_index."a | 21 |
| abstract_inverted_index.In | 126 |
| abstract_inverted_index.To | 29 |
| abstract_inverted_index.be | 10 |
| abstract_inverted_index.by | 139 |
| abstract_inverted_index.in | 54, 68, 99, 123, 152, 204 |
| abstract_inverted_index.is | 42, 76 |
| abstract_inverted_index.of | 2, 23, 39, 90, 109, 119, 136, 175 |
| abstract_inverted_index.so | 184 |
| abstract_inverted_index.to | 9, 20, 35, 131, 199 |
| abstract_inverted_index.we | 33, 59, 129, 146, 163, 188 |
| abstract_inverted_index.(1) | 145 |
| abstract_inverted_index.(2) | 161 |
| abstract_inverted_index.(3) | 186 |
| abstract_inverted_index.The | 0 |
| abstract_inverted_index.Two | 97 |
| abstract_inverted_index.and | 12, 26, 115, 155, 169, 192 |
| abstract_inverted_index.due | 19 |
| abstract_inverted_index.for | 15 |
| abstract_inverted_index.has | 6, 50, 179 |
| abstract_inverted_index.how | 198 |
| abstract_inverted_index.not | 180 |
| abstract_inverted_index.out | 171 |
| abstract_inverted_index.the | 55, 72, 84, 88, 106, 116, 133, 141, 149, 157, 165, 172, 190, 201, 205 |
| abstract_inverted_index.two | 142, 166 |
| abstract_inverted_index.yet | 47 |
| abstract_inverted_index.This | 45 |
| abstract_inverted_index.[3]. | 28 |
| abstract_inverted_index.also | 77 |
| abstract_inverted_index.been | 7, 52, 181 |
| abstract_inverted_index.even | 13, 94 |
| abstract_inverted_index.far; | 185 |
| abstract_inverted_index.give | 193 |
| abstract_inverted_index.have | 63 |
| abstract_inverted_index.kind | 38 |
| abstract_inverted_index.lack | 22 |
| abstract_inverted_index.more | 95 |
| abstract_inverted_index.need | 34 |
| abstract_inverted_index.role | 174 |
| abstract_inverted_index.some | 194 |
| abstract_inverted_index.such | 102 |
| abstract_inverted_index.them | 75 |
| abstract_inverted_index.then | 162 |
| abstract_inverted_index.this | 127 |
| abstract_inverted_index.used | 64 |
| abstract_inverted_index.well | 182 |
| abstract_inverted_index.what | 37 |
| abstract_inverted_index.Code: | 207 |
| abstract_inverted_index.These | 79 |
| abstract_inverted_index.basic | 46 |
| abstract_inverted_index.fair? | 44 |
| abstract_inverted_index.first | 147 |
| abstract_inverted_index.paper | 191 |
| abstract_inverted_index.point | 170 |
| abstract_inverted_index.rate, | 114 |
| abstract_inverted_index.setup | 41 |
| abstract_inverted_index.state | 1, 89, 135 |
| abstract_inverted_index.which | 178 |
| abstract_inverted_index.while | 71 |
| abstract_inverted_index.work, | 128 |
| abstract_inverted_index.barely | 51 |
| abstract_inverted_index.behind | 74 |
| abstract_inverted_index.effect | 108 |
| abstract_inverted_index.filter | 124 |
| abstract_inverted_index.first, | 32 |
| abstract_inverted_index.larger | 111 |
| abstract_inverted_index.neural | 3, 91 |
| abstract_inverted_index.papers | 62 |
| abstract_inverted_index.reason | 73 |
| abstract_inverted_index.unveil | 164 |
| abstract_inverted_index.while, | 17 |
| abstract_inverted_index.answer: | 36 |
| abstract_inverted_index.attempt | 130 |
| abstract_inverted_index.central | 173 |
| abstract_inverted_index.clarify | 148 |
| abstract_inverted_index.crucial | 48 |
| abstract_inverted_index.explain | 132 |
| abstract_inverted_index.further | 82 |
| abstract_inverted_index.future. | 206 |
| abstract_inverted_index.largely | 18 |
| abstract_inverted_index.network | 4, 92, 137, 176 |
| abstract_inverted_index.noticed | 8 |
| abstract_inverted_index.observe | 60 |
| abstract_inverted_index.pruning | 5, 69, 93, 100, 138, 153, 167, 202 |
| abstract_inverted_index.setups; | 160 |
| abstract_inverted_index.several | 61 |
| abstract_inverted_index.status: | 105 |
| abstract_inverted_index.unclear | 11 |
| abstract_inverted_index.weights | 122 |
| abstract_inverted_index.argument | 118 |
| abstract_inverted_index.conclude | 189 |
| abstract_inverted_index.concrete | 195 |
| abstract_inverted_index.elusive. | 78 |
| abstract_inverted_index.fairness | 150 |
| abstract_inverted_index.finally, | 187 |
| abstract_inverted_index.learning | 113 |
| abstract_inverted_index.metrics" | 27 |
| abstract_inverted_index.no-value | 117 |
| abstract_inverted_index.obscure. | 96 |
| abstract_inverted_index.pruning. | 125 |
| abstract_inverted_index.question | 49 |
| abstract_inverted_index.calibrate | 200 |
| abstract_inverted_index.clarified | 53 |
| abstract_inverted_index.confusing | 14, 104, 134 |
| abstract_inverted_index.distorted | 85 |
| abstract_inverted_index.mysteries | 98, 168 |
| abstract_inverted_index.principle | 151 |
| abstract_inverted_index.regarding | 197 |
| abstract_inverted_index.rendering | 87 |
| abstract_inverted_index.represent | 101 |
| abstract_inverted_index.summarize | 156 |
| abstract_inverted_index.(severely) | 65 |
| abstract_inverted_index.Meanwhile, | 58 |
| abstract_inverted_index.benchmarks | 25, 203 |
| abstract_inverted_index.community, | 56 |
| abstract_inverted_index.comparison | 40, 159 |
| abstract_inverted_index.considered | 43 |
| abstract_inverted_index.exacerbate | 83 |
| abstract_inverted_index.finetuning | 112 |
| abstract_inverted_index.inheriting | 120 |
| abstract_inverted_index.mysteries. | 143 |
| abstract_inverted_index.pretrained | 121 |
| abstract_inverted_index.recognized | 183 |
| abstract_inverted_index.benchmarks, | 31, 86 |
| abstract_inverted_index.experiments | 154 |
| abstract_inverted_index.standardize | 30 |
| abstract_inverted_index.sub-optimal | 66, 80 |
| abstract_inverted_index.suggestions | 196 |
| abstract_inverted_index.widely-used | 158 |
| abstract_inverted_index.demystifying | 140 |
| abstract_inverted_index.experiments, | 70 |
| abstract_inverted_index.standardized | 24 |
| abstract_inverted_index.Specifically, | 144 |
| abstract_inverted_index.trainability, | 177 |
| abstract_inverted_index.unfortunately. | 57 |
| abstract_inverted_index.hyper-parameters | 67, 81 |
| abstract_inverted_index.performance-boosting | 107 |
| abstract_inverted_index.https://github.com/mingsun-tse/why-the-state-of-pruning-so-confusing. | 208 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 4 |
| citation_normalized_percentile |