Finet: Using Fine-grained Batch Normalization to Train Light-weight Neural Networks Article Swipe
Chunjie Luo
,
Jianfeng Zhan
,
Lei Wang
,
Wanling Gao
·
YOU?
·
· 2020
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2005.06828
YOU?
·
· 2020
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2005.06828
To build light-weight network, we propose a new normalization, Fine-grained Batch Normalization (FBN). Different from Batch Normalization (BN), which normalizes the final summation of the weighted inputs, FBN normalizes the intermediate state of the summation. We propose a novel light-weight network based on FBN, called Finet. At training time, the convolutional layer with FBN can be seen as an inverted bottleneck mechanism. FBN can be fused into convolution at inference time. After fusion, Finet uses the standard convolution with equal channel width, thus makes the inference more efficient. On ImageNet classification dataset, Finet achieves the state-of-art performance (65.706% accuracy with 43M FLOPs, and 73.786% accuracy with 303M FLOPs), Moreover, experiments show that Finet is more efficient than other state-of-art light-weight networks.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2005.06828
- https://arxiv.org/pdf/2005.06828
- OA Status
- green
- Cited By
- 1
- References
- 36
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W3025005215
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W3025005215Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2005.06828Digital Object Identifier
- Title
-
Finet: Using Fine-grained Batch Normalization to Train Light-weight Neural NetworksWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2020Year of publication
- Publication date
-
2020-05-14Full publication date if available
- Authors
-
Chunjie Luo, Jianfeng Zhan, Lei Wang, Wanling GaoList of authors in order
- Landing page
-
https://arxiv.org/abs/2005.06828Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2005.06828Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2005.06828Direct OA link when available
- Concepts
-
Normalization (sociology), FLOPS, Inference, Convolutional neural network, Bottleneck, Computer science, Artificial intelligence, Convolution (computer science), Algorithm, Artificial neural network, Pattern recognition (psychology), Parallel computing, Sociology, Anthropology, Embedded systemTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
1Total citation count in OpenAlex
- Citations by year (recent)
-
2020: 1Per-year citation counts (last 5 years)
- References (count)
-
36Number of works referenced by this work
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W3025005215 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2005.06828 |
| ids.doi | https://doi.org/10.48550/arxiv.2005.06828 |
| ids.mag | 3025005215 |
| ids.openalex | https://openalex.org/W3025005215 |
| fwci | |
| type | preprint |
| title | Finet: Using Fine-grained Batch Normalization to Train Light-weight Neural Networks |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10036 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9998999834060669 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1707 |
| topics[0].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[0].display_name | Advanced Neural Network Applications |
| topics[1].id | https://openalex.org/T11307 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9990000128746033 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Domain Adaptation and Few-Shot Learning |
| topics[2].id | https://openalex.org/T10627 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9973999857902527 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1707 |
| topics[2].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[2].display_name | Advanced Image and Video Retrieval Techniques |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C136886441 |
| concepts[0].level | 2 |
| concepts[0].score | 0.8916943073272705 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q926129 |
| concepts[0].display_name | Normalization (sociology) |
| concepts[1].id | https://openalex.org/C3826847 |
| concepts[1].level | 2 |
| concepts[1].score | 0.8064562678337097 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q188768 |
| concepts[1].display_name | FLOPS |
| concepts[2].id | https://openalex.org/C2776214188 |
| concepts[2].level | 2 |
| concepts[2].score | 0.723701536655426 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q408386 |
| concepts[2].display_name | Inference |
| concepts[3].id | https://openalex.org/C81363708 |
| concepts[3].level | 2 |
| concepts[3].score | 0.6957817077636719 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q17084460 |
| concepts[3].display_name | Convolutional neural network |
| concepts[4].id | https://openalex.org/C2780513914 |
| concepts[4].level | 2 |
| concepts[4].score | 0.6547805070877075 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q18210350 |
| concepts[4].display_name | Bottleneck |
| concepts[5].id | https://openalex.org/C41008148 |
| concepts[5].level | 0 |
| concepts[5].score | 0.6451288461685181 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[5].display_name | Computer science |
| concepts[6].id | https://openalex.org/C154945302 |
| concepts[6].level | 1 |
| concepts[6].score | 0.5428310632705688 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[6].display_name | Artificial intelligence |
| concepts[7].id | https://openalex.org/C45347329 |
| concepts[7].level | 3 |
| concepts[7].score | 0.5363467931747437 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q5166604 |
| concepts[7].display_name | Convolution (computer science) |
| concepts[8].id | https://openalex.org/C11413529 |
| concepts[8].level | 1 |
| concepts[8].score | 0.4142169952392578 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q8366 |
| concepts[8].display_name | Algorithm |
| concepts[9].id | https://openalex.org/C50644808 |
| concepts[9].level | 2 |
| concepts[9].score | 0.370078444480896 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q192776 |
| concepts[9].display_name | Artificial neural network |
| concepts[10].id | https://openalex.org/C153180895 |
| concepts[10].level | 2 |
| concepts[10].score | 0.36129236221313477 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q7148389 |
| concepts[10].display_name | Pattern recognition (psychology) |
| concepts[11].id | https://openalex.org/C173608175 |
| concepts[11].level | 1 |
| concepts[11].score | 0.08507880568504333 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q232661 |
| concepts[11].display_name | Parallel computing |
| concepts[12].id | https://openalex.org/C144024400 |
| concepts[12].level | 0 |
| concepts[12].score | 0.0 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q21201 |
| concepts[12].display_name | Sociology |
| concepts[13].id | https://openalex.org/C19165224 |
| concepts[13].level | 1 |
| concepts[13].score | 0.0 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q23404 |
| concepts[13].display_name | Anthropology |
| concepts[14].id | https://openalex.org/C149635348 |
| concepts[14].level | 1 |
| concepts[14].score | 0.0 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q193040 |
| concepts[14].display_name | Embedded system |
| keywords[0].id | https://openalex.org/keywords/normalization |
| keywords[0].score | 0.8916943073272705 |
| keywords[0].display_name | Normalization (sociology) |
| keywords[1].id | https://openalex.org/keywords/flops |
| keywords[1].score | 0.8064562678337097 |
| keywords[1].display_name | FLOPS |
| keywords[2].id | https://openalex.org/keywords/inference |
| keywords[2].score | 0.723701536655426 |
| keywords[2].display_name | Inference |
| keywords[3].id | https://openalex.org/keywords/convolutional-neural-network |
| keywords[3].score | 0.6957817077636719 |
| keywords[3].display_name | Convolutional neural network |
| keywords[4].id | https://openalex.org/keywords/bottleneck |
| keywords[4].score | 0.6547805070877075 |
| keywords[4].display_name | Bottleneck |
| keywords[5].id | https://openalex.org/keywords/computer-science |
| keywords[5].score | 0.6451288461685181 |
| keywords[5].display_name | Computer science |
| keywords[6].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[6].score | 0.5428310632705688 |
| keywords[6].display_name | Artificial intelligence |
| keywords[7].id | https://openalex.org/keywords/convolution |
| keywords[7].score | 0.5363467931747437 |
| keywords[7].display_name | Convolution (computer science) |
| keywords[8].id | https://openalex.org/keywords/algorithm |
| keywords[8].score | 0.4142169952392578 |
| keywords[8].display_name | Algorithm |
| keywords[9].id | https://openalex.org/keywords/artificial-neural-network |
| keywords[9].score | 0.370078444480896 |
| keywords[9].display_name | Artificial neural network |
| keywords[10].id | https://openalex.org/keywords/pattern-recognition |
| keywords[10].score | 0.36129236221313477 |
| keywords[10].display_name | Pattern recognition (psychology) |
| keywords[11].id | https://openalex.org/keywords/parallel-computing |
| keywords[11].score | 0.08507880568504333 |
| keywords[11].display_name | Parallel computing |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2005.06828 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2005.06828 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2005.06828 |
| locations[1].id | doi:10.48550/arxiv.2005.06828 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2005.06828 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5033347503 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-6977-929X |
| authorships[0].author.display_name | Chunjie Luo |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Chunjie Luo |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5085963553 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-3728-6837 |
| authorships[1].author.display_name | Jianfeng Zhan |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Jianfeng Zhan |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5100435738 |
| authorships[2].author.orcid | https://orcid.org/0000-0001-5990-896X |
| authorships[2].author.display_name | Lei Wang |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Lei Wang |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5075110031 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-3911-9389 |
| authorships[3].author.display_name | Wanling Gao |
| authorships[3].author_position | last |
| authorships[3].raw_author_name | Wanling Gao |
| authorships[3].is_corresponding | False |
| has_content.pdf | True |
| has_content.grobid_xml | True |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2005.06828 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Finet: Using Fine-grained Batch Normalization to Train Light-weight Neural Networks |
| has_fulltext | True |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10036 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9998999834060669 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1707 |
| primary_topic.subfield.display_name | Computer Vision and Pattern Recognition |
| primary_topic.display_name | Advanced Neural Network Applications |
| related_works | https://openalex.org/W2595172197, https://openalex.org/W2084856301, https://openalex.org/W2127970246, https://openalex.org/W4382618745, https://openalex.org/W2885125400, https://openalex.org/W3005814217, https://openalex.org/W2995343971, https://openalex.org/W2982536526, https://openalex.org/W2964954556, https://openalex.org/W2992221004 |
| cited_by_count | 1 |
| counts_by_year[0].year | 2020 |
| counts_by_year[0].cited_by_count | 1 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2005.06828 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2005.06828 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2005.06828 |
| primary_location.id | pmh:oai:arXiv.org:2005.06828 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2005.06828 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2005.06828 |
| publication_date | 2020-05-14 |
| publication_year | 2020 |
| referenced_works | https://openalex.org/W2117539524, https://openalex.org/W2964068654, https://openalex.org/W2902302607, https://openalex.org/W2963373786, https://openalex.org/W2963685250, https://openalex.org/W2963454111, https://openalex.org/W2549139847, https://openalex.org/W2963836885, https://openalex.org/W2982083293, https://openalex.org/W2962839335, https://openalex.org/W2963446712, https://openalex.org/W2963125010, https://openalex.org/W2963163009, https://openalex.org/W2183341477, https://openalex.org/W1836465849, https://openalex.org/W2612445135, https://openalex.org/W2194775991, https://openalex.org/W2502312327, https://openalex.org/W2594833348, https://openalex.org/W2952258850, https://openalex.org/W2982012700, https://openalex.org/W2964081807, https://openalex.org/W2963304263, https://openalex.org/W2963918968, https://openalex.org/W2966623823, https://openalex.org/W2292729293, https://openalex.org/W2951720195, https://openalex.org/W2963755427, https://openalex.org/W3118608800, https://openalex.org/W2955425717, https://openalex.org/W2163605009, https://openalex.org/W2778955544, https://openalex.org/W2963399222, https://openalex.org/W2963993763, https://openalex.org/W2964169985, https://openalex.org/W2962835968 |
| referenced_works_count | 36 |
| abstract_inverted_index.a | 6, 37 |
| abstract_inverted_index.At | 46 |
| abstract_inverted_index.On | 88 |
| abstract_inverted_index.To | 0 |
| abstract_inverted_index.We | 35 |
| abstract_inverted_index.an | 58 |
| abstract_inverted_index.as | 57 |
| abstract_inverted_index.at | 68 |
| abstract_inverted_index.be | 55, 64 |
| abstract_inverted_index.is | 113 |
| abstract_inverted_index.of | 23, 32 |
| abstract_inverted_index.on | 42 |
| abstract_inverted_index.we | 4 |
| abstract_inverted_index.43M | 100 |
| abstract_inverted_index.FBN | 27, 53, 62 |
| abstract_inverted_index.and | 102 |
| abstract_inverted_index.can | 54, 63 |
| abstract_inverted_index.new | 7 |
| abstract_inverted_index.the | 20, 24, 29, 33, 49, 75, 84, 94 |
| abstract_inverted_index.303M | 106 |
| abstract_inverted_index.FBN, | 43 |
| abstract_inverted_index.from | 14 |
| abstract_inverted_index.into | 66 |
| abstract_inverted_index.more | 86, 114 |
| abstract_inverted_index.seen | 56 |
| abstract_inverted_index.show | 110 |
| abstract_inverted_index.than | 116 |
| abstract_inverted_index.that | 111 |
| abstract_inverted_index.thus | 82 |
| abstract_inverted_index.uses | 74 |
| abstract_inverted_index.with | 52, 78, 99, 105 |
| abstract_inverted_index.(BN), | 17 |
| abstract_inverted_index.After | 71 |
| abstract_inverted_index.Batch | 10, 15 |
| abstract_inverted_index.Finet | 73, 92, 112 |
| abstract_inverted_index.based | 41 |
| abstract_inverted_index.build | 1 |
| abstract_inverted_index.equal | 79 |
| abstract_inverted_index.final | 21 |
| abstract_inverted_index.fused | 65 |
| abstract_inverted_index.layer | 51 |
| abstract_inverted_index.makes | 83 |
| abstract_inverted_index.novel | 38 |
| abstract_inverted_index.other | 117 |
| abstract_inverted_index.state | 31 |
| abstract_inverted_index.time, | 48 |
| abstract_inverted_index.time. | 70 |
| abstract_inverted_index.which | 18 |
| abstract_inverted_index.(FBN). | 12 |
| abstract_inverted_index.FLOPs, | 101 |
| abstract_inverted_index.Finet. | 45 |
| abstract_inverted_index.called | 44 |
| abstract_inverted_index.width, | 81 |
| abstract_inverted_index.73.786% | 103 |
| abstract_inverted_index.FLOPs), | 107 |
| abstract_inverted_index.channel | 80 |
| abstract_inverted_index.fusion, | 72 |
| abstract_inverted_index.inputs, | 26 |
| abstract_inverted_index.network | 40 |
| abstract_inverted_index.propose | 5, 36 |
| abstract_inverted_index.(65.706% | 97 |
| abstract_inverted_index.ImageNet | 89 |
| abstract_inverted_index.accuracy | 98, 104 |
| abstract_inverted_index.achieves | 93 |
| abstract_inverted_index.dataset, | 91 |
| abstract_inverted_index.inverted | 59 |
| abstract_inverted_index.network, | 3 |
| abstract_inverted_index.standard | 76 |
| abstract_inverted_index.training | 47 |
| abstract_inverted_index.weighted | 25 |
| abstract_inverted_index.Different | 13 |
| abstract_inverted_index.Moreover, | 108 |
| abstract_inverted_index.efficient | 115 |
| abstract_inverted_index.inference | 69, 85 |
| abstract_inverted_index.networks. | 120 |
| abstract_inverted_index.summation | 22 |
| abstract_inverted_index.bottleneck | 60 |
| abstract_inverted_index.efficient. | 87 |
| abstract_inverted_index.mechanism. | 61 |
| abstract_inverted_index.normalizes | 19, 28 |
| abstract_inverted_index.summation. | 34 |
| abstract_inverted_index.convolution | 67, 77 |
| abstract_inverted_index.experiments | 109 |
| abstract_inverted_index.performance | 96 |
| abstract_inverted_index.Fine-grained | 9 |
| abstract_inverted_index.intermediate | 30 |
| abstract_inverted_index.light-weight | 2, 39, 119 |
| abstract_inverted_index.state-of-art | 95, 118 |
| abstract_inverted_index.Normalization | 11, 16 |
| abstract_inverted_index.convolutional | 50 |
| abstract_inverted_index.classification | 90 |
| abstract_inverted_index.normalization, | 8 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 4 |
| citation_normalized_percentile |