Accumulation Bit-Width Scaling For Ultra-Low Precision Training Of Deep Networks Article Swipe

PDF

Charbel Sakr , Naigang Wang , Chia‐Yu Chen , Jungwook Choi , Ankur Agrawal , Naresh R. Shanbhag , Kailash Gopalakrishnan ·

YOU? · · 2019 · Open Access · · DOI: https://doi.org/10.48550/arxiv.1901.06588

Efforts to reduce the numerical precision of computations in deep learning training have yielded systems that aggressively quantize weights and activations, yet employ wide high-precision accumulators for partial sums in inner-product operations to preserve the quality of convergence. The absence of any framework to analyze the precision requirements of partial sum accumulations results in conservative design choices. This imposes an upper-bound on the reduction of complexity of multiply-accumulate units. We present a statistical approach to analyze the impact of reduced accumulation precision on deep learning training. Observing that a bad choice for accumulation precision results in loss of information that manifests itself as a reduction in variance in an ensemble of partial sums, we derive a set of equations that relate this variance to the length of accumulation and the minimum number of bits needed for accumulation. We apply our analysis to three benchmark networks: CIFAR-10 ResNet 32, ImageNet ResNet 18 and ImageNet AlexNet. In each case, with accumulation precision set in accordance with our proposed equations, the networks successfully converge to the single precision floating-point baseline. We also show that reducing accumulation precision further degrades the quality of the trained network, proving that our equations produce tight bounds. Overall this analysis enables precise tailoring of computation hardware to the application, yielding area- and power-optimal systems.

Related Topics

Benchmark (Surveying)

Artificial Intelligence

Concepts

Benchmark (surveying) Computer science Computation Convergence (economics) Variance (accounting) Set (abstract data type) Reduction (mathematics) Algorithm Variance reduction Scaling Deep learning Computer engineering Artificial intelligence Monte Carlo method Mathematics Statistics Accounting Geography Economic growth Business Economics Programming language Geodesy Geometry

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/1901.06588
PDF: https://arxiv.org/pdf/1901.06588
OA Status: green
Related Works: 20
OpenAlex ID: https://openalex.org/W2912180158

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W2912180158

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.1901.06588

Digital Object Identifier
Title: Accumulation Bit-Width Scaling For Ultra-Low Precision Training Of Deep Networks

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2019

Year of publication
Publication date: 2019-01-19

Full publication date if available
Authors: Charbel Sakr, Naigang Wang, Chia‐Yu Chen, Jungwook Choi, Ankur Agrawal, Naresh R. Shanbhag, Kailash Gopalakrishnan

List of authors in order
Landing page: https://arxiv.org/abs/1901.06588

Publisher landing page
PDF URL: https://arxiv.org/pdf/1901.06588

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/1901.06588

Direct OA link when available
Concepts: Benchmark (surveying), Computer science, Computation, Convergence (economics), Variance (accounting), Set (abstract data type), Reduction (mathematics), Algorithm, Variance reduction, Scaling, Deep learning, Computer engineering, Artificial intelligence, Monte Carlo method, Mathematics, Statistics, Accounting, Geography, Economic growth, Business, Economics, Programming language, Geodesy, Geometry

Top concepts (fields/topics) attached by OpenAlex
Cited by: 0

Total citation count in OpenAlex
Related works (count): 20

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W2912180158
doi	https://doi.org/10.48550/arxiv.1901.06588
ids.doi	https://doi.org/10.48550/arxiv.1901.06588
ids.mag	2912180158
ids.openalex	https://openalex.org/W2912180158
fwci
type	preprint
title	Accumulation Bit-Width Scaling For Ultra-Low Precision Training Of Deep Networks
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T10036
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9961000084877014
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1707
topics[0].subfield.display_name	Computer Vision and Pattern Recognition
topics[0].display_name	Advanced Neural Network Applications
topics[1].id	https://openalex.org/T12072
topics[1].field.id	https://openalex.org/fields/17
topics[1].field.display_name	Computer Science
topics[1].score	0.9869999885559082
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/1702
topics[1].subfield.display_name	Artificial Intelligence
topics[1].display_name	Machine Learning and Algorithms
topics[2].id	https://openalex.org/T10052
topics[2].field.id	https://openalex.org/fields/17
topics[2].field.display_name	Computer Science
topics[2].score	0.9764000177383423
topics[2].domain.id	https://openalex.org/domains/3
topics[2].domain.display_name	Physical Sciences
topics[2].subfield.id	https://openalex.org/subfields/1707
topics[2].subfield.display_name	Computer Vision and Pattern Recognition
topics[2].display_name	Medical Image Segmentation Techniques
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C185798385
concepts[0].level	2
concepts[0].score	0.7222636342048645
concepts[0].wikidata	https://www.wikidata.org/wiki/Q1161707
concepts[0].display_name	Benchmark (surveying)
concepts[1].id	https://openalex.org/C41008148
concepts[1].level	0
concepts[1].score	0.7190945148468018
concepts[1].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[1].display_name	Computer science
concepts[2].id	https://openalex.org/C45374587
concepts[2].level	2
concepts[2].score	0.5788069367408752
concepts[2].wikidata	https://www.wikidata.org/wiki/Q12525525
concepts[2].display_name	Computation
concepts[3].id	https://openalex.org/C2777303404
concepts[3].level	2
concepts[3].score	0.5681020617485046
concepts[3].wikidata	https://www.wikidata.org/wiki/Q759757
concepts[3].display_name	Convergence (economics)
concepts[4].id	https://openalex.org/C196083921
concepts[4].level	2
concepts[4].score	0.5552449226379395
concepts[4].wikidata	https://www.wikidata.org/wiki/Q7915758
concepts[4].display_name	Variance (accounting)
concepts[5].id	https://openalex.org/C177264268
concepts[5].level	2
concepts[5].score	0.5531225204467773
concepts[5].wikidata	https://www.wikidata.org/wiki/Q1514741
concepts[5].display_name	Set (abstract data type)
concepts[6].id	https://openalex.org/C111335779
concepts[6].level	2
concepts[6].score	0.548845648765564
concepts[6].wikidata	https://www.wikidata.org/wiki/Q3454686
concepts[6].display_name	Reduction (mathematics)
concepts[7].id	https://openalex.org/C11413529
concepts[7].level	1
concepts[7].score	0.5315344929695129
concepts[7].wikidata	https://www.wikidata.org/wiki/Q8366
concepts[7].display_name	Algorithm
concepts[8].id	https://openalex.org/C62644790
concepts[8].level	3
concepts[8].score	0.4893800616264343
concepts[8].wikidata	https://www.wikidata.org/wiki/Q3454689
concepts[8].display_name	Variance reduction
concepts[9].id	https://openalex.org/C99844830
concepts[9].level	2
concepts[9].score	0.4552118480205536
concepts[9].wikidata	https://www.wikidata.org/wiki/Q102441924
concepts[9].display_name	Scaling
concepts[10].id	https://openalex.org/C108583219
concepts[10].level	2
concepts[10].score	0.42108145356178284
concepts[10].wikidata	https://www.wikidata.org/wiki/Q197536
concepts[10].display_name	Deep learning
concepts[11].id	https://openalex.org/C113775141
concepts[11].level	1
concepts[11].score	0.33620208501815796
concepts[11].wikidata	https://www.wikidata.org/wiki/Q428691
concepts[11].display_name	Computer engineering
concepts[12].id	https://openalex.org/C154945302
concepts[12].level	1
concepts[12].score	0.3361746072769165
concepts[12].wikidata	https://www.wikidata.org/wiki/Q11660
concepts[12].display_name	Artificial intelligence
concepts[13].id	https://openalex.org/C19499675
concepts[13].level	2
concepts[13].score	0.21661034226417542
concepts[13].wikidata	https://www.wikidata.org/wiki/Q232207
concepts[13].display_name	Monte Carlo method
concepts[14].id	https://openalex.org/C33923547
concepts[14].level	0
concepts[14].score	0.20981448888778687
concepts[14].wikidata	https://www.wikidata.org/wiki/Q395
concepts[14].display_name	Mathematics
concepts[15].id	https://openalex.org/C105795698
concepts[15].level	1
concepts[15].score	0.1312398612499237
concepts[15].wikidata	https://www.wikidata.org/wiki/Q12483
concepts[15].display_name	Statistics
concepts[16].id	https://openalex.org/C121955636
concepts[16].level	1
concepts[16].score	0.0
concepts[16].wikidata	https://www.wikidata.org/wiki/Q4116214
concepts[16].display_name	Accounting
concepts[17].id	https://openalex.org/C205649164
concepts[17].level	0
concepts[17].score	0.0
concepts[17].wikidata	https://www.wikidata.org/wiki/Q1071
concepts[17].display_name	Geography
concepts[18].id	https://openalex.org/C50522688
concepts[18].level	1
concepts[18].score	0.0
concepts[18].wikidata	https://www.wikidata.org/wiki/Q189833
concepts[18].display_name	Economic growth
concepts[19].id	https://openalex.org/C144133560
concepts[19].level	0
concepts[19].score	0.0
concepts[19].wikidata	https://www.wikidata.org/wiki/Q4830453
concepts[19].display_name	Business
concepts[20].id	https://openalex.org/C162324750
concepts[20].level	0
concepts[20].score	0.0
concepts[20].wikidata	https://www.wikidata.org/wiki/Q8134
concepts[20].display_name	Economics
concepts[21].id	https://openalex.org/C199360897
concepts[21].level	1
concepts[21].score	0.0
concepts[21].wikidata	https://www.wikidata.org/wiki/Q9143
concepts[21].display_name	Programming language
concepts[22].id	https://openalex.org/C13280743
concepts[22].level	1
concepts[22].score	0.0
concepts[22].wikidata	https://www.wikidata.org/wiki/Q131089
concepts[22].display_name	Geodesy
concepts[23].id	https://openalex.org/C2524010
concepts[23].level	1
concepts[23].score	0.0
concepts[23].wikidata	https://www.wikidata.org/wiki/Q8087
concepts[23].display_name	Geometry
keywords[0].id	https://openalex.org/keywords/benchmark
keywords[0].score	0.7222636342048645
keywords[0].display_name	Benchmark (surveying)
keywords[1].id	https://openalex.org/keywords/computer-science
keywords[1].score	0.7190945148468018
keywords[1].display_name	Computer science
keywords[2].id	https://openalex.org/keywords/computation
keywords[2].score	0.5788069367408752
keywords[2].display_name	Computation
keywords[3].id	https://openalex.org/keywords/convergence
keywords[3].score	0.5681020617485046
keywords[3].display_name	Convergence (economics)
keywords[4].id	https://openalex.org/keywords/variance
keywords[4].score	0.5552449226379395
keywords[4].display_name	Variance (accounting)
keywords[5].id	https://openalex.org/keywords/set
keywords[5].score	0.5531225204467773
keywords[5].display_name	Set (abstract data type)
keywords[6].id	https://openalex.org/keywords/reduction
keywords[6].score	0.548845648765564
keywords[6].display_name	Reduction (mathematics)
keywords[7].id	https://openalex.org/keywords/algorithm
keywords[7].score	0.5315344929695129
keywords[7].display_name	Algorithm
keywords[8].id	https://openalex.org/keywords/variance-reduction
keywords[8].score	0.4893800616264343
keywords[8].display_name	Variance reduction
keywords[9].id	https://openalex.org/keywords/scaling
keywords[9].score	0.4552118480205536
keywords[9].display_name	Scaling
keywords[10].id	https://openalex.org/keywords/deep-learning
keywords[10].score	0.42108145356178284
keywords[10].display_name	Deep learning
keywords[11].id	https://openalex.org/keywords/computer-engineering
keywords[11].score	0.33620208501815796
keywords[11].display_name	Computer engineering
keywords[12].id	https://openalex.org/keywords/artificial-intelligence
keywords[12].score	0.3361746072769165
keywords[12].display_name	Artificial intelligence
keywords[13].id	https://openalex.org/keywords/monte-carlo-method
keywords[13].score	0.21661034226417542
keywords[13].display_name	Monte Carlo method
keywords[14].id	https://openalex.org/keywords/mathematics
keywords[14].score	0.20981448888778687
keywords[14].display_name	Mathematics
keywords[15].id	https://openalex.org/keywords/statistics
keywords[15].score	0.1312398612499237
keywords[15].display_name	Statistics
language	en
locations[0].id	pmh:oai:arXiv.org:1901.06588
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/1901.06588
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/1901.06588
locations[1].id	mag:2912180158
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license
locations[1].pdf_url
locations[1].version	submittedVersion
locations[1].raw_type
locations[1].license_id
locations[1].is_accepted	False
locations[1].is_published	False
locations[1].raw_source_name	arXiv (Cornell University)
locations[1].landing_page_url	https://arxiv.org/pdf/1901.06588
locations[2].id	doi:10.48550/arxiv.1901.06588
locations[2].is_oa	True
locations[2].source.id	https://openalex.org/S4306400194
locations[2].source.issn
locations[2].source.type	repository
locations[2].source.is_oa	True
locations[2].source.issn_l
locations[2].source.is_core	False
locations[2].source.is_in_doaj	False
locations[2].source.display_name	arXiv (Cornell University)
locations[2].source.host_organization	https://openalex.org/I205783295
locations[2].source.host_organization_name	Cornell University
locations[2].source.host_organization_lineage	https://openalex.org/I205783295
locations[2].license
locations[2].pdf_url
locations[2].version
locations[2].raw_type	article
locations[2].license_id
locations[2].is_accepted	False
locations[2].is_published
locations[2].raw_source_name
locations[2].landing_page_url	https://doi.org/10.48550/arxiv.1901.06588
locations[3].id	mag:2963131426
locations[3].is_oa	True
locations[3].source.id	https://openalex.org/S4306400194
locations[3].source.issn
locations[3].source.type	repository
locations[3].source.is_oa	True
locations[3].source.issn_l
locations[3].source.is_core	False
locations[3].source.is_in_doaj	False
locations[3].source.display_name	arXiv (Cornell University)
locations[3].source.host_organization	https://openalex.org/I205783295
locations[3].source.host_organization_name	Cornell University
locations[3].source.host_organization_lineage	https://openalex.org/I205783295
locations[3].license
locations[3].pdf_url
locations[3].version
locations[3].raw_type
locations[3].license_id
locations[3].is_accepted	False
locations[3].is_published
locations[3].raw_source_name	arXiv (Cornell University)
locations[3].landing_page_url	https://arxiv.org/pdf/1901.06588.pdf
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5047303294
authorships[0].author.orcid	https://orcid.org/0000-0001-5641-0541
authorships[0].author.display_name	Charbel Sakr
authorships[0].countries	US
authorships[0].affiliations[0].raw_affiliation_string	IBM Research
authorships[0].affiliations[1].institution_ids	https://openalex.org/I157725225
authorships[0].affiliations[1].raw_affiliation_string	University of Illinois at Urbana-Champaign
authorships[0].institutions[0].id	https://openalex.org/I157725225
authorships[0].institutions[0].ror	https://ror.org/047426m28
authorships[0].institutions[0].type	education
authorships[0].institutions[0].lineage	https://openalex.org/I157725225
authorships[0].institutions[0].country_code	US
authorships[0].institutions[0].display_name	University of Illinois Urbana-Champaign
authorships[0].author_position	first
authorships[0].raw_author_name	Charbel Sakr
authorships[0].is_corresponding	False
authorships[0].raw_affiliation_strings	IBM Research, University of Illinois at Urbana-Champaign
authorships[1].author.id	https://openalex.org/A5082043392
authorships[1].author.orcid	https://orcid.org/0000-0001-7664-0061
authorships[1].author.display_name	Naigang Wang
authorships[1].affiliations[0].raw_affiliation_string	IBM
authorships[1].author_position	middle
authorships[1].raw_author_name	Naigang Wang
authorships[1].is_corresponding	False
authorships[1].raw_affiliation_strings	IBM
authorships[2].author.id	https://openalex.org/A5101406569
authorships[2].author.orcid	https://orcid.org/0000-0001-5542-7149
authorships[2].author.display_name	Chia‐Yu Chen
authorships[2].affiliations[0].raw_affiliation_string	IBM
authorships[2].author_position	middle
authorships[2].raw_author_name	Chia-Yu Chen
authorships[2].is_corresponding	False
authorships[2].raw_affiliation_strings	IBM
authorships[3].author.id	https://openalex.org/A5078440061
authorships[3].author.orcid	https://orcid.org/0000-0002-3075-8694
authorships[3].author.display_name	Jungwook Choi
authorships[3].affiliations[0].raw_affiliation_string	DEPARTMENT OF ELECTRONIC ENGINEERING
authorships[3].author_position	middle
authorships[3].raw_author_name	Jungwook Choi
authorships[3].is_corresponding	False
authorships[3].raw_affiliation_strings	DEPARTMENT OF ELECTRONIC ENGINEERING
authorships[4].author.id	https://openalex.org/A5103063911
authorships[4].author.orcid	https://orcid.org/0000-0002-4389-5911
authorships[4].author.display_name	Ankur Agrawal
authorships[4].affiliations[0].raw_affiliation_string	IBM
authorships[4].author_position	middle
authorships[4].raw_author_name	Ankur Agrawal
authorships[4].is_corresponding	False
authorships[4].raw_affiliation_strings	IBM
authorships[5].author.id	https://openalex.org/A5057014407
authorships[5].author.orcid	https://orcid.org/0000-0002-4323-9164
authorships[5].author.display_name	Naresh R. Shanbhag
authorships[5].countries	US
authorships[5].affiliations[0].institution_ids	https://openalex.org/I157725225
authorships[5].affiliations[0].raw_affiliation_string	***University of Illinois at Urbana-Champaign
authorships[5].institutions[0].id	https://openalex.org/I157725225
authorships[5].institutions[0].ror	https://ror.org/047426m28
authorships[5].institutions[0].type	education
authorships[5].institutions[0].lineage	https://openalex.org/I157725225
authorships[5].institutions[0].country_code	US
authorships[5].institutions[0].display_name	University of Illinois Urbana-Champaign
authorships[5].author_position	middle
authorships[5].raw_author_name	Naresh Shanbhag
authorships[5].is_corresponding	False
authorships[5].raw_affiliation_strings	***University of Illinois at Urbana-Champaign
authorships[6].author.id	https://openalex.org/A5101429456
authorships[6].author.orcid	https://orcid.org/0000-0002-8952-0875
authorships[6].author.display_name	Kailash Gopalakrishnan
authorships[6].affiliations[0].raw_affiliation_string	IBM
authorships[6].author_position	last
authorships[6].raw_author_name	Kailash Gopalakrishnan
authorships[6].is_corresponding	False
authorships[6].raw_affiliation_strings	IBM
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/1901.06588
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Accumulation Bit-Width Scaling For Ultra-Low Precision Training Of Deep Networks
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T10036
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9961000084877014
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1707
primary_topic.subfield.display_name	Computer Vision and Pattern Recognition
primary_topic.display_name	Advanced Neural Network Applications
related_works	https://openalex.org/W2963131426, https://openalex.org/W2890616119, https://openalex.org/W2946955515, https://openalex.org/W2763421725, https://openalex.org/W2890770683, https://openalex.org/W2584314014, https://openalex.org/W3120250747, https://openalex.org/W2944508492, https://openalex.org/W3133264013, https://openalex.org/W2994782216, https://openalex.org/W2940542208, https://openalex.org/W3129181244, https://openalex.org/W3040522390, https://openalex.org/W3008507360, https://openalex.org/W2964472257, https://openalex.org/W3173842498, https://openalex.org/W3123666279, https://openalex.org/W2799243248, https://openalex.org/W2726743825, https://openalex.org/W2737685714
cited_by_count	0
locations_count	4
best_oa_location.id	pmh:oai:arXiv.org:1901.06588
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/1901.06588
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/1901.06588
primary_location.id	pmh:oai:arXiv.org:1901.06588
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/1901.06588
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/1901.06588
publication_date	2019-01-19
publication_year	2019
referenced_works_count	0
abstract_inverted_index.a	71, 88, 103, 115
abstract_inverted_index.18	150
abstract_inverted_index.In	154
abstract_inverted_index.We	69, 137, 177
abstract_inverted_index.an	59, 108
abstract_inverted_index.as	102
abstract_inverted_index.in	8, 29, 53, 95, 105, 107, 161
abstract_inverted_index.of	6, 36, 40, 48, 64, 66, 78, 97, 110, 117, 126, 132, 188, 205
abstract_inverted_index.on	61, 82
abstract_inverted_index.to	1, 32, 43, 74, 123, 141, 171, 208
abstract_inverted_index.we	113
abstract_inverted_index.32,	147
abstract_inverted_index.The	38
abstract_inverted_index.and	19, 128, 151, 213
abstract_inverted_index.any	41
abstract_inverted_index.bad	89
abstract_inverted_index.for	26, 91, 135
abstract_inverted_index.our	139, 164, 194
abstract_inverted_index.set	116, 160
abstract_inverted_index.sum	50
abstract_inverted_index.the	3, 34, 45, 62, 76, 124, 129, 167, 172, 186, 189, 209
abstract_inverted_index.yet	21
abstract_inverted_index.This	57
abstract_inverted_index.also	178
abstract_inverted_index.bits	133
abstract_inverted_index.deep	9, 83
abstract_inverted_index.each	155
abstract_inverted_index.have	12
abstract_inverted_index.loss	96
abstract_inverted_index.show	179
abstract_inverted_index.sums	28
abstract_inverted_index.that	15, 87, 99, 119, 180, 193
abstract_inverted_index.this	121, 200
abstract_inverted_index.wide	23
abstract_inverted_index.with	157, 163
abstract_inverted_index.apply	138
abstract_inverted_index.area-	212
abstract_inverted_index.case,	156
abstract_inverted_index.sums,	112
abstract_inverted_index.three	142
abstract_inverted_index.tight	197
abstract_inverted_index.ResNet	146, 149
abstract_inverted_index.choice	90
abstract_inverted_index.derive	114
abstract_inverted_index.design	55
abstract_inverted_index.employ	22
abstract_inverted_index.impact	77
abstract_inverted_index.itself	101
abstract_inverted_index.length	125
abstract_inverted_index.needed	134
abstract_inverted_index.number	131
abstract_inverted_index.reduce	2
abstract_inverted_index.relate	120
abstract_inverted_index.single	173
abstract_inverted_index.units.	68
abstract_inverted_index.Efforts	0
abstract_inverted_index.Overall	199
abstract_inverted_index.absence	39
abstract_inverted_index.analyze	44, 75
abstract_inverted_index.bounds.	198
abstract_inverted_index.enables	202
abstract_inverted_index.further	184
abstract_inverted_index.imposes	58
abstract_inverted_index.minimum	130
abstract_inverted_index.partial	27, 49, 111
abstract_inverted_index.precise	203
abstract_inverted_index.present	70
abstract_inverted_index.produce	196
abstract_inverted_index.proving	192
abstract_inverted_index.quality	35, 187
abstract_inverted_index.reduced	79
abstract_inverted_index.results	52, 94
abstract_inverted_index.systems	14
abstract_inverted_index.trained	190
abstract_inverted_index.weights	18
abstract_inverted_index.yielded	13
abstract_inverted_index.AlexNet.	153
abstract_inverted_index.CIFAR-10	145
abstract_inverted_index.ImageNet	148, 152
abstract_inverted_index.analysis	140, 201
abstract_inverted_index.approach	73
abstract_inverted_index.choices.	56
abstract_inverted_index.converge	170
abstract_inverted_index.degrades	185
abstract_inverted_index.ensemble	109
abstract_inverted_index.hardware	207
abstract_inverted_index.learning	10, 84
abstract_inverted_index.network,	191
abstract_inverted_index.networks	168
abstract_inverted_index.preserve	33
abstract_inverted_index.proposed	165
abstract_inverted_index.quantize	17
abstract_inverted_index.reducing	181
abstract_inverted_index.systems.	215
abstract_inverted_index.training	11
abstract_inverted_index.variance	106, 122
abstract_inverted_index.yielding	211
abstract_inverted_index.Observing	86
abstract_inverted_index.baseline.	176
abstract_inverted_index.benchmark	143
abstract_inverted_index.equations	118, 195
abstract_inverted_index.framework	42
abstract_inverted_index.manifests	100
abstract_inverted_index.networks:	144
abstract_inverted_index.numerical	4
abstract_inverted_index.precision	5, 46, 81, 93, 159, 174, 183
abstract_inverted_index.reduction	63, 104
abstract_inverted_index.tailoring	204
abstract_inverted_index.training.	85
abstract_inverted_index.accordance	162
abstract_inverted_index.complexity	65
abstract_inverted_index.equations,	166
abstract_inverted_index.operations	31
abstract_inverted_index.computation	206
abstract_inverted_index.information	98
abstract_inverted_index.statistical	72
abstract_inverted_index.upper-bound	60
abstract_inverted_index.accumulation	80, 92, 127, 158, 182
abstract_inverted_index.accumulators	25
abstract_inverted_index.activations,	20
abstract_inverted_index.aggressively	16
abstract_inverted_index.application,	210
abstract_inverted_index.computations	7
abstract_inverted_index.conservative	54
abstract_inverted_index.convergence.	37
abstract_inverted_index.requirements	47
abstract_inverted_index.successfully	169
abstract_inverted_index.accumulation.	136
abstract_inverted_index.accumulations	51
abstract_inverted_index.inner-product	30
abstract_inverted_index.power-optimal	214
abstract_inverted_index.floating-point	175
abstract_inverted_index.high-precision	24
abstract_inverted_index.multiply-accumulate	67
cited_by_percentile_year
countries_distinct_count	1
institutions_distinct_count	7
citation_normalized_percentile