Non-Determinism in TensorFlow ResNets Article Swipe
Miguel Morin
,
Matthew Willetts
·
YOU?
·
· 2020
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2001.11396
YOU?
·
· 2020
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2001.11396
We show that the stochasticity in training ResNets for image classification on GPUs in TensorFlow is dominated by the non-determinism from GPUs, rather than by the initialisation of the weights and biases of the network or by the sequence of minibatches given. The standard deviation of test set accuracy is 0.02 with fixed seeds, compared to 0.027 with different seeds---nearly 74\% of the standard deviation of a ResNet model is non-deterministic. For test set loss the ratio of standard deviations is more than 80\%. These results call for more robust evaluation strategies of deep learning models, as a significant amount of the variation in results across runs can arise simply from GPU randomness.
Related Topics
Concepts
Randomness
Standard deviation
Determinism
Set (abstract data type)
Computer science
Test set
Image (mathematics)
Standard Model (mathematical formulation)
Algorithm
Sequence (biology)
Artificial intelligence
Mathematics
Statistics
Physics
Quantum mechanics
History
Programming language
Archaeology
Genetics
Biology
Gauge (firearms)
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2001.11396
- https://arxiv.org/pdf/2001.11396
- OA Status
- green
- Cited By
- 11
- References
- 3
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W3003546693
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W3003546693Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2001.11396Digital Object Identifier
- Title
-
Non-Determinism in TensorFlow ResNetsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2020Year of publication
- Publication date
-
2020-01-30Full publication date if available
- Authors
-
Miguel Morin, Matthew WillettsList of authors in order
- Landing page
-
https://arxiv.org/abs/2001.11396Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2001.11396Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2001.11396Direct OA link when available
- Concepts
-
Randomness, Standard deviation, Determinism, Set (abstract data type), Computer science, Test set, Image (mathematics), Standard Model (mathematical formulation), Algorithm, Sequence (biology), Artificial intelligence, Mathematics, Statistics, Physics, Quantum mechanics, History, Programming language, Archaeology, Genetics, Biology, Gauge (firearms)Top concepts (fields/topics) attached by OpenAlex
- Cited by
-
11Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 1, 2024: 2, 2023: 2, 2022: 3, 2021: 2Per-year citation counts (last 5 years)
- References (count)
-
3Number of works referenced by this work
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W3003546693 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2001.11396 |
| ids.doi | https://doi.org/10.48550/arxiv.2001.11396 |
| ids.mag | 3003546693 |
| ids.openalex | https://openalex.org/W3003546693 |
| fwci | |
| type | preprint |
| title | Non-Determinism in TensorFlow ResNets |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10036 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9998999834060669 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1707 |
| topics[0].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[0].display_name | Advanced Neural Network Applications |
| topics[1].id | https://openalex.org/T11689 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9994999766349792 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Adversarial Robustness in Machine Learning |
| topics[2].id | https://openalex.org/T11307 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9987000226974487 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Domain Adaptation and Few-Shot Learning |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C125112378 |
| concepts[0].level | 2 |
| concepts[0].score | 0.8354381322860718 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q176640 |
| concepts[0].display_name | Randomness |
| concepts[1].id | https://openalex.org/C22679943 |
| concepts[1].level | 2 |
| concepts[1].score | 0.7067040205001831 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q159375 |
| concepts[1].display_name | Standard deviation |
| concepts[2].id | https://openalex.org/C192183473 |
| concepts[2].level | 2 |
| concepts[2].score | 0.6683692932128906 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q131133 |
| concepts[2].display_name | Determinism |
| concepts[3].id | https://openalex.org/C177264268 |
| concepts[3].level | 2 |
| concepts[3].score | 0.6519408822059631 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q1514741 |
| concepts[3].display_name | Set (abstract data type) |
| concepts[4].id | https://openalex.org/C41008148 |
| concepts[4].level | 0 |
| concepts[4].score | 0.6324559450149536 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[4].display_name | Computer science |
| concepts[5].id | https://openalex.org/C169903167 |
| concepts[5].level | 2 |
| concepts[5].score | 0.5822446942329407 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q3985153 |
| concepts[5].display_name | Test set |
| concepts[6].id | https://openalex.org/C115961682 |
| concepts[6].level | 2 |
| concepts[6].score | 0.5541396737098694 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q860623 |
| concepts[6].display_name | Image (mathematics) |
| concepts[7].id | https://openalex.org/C101454708 |
| concepts[7].level | 3 |
| concepts[7].score | 0.511046290397644 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q17106019 |
| concepts[7].display_name | Standard Model (mathematical formulation) |
| concepts[8].id | https://openalex.org/C11413529 |
| concepts[8].level | 1 |
| concepts[8].score | 0.5093194246292114 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q8366 |
| concepts[8].display_name | Algorithm |
| concepts[9].id | https://openalex.org/C2778112365 |
| concepts[9].level | 2 |
| concepts[9].score | 0.48329856991767883 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q3511065 |
| concepts[9].display_name | Sequence (biology) |
| concepts[10].id | https://openalex.org/C154945302 |
| concepts[10].level | 1 |
| concepts[10].score | 0.4386480450630188 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[10].display_name | Artificial intelligence |
| concepts[11].id | https://openalex.org/C33923547 |
| concepts[11].level | 0 |
| concepts[11].score | 0.256845086812973 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[11].display_name | Mathematics |
| concepts[12].id | https://openalex.org/C105795698 |
| concepts[12].level | 1 |
| concepts[12].score | 0.23877376317977905 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q12483 |
| concepts[12].display_name | Statistics |
| concepts[13].id | https://openalex.org/C121332964 |
| concepts[13].level | 0 |
| concepts[13].score | 0.07585188746452332 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q413 |
| concepts[13].display_name | Physics |
| concepts[14].id | https://openalex.org/C62520636 |
| concepts[14].level | 1 |
| concepts[14].score | 0.0 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q944 |
| concepts[14].display_name | Quantum mechanics |
| concepts[15].id | https://openalex.org/C95457728 |
| concepts[15].level | 0 |
| concepts[15].score | 0.0 |
| concepts[15].wikidata | https://www.wikidata.org/wiki/Q309 |
| concepts[15].display_name | History |
| concepts[16].id | https://openalex.org/C199360897 |
| concepts[16].level | 1 |
| concepts[16].score | 0.0 |
| concepts[16].wikidata | https://www.wikidata.org/wiki/Q9143 |
| concepts[16].display_name | Programming language |
| concepts[17].id | https://openalex.org/C166957645 |
| concepts[17].level | 1 |
| concepts[17].score | 0.0 |
| concepts[17].wikidata | https://www.wikidata.org/wiki/Q23498 |
| concepts[17].display_name | Archaeology |
| concepts[18].id | https://openalex.org/C54355233 |
| concepts[18].level | 1 |
| concepts[18].score | 0.0 |
| concepts[18].wikidata | https://www.wikidata.org/wiki/Q7162 |
| concepts[18].display_name | Genetics |
| concepts[19].id | https://openalex.org/C86803240 |
| concepts[19].level | 0 |
| concepts[19].score | 0.0 |
| concepts[19].wikidata | https://www.wikidata.org/wiki/Q420 |
| concepts[19].display_name | Biology |
| concepts[20].id | https://openalex.org/C40976572 |
| concepts[20].level | 2 |
| concepts[20].score | 0.0 |
| concepts[20].wikidata | https://www.wikidata.org/wiki/Q2330873 |
| concepts[20].display_name | Gauge (firearms) |
| keywords[0].id | https://openalex.org/keywords/randomness |
| keywords[0].score | 0.8354381322860718 |
| keywords[0].display_name | Randomness |
| keywords[1].id | https://openalex.org/keywords/standard-deviation |
| keywords[1].score | 0.7067040205001831 |
| keywords[1].display_name | Standard deviation |
| keywords[2].id | https://openalex.org/keywords/determinism |
| keywords[2].score | 0.6683692932128906 |
| keywords[2].display_name | Determinism |
| keywords[3].id | https://openalex.org/keywords/set |
| keywords[3].score | 0.6519408822059631 |
| keywords[3].display_name | Set (abstract data type) |
| keywords[4].id | https://openalex.org/keywords/computer-science |
| keywords[4].score | 0.6324559450149536 |
| keywords[4].display_name | Computer science |
| keywords[5].id | https://openalex.org/keywords/test-set |
| keywords[5].score | 0.5822446942329407 |
| keywords[5].display_name | Test set |
| keywords[6].id | https://openalex.org/keywords/image |
| keywords[6].score | 0.5541396737098694 |
| keywords[6].display_name | Image (mathematics) |
| keywords[7].id | https://openalex.org/keywords/standard-model |
| keywords[7].score | 0.511046290397644 |
| keywords[7].display_name | Standard Model (mathematical formulation) |
| keywords[8].id | https://openalex.org/keywords/algorithm |
| keywords[8].score | 0.5093194246292114 |
| keywords[8].display_name | Algorithm |
| keywords[9].id | https://openalex.org/keywords/sequence |
| keywords[9].score | 0.48329856991767883 |
| keywords[9].display_name | Sequence (biology) |
| keywords[10].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[10].score | 0.4386480450630188 |
| keywords[10].display_name | Artificial intelligence |
| keywords[11].id | https://openalex.org/keywords/mathematics |
| keywords[11].score | 0.256845086812973 |
| keywords[11].display_name | Mathematics |
| keywords[12].id | https://openalex.org/keywords/statistics |
| keywords[12].score | 0.23877376317977905 |
| keywords[12].display_name | Statistics |
| keywords[13].id | https://openalex.org/keywords/physics |
| keywords[13].score | 0.07585188746452332 |
| keywords[13].display_name | Physics |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2001.11396 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2001.11396 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2001.11396 |
| locations[1].id | doi:10.48550/arxiv.2001.11396 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2001.11396 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5085546836 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Miguel Morin |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Miguel Morin |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5019067441 |
| authorships[1].author.orcid | https://orcid.org/0000-0001-6585-3950 |
| authorships[1].author.display_name | Matthew Willetts |
| authorships[1].author_position | last |
| authorships[1].raw_author_name | Matthew Willetts |
| authorships[1].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2001.11396 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Non-Determinism in TensorFlow ResNets |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10036 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9998999834060669 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1707 |
| primary_topic.subfield.display_name | Computer Vision and Pattern Recognition |
| primary_topic.display_name | Advanced Neural Network Applications |
| related_works | https://openalex.org/W3034924094, https://openalex.org/W3094954546, https://openalex.org/W2122524456, https://openalex.org/W1488708774, https://openalex.org/W1982811510, https://openalex.org/W4391100477, https://openalex.org/W2402189625, https://openalex.org/W4327779705, https://openalex.org/W4310560702, https://openalex.org/W2105836965 |
| cited_by_count | 11 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 1 |
| counts_by_year[1].year | 2024 |
| counts_by_year[1].cited_by_count | 2 |
| counts_by_year[2].year | 2023 |
| counts_by_year[2].cited_by_count | 2 |
| counts_by_year[3].year | 2022 |
| counts_by_year[3].cited_by_count | 3 |
| counts_by_year[4].year | 2021 |
| counts_by_year[4].cited_by_count | 2 |
| counts_by_year[5].year | 2020 |
| counts_by_year[5].cited_by_count | 1 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2001.11396 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2001.11396 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2001.11396 |
| primary_location.id | pmh:oai:arXiv.org:2001.11396 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2001.11396 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2001.11396 |
| publication_date | 2020-01-30 |
| publication_year | 2020 |
| referenced_works | https://openalex.org/W2271840356, https://openalex.org/W3118608800, https://openalex.org/W2194775991 |
| referenced_works_count | 3 |
| abstract_inverted_index.a | 66, 97 |
| abstract_inverted_index.We | 0 |
| abstract_inverted_index.as | 96 |
| abstract_inverted_index.by | 17, 24, 36 |
| abstract_inverted_index.in | 5, 13, 103 |
| abstract_inverted_index.is | 15, 49, 69, 80 |
| abstract_inverted_index.of | 27, 32, 39, 45, 61, 65, 77, 92, 100 |
| abstract_inverted_index.on | 11 |
| abstract_inverted_index.or | 35 |
| abstract_inverted_index.to | 55 |
| abstract_inverted_index.For | 71 |
| abstract_inverted_index.GPU | 111 |
| abstract_inverted_index.The | 42 |
| abstract_inverted_index.and | 30 |
| abstract_inverted_index.can | 107 |
| abstract_inverted_index.for | 8, 87 |
| abstract_inverted_index.set | 47, 73 |
| abstract_inverted_index.the | 3, 18, 25, 28, 33, 37, 62, 75, 101 |
| abstract_inverted_index.0.02 | 50 |
| abstract_inverted_index.74\% | 60 |
| abstract_inverted_index.GPUs | 12 |
| abstract_inverted_index.call | 86 |
| abstract_inverted_index.deep | 93 |
| abstract_inverted_index.from | 20, 110 |
| abstract_inverted_index.loss | 74 |
| abstract_inverted_index.more | 81, 88 |
| abstract_inverted_index.runs | 106 |
| abstract_inverted_index.show | 1 |
| abstract_inverted_index.test | 46, 72 |
| abstract_inverted_index.than | 23, 82 |
| abstract_inverted_index.that | 2 |
| abstract_inverted_index.with | 51, 57 |
| abstract_inverted_index.0.027 | 56 |
| abstract_inverted_index.80\%. | 83 |
| abstract_inverted_index.GPUs, | 21 |
| abstract_inverted_index.These | 84 |
| abstract_inverted_index.arise | 108 |
| abstract_inverted_index.fixed | 52 |
| abstract_inverted_index.image | 9 |
| abstract_inverted_index.model | 68 |
| abstract_inverted_index.ratio | 76 |
| abstract_inverted_index.ResNet | 67 |
| abstract_inverted_index.across | 105 |
| abstract_inverted_index.amount | 99 |
| abstract_inverted_index.biases | 31 |
| abstract_inverted_index.given. | 41 |
| abstract_inverted_index.rather | 22 |
| abstract_inverted_index.robust | 89 |
| abstract_inverted_index.seeds, | 53 |
| abstract_inverted_index.simply | 109 |
| abstract_inverted_index.ResNets | 7 |
| abstract_inverted_index.models, | 95 |
| abstract_inverted_index.network | 34 |
| abstract_inverted_index.results | 85, 104 |
| abstract_inverted_index.weights | 29 |
| abstract_inverted_index.accuracy | 48 |
| abstract_inverted_index.compared | 54 |
| abstract_inverted_index.learning | 94 |
| abstract_inverted_index.sequence | 38 |
| abstract_inverted_index.standard | 43, 63, 78 |
| abstract_inverted_index.training | 6 |
| abstract_inverted_index.deviation | 44, 64 |
| abstract_inverted_index.different | 58 |
| abstract_inverted_index.dominated | 16 |
| abstract_inverted_index.variation | 102 |
| abstract_inverted_index.TensorFlow | 14 |
| abstract_inverted_index.deviations | 79 |
| abstract_inverted_index.evaluation | 90 |
| abstract_inverted_index.strategies | 91 |
| abstract_inverted_index.minibatches | 40 |
| abstract_inverted_index.randomness. | 112 |
| abstract_inverted_index.significant | 98 |
| abstract_inverted_index.stochasticity | 4 |
| abstract_inverted_index.classification | 10 |
| abstract_inverted_index.initialisation | 26 |
| abstract_inverted_index.seeds---nearly | 59 |
| abstract_inverted_index.non-determinism | 19 |
| abstract_inverted_index.non-deterministic. | 70 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 2 |
| citation_normalized_percentile |