Anytime MiniBatch: Exploiting Stragglers in Online Distributed\n Optimization Article Swipe
Nuwan S. Ferdinand
,
Haider Al-Lawati
,
Stark C. Draper
,
Matthew Nokleby
·
YOU?
·
· 2020
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2006.05752
YOU?
·
· 2020
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2006.05752
Distributed optimization is vital in solving large-scale machine learning\nproblems. A widely-shared feature of distributed optimization techniques is the\nrequirement that all nodes complete their assigned tasks in each computational\nepoch before the system can proceed to the next epoch. In such settings, slow\nnodes, called stragglers, can greatly slow progress. To mitigate the impact of\nstragglers, we propose an online distributed optimization method called Anytime\nMinibatch. In this approach, all nodes are given a fixed time to compute the\ngradients of as many data samples as possible. The result is a variable\nper-node minibatch size. Workers then get a fixed communication time to average\ntheir minibatch gradients via several rounds of consensus, which are then used\nto update primal variables via dual averaging. Anytime Minibatch prevents\nstragglers from holding up the system without wasting the work that stragglers\ncan complete. We present a convergence analysis and analyze the wall time\nperformance. Our numerical results show that our approach is up to 1.5 times\nfaster in Amazon EC2 and it is up to five times faster when there is greater\nvariability in compute node performance.\n
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2006.05752
- https://arxiv.org/pdf/2006.05752
- OA Status
- green
- Cited By
- 26
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4287759323
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4287759323Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2006.05752Digital Object Identifier
- Title
-
Anytime MiniBatch: Exploiting Stragglers in Online Distributed\n OptimizationWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2020Year of publication
- Publication date
-
2020-06-10Full publication date if available
- Authors
-
Nuwan S. Ferdinand, Haider Al-Lawati, Stark C. Draper, Matthew NoklebyList of authors in order
- Landing page
-
https://arxiv.org/abs/2006.05752Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2006.05752Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2006.05752Direct OA link when available
- Concepts
-
Computer science, Node (physics), Epoch (astronomy), Convergence (economics), Optimization problem, Distributed computing, Algorithm, Stars, Structural engineering, Economics, Engineering, Economic growth, Computer visionTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
26Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 1, 2024: 3, 2023: 5, 2022: 5, 2021: 6Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4287759323 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2006.05752 |
| ids.openalex | https://openalex.org/W4287759323 |
| fwci | 3.08405052 |
| type | preprint |
| title | Anytime MiniBatch: Exploiting Stragglers in Online Distributed\n Optimization |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T11612 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9991999864578247 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Stochastic Gradient Optimization Techniques |
| topics[1].id | https://openalex.org/T10249 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9937999844551086 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1705 |
| topics[1].subfield.display_name | Computer Networks and Communications |
| topics[1].display_name | Distributed Control Multi-Agent Systems |
| topics[2].id | https://openalex.org/T12101 |
| topics[2].field.id | https://openalex.org/fields/18 |
| topics[2].field.display_name | Decision Sciences |
| topics[2].score | 0.9898999929428101 |
| topics[2].domain.id | https://openalex.org/domains/2 |
| topics[2].domain.display_name | Social Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1803 |
| topics[2].subfield.display_name | Management Science and Operations Research |
| topics[2].display_name | Advanced Bandit Algorithms Research |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.7955117225646973 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C62611344 |
| concepts[1].level | 2 |
| concepts[1].score | 0.6312267780303955 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q1062658 |
| concepts[1].display_name | Node (physics) |
| concepts[2].id | https://openalex.org/C2780317896 |
| concepts[2].level | 3 |
| concepts[2].score | 0.5593911409378052 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q2703 |
| concepts[2].display_name | Epoch (astronomy) |
| concepts[3].id | https://openalex.org/C2777303404 |
| concepts[3].level | 2 |
| concepts[3].score | 0.5074695944786072 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q759757 |
| concepts[3].display_name | Convergence (economics) |
| concepts[4].id | https://openalex.org/C137836250 |
| concepts[4].level | 2 |
| concepts[4].score | 0.44082680344581604 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q984063 |
| concepts[4].display_name | Optimization problem |
| concepts[5].id | https://openalex.org/C120314980 |
| concepts[5].level | 1 |
| concepts[5].score | 0.4191340506076813 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q180634 |
| concepts[5].display_name | Distributed computing |
| concepts[6].id | https://openalex.org/C11413529 |
| concepts[6].level | 1 |
| concepts[6].score | 0.2353413701057434 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q8366 |
| concepts[6].display_name | Algorithm |
| concepts[7].id | https://openalex.org/C150846664 |
| concepts[7].level | 2 |
| concepts[7].score | 0.0 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q7602306 |
| concepts[7].display_name | Stars |
| concepts[8].id | https://openalex.org/C66938386 |
| concepts[8].level | 1 |
| concepts[8].score | 0.0 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q633538 |
| concepts[8].display_name | Structural engineering |
| concepts[9].id | https://openalex.org/C162324750 |
| concepts[9].level | 0 |
| concepts[9].score | 0.0 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q8134 |
| concepts[9].display_name | Economics |
| concepts[10].id | https://openalex.org/C127413603 |
| concepts[10].level | 0 |
| concepts[10].score | 0.0 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q11023 |
| concepts[10].display_name | Engineering |
| concepts[11].id | https://openalex.org/C50522688 |
| concepts[11].level | 1 |
| concepts[11].score | 0.0 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q189833 |
| concepts[11].display_name | Economic growth |
| concepts[12].id | https://openalex.org/C31972630 |
| concepts[12].level | 1 |
| concepts[12].score | 0.0 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q844240 |
| concepts[12].display_name | Computer vision |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.7955117225646973 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/node |
| keywords[1].score | 0.6312267780303955 |
| keywords[1].display_name | Node (physics) |
| keywords[2].id | https://openalex.org/keywords/epoch |
| keywords[2].score | 0.5593911409378052 |
| keywords[2].display_name | Epoch (astronomy) |
| keywords[3].id | https://openalex.org/keywords/convergence |
| keywords[3].score | 0.5074695944786072 |
| keywords[3].display_name | Convergence (economics) |
| keywords[4].id | https://openalex.org/keywords/optimization-problem |
| keywords[4].score | 0.44082680344581604 |
| keywords[4].display_name | Optimization problem |
| keywords[5].id | https://openalex.org/keywords/distributed-computing |
| keywords[5].score | 0.4191340506076813 |
| keywords[5].display_name | Distributed computing |
| keywords[6].id | https://openalex.org/keywords/algorithm |
| keywords[6].score | 0.2353413701057434 |
| keywords[6].display_name | Algorithm |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2006.05752 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2006.05752 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2006.05752 |
| indexed_in | arxiv |
| authorships[0].author.id | https://openalex.org/A5033715524 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-4043-1150 |
| authorships[0].author.display_name | Nuwan S. Ferdinand |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Ferdinand, Nuwan |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5110928091 |
| authorships[1].author.orcid | |
| authorships[1].author.display_name | Haider Al-Lawati |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Al-Lawati, Haider |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5029182703 |
| authorships[2].author.orcid | https://orcid.org/0000-0001-8100-5599 |
| authorships[2].author.display_name | Stark C. Draper |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Draper, Stark C. |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5068254472 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-3454-4212 |
| authorships[3].author.display_name | Matthew Nokleby |
| authorships[3].author_position | last |
| authorships[3].raw_author_name | Nokleby, Matthew |
| authorships[3].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2006.05752 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Anytime MiniBatch: Exploiting Stragglers in Online Distributed\n Optimization |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-10-10T17:16:08.811792 |
| primary_topic.id | https://openalex.org/T11612 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9991999864578247 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Stochastic Gradient Optimization Techniques |
| related_works | https://openalex.org/W2392871747, https://openalex.org/W2391035523, https://openalex.org/W3122679363, https://openalex.org/W4233937365, https://openalex.org/W2392463700, https://openalex.org/W2356534600, https://openalex.org/W2350450974, https://openalex.org/W1969947026, https://openalex.org/W265475347, https://openalex.org/W950417808 |
| cited_by_count | 26 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 1 |
| counts_by_year[1].year | 2024 |
| counts_by_year[1].cited_by_count | 3 |
| counts_by_year[2].year | 2023 |
| counts_by_year[2].cited_by_count | 5 |
| counts_by_year[3].year | 2022 |
| counts_by_year[3].cited_by_count | 5 |
| counts_by_year[4].year | 2021 |
| counts_by_year[4].cited_by_count | 6 |
| counts_by_year[5].year | 2020 |
| counts_by_year[5].cited_by_count | 5 |
| counts_by_year[6].year | 2019 |
| counts_by_year[6].cited_by_count | 1 |
| locations_count | 1 |
| best_oa_location.id | pmh:oai:arXiv.org:2006.05752 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2006.05752 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2006.05752 |
| primary_location.id | pmh:oai:arXiv.org:2006.05752 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2006.05752 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2006.05752 |
| publication_date | 2020-06-10 |
| publication_year | 2020 |
| referenced_works_count | 0 |
| abstract_inverted_index.A | 9 |
| abstract_inverted_index.a | 68, 84, 91, 131 |
| abstract_inverted_index.In | 37, 61 |
| abstract_inverted_index.To | 47 |
| abstract_inverted_index.We | 129 |
| abstract_inverted_index.an | 54 |
| abstract_inverted_index.as | 75, 79 |
| abstract_inverted_index.in | 4, 25, 151, 166 |
| abstract_inverted_index.is | 2, 16, 83, 146, 156, 164 |
| abstract_inverted_index.it | 155 |
| abstract_inverted_index.of | 12, 74, 102 |
| abstract_inverted_index.to | 33, 71, 95, 148, 158 |
| abstract_inverted_index.up | 119, 147, 157 |
| abstract_inverted_index.we | 52 |
| abstract_inverted_index.1.5 | 149 |
| abstract_inverted_index.EC2 | 153 |
| abstract_inverted_index.Our | 139 |
| abstract_inverted_index.The | 81 |
| abstract_inverted_index.all | 19, 64 |
| abstract_inverted_index.and | 134, 154 |
| abstract_inverted_index.are | 66, 105 |
| abstract_inverted_index.can | 31, 43 |
| abstract_inverted_index.get | 90 |
| abstract_inverted_index.our | 144 |
| abstract_inverted_index.the | 29, 34, 49, 120, 124, 136 |
| abstract_inverted_index.via | 99, 111 |
| abstract_inverted_index.data | 77 |
| abstract_inverted_index.dual | 112 |
| abstract_inverted_index.each | 26 |
| abstract_inverted_index.five | 159 |
| abstract_inverted_index.from | 117 |
| abstract_inverted_index.many | 76 |
| abstract_inverted_index.next | 35 |
| abstract_inverted_index.node | 168 |
| abstract_inverted_index.show | 142 |
| abstract_inverted_index.slow | 45 |
| abstract_inverted_index.such | 38 |
| abstract_inverted_index.that | 18, 126, 143 |
| abstract_inverted_index.then | 89, 106 |
| abstract_inverted_index.this | 62 |
| abstract_inverted_index.time | 70, 94 |
| abstract_inverted_index.wall | 137 |
| abstract_inverted_index.when | 162 |
| abstract_inverted_index.work | 125 |
| abstract_inverted_index.fixed | 69, 92 |
| abstract_inverted_index.given | 67 |
| abstract_inverted_index.nodes | 20, 65 |
| abstract_inverted_index.size. | 87 |
| abstract_inverted_index.tasks | 24 |
| abstract_inverted_index.their | 22 |
| abstract_inverted_index.there | 163 |
| abstract_inverted_index.times | 160 |
| abstract_inverted_index.vital | 3 |
| abstract_inverted_index.which | 104 |
| abstract_inverted_index.Amazon | 152 |
| abstract_inverted_index.before | 28 |
| abstract_inverted_index.called | 41, 59 |
| abstract_inverted_index.epoch. | 36 |
| abstract_inverted_index.faster | 161 |
| abstract_inverted_index.impact | 50 |
| abstract_inverted_index.method | 58 |
| abstract_inverted_index.online | 55 |
| abstract_inverted_index.primal | 109 |
| abstract_inverted_index.result | 82 |
| abstract_inverted_index.rounds | 101 |
| abstract_inverted_index.system | 30, 121 |
| abstract_inverted_index.update | 108 |
| abstract_inverted_index.Anytime | 114 |
| abstract_inverted_index.Workers | 88 |
| abstract_inverted_index.analyze | 135 |
| abstract_inverted_index.compute | 72, 167 |
| abstract_inverted_index.feature | 11 |
| abstract_inverted_index.greatly | 44 |
| abstract_inverted_index.holding | 118 |
| abstract_inverted_index.machine | 7 |
| abstract_inverted_index.present | 130 |
| abstract_inverted_index.proceed | 32 |
| abstract_inverted_index.propose | 53 |
| abstract_inverted_index.results | 141 |
| abstract_inverted_index.samples | 78 |
| abstract_inverted_index.several | 100 |
| abstract_inverted_index.solving | 5 |
| abstract_inverted_index.wasting | 123 |
| abstract_inverted_index.without | 122 |
| abstract_inverted_index.analysis | 133 |
| abstract_inverted_index.approach | 145 |
| abstract_inverted_index.assigned | 23 |
| abstract_inverted_index.complete | 21 |
| abstract_inverted_index.mitigate | 48 |
| abstract_inverted_index.used\nto | 107 |
| abstract_inverted_index.Minibatch | 115 |
| abstract_inverted_index.approach, | 63 |
| abstract_inverted_index.complete. | 128 |
| abstract_inverted_index.gradients | 98 |
| abstract_inverted_index.minibatch | 86, 97 |
| abstract_inverted_index.numerical | 140 |
| abstract_inverted_index.possible. | 80 |
| abstract_inverted_index.progress. | 46 |
| abstract_inverted_index.settings, | 39 |
| abstract_inverted_index.variables | 110 |
| abstract_inverted_index.averaging. | 113 |
| abstract_inverted_index.consensus, | 103 |
| abstract_inverted_index.techniques | 15 |
| abstract_inverted_index.Distributed | 0 |
| abstract_inverted_index.convergence | 132 |
| abstract_inverted_index.distributed | 13, 56 |
| abstract_inverted_index.large-scale | 6 |
| abstract_inverted_index.stragglers, | 42 |
| abstract_inverted_index.optimization | 1, 14, 57 |
| abstract_inverted_index.slow\nnodes, | 40 |
| abstract_inverted_index.communication | 93 |
| abstract_inverted_index.times\nfaster | 150 |
| abstract_inverted_index.widely-shared | 10 |
| abstract_inverted_index.average\ntheir | 96 |
| abstract_inverted_index.performance.\n | 169 |
| abstract_inverted_index.the\ngradients | 73 |
| abstract_inverted_index.of\nstragglers, | 51 |
| abstract_inverted_index.stragglers\ncan | 127 |
| abstract_inverted_index.the\nrequirement | 17 |
| abstract_inverted_index.time\nperformance. | 138 |
| abstract_inverted_index.variable\nper-node | 85 |
| abstract_inverted_index.Anytime\nMinibatch. | 60 |
| abstract_inverted_index.learning\nproblems. | 8 |
| abstract_inverted_index.computational\nepoch | 27 |
| abstract_inverted_index.greater\nvariability | 165 |
| abstract_inverted_index.prevents\nstragglers | 116 |
| cited_by_percentile_year.max | 98 |
| cited_by_percentile_year.min | 90 |
| countries_distinct_count | 0 |
| institutions_distinct_count | 4 |
| citation_normalized_percentile.value | 0.92728013 |
| citation_normalized_percentile.is_in_top_1_percent | False |
| citation_normalized_percentile.is_in_top_10_percent | True |