Erasure Coded Neural Network Inference via Fisher Averaging Article Swipe

PDF

Divyansh Jhunjhunwala , Neharika Jali , Gauri Joshi , Shiqiang Wang ·

YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2409.01420

Erasure-coded computing has been successfully used in cloud systems to reduce tail latency caused by factors such as straggling servers and heterogeneous traffic variations. A majority of cloud computing traffic now consists of inference on neural networks on shared resources where the response time of inference queries is also adversely affected by the same factors. However, current erasure coding techniques are largely focused on linear computations such as matrix-vector and matrix-matrix multiplications and hence do not work for the highly non-linear neural network functions. In this paper, we seek to design a method to code over neural networks, that is, given two or more neural network models, how to construct a coded model whose output is a linear combination of the outputs of the given neural networks. We formulate the problem as a KL barycenter problem and propose a practical algorithm COIN that leverages the diagonal Fisher information to create a coded model that approximately outputs the desired linear combination of outputs. We conduct experiments to perform erasure coding over neural networks trained on real-world vision datasets and show that the accuracy of the decoded outputs using COIN is significantly higher than other baselines while being extremely compute-efficient.

Related Topics

Erasure (Duo)

Computer Science

Artificial Intelligence

Algorithm

Programming Language

Concepts

Erasure Inference Computer science Artificial neural network Artificial intelligence Algorithm Pattern recognition (psychology) Programming language

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2409.01420
PDF: https://arxiv.org/pdf/2409.01420
OA Status: green
Related Works: 10
OpenAlex ID: https://openalex.org/W4403161375

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4403161375

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2409.01420

Digital Object Identifier
Title: Erasure Coded Neural Network Inference via Fisher Averaging

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2024

Year of publication
Publication date: 2024-09-02

Full publication date if available
Authors: Divyansh Jhunjhunwala, Neharika Jali, Gauri Joshi, Shiqiang Wang

List of authors in order
Landing page: https://arxiv.org/abs/2409.01420

Publisher landing page
PDF URL: https://arxiv.org/pdf/2409.01420

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2409.01420

Direct OA link when available
Concepts: Erasure, Inference, Computer science, Artificial neural network, Artificial intelligence, Algorithm, Pattern recognition (psychology), Programming language

Top concepts (fields/topics) attached by OpenAlex
Cited by: 0

Total citation count in OpenAlex
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4403161375
doi	https://doi.org/10.48550/arxiv.2409.01420
ids.doi	https://doi.org/10.48550/arxiv.2409.01420
ids.openalex	https://openalex.org/W4403161375
fwci
type	preprint
title	Erasure Coded Neural Network Inference via Fisher Averaging
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T10320
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9958000183105469
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1702
topics[0].subfield.display_name	Artificial Intelligence
topics[0].display_name	Neural Networks and Applications
topics[1].id	https://openalex.org/T10862
topics[1].field.id	https://openalex.org/fields/17
topics[1].field.display_name	Computer Science
topics[1].score	0.9761999845504761
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/1702
topics[1].subfield.display_name	Artificial Intelligence
topics[1].display_name	AI in cancer detection
topics[2].id	https://openalex.org/T12702
topics[2].field.id	https://openalex.org/fields/28
topics[2].field.display_name	Neuroscience
topics[2].score	0.9417999982833862
topics[2].domain.id	https://openalex.org/domains/1
topics[2].domain.display_name	Life Sciences
topics[2].subfield.id	https://openalex.org/subfields/2808
topics[2].subfield.display_name	Neurology
topics[2].display_name	Brain Tumor Detection and Classification
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C2778790127
concepts[0].level	2
concepts[0].score	0.823685348033905
concepts[0].wikidata	https://www.wikidata.org/wiki/Q484885
concepts[0].display_name	Erasure
concepts[1].id	https://openalex.org/C2776214188
concepts[1].level	2
concepts[1].score	0.743103563785553
concepts[1].wikidata	https://www.wikidata.org/wiki/Q408386
concepts[1].display_name	Inference
concepts[2].id	https://openalex.org/C41008148
concepts[2].level	0
concepts[2].score	0.625018835067749
concepts[2].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[2].display_name	Computer science
concepts[3].id	https://openalex.org/C50644808
concepts[3].level	2
concepts[3].score	0.4736332297325134
concepts[3].wikidata	https://www.wikidata.org/wiki/Q192776
concepts[3].display_name	Artificial neural network
concepts[4].id	https://openalex.org/C154945302
concepts[4].level	1
concepts[4].score	0.41502851247787476
concepts[4].wikidata	https://www.wikidata.org/wiki/Q11660
concepts[4].display_name	Artificial intelligence
concepts[5].id	https://openalex.org/C11413529
concepts[5].level	1
concepts[5].score	0.40724682807922363
concepts[5].wikidata	https://www.wikidata.org/wiki/Q8366
concepts[5].display_name	Algorithm
concepts[6].id	https://openalex.org/C153180895
concepts[6].level	2
concepts[6].score	0.36687567830085754
concepts[6].wikidata	https://www.wikidata.org/wiki/Q7148389
concepts[6].display_name	Pattern recognition (psychology)
concepts[7].id	https://openalex.org/C199360897
concepts[7].level	1
concepts[7].score	0.054222673177719116
concepts[7].wikidata	https://www.wikidata.org/wiki/Q9143
concepts[7].display_name	Programming language
keywords[0].id	https://openalex.org/keywords/erasure
keywords[0].score	0.823685348033905
keywords[0].display_name	Erasure
keywords[1].id	https://openalex.org/keywords/inference
keywords[1].score	0.743103563785553
keywords[1].display_name	Inference
keywords[2].id	https://openalex.org/keywords/computer-science
keywords[2].score	0.625018835067749
keywords[2].display_name	Computer science
keywords[3].id	https://openalex.org/keywords/artificial-neural-network
keywords[3].score	0.4736332297325134
keywords[3].display_name	Artificial neural network
keywords[4].id	https://openalex.org/keywords/artificial-intelligence
keywords[4].score	0.41502851247787476
keywords[4].display_name	Artificial intelligence
keywords[5].id	https://openalex.org/keywords/algorithm
keywords[5].score	0.40724682807922363
keywords[5].display_name	Algorithm
keywords[6].id	https://openalex.org/keywords/pattern-recognition
keywords[6].score	0.36687567830085754
keywords[6].display_name	Pattern recognition (psychology)
keywords[7].id	https://openalex.org/keywords/programming-language
keywords[7].score	0.054222673177719116
keywords[7].display_name	Programming language
language	en
locations[0].id	pmh:oai:arXiv.org:2409.01420
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license	cc-by-sa
locations[0].pdf_url	https://arxiv.org/pdf/2409.01420
locations[0].version	submittedVersion
locations[0].raw_type
locations[0].license_id	https://openalex.org/licenses/cc-by-sa
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2409.01420
locations[1].id	doi:10.48550/arxiv.2409.01420
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2409.01420
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5079408504
authorships[0].author.orcid
authorships[0].author.display_name	Divyansh Jhunjhunwala
authorships[0].author_position	first
authorships[0].raw_author_name	Jhunjhunwala, Divyansh
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5093868335
authorships[1].author.orcid
authorships[1].author.display_name	Neharika Jali
authorships[1].author_position	middle
authorships[1].raw_author_name	Jali, Neharika
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5067441201
authorships[2].author.orcid	https://orcid.org/0000-0002-6372-9697
authorships[2].author.display_name	Gauri Joshi
authorships[2].author_position	middle
authorships[2].raw_author_name	Joshi, Gauri
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5100443968
authorships[3].author.orcid	https://orcid.org/0000-0003-2090-5512
authorships[3].author.display_name	Shiqiang Wang
authorships[3].author_position	last
authorships[3].raw_author_name	Wang, Shiqiang
authorships[3].is_corresponding	False
has_content.pdf	True
has_content.grobid_xml	True
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2409.01420
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Erasure Coded Neural Network Inference via Fisher Averaging
has_fulltext	True
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T10320
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9958000183105469
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1702
primary_topic.subfield.display_name	Artificial Intelligence
primary_topic.display_name	Neural Networks and Applications
related_works	https://openalex.org/W2072016134, https://openalex.org/W2351168932, https://openalex.org/W2108143034, https://openalex.org/W2465690287, https://openalex.org/W2376389741, https://openalex.org/W1997850251, https://openalex.org/W2084549431, https://openalex.org/W913430005, https://openalex.org/W1509695971, https://openalex.org/W1985021129
cited_by_count	0
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2409.01420
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license	cc-by-sa
best_oa_location.pdf_url	https://arxiv.org/pdf/2409.01420
best_oa_location.version	submittedVersion
best_oa_location.raw_type
best_oa_location.license_id	https://openalex.org/licenses/cc-by-sa
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2409.01420
primary_location.id	pmh:oai:arXiv.org:2409.01420
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license	cc-by-sa
primary_location.pdf_url	https://arxiv.org/pdf/2409.01420
primary_location.version	submittedVersion
primary_location.raw_type
primary_location.license_id	https://openalex.org/licenses/cc-by-sa
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2409.01420
publication_date	2024-09-02
publication_year	2024
referenced_works_count	0
abstract_inverted_index.A	24
abstract_inverted_index.a	91, 110, 116, 132, 138, 150
abstract_inverted_index.In	84
abstract_inverted_index.KL	133
abstract_inverted_index.We	127, 162
abstract_inverted_index.as	17, 67, 131
abstract_inverted_index.by	14, 51
abstract_inverted_index.do	74
abstract_inverted_index.in	6
abstract_inverted_index.is	47, 115, 188
abstract_inverted_index.of	26, 32, 44, 119, 122, 160, 182
abstract_inverted_index.on	34, 37, 63, 173
abstract_inverted_index.or	102
abstract_inverted_index.to	9, 89, 93, 108, 148, 165
abstract_inverted_index.we	87
abstract_inverted_index.and	20, 69, 72, 136, 177
abstract_inverted_index.are	60
abstract_inverted_index.for	77
abstract_inverted_index.has	2
abstract_inverted_index.how	107
abstract_inverted_index.is,	99
abstract_inverted_index.not	75
abstract_inverted_index.now	30
abstract_inverted_index.the	41, 52, 78, 120, 123, 129, 144, 156, 180, 183
abstract_inverted_index.two	101
abstract_inverted_index.COIN	141, 187
abstract_inverted_index.also	48
abstract_inverted_index.been	3
abstract_inverted_index.code	94
abstract_inverted_index.more	103
abstract_inverted_index.over	95, 169
abstract_inverted_index.same	53
abstract_inverted_index.seek	88
abstract_inverted_index.show	178
abstract_inverted_index.such	16, 66
abstract_inverted_index.tail	11
abstract_inverted_index.than	191
abstract_inverted_index.that	98, 142, 153, 179
abstract_inverted_index.this	85
abstract_inverted_index.time	43
abstract_inverted_index.used	5
abstract_inverted_index.work	76
abstract_inverted_index.being	195
abstract_inverted_index.cloud	7, 27
abstract_inverted_index.coded	111, 151
abstract_inverted_index.given	100, 124
abstract_inverted_index.hence	73
abstract_inverted_index.model	112, 152
abstract_inverted_index.other	192
abstract_inverted_index.using	186
abstract_inverted_index.where	40
abstract_inverted_index.while	194
abstract_inverted_index.whose	113
abstract_inverted_index.Fisher	146
abstract_inverted_index.caused	13
abstract_inverted_index.coding	58, 168
abstract_inverted_index.create	149
abstract_inverted_index.design	90
abstract_inverted_index.higher	190
abstract_inverted_index.highly	79
abstract_inverted_index.linear	64, 117, 158
abstract_inverted_index.method	92
abstract_inverted_index.neural	35, 81, 96, 104, 125, 170
abstract_inverted_index.output	114
abstract_inverted_index.paper,	86
abstract_inverted_index.reduce	10
abstract_inverted_index.shared	38
abstract_inverted_index.vision	175
abstract_inverted_index.conduct	163
abstract_inverted_index.current	56
abstract_inverted_index.decoded	184
abstract_inverted_index.desired	157
abstract_inverted_index.erasure	57, 167
abstract_inverted_index.factors	15
abstract_inverted_index.focused	62
abstract_inverted_index.largely	61
abstract_inverted_index.latency	12
abstract_inverted_index.models,	106
abstract_inverted_index.network	82, 105
abstract_inverted_index.outputs	121, 155, 185
abstract_inverted_index.perform	166
abstract_inverted_index.problem	130, 135
abstract_inverted_index.propose	137
abstract_inverted_index.queries	46
abstract_inverted_index.servers	19
abstract_inverted_index.systems	8
abstract_inverted_index.traffic	22, 29
abstract_inverted_index.trained	172
abstract_inverted_index.However,	55
abstract_inverted_index.accuracy	181
abstract_inverted_index.affected	50
abstract_inverted_index.consists	31
abstract_inverted_index.datasets	176
abstract_inverted_index.diagonal	145
abstract_inverted_index.factors.	54
abstract_inverted_index.majority	25
abstract_inverted_index.networks	36, 171
abstract_inverted_index.outputs.	161
abstract_inverted_index.response	42
abstract_inverted_index.adversely	49
abstract_inverted_index.algorithm	140
abstract_inverted_index.baselines	193
abstract_inverted_index.computing	1, 28
abstract_inverted_index.construct	109
abstract_inverted_index.extremely	196
abstract_inverted_index.formulate	128
abstract_inverted_index.inference	33, 45
abstract_inverted_index.leverages	143
abstract_inverted_index.networks,	97
abstract_inverted_index.networks.	126
abstract_inverted_index.practical	139
abstract_inverted_index.resources	39
abstract_inverted_index.barycenter	134
abstract_inverted_index.functions.	83
abstract_inverted_index.non-linear	80
abstract_inverted_index.real-world	174
abstract_inverted_index.straggling	18
abstract_inverted_index.techniques	59
abstract_inverted_index.combination	118, 159
abstract_inverted_index.experiments	164
abstract_inverted_index.information	147
abstract_inverted_index.variations.	23
abstract_inverted_index.computations	65
abstract_inverted_index.successfully	4
abstract_inverted_index.Erasure-coded	0
abstract_inverted_index.approximately	154
abstract_inverted_index.heterogeneous	21
abstract_inverted_index.matrix-matrix	70
abstract_inverted_index.matrix-vector	68
abstract_inverted_index.significantly	189
abstract_inverted_index.multiplications	71
abstract_inverted_index.compute-efficient.	197
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	4
citation_normalized_percentile