Cost-Driven Hardware-Software Co-Optimization of Machine Learning Pipelines Article Swipe

PDF

Ravit Sharma , Wojciech Romaszkan , Feiqian Zhu , Puneet Gupta ·

YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2310.07940

Researchers have long touted a vision of the future enabled by a proliferation of internet-of-things devices, including smart sensors, homes, and cities. Increasingly, embedding intelligence in such devices involves the use of deep neural networks. However, their storage and processing requirements make them prohibitive for cheap, off-the-shelf platforms. Overcoming those requirements is necessary for enabling widely-applicable smart devices. While many ways of making models smaller and more efficient have been developed, there is a lack of understanding of which ones are best suited for particular scenarios. More importantly for edge platforms, those choices cannot be analyzed in isolation from cost and user experience. In this work, we holistically explore how quantization, model scaling, and multi-modality interact with system components such as memory, sensors, and processors. We perform this hardware/software co-design from the cost, latency, and user-experience perspective, and develop a set of guidelines for optimal system design and model deployment for the most cost-constrained platforms. We demonstrate our approach using an end-to-end, on-device, biometric user authentication system using a $20 ESP-EYE board.

Related Topics

Computer Science

Embedded System

Lock And Key

Human–Computer Interaction

Software Engineering

Computer Security

Concepts

Computer science Software deployment Embedded system Software Cloud computing Key (lock) Human–computer interaction Software engineering Computer security Operating system

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2310.07940
PDF: https://arxiv.org/pdf/2310.07940
OA Status: green
Related Works: 10
OpenAlex ID: https://openalex.org/W4387635060

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4387635060

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2310.07940

Digital Object Identifier
Title: Cost-Driven Hardware-Software Co-Optimization of Machine Learning Pipelines

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2023

Year of publication
Publication date: 2023-10-11

Full publication date if available
Authors: Ravit Sharma, Wojciech Romaszkan, Feiqian Zhu, Puneet Gupta

List of authors in order
Landing page: https://arxiv.org/abs/2310.07940

Publisher landing page
PDF URL: https://arxiv.org/pdf/2310.07940

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2310.07940

Direct OA link when available
Concepts: Computer science, Software deployment, Embedded system, Software, Cloud computing, Key (lock), Human–computer interaction, Software engineering, Computer security, Operating system

Top concepts (fields/topics) attached by OpenAlex
Cited by: 0

Total citation count in OpenAlex
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4387635060
doi	https://doi.org/10.48550/arxiv.2310.07940
ids.doi	https://doi.org/10.48550/arxiv.2310.07940
ids.openalex	https://openalex.org/W4387635060
fwci
type	preprint
title	Cost-Driven Hardware-Software Co-Optimization of Machine Learning Pipelines
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T12238
topics[0].field.id	https://openalex.org/fields/22
topics[0].field.display_name	Engineering
topics[0].score	0.913100004196167
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/2208
topics[0].subfield.display_name	Electrical and Electronic Engineering
topics[0].display_name	Green IT and Sustainability
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C41008148
concepts[0].level	0
concepts[0].score	0.7307929992675781
concepts[0].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[0].display_name	Computer science
concepts[1].id	https://openalex.org/C105339364
concepts[1].level	2
concepts[1].score	0.642467737197876
concepts[1].wikidata	https://www.wikidata.org/wiki/Q2297740
concepts[1].display_name	Software deployment
concepts[2].id	https://openalex.org/C149635348
concepts[2].level	1
concepts[2].score	0.49372801184654236
concepts[2].wikidata	https://www.wikidata.org/wiki/Q193040
concepts[2].display_name	Embedded system
concepts[3].id	https://openalex.org/C2777904410
concepts[3].level	2
concepts[3].score	0.4848726689815521
concepts[3].wikidata	https://www.wikidata.org/wiki/Q7397
concepts[3].display_name	Software
concepts[4].id	https://openalex.org/C79974875
concepts[4].level	2
concepts[4].score	0.4287775754928589
concepts[4].wikidata	https://www.wikidata.org/wiki/Q483639
concepts[4].display_name	Cloud computing
concepts[5].id	https://openalex.org/C26517878
concepts[5].level	2
concepts[5].score	0.4282703101634979
concepts[5].wikidata	https://www.wikidata.org/wiki/Q228039
concepts[5].display_name	Key (lock)
concepts[6].id	https://openalex.org/C107457646
concepts[6].level	1
concepts[6].score	0.37471896409988403
concepts[6].wikidata	https://www.wikidata.org/wiki/Q207434
concepts[6].display_name	Human–computer interaction
concepts[7].id	https://openalex.org/C115903868
concepts[7].level	1
concepts[7].score	0.29328906536102295
concepts[7].wikidata	https://www.wikidata.org/wiki/Q80993
concepts[7].display_name	Software engineering
concepts[8].id	https://openalex.org/C38652104
concepts[8].level	1
concepts[8].score	0.1961134374141693
concepts[8].wikidata	https://www.wikidata.org/wiki/Q3510521
concepts[8].display_name	Computer security
concepts[9].id	https://openalex.org/C111919701
concepts[9].level	1
concepts[9].score	0.15441393852233887
concepts[9].wikidata	https://www.wikidata.org/wiki/Q9135
concepts[9].display_name	Operating system
keywords[0].id	https://openalex.org/keywords/computer-science
keywords[0].score	0.7307929992675781
keywords[0].display_name	Computer science
keywords[1].id	https://openalex.org/keywords/software-deployment
keywords[1].score	0.642467737197876
keywords[1].display_name	Software deployment
keywords[2].id	https://openalex.org/keywords/embedded-system
keywords[2].score	0.49372801184654236
keywords[2].display_name	Embedded system
keywords[3].id	https://openalex.org/keywords/software
keywords[3].score	0.4848726689815521
keywords[3].display_name	Software
keywords[4].id	https://openalex.org/keywords/cloud-computing
keywords[4].score	0.4287775754928589
keywords[4].display_name	Cloud computing
keywords[5].id	https://openalex.org/keywords/key
keywords[5].score	0.4282703101634979
keywords[5].display_name	Key (lock)
keywords[6].id	https://openalex.org/keywords/human–computer-interaction
keywords[6].score	0.37471896409988403
keywords[6].display_name	Human–computer interaction
keywords[7].id	https://openalex.org/keywords/software-engineering
keywords[7].score	0.29328906536102295
keywords[7].display_name	Software engineering
keywords[8].id	https://openalex.org/keywords/computer-security
keywords[8].score	0.1961134374141693
keywords[8].display_name	Computer security
keywords[9].id	https://openalex.org/keywords/operating-system
keywords[9].score	0.15441393852233887
keywords[9].display_name	Operating system
language	en
locations[0].id	pmh:oai:arXiv.org:2310.07940
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2310.07940
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2310.07940
locations[1].id	doi:10.48550/arxiv.2310.07940
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license	cc-by
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id	https://openalex.org/licenses/cc-by
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2310.07940
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5102567257
authorships[0].author.orcid
authorships[0].author.display_name	Ravit Sharma
authorships[0].author_position	first
authorships[0].raw_author_name	Sharma, Ravit
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5084220043
authorships[1].author.orcid	https://orcid.org/0000-0003-0906-7079
authorships[1].author.display_name	Wojciech Romaszkan
authorships[1].author_position	middle
authorships[1].raw_author_name	Romaszkan, Wojciech
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5111056495
authorships[2].author.orcid
authorships[2].author.display_name	Feiqian Zhu
authorships[2].author_position	middle
authorships[2].raw_author_name	Zhu, Feiqian
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5084229134
authorships[3].author.orcid	https://orcid.org/0000-0002-6188-1134
authorships[3].author.display_name	Puneet Gupta
authorships[3].author_position	last
authorships[3].raw_author_name	Gupta, Puneet
authorships[3].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2310.07940
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Cost-Driven Hardware-Software Co-Optimization of Machine Learning Pipelines
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T12238
primary_topic.field.id	https://openalex.org/fields/22
primary_topic.field.display_name	Engineering
primary_topic.score	0.913100004196167
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/2208
primary_topic.subfield.display_name	Electrical and Electronic Engineering
primary_topic.display_name	Green IT and Sustainability
related_works	https://openalex.org/W2770234245, https://openalex.org/W96612179, https://openalex.org/W4229499248, https://openalex.org/W2566006169, https://openalex.org/W2987774938, https://openalex.org/W4256492088, https://openalex.org/W632915154, https://openalex.org/W2055733372, https://openalex.org/W3022067003, https://openalex.org/W4378874356
cited_by_count	0
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2310.07940
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2310.07940
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2310.07940
primary_location.id	pmh:oai:arXiv.org:2310.07940
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2310.07940
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2310.07940
publication_date	2023-10-11
publication_year	2023
referenced_works_count	0
abstract_inverted_index.a	4, 11, 73, 139, 168
abstract_inverted_index.In	103
abstract_inverted_index.We	125, 155
abstract_inverted_index.an	160
abstract_inverted_index.as	120
abstract_inverted_index.be	94
abstract_inverted_index.by	10
abstract_inverted_index.in	25, 96
abstract_inverted_index.is	51, 72
abstract_inverted_index.of	6, 13, 31, 61, 75, 77, 141
abstract_inverted_index.we	106
abstract_inverted_index.$20	169
abstract_inverted_index.and	20, 38, 65, 100, 113, 123, 134, 137, 147
abstract_inverted_index.are	80
abstract_inverted_index.for	44, 53, 83, 88, 143, 150
abstract_inverted_index.how	109
abstract_inverted_index.our	157
abstract_inverted_index.set	140
abstract_inverted_index.the	7, 29, 131, 151
abstract_inverted_index.use	30
abstract_inverted_index.More	86
abstract_inverted_index.been	69
abstract_inverted_index.best	81
abstract_inverted_index.cost	99
abstract_inverted_index.deep	32
abstract_inverted_index.edge	89
abstract_inverted_index.from	98, 130
abstract_inverted_index.have	1, 68
abstract_inverted_index.lack	74
abstract_inverted_index.long	2
abstract_inverted_index.make	41
abstract_inverted_index.many	59
abstract_inverted_index.more	66
abstract_inverted_index.most	152
abstract_inverted_index.ones	79
abstract_inverted_index.such	26, 119
abstract_inverted_index.them	42
abstract_inverted_index.this	104, 127
abstract_inverted_index.user	101, 164
abstract_inverted_index.ways	60
abstract_inverted_index.with	116
abstract_inverted_index.While	58
abstract_inverted_index.cost,	132
abstract_inverted_index.model	111, 148
abstract_inverted_index.smart	17, 56
abstract_inverted_index.their	36
abstract_inverted_index.there	71
abstract_inverted_index.those	49, 91
abstract_inverted_index.using	159, 167
abstract_inverted_index.which	78
abstract_inverted_index.work,	105
abstract_inverted_index.board.	171
abstract_inverted_index.cannot	93
abstract_inverted_index.cheap,	45
abstract_inverted_index.design	146
abstract_inverted_index.future	8
abstract_inverted_index.homes,	19
abstract_inverted_index.making	62
abstract_inverted_index.models	63
abstract_inverted_index.neural	33
abstract_inverted_index.suited	82
abstract_inverted_index.system	117, 145, 166
abstract_inverted_index.touted	3
abstract_inverted_index.vision	5
abstract_inverted_index.ESP-EYE	170
abstract_inverted_index.choices	92
abstract_inverted_index.cities.	21
abstract_inverted_index.develop	138
abstract_inverted_index.devices	27
abstract_inverted_index.enabled	9
abstract_inverted_index.explore	108
abstract_inverted_index.memory,	121
abstract_inverted_index.optimal	144
abstract_inverted_index.perform	126
abstract_inverted_index.smaller	64
abstract_inverted_index.storage	37
abstract_inverted_index.However,	35
abstract_inverted_index.analyzed	95
abstract_inverted_index.approach	158
abstract_inverted_index.devices,	15
abstract_inverted_index.devices.	57
abstract_inverted_index.enabling	54
abstract_inverted_index.interact	115
abstract_inverted_index.involves	28
abstract_inverted_index.latency,	133
abstract_inverted_index.scaling,	112
abstract_inverted_index.sensors,	18, 122
abstract_inverted_index.biometric	163
abstract_inverted_index.co-design	129
abstract_inverted_index.efficient	67
abstract_inverted_index.embedding	23
abstract_inverted_index.including	16
abstract_inverted_index.isolation	97
abstract_inverted_index.necessary	52
abstract_inverted_index.networks.	34
abstract_inverted_index.Overcoming	48
abstract_inverted_index.components	118
abstract_inverted_index.deployment	149
abstract_inverted_index.developed,	70
abstract_inverted_index.guidelines	142
abstract_inverted_index.on-device,	162
abstract_inverted_index.particular	84
abstract_inverted_index.platforms,	90
abstract_inverted_index.platforms.	47, 154
abstract_inverted_index.processing	39
abstract_inverted_index.scenarios.	85
abstract_inverted_index.Researchers	0
abstract_inverted_index.demonstrate	156
abstract_inverted_index.end-to-end,	161
abstract_inverted_index.experience.	102
abstract_inverted_index.importantly	87
abstract_inverted_index.processors.	124
abstract_inverted_index.prohibitive	43
abstract_inverted_index.holistically	107
abstract_inverted_index.intelligence	24
abstract_inverted_index.perspective,	136
abstract_inverted_index.requirements	40, 50
abstract_inverted_index.Increasingly,	22
abstract_inverted_index.off-the-shelf	46
abstract_inverted_index.proliferation	12
abstract_inverted_index.quantization,	110
abstract_inverted_index.understanding	76
abstract_inverted_index.authentication	165
abstract_inverted_index.multi-modality	114
abstract_inverted_index.user-experience	135
abstract_inverted_index.cost-constrained	153
abstract_inverted_index.hardware/software	128
abstract_inverted_index.widely-applicable	55
abstract_inverted_index.internet-of-things	14
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	4
sustainable_development_goals[0].id	https://metadata.un.org/sdg/11
sustainable_development_goals[0].score	0.4099999964237213
sustainable_development_goals[0].display_name	Sustainable cities and communities
citation_normalized_percentile