Quality at the Tail of Machine Learning Inference Article Swipe

PDF

Zhengxin Yang , Wanling Gao , Chunjie Luo , Lei Wang , Jianfeng Zhan ·

YOU? · · 2022 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2212.13925

Machine learning inference should be subject to stringent inference time constraints while ensuring high inference quality, especially in safety-critical (e.g., autonomous driving) and mission-critical (e.g., emotion recognition) contexts. Neglecting either aspect can lead to severe consequences, such as loss of life and property damage. Many studies lack a comprehensive consideration of these metrics, leading to incomplete or misleading evaluations. The study unveils a counterintuitive revelation: deep learning inference quality exhibits fluctuations due to inference time. To depict this phenomenon, the authors coin a new term, "tail quality," providing a more comprehensive evaluation, and overcoming conventional metric limitations. Moreover, the research proposes an initial evaluation framework to analyze factors affecting quality fluctuations, facilitating the prediction of the potential distribution of inference quality. The effectiveness of the evaluation framework is validated through experiments conducted on deep learning models for three different tasks across four systems.

Related Topics

Business

Environmental Science

Physics

Quantum Mechanics

Concepts

Quality (philosophy) Business Environmental science Physics Quantum mechanics

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2212.13925
PDF: https://arxiv.org/pdf/2212.13925
OA Status: green
Related Works: 10
OpenAlex ID: https://openalex.org/W4313303747

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4313303747

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2212.13925

Digital Object Identifier
Title: Quality at the Tail of Machine Learning Inference

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2022

Year of publication
Publication date: 2022-12-25

Full publication date if available
Authors: Zhengxin Yang, Wanling Gao, Chunjie Luo, Lei Wang, Jianfeng Zhan

List of authors in order
Landing page: https://arxiv.org/abs/2212.13925

Publisher landing page
PDF URL: https://arxiv.org/pdf/2212.13925

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2212.13925

Direct OA link when available
Concepts: Quality (philosophy), Business, Environmental science, Physics, Quantum mechanics

Top concepts (fields/topics) attached by OpenAlex
Cited by: 0

Total citation count in OpenAlex
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4313303747
doi	https://doi.org/10.48550/arxiv.2212.13925
ids.doi	https://doi.org/10.48550/arxiv.2212.13925
ids.openalex	https://openalex.org/W4313303747
fwci
type	preprint
title	Quality at the Tail of Machine Learning Inference
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C2779530757
concepts[0].level	2
concepts[0].score	0.6315054893493652
concepts[0].wikidata	https://www.wikidata.org/wiki/Q1207505
concepts[0].display_name	Quality (philosophy)
concepts[1].id	https://openalex.org/C144133560
concepts[1].level	0
concepts[1].score	0.3913952708244324
concepts[1].wikidata	https://www.wikidata.org/wiki/Q4830453
concepts[1].display_name	Business
concepts[2].id	https://openalex.org/C39432304
concepts[2].level	0
concepts[2].score	0.3739609122276306
concepts[2].wikidata	https://www.wikidata.org/wiki/Q188847
concepts[2].display_name	Environmental science
concepts[3].id	https://openalex.org/C121332964
concepts[3].level	0
concepts[3].score	0.25492212176322937
concepts[3].wikidata	https://www.wikidata.org/wiki/Q413
concepts[3].display_name	Physics
concepts[4].id	https://openalex.org/C62520636
concepts[4].level	1
concepts[4].score	0.0
concepts[4].wikidata	https://www.wikidata.org/wiki/Q944
concepts[4].display_name	Quantum mechanics
keywords[0].id	https://openalex.org/keywords/quality
keywords[0].score	0.6315054893493652
keywords[0].display_name	Quality (philosophy)
keywords[1].id	https://openalex.org/keywords/business
keywords[1].score	0.3913952708244324
keywords[1].display_name	Business
keywords[2].id	https://openalex.org/keywords/environmental-science
keywords[2].score	0.3739609122276306
keywords[2].display_name	Environmental science
keywords[3].id	https://openalex.org/keywords/physics
keywords[3].score	0.25492212176322937
keywords[3].display_name	Physics
language	en
locations[0].id	pmh:oai:arXiv.org:2212.13925
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2212.13925
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2212.13925
locations[1].id	doi:10.48550/arxiv.2212.13925
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2212.13925
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5003256733
authorships[0].author.orcid	https://orcid.org/0000-0002-3160-0784
authorships[0].author.display_name	Zhengxin Yang
authorships[0].author_position	first
authorships[0].raw_author_name	Yang, Zhengxin
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5075110031
authorships[1].author.orcid	https://orcid.org/0000-0002-3911-9389
authorships[1].author.display_name	Wanling Gao
authorships[1].author_position	middle
authorships[1].raw_author_name	Gao, Wanling
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5033347503
authorships[2].author.orcid	https://orcid.org/0000-0002-6977-929X
authorships[2].author.display_name	Chunjie Luo
authorships[2].author_position	middle
authorships[2].raw_author_name	Luo, Chunjie
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5100435848
authorships[3].author.orcid	https://orcid.org/0000-0002-0961-0441
authorships[3].author.display_name	Lei Wang
authorships[3].author_position	middle
authorships[3].raw_author_name	Wang, Lei
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5085963553
authorships[4].author.orcid	https://orcid.org/0000-0002-3728-6837
authorships[4].author.display_name	Jianfeng Zhan
authorships[4].author_position	last
authorships[4].raw_author_name	Zhan, Jianfeng
authorships[4].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2212.13925
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2023-01-06T00:00:00
display_name	Quality at the Tail of Machine Learning Inference
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic
related_works	https://openalex.org/W4391375266, https://openalex.org/W2384667405, https://openalex.org/W2373216116, https://openalex.org/W2362665430, https://openalex.org/W3123396266, https://openalex.org/W4323667137, https://openalex.org/W3007404728, https://openalex.org/W2352466142, https://openalex.org/W2375981319, https://openalex.org/W2562923059
cited_by_count	0
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2212.13925
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2212.13925
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2212.13925
primary_location.id	pmh:oai:arXiv.org:2212.13925
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2212.13925
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2212.13925
publication_date	2022-12-25
publication_year	2022
referenced_works_count	0
abstract_inverted_index.a	47, 62, 82, 88
abstract_inverted_index.To	75
abstract_inverted_index.an	101
abstract_inverted_index.as	37
abstract_inverted_index.be	4
abstract_inverted_index.in	17
abstract_inverted_index.is	127
abstract_inverted_index.of	39, 50, 114, 118, 123
abstract_inverted_index.on	132
abstract_inverted_index.or	56
abstract_inverted_index.to	6, 33, 54, 72, 105
abstract_inverted_index.The	59, 121
abstract_inverted_index.and	22, 41, 92
abstract_inverted_index.can	31
abstract_inverted_index.due	71
abstract_inverted_index.for	136
abstract_inverted_index.new	83
abstract_inverted_index.the	79, 98, 112, 115, 124
abstract_inverted_index.Many	44
abstract_inverted_index.coin	81
abstract_inverted_index.deep	65, 133
abstract_inverted_index.four	141
abstract_inverted_index.high	13
abstract_inverted_index.lack	46
abstract_inverted_index.lead	32
abstract_inverted_index.life	40
abstract_inverted_index.loss	38
abstract_inverted_index.more	89
abstract_inverted_index.such	36
abstract_inverted_index.this	77
abstract_inverted_index.time	9
abstract_inverted_index."tail	85
abstract_inverted_index.study	60
abstract_inverted_index.tasks	139
abstract_inverted_index.term,	84
abstract_inverted_index.these	51
abstract_inverted_index.three	137
abstract_inverted_index.time.	74
abstract_inverted_index.while	11
abstract_inverted_index.(e.g.,	19, 24
abstract_inverted_index.across	140
abstract_inverted_index.aspect	30
abstract_inverted_index.depict	76
abstract_inverted_index.either	29
abstract_inverted_index.metric	95
abstract_inverted_index.models	135
abstract_inverted_index.severe	34
abstract_inverted_index.should	3
abstract_inverted_index.Machine	0
abstract_inverted_index.analyze	106
abstract_inverted_index.authors	80
abstract_inverted_index.damage.	43
abstract_inverted_index.emotion	25
abstract_inverted_index.factors	107
abstract_inverted_index.initial	102
abstract_inverted_index.leading	53
abstract_inverted_index.quality	68, 109
abstract_inverted_index.studies	45
abstract_inverted_index.subject	5
abstract_inverted_index.through	129
abstract_inverted_index.unveils	61
abstract_inverted_index.driving)	21
abstract_inverted_index.ensuring	12
abstract_inverted_index.exhibits	69
abstract_inverted_index.learning	1, 66, 134
abstract_inverted_index.metrics,	52
abstract_inverted_index.property	42
abstract_inverted_index.proposes	100
abstract_inverted_index.quality,	15
abstract_inverted_index.quality.	120
abstract_inverted_index.research	99
abstract_inverted_index.systems.	142
abstract_inverted_index.Moreover,	97
abstract_inverted_index.affecting	108
abstract_inverted_index.conducted	131
abstract_inverted_index.contexts.	27
abstract_inverted_index.different	138
abstract_inverted_index.framework	104, 126
abstract_inverted_index.inference	2, 8, 14, 67, 73, 119
abstract_inverted_index.potential	116
abstract_inverted_index.providing	87
abstract_inverted_index.quality,"	86
abstract_inverted_index.stringent	7
abstract_inverted_index.validated	128
abstract_inverted_index.Neglecting	28
abstract_inverted_index.autonomous	20
abstract_inverted_index.especially	16
abstract_inverted_index.evaluation	103, 125
abstract_inverted_index.incomplete	55
abstract_inverted_index.misleading	57
abstract_inverted_index.overcoming	93
abstract_inverted_index.prediction	113
abstract_inverted_index.constraints	10
abstract_inverted_index.evaluation,	91
abstract_inverted_index.experiments	130
abstract_inverted_index.phenomenon,	78
abstract_inverted_index.revelation:	64
abstract_inverted_index.conventional	94
abstract_inverted_index.distribution	117
abstract_inverted_index.evaluations.	58
abstract_inverted_index.facilitating	111
abstract_inverted_index.fluctuations	70
abstract_inverted_index.limitations.	96
abstract_inverted_index.recognition)	26
abstract_inverted_index.comprehensive	48, 90
abstract_inverted_index.consequences,	35
abstract_inverted_index.consideration	49
abstract_inverted_index.effectiveness	122
abstract_inverted_index.fluctuations,	110
abstract_inverted_index.safety-critical	18
abstract_inverted_index.counterintuitive	63
abstract_inverted_index.mission-critical	23
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	5
sustainable_development_goals[0].id	https://metadata.un.org/sdg/16
sustainable_development_goals[0].score	0.6000000238418579
sustainable_development_goals[0].display_name	Peace, Justice and strong institutions
citation_normalized_percentile