Constrained Instance and Class Reweighting for Robust Learning under\n Label Noise Article Swipe

PDF

YOU? · · 2021 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2111.05428

Deep neural networks have shown impressive performance in supervised\nlearning, enabled by their ability to fit well to the provided training data.\nHowever, their performance is largely dependent on the quality of the training\ndata and often degrades in the presence of noise. We propose a principled\napproach for tackling label noise with the aim of assigning importance weights\nto individual instances and class labels. Our method works by formulating a\nclass of constrained optimization problems that yield simple closed form\nupdates for these importance weights. The proposed optimization problems are\nsolved per mini-batch which obviates the need of storing and updating the\nweights over the full dataset. Our optimization framework also provides a\ntheoretical perspective on existing label smoothing heuristics for addressing\nlabel noise (such as label bootstrapping). We evaluate our method on several\nbenchmark datasets and observe considerable performance gains in the presence\nof label noise.\n

Related Topics

Computer Science

Machine Learning

Benchmark (Surveying)

Artificial Intelligence

Overfitting

Perspective (Graphical)

Concepts

Bootstrapping (finance) Computer science Noise (video) Machine learning Smoothing Heuristics Benchmark (surveying) Class (philosophy) Artificial intelligence Overfitting Perspective (graphical) Artificial neural network Data mining Mathematics Image (mathematics) Operating system Computer vision Econometrics Geography Geodesy

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2111.05428
PDF: https://arxiv.org/pdf/2111.05428
OA Status: green
Cited By: 6
Related Works: 10
OpenAlex ID: https://openalex.org/W4225463537

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4225463537

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2111.05428

Digital Object Identifier
Title: Constrained Instance and Class Reweighting for Robust Learning under\n Label Noise

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2021

Year of publication
Publication date: 2021-11-09

Full publication date if available
Authors: Abhishek Kumar, Ehsan Amid

List of authors in order
Landing page: https://arxiv.org/abs/2111.05428

Publisher landing page
PDF URL: https://arxiv.org/pdf/2111.05428

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2111.05428

Direct OA link when available
Concepts: Bootstrapping (finance), Computer science, Noise (video), Machine learning, Smoothing, Heuristics, Benchmark (surveying), Class (philosophy), Artificial intelligence, Overfitting, Perspective (graphical), Artificial neural network, Data mining, Mathematics, Image (mathematics), Operating system, Computer vision, Econometrics, Geography, Geodesy

Top concepts (fields/topics) attached by OpenAlex
Cited by: 6

Total citation count in OpenAlex
Citations by year (recent): 2023: 4, 2022: 2

Per-year citation counts (last 5 years)
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4225463537
doi	https://doi.org/10.48550/arxiv.2111.05428
ids.openalex	https://openalex.org/W4225463537
fwci	0.84662145
type	preprint
title	Constrained Instance and Class Reweighting for Robust Learning under\n Label Noise
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T12535
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9940000176429749
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1702
topics[0].subfield.display_name	Artificial Intelligence
topics[0].display_name	Machine Learning and Data Classification
topics[1].id	https://openalex.org/T11606
topics[1].field.id	https://openalex.org/fields/22
topics[1].field.display_name	Engineering
topics[1].score	0.9380000233650208
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/2205
topics[1].subfield.display_name	Civil and Structural Engineering
topics[1].display_name	Infrastructure Maintenance and Monitoring
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C207609745
concepts[0].level	2
concepts[0].score	0.7385431528091431
concepts[0].wikidata	https://www.wikidata.org/wiki/Q4944086
concepts[0].display_name	Bootstrapping (finance)
concepts[1].id	https://openalex.org/C41008148
concepts[1].level	0
concepts[1].score	0.7362393140792847
concepts[1].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[1].display_name	Computer science
concepts[2].id	https://openalex.org/C99498987
concepts[2].level	3
concepts[2].score	0.7077111601829529
concepts[2].wikidata	https://www.wikidata.org/wiki/Q2210247
concepts[2].display_name	Noise (video)
concepts[3].id	https://openalex.org/C119857082
concepts[3].level	1
concepts[3].score	0.6922382116317749
concepts[3].wikidata	https://www.wikidata.org/wiki/Q2539
concepts[3].display_name	Machine learning
concepts[4].id	https://openalex.org/C3770464
concepts[4].level	2
concepts[4].score	0.665520966053009
concepts[4].wikidata	https://www.wikidata.org/wiki/Q775963
concepts[4].display_name	Smoothing
concepts[5].id	https://openalex.org/C127705205
concepts[5].level	2
concepts[5].score	0.6635363101959229
concepts[5].wikidata	https://www.wikidata.org/wiki/Q5748245
concepts[5].display_name	Heuristics
concepts[6].id	https://openalex.org/C185798385
concepts[6].level	2
concepts[6].score	0.6214388012886047
concepts[6].wikidata	https://www.wikidata.org/wiki/Q1161707
concepts[6].display_name	Benchmark (surveying)
concepts[7].id	https://openalex.org/C2777212361
concepts[7].level	2
concepts[7].score	0.6173864603042603
concepts[7].wikidata	https://www.wikidata.org/wiki/Q5127848
concepts[7].display_name	Class (philosophy)
concepts[8].id	https://openalex.org/C154945302
concepts[8].level	1
concepts[8].score	0.6119158864021301
concepts[8].wikidata	https://www.wikidata.org/wiki/Q11660
concepts[8].display_name	Artificial intelligence
concepts[9].id	https://openalex.org/C22019652
concepts[9].level	3
concepts[9].score	0.5655351877212524
concepts[9].wikidata	https://www.wikidata.org/wiki/Q331309
concepts[9].display_name	Overfitting
concepts[10].id	https://openalex.org/C12713177
concepts[10].level	2
concepts[10].score	0.4541834890842438
concepts[10].wikidata	https://www.wikidata.org/wiki/Q1900281
concepts[10].display_name	Perspective (graphical)
concepts[11].id	https://openalex.org/C50644808
concepts[11].level	2
concepts[11].score	0.38118696212768555
concepts[11].wikidata	https://www.wikidata.org/wiki/Q192776
concepts[11].display_name	Artificial neural network
concepts[12].id	https://openalex.org/C124101348
concepts[12].level	1
concepts[12].score	0.35557907819747925
concepts[12].wikidata	https://www.wikidata.org/wiki/Q172491
concepts[12].display_name	Data mining
concepts[13].id	https://openalex.org/C33923547
concepts[13].level	0
concepts[13].score	0.14575359225273132
concepts[13].wikidata	https://www.wikidata.org/wiki/Q395
concepts[13].display_name	Mathematics
concepts[14].id	https://openalex.org/C115961682
concepts[14].level	2
concepts[14].score	0.0
concepts[14].wikidata	https://www.wikidata.org/wiki/Q860623
concepts[14].display_name	Image (mathematics)
concepts[15].id	https://openalex.org/C111919701
concepts[15].level	1
concepts[15].score	0.0
concepts[15].wikidata	https://www.wikidata.org/wiki/Q9135
concepts[15].display_name	Operating system
concepts[16].id	https://openalex.org/C31972630
concepts[16].level	1
concepts[16].score	0.0
concepts[16].wikidata	https://www.wikidata.org/wiki/Q844240
concepts[16].display_name	Computer vision
concepts[17].id	https://openalex.org/C149782125
concepts[17].level	1
concepts[17].score	0.0
concepts[17].wikidata	https://www.wikidata.org/wiki/Q160039
concepts[17].display_name	Econometrics
concepts[18].id	https://openalex.org/C205649164
concepts[18].level	0
concepts[18].score	0.0
concepts[18].wikidata	https://www.wikidata.org/wiki/Q1071
concepts[18].display_name	Geography
concepts[19].id	https://openalex.org/C13280743
concepts[19].level	1
concepts[19].score	0.0
concepts[19].wikidata	https://www.wikidata.org/wiki/Q131089
concepts[19].display_name	Geodesy
keywords[0].id	https://openalex.org/keywords/bootstrapping
keywords[0].score	0.7385431528091431
keywords[0].display_name	Bootstrapping (finance)
keywords[1].id	https://openalex.org/keywords/computer-science
keywords[1].score	0.7362393140792847
keywords[1].display_name	Computer science
keywords[2].id	https://openalex.org/keywords/noise
keywords[2].score	0.7077111601829529
keywords[2].display_name	Noise (video)
keywords[3].id	https://openalex.org/keywords/machine-learning
keywords[3].score	0.6922382116317749
keywords[3].display_name	Machine learning
keywords[4].id	https://openalex.org/keywords/smoothing
keywords[4].score	0.665520966053009
keywords[4].display_name	Smoothing
keywords[5].id	https://openalex.org/keywords/heuristics
keywords[5].score	0.6635363101959229
keywords[5].display_name	Heuristics
keywords[6].id	https://openalex.org/keywords/benchmark
keywords[6].score	0.6214388012886047
keywords[6].display_name	Benchmark (surveying)
keywords[7].id	https://openalex.org/keywords/class
keywords[7].score	0.6173864603042603
keywords[7].display_name	Class (philosophy)
keywords[8].id	https://openalex.org/keywords/artificial-intelligence
keywords[8].score	0.6119158864021301
keywords[8].display_name	Artificial intelligence
keywords[9].id	https://openalex.org/keywords/overfitting
keywords[9].score	0.5655351877212524
keywords[9].display_name	Overfitting
keywords[10].id	https://openalex.org/keywords/perspective
keywords[10].score	0.4541834890842438
keywords[10].display_name	Perspective (graphical)
keywords[11].id	https://openalex.org/keywords/artificial-neural-network
keywords[11].score	0.38118696212768555
keywords[11].display_name	Artificial neural network
keywords[12].id	https://openalex.org/keywords/data-mining
keywords[12].score	0.35557907819747925
keywords[12].display_name	Data mining
keywords[13].id	https://openalex.org/keywords/mathematics
keywords[13].score	0.14575359225273132
keywords[13].display_name	Mathematics
language	en
locations[0].id	pmh:oai:arXiv.org:2111.05428
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2111.05428
locations[0].version	submittedVersion
locations[0].raw_type
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2111.05428
indexed_in	arxiv
authorships[0].author.id	https://openalex.org/A5100403739
authorships[0].author.orcid	https://orcid.org/0000-0002-8848-2889
authorships[0].author.display_name	Abhishek Kumar
authorships[0].author_position	first
authorships[0].raw_author_name	Kumar, Abhishek
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5056776503
authorships[1].author.orcid	https://orcid.org/0000-0001-6097-0226
authorships[1].author.display_name	Ehsan Amid
authorships[1].author_position	last
authorships[1].raw_author_name	Amid, Ehsan
authorships[1].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2111.05428
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Constrained Instance and Class Reweighting for Robust Learning under\n Label Noise
has_fulltext	False
is_retracted	False
updated_date	2025-10-10T17:16:08.811792
primary_topic.id	https://openalex.org/T12535
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9940000176429749
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1702
primary_topic.subfield.display_name	Artificial Intelligence
primary_topic.display_name	Machine Learning and Data Classification
related_works	https://openalex.org/W4362597605, https://openalex.org/W1574414179, https://openalex.org/W3009056573, https://openalex.org/W4297676672, https://openalex.org/W2922073769, https://openalex.org/W4281702477, https://openalex.org/W2490526372, https://openalex.org/W4376166922, https://openalex.org/W2913541211, https://openalex.org/W2020510022
cited_by_count	6
counts_by_year[0].year	2023
counts_by_year[0].cited_by_count	4
counts_by_year[1].year	2022
counts_by_year[1].cited_by_count	2
locations_count	1
best_oa_location.id	pmh:oai:arXiv.org:2111.05428
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2111.05428
best_oa_location.version	submittedVersion
best_oa_location.raw_type
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2111.05428
primary_location.id	pmh:oai:arXiv.org:2111.05428
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2111.05428
primary_location.version	submittedVersion
primary_location.raw_type
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2111.05428
publication_date	2021-11-09
publication_year	2021
referenced_works_count	0
abstract_inverted_index.a	42
abstract_inverted_index.We	40, 118
abstract_inverted_index.as	115
abstract_inverted_index.by	10, 63
abstract_inverted_index.in	7, 35, 130
abstract_inverted_index.is	23
abstract_inverted_index.of	29, 38, 51, 66, 90
abstract_inverted_index.on	26, 106, 122
abstract_inverted_index.to	13, 16
abstract_inverted_index.Our	60, 99
abstract_inverted_index.The	79
abstract_inverted_index.aim	50
abstract_inverted_index.and	32, 57, 92, 125
abstract_inverted_index.fit	14
abstract_inverted_index.for	44, 75, 111
abstract_inverted_index.our	120
abstract_inverted_index.per	84
abstract_inverted_index.the	17, 27, 30, 36, 49, 88, 96, 131
abstract_inverted_index.Deep	0
abstract_inverted_index.also	102
abstract_inverted_index.full	97
abstract_inverted_index.have	3
abstract_inverted_index.need	89
abstract_inverted_index.over	95
abstract_inverted_index.that	70
abstract_inverted_index.well	15
abstract_inverted_index.with	48
abstract_inverted_index.(such	114
abstract_inverted_index.class	58
abstract_inverted_index.gains	129
abstract_inverted_index.label	46, 108, 116, 133
abstract_inverted_index.noise	47, 113
abstract_inverted_index.often	33
abstract_inverted_index.shown	4
abstract_inverted_index.their	11, 21
abstract_inverted_index.these	76
abstract_inverted_index.which	86
abstract_inverted_index.works	62
abstract_inverted_index.yield	71
abstract_inverted_index.closed	73
abstract_inverted_index.method	61, 121
abstract_inverted_index.neural	1
abstract_inverted_index.noise.	39
abstract_inverted_index.simple	72
abstract_inverted_index.ability	12
abstract_inverted_index.enabled	9
abstract_inverted_index.labels.	59
abstract_inverted_index.largely	24
abstract_inverted_index.observe	126
abstract_inverted_index.propose	41
abstract_inverted_index.quality	28
abstract_inverted_index.storing	91
abstract_inverted_index.a\nclass	65
abstract_inverted_index.dataset.	98
abstract_inverted_index.datasets	124
abstract_inverted_index.degrades	34
abstract_inverted_index.evaluate	119
abstract_inverted_index.existing	107
abstract_inverted_index.networks	2
abstract_inverted_index.noise.\n	134
abstract_inverted_index.obviates	87
abstract_inverted_index.presence	37
abstract_inverted_index.problems	69, 82
abstract_inverted_index.proposed	80
abstract_inverted_index.provided	18
abstract_inverted_index.provides	103
abstract_inverted_index.tackling	45
abstract_inverted_index.training	19
abstract_inverted_index.updating	93
abstract_inverted_index.weights.	78
abstract_inverted_index.assigning	52
abstract_inverted_index.dependent	25
abstract_inverted_index.framework	101
abstract_inverted_index.instances	56
abstract_inverted_index.smoothing	109
abstract_inverted_index.heuristics	110
abstract_inverted_index.importance	53, 77
abstract_inverted_index.impressive	5
abstract_inverted_index.individual	55
abstract_inverted_index.mini-batch	85
abstract_inverted_index.are\nsolved	83
abstract_inverted_index.constrained	67
abstract_inverted_index.formulating	64
abstract_inverted_index.performance	6, 22, 128
abstract_inverted_index.perspective	105
abstract_inverted_index.weights\nto	54
abstract_inverted_index.considerable	127
abstract_inverted_index.optimization	68, 81, 100
abstract_inverted_index.presence\nof	132
abstract_inverted_index.the\nweights	94
abstract_inverted_index.form\nupdates	74
abstract_inverted_index.a\ntheoretical	104
abstract_inverted_index.training\ndata	31
abstract_inverted_index.bootstrapping).	117
abstract_inverted_index.data.\nHowever,	20
abstract_inverted_index.addressing\nlabel	112
abstract_inverted_index.several\nbenchmark	123
abstract_inverted_index.principled\napproach	43
abstract_inverted_index.supervised\nlearning,	8
cited_by_percentile_year.max	97
cited_by_percentile_year.min	94
countries_distinct_count	0
institutions_distinct_count	2
citation_normalized_percentile.value	0.79792482
citation_normalized_percentile.is_in_top_1_percent	False
citation_normalized_percentile.is_in_top_10_percent	False