Selective classification using a robust meta-learning approach Article Swipe

MAKE IMPACT

Nishant Jain , Pradeep Shenoy · YOU? · · 2022 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2212.05987

Predictive uncertainty-a model's self awareness regarding its accuracy on an input-is key for both building robust models via training interventions and for test-time applications such as selective classification. We propose a novel instance-conditioned reweighting approach that captures predictive uncertainty using an auxiliary network and unifies these train- and test-time applications. The auxiliary network is trained using a meta-objective in a bilevel optimization framework. A key contribution of our proposal is the meta-objective of minimizing the dropout variance, an approximation of Bayesian Predictive uncertainty. We show in controlled experiments that we effectively capture the diverse specific notions of uncertainty through this meta-objective, while previous approaches only capture certain aspects. These results translate to significant gains in real-world settings-selective classification, label noise, domain adaptation, calibration-and across datasets-Imagenet, Cifar100, diabetic retinopathy, Camelyon, WILDs, Imagenet-C,-A,-R, Clothing1M, etc. For Diabetic Retinopathy, we see upto 3.4%/3.3% accuracy and AUC gains over SOTA in selective classification. We also improve upon large-scale pretrained models such as PLEX.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2212.05987

PDF

https://arxiv.org/pdf/2212.05987

OA Status

green

Cited By

1

Related Works

10

OpenAlex ID

https://openalex.org/W4311431950

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4311431950

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2212.05987

Digital Object Identifier

Title

Selective classification using a robust meta-learning approach

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2022

Year of publication

Publication date

2022-12-12

Full publication date if available

Authors

Nishant Jain, Pradeep Shenoy

List of authors in order

Landing page

https://arxiv.org/abs/2212.05987

Publisher landing page

PDF URL

https://arxiv.org/pdf/2212.05987

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2212.05987

Direct OA link when available

Concepts

Computer science, Machine learning, Artificial intelligence, Bayesian optimization, Key (lock), Dropout (neural networks), Bayesian probability, Calibration, Mathematics, Statistics, Computer security

Top concepts (fields/topics) attached by OpenAlex

Cited by

1

Total citation count in OpenAlex

Citations by year (recent)

2023: 1

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload