Shortcut Learning Susceptibility in Vision Classifiers Article Swipe

MAKE IMPACT

Pirzada Suhail , Amit Sethi · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2502.09150

Shortcut learning, where machine learning models exploit spurious correlations in data instead of capturing meaningful features, poses a significant challenge to building robust and generalizable models. This phenomenon is prevalent across various machine learning applications, including vision, natural language processing, and speech recognition, where models may find unintended cues that minimize training loss but fail to capture the underlying structure of the data. Vision classifiers based on Convolutional Neural Networks (CNNs), Multi-Layer Perceptrons (MLPs), and Vision Transformers (ViTs) leverage distinct architectural principles to process spatial and structural information, making them differently susceptible to shortcut learning. In this study, we systematically evaluate these architectures by introducing deliberate shortcuts into the dataset that are correlated with class labels both positionally and via intensity, creating a controlled setup to assess whether models rely on these artificial cues or learn actual distinguishing features. We perform both quantitative evaluation by training on the shortcut-modified dataset and testing on two different test sets-one containing the same shortcuts and another without them-to determine the extent of reliance on shortcuts. Additionally, qualitative evaluation is performed using network inversion-based reconstruction techniques to analyze what the models internalize in their weights, aiming to reconstruct the training data as perceived by the classifiers. Further, we evaluate susceptibility to shortcut learning across different learning rates. Our analysis reveals that CNNs at lower learning rates tend to be more reserved against entirely picking up shortcut features, while ViTs, particularly those without positional encodings, almost entirely ignore the distinctive image features in the presence of shortcuts.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2502.09150

PDF

https://arxiv.org/pdf/2502.09150

OA Status

green

Related Works

10

OpenAlex ID

https://openalex.org/W4407571876

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4407571876

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2502.09150

Digital Object Identifier

Title

Shortcut Learning Susceptibility in Vision Classifiers

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2025

Year of publication

Publication date

2025-02-13

Full publication date if available

Authors

Pirzada Suhail, Amit Sethi

List of authors in order

Landing page

https://arxiv.org/abs/2502.09150

Publisher landing page

PDF URL

https://arxiv.org/pdf/2502.09150

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2502.09150

Direct OA link when available

Concepts

Artificial intelligence, Computer science, Machine learning, Computer vision

Top concepts (fields/topics) attached by OpenAlex

Cited by

0

Total citation count in OpenAlex

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload