Probabilistic Attention based on Gaussian Processes for Deep Multiple Instance Learning Article Swipe

MAKE IMPACT

Arne Schmidt , Pablo Morales-Álvarez , Rafael Molina · YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2302.04061

Multiple Instance Learning (MIL) is a weakly supervised learning paradigm that is becoming increasingly popular because it requires less labeling effort than fully supervised methods. This is especially interesting for areas where the creation of large annotated datasets remains challenging, as in medicine. Although recent deep learning MIL approaches have obtained state-of-the-art results, they are fully deterministic and do not provide uncertainty estimations for the predictions. In this work, we introduce the Attention Gaussian Process (AGP) model, a novel probabilistic attention mechanism based on Gaussian Processes for deep MIL. AGP provides accurate bag-level predictions as well as instance-level explainability, and can be trained end-to-end. Moreover, its probabilistic nature guarantees robustness to overfitting on small datasets and uncertainty estimations for the predictions. The latter is especially important in medical applications, where decisions have a direct impact on the patient's health. The proposed model is validated experimentally as follows. First, its behavior is illustrated in two synthetic MIL experiments based on the well-known MNIST and CIFAR-10 datasets, respectively. Then, it is evaluated in three different real-world cancer detection experiments. AGP outperforms state-of-the-art MIL approaches, including deterministic deep learning ones. It shows a strong performance even on a small dataset with less than 100 labels and generalizes better than competing methods on an external test set. Moreover, we experimentally show that predictive uncertainty correlates with the risk of wrong predictions, and therefore it is a good indicator of reliability in practice. Our code is publicly available.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2302.04061

PDF

https://arxiv.org/pdf/2302.04061

OA Status

green

Cited By

1

Related Works

10

OpenAlex ID

https://openalex.org/W4319793270

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4319793270

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2302.04061

Digital Object Identifier

Title

Probabilistic Attention based on Gaussian Processes for Deep Multiple Instance Learning

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2023

Year of publication

Publication date

2023-02-08

Full publication date if available

Authors

Arne Schmidt, Pablo Morales-Álvarez, Rafael Molina

List of authors in order

Landing page

https://arxiv.org/abs/2302.04061

Publisher landing page

PDF URL

https://arxiv.org/pdf/2302.04061

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2302.04061

Direct OA link when available

Concepts

Overfitting, Artificial intelligence, Computer science, MNIST database, Probabilistic logic, Machine learning, Robustness (evolution), Deep learning, Gaussian process, Gaussian, Artificial neural network, Chemistry, Gene, Quantum mechanics, Physics, Biochemistry

Top concepts (fields/topics) attached by OpenAlex

Cited by

1

Total citation count in OpenAlex

Citations by year (recent)

2024: 1

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload