Feature Preserving Shrinkage on Bayesian Neural Networks via the R2D2 Prior Article Swipe

MAKE IMPACT

Tsai Hor Chan , Ding Zhang , Guosheng Yin , Lequan Yu · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2505.18280

Bayesian neural networks (BNNs) treat neural network weights as random variables, which aim to provide posterior uncertainty estimates and avoid overfitting by performing inference on the posterior weights. However, the selection of appropriate prior distributions remains a challenging task, and BNNs may suffer from catastrophic inflated variance or poor predictive performance when poor choices are made for the priors. Existing BNN designs apply different priors to weights, while the behaviours of these priors make it difficult to sufficiently shrink noisy signals or they are prone to overshrinking important signals in the weights. To alleviate this problem, we propose a novel R2D2-Net, which imposes the R^2-induced Dirichlet Decomposition (R2D2) prior to the BNN weights. The R2D2-Net can effectively shrink irrelevant coefficients towards zero, while preventing key features from over-shrinkage. To approximate the posterior distribution of weights more accurately, we further propose a variational Gibbs inference algorithm that combines the Gibbs updating procedure and gradient-based optimization. This strategy enhances stability and consistency in estimation when the variational objective involving the shrinkage parameters is non-convex. We also analyze the evidence lower bound (ELBO) and the posterior concentration rates from a theoretical perspective. Experiments on both natural and medical image classification and uncertainty estimation tasks demonstrate satisfactory performance of our method.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2505.18280

PDF

https://arxiv.org/pdf/2505.18280

OA Status

green

OpenAlex ID

https://openalex.org/W4414581395

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4414581395

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2505.18280

Digital Object Identifier

Title

Feature Preserving Shrinkage on Bayesian Neural Networks via the R2D2 Prior

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2025

Year of publication

Publication date

2025-05-23

Full publication date if available

Authors

Tsai Hor Chan, Ding Zhang, Guosheng Yin, Lequan Yu

List of authors in order

Landing page

https://arxiv.org/abs/2505.18280

Publisher landing page

PDF URL

https://arxiv.org/pdf/2505.18280

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2505.18280

Direct OA link when available

Cited by

0

Total citation count in OpenAlex

Full payload