Efficient Network Automatic Relevance Determination Article Swipe

MAKE IMPACT

Hongwei Zhang , Ziqi Ye , Xinyuan Wang , Xin Guo , Zenglin Xu , Yuan Cheng , Zixin Hu , Yuan Qi · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2506.12352

We propose Network Automatic Relevance Determination (NARD), an extension of ARD for linearly probabilistic models, to simultaneously model sparse relationships between inputs $X \in \mathbb R^{d \times N}$ and outputs $Y \in \mathbb R^{m \times N}$, while capturing the correlation structure among the $Y$. NARD employs a matrix normal prior which contains a sparsity-inducing parameter to identify and discard irrelevant features, thereby promoting sparsity in the model. Algorithmically, it iteratively updates both the precision matrix and the relationship between $Y$ and the refined inputs. To mitigate the computational inefficiencies of the $\mathcal O(m^3 + d^3)$ cost per iteration, we introduce Sequential NARD, which evaluates features sequentially, and a Surrogate Function Method, leveraging an efficient approximation of the marginal likelihood and simplifying the calculation of determinant and inverse of an intermediate matrix. Combining the Sequential update with the Surrogate Function method further reduces computational costs. The computational complexity per iteration for these three methods is reduced to $\mathcal O(m^3+p^3)$, $\mathcal O(m^3 + d^2)$, $\mathcal O(m^3+p^2)$, respectively, where $p \ll d$ is the final number of features in the model. Our methods demonstrate significant improvements in computational efficiency with comparable performance on both synthetic and real-world datasets.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2506.12352

PDF

https://arxiv.org/pdf/2506.12352

OA Status

green

OpenAlex ID

https://openalex.org/W4415111445

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4415111445

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2506.12352

Digital Object Identifier

Title

Efficient Network Automatic Relevance Determination

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2025

Year of publication

Publication date

2025-06-14

Full publication date if available

Authors

Hongwei Zhang, Ziqi Ye, Xinyuan Wang, Xin Guo, Zenglin Xu, Yuan Cheng, Zixin Hu, Yuan Qi

List of authors in order

Landing page

https://arxiv.org/abs/2506.12352

Publisher landing page

PDF URL

https://arxiv.org/pdf/2506.12352

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2506.12352

Direct OA link when available

Cited by

0

Total citation count in OpenAlex

Full payload