Fast optimization of common basis for matrix set through Common Singular Value Decomposition Article Swipe

MAKE IMPACT

Jarek Duda · YOU? · · 2022 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2204.08242

SVD (singular value decomposition) is one of the basic tools of machine learning, allowing to optimize basis for a given matrix. However, sometimes we have a set of matrices $\{A_k\}_k$ instead, and would like to optimize a single common basis for them: find orthogonal matrices $U$, $V$, such that $\{U^T A_k V\}$ set of matrices is somehow simpler. For example DCT-II is orthonormal basis of functions commonly used in image/video compression - as discussed here, this kind of basis can be quickly automatically optimized for a given dataset. While also discussed gradient descent optimization might be computationally costly, there is proposed CSVD (common SVD): fast general approach based on SVD. Specifically, we choose $U$ as built of eigenvectors of $\sum_i (w_k)^q (A_k A_k^T)^p$ and $V$ of $\sum_k (w_k)^q (A_k^T A_k)^p$, where $w_k$ are their weights, $p,q>0$ are some chosen powers e.g. 1/2, optionally with normalization e.g. $A \to A - rc^T$ where $r_i=\sum_j A_{ij}, c_j =\sum_i A_{ij}$.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2204.08242

PDF

https://arxiv.org/pdf/2204.08242

OA Status

green

Related Works

10

OpenAlex ID

https://openalex.org/W4224253531

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4224253531

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2204.08242

Digital Object Identifier

Title

Fast optimization of common basis for matrix set through Common Singular Value Decomposition

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2022

Year of publication

Publication date

2022-04-18

Full publication date if available

Authors

Jarek Duda

List of authors in order

Landing page

https://arxiv.org/abs/2204.08242

Publisher landing page

PDF URL

https://arxiv.org/pdf/2204.08242

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2204.08242

Direct OA link when available

Concepts

Singular value decomposition, Orthonormal basis, Basis (linear algebra), Eigenvalues and eigenvectors, Normalization (sociology), Combinatorics, Orthogonal matrix, Mathematics, Matrix (chemical analysis), Singular value, Set (abstract data type), Gradient descent, Algorithm, Basis function, Value (mathematics), Orthogonal basis, Discrete mathematics, Computer science, Mathematical analysis, Artificial intelligence, Geometry, Physics, Statistics, Artificial neural network, Sociology, Materials science, Quantum mechanics, Composite material, Programming language, Anthropology

Top concepts (fields/topics) attached by OpenAlex

Cited by

0

Total citation count in OpenAlex

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload