On unsupervised projections and second order signals Article Swipe

MAKE IMPACT

Thomas Lartigue , Sach Mukherjee · YOU? · · 2022 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2204.05139

Linear projections are widely used in the analysis of high-dimensional data. In unsupervised settings where the data harbour latent classes/clusters, the question of whether class discriminatory signals are retained under projection is crucial. In the case of mean differences between classes, this question has been well studied. However, in many contemporary applications, notably in biomedicine, group differences at the level of covariance or graphical model structure are important. Motivated by such applications, in this paper we ask whether linear projections can preserve differences in second order structure between latent groups. We focus on unsupervised projections, which can be computed without knowledge of class labels. We discuss a simple theoretical framework to study the behaviour of such projections which we use to inform an analysis via quasi-exhaustive enumeration. This allows us to consider the performance, over more than a hundred thousand sets of data-generating population parameters, of two popular projections, namely random projections (RP) and Principal Component Analysis (PCA). Across this broad range of regimes, PCA turns out to be more effective at retaining second order signals than RP and is often even competitive with supervised projection. We complement these results with fully empirical experiments showing 0-1 loss using simulated and real data. We study also the effect of projection dimension, drawing attention to a bias-variance trade-off in this respect. Our results show that PCA can indeed be a suitable first-step for unsupervised analysis, including in cases where differential covariance or graphical model structure are of interest.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2204.05139

PDF

https://arxiv.org/pdf/2204.05139

OA Status

green

Related Works

10

OpenAlex ID

https://openalex.org/W4223630185

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4223630185

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2204.05139

Digital Object Identifier

Title

On unsupervised projections and second order signals

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2022

Year of publication

Publication date

2022-04-11

Full publication date if available

Authors

Thomas Lartigue, Sach Mukherjee

List of authors in order

Landing page

https://arxiv.org/abs/2204.05139

Publisher landing page

PDF URL

https://arxiv.org/pdf/2204.05139

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2204.05139

Direct OA link when available

Concepts

Principal component analysis, Projection (relational algebra), Covariance, Artificial intelligence, Pattern recognition (psychology), Complement (music), Computer science, Dimension (graph theory), Mathematics, Population, Machine learning, Statistics, Algorithm, Gene, Pure mathematics, Sociology, Chemistry, Biochemistry, Complementation, Demography, Phenotype

Top concepts (fields/topics) attached by OpenAlex

Cited by

0

Total citation count in OpenAlex

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload