Optimal Variable Clustering for High-Dimensional Matrix Valued Data Article Swipe

MAKE IMPACT

Inbeom Lee , Siyi Deng , Yang Ning · YOU? · · 2021 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2112.12909

Matrix valued data has become increasingly prevalent in many applications. Most of the existing clustering methods for this type of data are tailored to the mean model and do not account for the dependence structure of the features, which can be very informative, especially in high-dimensional settings or when mean information is not available. To extract the information from the dependence structure for clustering, we propose a new latent variable model for the features arranged in matrix form, with some unknown membership matrices representing the clusters for the rows and columns. Under this model, we further propose a class of hierarchical clustering algorithms using the difference of a weighted covariance matrix as the dissimilarity measure. Theoretically, we show that under mild conditions, our algorithm attains clustering consistency in the high-dimensional setting. While this consistency result holds for our algorithm with a broad class of weighted covariance matrices, the conditions for this result depend on the choice of the weight. To investigate how the weight affects the theoretical performance of our algorithm, we establish the minimax lower bound for clustering under our latent variable model in terms of some cluster separation metric. Given these results, we identify the optimal weight in the sense that using this weight guarantees our algorithm to be minimax rate-optimal. The practical implementation of our algorithm with the optimal weight is also discussed. Simulation studies show that our algorithm performs better than existing methods in terms of the adjusted Rand index (ARI). The method is applied to a genomic dataset and yields meaningful interpretations.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2112.12909

PDF

https://arxiv.org/pdf/2112.12909

OA Status

green

Cited By

1

Related Works

10

OpenAlex ID

https://openalex.org/W4200630292

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4200630292

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2112.12909

Digital Object Identifier

Title

Optimal Variable Clustering for High-Dimensional Matrix Valued Data

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2021

Year of publication

Publication date

2021-12-24

Full publication date if available

Authors

Inbeom Lee, Siyi Deng, Yang Ning

List of authors in order

Landing page

https://arxiv.org/abs/2112.12909

Publisher landing page

PDF URL

https://arxiv.org/pdf/2112.12909

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2112.12909

Direct OA link when available

Concepts

Cluster analysis, Minimax, Mathematics, Covariance, Covariance matrix, Matrix (chemical analysis), Consistency (knowledge bases), k-medians clustering, Metric (unit), Algorithm, CURE data clustering algorithm, Variable (mathematics), Latent class model, Correlation clustering, Computer science, Data mining, Mathematical optimization, Artificial intelligence, Statistics, Operations management, Economics, Materials science, Mathematical analysis, Composite material

Top concepts (fields/topics) attached by OpenAlex

Cited by

1

Total citation count in OpenAlex

Citations by year (recent)

2023: 1

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload