Multiview Transformer: Rethinking Spatial Information in Hyperspectral Image Classification Article Swipe

MAKE IMPACT

Jie Zhang , Yongshan Zhang , Yicong Zhou · YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2310.07186

Identifying the land cover category for each pixel in a hyperspectral image (HSI) relies on spectral and spatial information. An HSI cuboid with a specific patch size is utilized to extract spatial-spectral feature representation for the central pixel. In this article, we investigate that scene-specific but not essential correlations may be recorded in an HSI cuboid. This additional information improves the model performance on existing HSI datasets and makes it hard to properly evaluate the ability of a model. We refer to this problem as the spatial overfitting issue and utilize strict experimental settings to avoid it. We further propose a multiview transformer for HSI classification, which consists of multiview principal component analysis (MPCA), spectral encoder-decoder (SED), and spatial-pooling tokenization transformer (SPTT). MPCA performs dimension reduction on an HSI via constructing spectral multiview observations and applying PCA on each view data to extract low-dimensional view representation. The combination of view representations, named multiview representation, is the dimension reduction output of the MPCA. To aggregate the multiview information, a fully-convolutional SED with a U-shape in spectral dimension is introduced to extract a multiview feature map. SPTT transforms the multiview features into tokens using the spatial-pooling tokenization strategy and learns robust and discriminative spatial-spectral features for land cover identification. Classification is conducted with a linear classifier. Experiments on three HSI datasets with rigid settings demonstrate the superiority of the proposed multiview transformer over the state-of-the-art methods.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2310.07186

PDF

https://arxiv.org/pdf/2310.07186

OA Status

green

Related Works

10

OpenAlex ID

https://openalex.org/W4387623393

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4387623393

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2310.07186

Digital Object Identifier

Title

Multiview Transformer: Rethinking Spatial Information in Hyperspectral Image Classification

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2023

Year of publication

Publication date

2023-10-11

Full publication date if available

Authors

Jie Zhang, Yongshan Zhang, Yicong Zhou

List of authors in order

Landing page

https://arxiv.org/abs/2310.07186

Publisher landing page

PDF URL

https://arxiv.org/pdf/2310.07186

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2310.07186

Direct OA link when available

Concepts

Artificial intelligence, Hyperspectral imaging, Computer science, Pattern recognition (psychology), Dimensionality reduction, Spatial analysis, Pixel, Feature extraction, Computer vision, Discriminative model, Remote sensing, Geography

Top concepts (fields/topics) attached by OpenAlex

Cited by

0

Total citation count in OpenAlex

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload