Channel and Spatial Relation-Propagation Network for RGB-Thermal Semantic Segmentation Article Swipe

MAKE IMPACT

Zikun Zhou , Shukun Wu , Zhu Guoqing , Hongpeng Wang , Zhenyu He · YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2308.12534

RGB-Thermal (RGB-T) semantic segmentation has shown great potential in handling low-light conditions where RGB-based segmentation is hindered by poor RGB imaging quality. The key to RGB-T semantic segmentation is to effectively leverage the complementarity nature of RGB and thermal images. Most existing algorithms fuse RGB and thermal information in feature space via concatenation, element-wise summation, or attention operations in either unidirectional enhancement or bidirectional aggregation manners. However, they usually overlook the modality gap between RGB and thermal images during feature fusion, resulting in modality-specific information from one modality contaminating the other. In this paper, we propose a Channel and Spatial Relation-Propagation Network (CSRPNet) for RGB-T semantic segmentation, which propagates only modality-shared information across different modalities and alleviates the modality-specific information contamination issue. Our CSRPNet first performs relation-propagation in channel and spatial dimensions to capture the modality-shared features from the RGB and thermal features. CSRPNet then aggregates the modality-shared features captured from one modality with the input feature from the other modality to enhance the input feature without the contamination issue. While being fused together, the enhanced RGB and thermal features will be also fed into the subsequent RGB or thermal feature extraction layers for interactive feature fusion, respectively. We also introduce a dual-path cascaded feature refinement module that aggregates multi-layer features to produce two refined features for semantic and boundary prediction. Extensive experimental results demonstrate that CSRPNet performs favorably against state-of-the-art algorithms.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2308.12534

PDF

https://arxiv.org/pdf/2308.12534

OA Status

green

Cited By

2

Related Works

10

OpenAlex ID

https://openalex.org/W4386185159

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4386185159

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2308.12534

Digital Object Identifier

Title

Channel and Spatial Relation-Propagation Network for RGB-Thermal Semantic Segmentation

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2023

Year of publication

Publication date

2023-08-24

Full publication date if available

Authors

Zikun Zhou, Shukun Wu, Zhu Guoqing, Hongpeng Wang, Zhenyu He

List of authors in order

Landing page

https://arxiv.org/abs/2308.12534

Publisher landing page

PDF URL

https://arxiv.org/pdf/2308.12534

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2308.12534

Direct OA link when available

Concepts

RGB color model, Artificial intelligence, Computer science, Segmentation, Feature (linguistics), Computer vision, Modality (human–computer interaction), Pattern recognition (psychology), Channel (broadcasting), Computer network, Philosophy, Linguistics

Top concepts (fields/topics) attached by OpenAlex

Cited by

2

Total citation count in OpenAlex

Citations by year (recent)

2024: 2

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload