Self-Supervised Multi-Modality Learning for Multi-Label Skin Lesion Classification Article Swipe

MAKE IMPACT

Hao Wang , Euijoon Ahn , Lei Bi , Jin‐Man Kim · YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2310.18583

The clinical diagnosis of skin lesion involves the analysis of dermoscopic and clinical modalities. Dermoscopic images provide a detailed view of the surface structures whereas clinical images offer a complementary macroscopic information. The visual diagnosis of melanoma is also based on seven-point checklist which involves identifying different visual attributes. Recently, supervised learning approaches such as convolutional neural networks (CNNs) have shown great performances using both dermoscopic and clinical modalities (Multi-modality). The seven different visual attributes in the checklist are also used to further improve the the diagnosis. The performances of these approaches, however, are still reliant on the availability of large-scaled labeled data. The acquisition of annotated dataset is an expensive and time-consuming task, more so with annotating multi-attributes. To overcome this limitation, we propose a self-supervised learning (SSL) algorithm for multi-modality skin lesion classification. Our algorithm enables the multi-modality learning by maximizing the similarities between paired dermoscopic and clinical images from different views. In addition, we generate surrogate pseudo-multi-labels that represent seven attributes via clustering analysis. We also propose a label-relation-aware module to refine each pseudo-label embedding and capture the interrelationships between pseudo-multi-labels. We validated the effectiveness of our algorithm using well-benchmarked seven-point skin lesion dataset. Our results show that our algorithm achieved better performances than other state-of-the-art SSL counterparts.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2310.18583

PDF

https://arxiv.org/pdf/2310.18583

OA Status

green

Cited By

1

Related Works

10

OpenAlex ID

https://openalex.org/W4388092626

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4388092626

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2310.18583

Digital Object Identifier

Title

Self-Supervised Multi-Modality Learning for Multi-Label Skin Lesion Classification

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2023

Year of publication

Publication date

2023-10-28

Full publication date if available

Authors

Hao Wang, Euijoon Ahn, Lei Bi, Jin‐Man Kim

List of authors in order

Landing page

https://arxiv.org/abs/2310.18583

Publisher landing page

PDF URL

https://arxiv.org/pdf/2310.18583

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2310.18583

Direct OA link when available

Concepts

Artificial intelligence, Modality (human–computer interaction), Computer science, Pattern recognition (psychology), Convolutional neural network, Modalities, Machine learning, Skin lesion, Medicine, Pathology, Social science, Sociology

Top concepts (fields/topics) attached by OpenAlex

Cited by

1

Total citation count in OpenAlex

Citations by year (recent)

2024: 1

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload