DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation Article Swipe

MAKE IMPACT

Lukas Hoyer , Dengxin Dai , Luc Van Gool · YOU? · · 2021 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2111.14887

As acquiring pixel-wise annotations of real-world images for semantic segmentation is a costly process, a model can instead be trained with more accessible synthetic data and adapted to real images without requiring their annotations. This process is studied in unsupervised domain adaptation (UDA). Even though a large number of methods propose new adaptation strategies, they are mostly based on outdated network architectures. As the influence of recent network architectures has not been systematically studied, we first benchmark different network architectures for UDA and newly reveal the potential of Transformers for UDA semantic segmentation. Based on the findings, we propose a novel UDA method, DAFormer. The network architecture of DAFormer consists of a Transformer encoder and a multi-level context-aware feature fusion decoder. It is enabled by three simple but crucial training strategies to stabilize the training and to avoid overfitting to the source domain: While (1) Rare Class Sampling on the source domain improves the quality of the pseudo-labels by mitigating the confirmation bias of self-training toward common classes, (2) a Thing-Class ImageNet Feature Distance and (3) a learning rate warmup promote feature transfer from ImageNet pretraining. DAFormer represents a major advance in UDA. It improves the state of the art by 10.8 mIoU for GTA-to-Cityscapes and 5.4 mIoU for Synthia-to-Cityscapes and enables learning even difficult classes such as train, bus, and truck well. The implementation is available at https://github.com/lhoyer/DAFormer.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2111.14887

PDF

https://arxiv.org/pdf/2111.14887

OA Status

green

Cited By

1

Related Works

10

OpenAlex ID

https://openalex.org/W4226229973

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4226229973

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2111.14887

Digital Object Identifier

Title

DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2021

Year of publication

Publication date

2021-11-29

Full publication date if available

Authors

Lukas Hoyer, Dengxin Dai, Luc Van Gool

List of authors in order

Landing page

https://arxiv.org/abs/2111.14887

Publisher landing page

PDF URL

https://arxiv.org/pdf/2111.14887

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2111.14887

Direct OA link when available

Concepts

Computer science, Artificial intelligence, Segmentation, Overfitting, Transformer, Feature (linguistics), Feature learning, Machine learning, Network architecture, Encoder, Artificial neural network, Pattern recognition (psychology), Engineering, Computer network, Electrical engineering, Philosophy, Linguistics, Operating system, Voltage

Top concepts (fields/topics) attached by OpenAlex

Cited by

1

Total citation count in OpenAlex

Citations by year (recent)

2025: 1

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload