Vision Transformer with Convolutions Architecture Search Article Swipe

MAKE IMPACT

Haichao Zhang , Kuangrong Hao , Witold Pedrycz , Lei Gao , Xue‐song Tang , Bing Wei · YOU? · · 2022 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2203.10435

Transformers exhibit great advantages in handling computer vision tasks. They model image classification tasks by utilizing a multi-head attention mechanism to process a series of patches consisting of split images. However, for complex tasks, Transformer in computer vision not only requires inheriting a bit of dynamic attention and global context, but also needs to introduce features concerning noise reduction, shifting, and scaling invariance of objects. Therefore, here we take a step forward to study the structural characteristics of Transformer and convolution and propose an architecture search method-Vision Transformer with Convolutions Architecture Search (VTCAS). The high-performance backbone network searched by VTCAS introduces the desirable features of convolutional neural networks into the Transformer architecture while maintaining the benefits of the multi-head attention mechanism. The searched block-based backbone network can extract feature maps at different scales. These features are compatible with a wider range of visual tasks, such as image classification (32 M parameters, 82.0% Top-1 accuracy on ImageNet-1K) and object detection (50.4% mAP on COCO2017). The proposed topology based on the multi-head attention mechanism and CNN adaptively associates relational features of pixels with multi-scale features of objects. It enhances the robustness of the neural network for object recognition, especially in the low illumination indoor scene.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2203.10435

PDF

https://arxiv.org/pdf/2203.10435

OA Status

green

Cited By

5

Related Works

10

OpenAlex ID

https://openalex.org/W4221167449

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4221167449

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2203.10435

Digital Object Identifier

Title

Vision Transformer with Convolutions Architecture Search

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2022

Year of publication

Publication date

2022-03-20

Full publication date if available

Authors

Haichao Zhang, Kuangrong Hao, Witold Pedrycz, Lei Gao, Xue‐song Tang, Bing Wei

List of authors in order

Landing page

https://arxiv.org/abs/2203.10435

Publisher landing page

PDF URL

https://arxiv.org/pdf/2203.10435

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2203.10435

Direct OA link when available

Concepts

Computer science, Artificial intelligence, Transformer, Convolutional neural network, Robustness (evolution), Computer vision, Architecture, Pixel, Pattern recognition (psychology), Cognitive neuroscience of visual object recognition, Feature extraction, Engineering, Biochemistry, Electrical engineering, Voltage, Visual arts, Gene, Art, Chemistry

Top concepts (fields/topics) attached by OpenAlex

Cited by

5

Total citation count in OpenAlex

Citations by year (recent)

2024: 2, 2023: 2, 2022: 1

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload