Stacked BNAS: Rethinking Broad Convolutional Neural Network for Neural Architecture Search Article Swipe

MAKE IMPACT

Zixiang Ding , Yaran Chen , Nannan Li , Dongbin Zhao · YOU? · · 2021 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2111.07722

Different from other deep scalable architecture-based NAS approaches, Broad Neural Architecture Search (BNAS) proposes a broad scalable architecture which consists of convolution and enhancement blocks, dubbed Broad Convolutional Neural Network (BCNN), as the search space for amazing efficiency improvement. BCNN reuses the topologies of cells in the convolution block so that BNAS can employ few cells for efficient search. Moreover, multi-scale feature fusion and knowledge embedding are proposed to improve the performance of BCNN with shallow topology. However, BNAS suffers some drawbacks: 1) insufficient representation diversity for feature fusion and enhancement and 2) time consumption of knowledge embedding design by human experts. This paper proposes Stacked BNAS, whose search space is a developed broad scalable architecture named Stacked BCNN, with better performance than BNAS. On the one hand, Stacked BCNN treats mini BCNN as a basic block to preserve comprehensive representation and deliver powerful feature extraction ability. For multi-scale feature enhancement, each mini BCNN feeds the outputs of deep and broad cells to the enhancement cell. For multi-scale feature fusion, each mini BCNN feeds the outputs of deep, broad and enhancement cells to the output node. On the other hand, Knowledge Embedding Search (KES) is proposed to learn appropriate knowledge embeddings in a differentiable way. Moreover, the basic unit of KES is an over-parameterized knowledge embedding module that consists of all possible candidate knowledge embeddings. Experimental results show that 1) Stacked BNAS obtains better performance than BNAS-v2 on both CIFAR-10 and ImageNet, 2) the proposed KES algorithm contributes to reducing the parameters of the learned architecture with satisfactory performance, and 3) Stacked BNAS delivers a state-of-the-art efficiency of 0.02 GPU days.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2111.07722

PDF

https://arxiv.org/pdf/2111.07722

OA Status

green

Cited By

1

Related Works

10

OpenAlex ID

https://openalex.org/W4226530147

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4226530147

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2111.07722

Digital Object Identifier

Title

Stacked BNAS: Rethinking Broad Convolutional Neural Network for Neural Architecture Search

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2021

Year of publication

Publication date

2021-11-15

Full publication date if available

Authors

Zixiang Ding, Yaran Chen, Nannan Li, Dongbin Zhao

List of authors in order

Landing page

https://arxiv.org/abs/2111.07722

Publisher landing page

PDF URL

https://arxiv.org/pdf/2111.07722

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2111.07722

Direct OA link when available

Concepts

Computer science, Convolutional neural network, Scalability, Embedding, Block (permutation group theory), Feature (linguistics), Artificial intelligence, Convolution (computer science), Pattern recognition (psychology), Artificial neural network, Mathematics, Philosophy, Geometry, Linguistics, Database

Top concepts (fields/topics) attached by OpenAlex

Cited by

1

Total citation count in OpenAlex

Citations by year (recent)

2023: 1

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload