DCP-NAS: Discrepant Child-Parent Neural Architecture Search for 1-bit CNNs Article Swipe

MAKE IMPACT

Yanjing Li , Sheng Xu , Xianbin Cao , Li’an Zhuo , Baochang Zhang , Tian Wang , Guodong Guo · YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2306.15390

Neural architecture search (NAS) proves to be among the effective approaches for many tasks by generating an application-adaptive neural architecture, which is still challenged by high computational cost and memory consumption. At the same time, 1-bit convolutional neural networks (CNNs) with binary weights and activations show their potential for resource-limited embedded devices. One natural approach is to use 1-bit CNNs to reduce the computation and memory cost of NAS by taking advantage of the strengths of each in a unified framework, while searching the 1-bit CNNs is more challenging due to the more complicated processes involved. In this paper, we introduce Discrepant Child-Parent Neural Architecture Search (DCP-NAS) to efficiently search 1-bit CNNs, based on a new framework of searching the 1-bit model (Child) under the supervision of a real-valued model (Parent). Particularly, we first utilize a Parent model to calculate a tangent direction, based on which the tangent propagation method is introduced to search the optimized 1-bit Child. We further observe a coupling relationship between the weights and architecture parameters existing in such differentiable frameworks. To address the issue, we propose a decoupled optimization method to search an optimized architecture. Extensive experiments demonstrate that our DCP-NAS achieves much better results than prior arts on both CIFAR-10 and ImageNet datasets. In particular, the backbones achieved by our DCP-NAS achieve strong generalization performance on person re-identification and object detection.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2306.15390

PDF

https://arxiv.org/pdf/2306.15390

OA Status

green

Cited By

1

Related Works

10

OpenAlex ID

https://openalex.org/W4382491647

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4382491647

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2306.15390

Digital Object Identifier

Title

DCP-NAS: Discrepant Child-Parent Neural Architecture Search for 1-bit CNNs

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2023

Year of publication

Publication date

2023-06-27

Full publication date if available

Authors

Yanjing Li, Sheng Xu, Xianbin Cao, Li’an Zhuo, Baochang Zhang, Tian Wang, Guodong Guo

List of authors in order

Landing page

https://arxiv.org/abs/2306.15390

Publisher landing page

PDF URL

https://arxiv.org/pdf/2306.15390

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2306.15390

Direct OA link when available

Concepts

Computer science, Convolutional neural network, Generalization, Artificial neural network, Architecture, Artificial intelligence, Binary number, Theoretical computer science, Machine learning, Computer engineering, Arithmetic, Mathematics, Art, Visual arts, Mathematical analysis

Top concepts (fields/topics) attached by OpenAlex

Cited by

1

Total citation count in OpenAlex

Citations by year (recent)

2024: 1

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload