SwiftNet: Using Graph Propagation as Meta-knowledge to Search Highly Representative Neural Architectures Article Swipe

MAKE IMPACT

Hsin-Pai Cheng , Tunhou Zhang , Yukun Yang , Feng Yan , Shiyu Li , Harris Teague , Hai Li , Yiran Chen · YOU? · · 2019 · Open Access · · DOI: https://doi.org/10.48550/arxiv.1906.08305

Designing neural architectures for edge devices is subject to constraints of accuracy, inference latency, and computational cost. Traditionally, researchers manually craft deep neural networks to meet the needs of mobile devices. Neural Architecture Search (NAS) was proposed to automate the neural architecture design without requiring extensive domain expertise and significant manual efforts. Recent works utilized NAS to design mobile models by taking into account hardware constraints and achieved state-of-the-art accuracy with fewer parameters and less computational cost measured in Multiply-accumulates (MACs). To find highly compact neural architectures, existing works relies on predefined cells and directly applying width multiplier, which may potentially limit the model flexibility, reduce the useful feature map information, and cause accuracy drop. To conquer this issue, we propose GRAM(GRAph propagation as Meta-knowledge) that adopts fine-grained (node-wise) search method and accumulates the knowledge learned in updates into a meta-graph. As a result, GRAM can enable more flexible search space and achieve higher search efficiency. Without the constraints of predefined cell or blocks, we propose a new structure-level pruning method to remove redundant operations in neural architectures. SwiftNet, which is a set of models discovered by GRAM, outperforms MobileNet-V2 by 2.15x higher accuracy density and 2.42x faster with similar accuracy. Compared with FBNet, SwiftNet reduces the search cost by 26x and achieves 2.35x higher accuracy density and 1.47x speedup while preserving similar accuracy. SwiftNetcan obtain 63.28% top-1 accuracy on ImageNet-1K with only 53M MACs and 2.07M parameters. The corresponding inference latency is only 19.09 ms on Google Pixel 1.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/1906.08305

PDF

https://arxiv.org/pdf/1906.08305

OA Status

green

Cited By

8

References

36

Related Works

10

OpenAlex ID

https://openalex.org/W2951427637

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W2951427637

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.1906.08305

Digital Object Identifier

Title

SwiftNet: Using Graph Propagation as Meta-knowledge to Search Highly Representative Neural Architectures

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2019

Year of publication

Publication date

2019-06-19

Full publication date if available

Authors

Hsin-Pai Cheng, Tunhou Zhang, Yukun Yang, Feng Yan, Shiyu Li, Harris Teague, Hai Li, Yiran Chen

List of authors in order

Landing page

https://arxiv.org/abs/1906.08305

Publisher landing page

PDF URL

https://arxiv.org/pdf/1906.08305

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/1906.08305

Direct OA link when available

Concepts

Computer science, Knowledge graph, Graph, Artificial neural network, Artificial intelligence, Theoretical computer science

Top concepts (fields/topics) attached by OpenAlex

Cited by

8

Total citation count in OpenAlex

Citations by year (recent)

2023: 1, 2021: 3, 2020: 3, 2019: 1

Per-year citation counts (last 5 years)

References (count)

36

Number of works referenced by this work

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload