ICDARTS: Improving the Stability and Performance of Cyclic DARTS Article Swipe

MAKE IMPACT

Emily Herron , Derek Rose , Steven R. Young · YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2309.00664

This work introduces improvements to the stability and generalizability of Cyclic DARTS (CDARTS). CDARTS is a Differentiable Architecture Search (DARTS)-based approach to neural architecture search (NAS) that uses a cyclic feedback mechanism to train search and evaluation networks concurrently. This training protocol aims to optimize the search process by enforcing that the search and evaluation networks produce similar outputs. However, CDARTS introduces a loss function for the evaluation network that is dependent on the search network. The dissimilarity between the loss functions used by the evaluation networks during the search and retraining phases results in a search-phase evaluation network that is a sub-optimal proxy for the final evaluation network that is utilized during retraining. We present ICDARTS, a revised approach that eliminates the dependency of the evaluation network weights upon those of the search network, along with a modified process for discretizing the search network's \textit{zero} operations that allows these operations to be retained in the final evaluation networks. We pair the results of these changes with ablation studies on ICDARTS' algorithm and network template. Finally, we explore methods for expanding the search space of ICDARTS by expanding its operation set and exploring alternate methods for discretizing its continuous search cells. These experiments resulted in networks with improved generalizability and the implementation of a novel method for incorporating a dynamic search space into ICDARTS.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2309.00664

PDF

https://arxiv.org/pdf/2309.00664

OA Status

green

Related Works

10

OpenAlex ID

https://openalex.org/W4386494222

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4386494222

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2309.00664

Digital Object Identifier

Title

ICDARTS: Improving the Stability and Performance of Cyclic DARTS

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2023

Year of publication

Publication date

2023-09-01

Full publication date if available

Authors

Emily Herron, Derek Rose, Steven R. Young

List of authors in order

Landing page

https://arxiv.org/abs/2309.00664

Publisher landing page

PDF URL

https://arxiv.org/pdf/2309.00664

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2309.00664

Direct OA link when available

Concepts

Computer science, Network architecture, Tabu search, Search algorithm, Beam search, Process (computing), Artificial neural network, Set (abstract data type), Artificial intelligence, Machine learning, Data mining, Algorithm, Computer network, Programming language, Operating system

Top concepts (fields/topics) attached by OpenAlex

Cited by

0

Total citation count in OpenAlex

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload