ICDARTS: Improving the Stability of Cyclic DARTS

Cyclic DARTS (CDARTS) is a Differentiable Architecture Search (DARTS)-based approach to neural architecture search (NAS) that uses a cyclic feedback mechanism to train search and evaluation networks concurrently. This training protocol aims to optimize the search process and evaluate the deep evalua...

Full description

Saved in:
Bibliographic Details
Published in2022 21st IEEE International Conference on Machine Learning and Applications (ICMLA) pp. 1055 - 1062
Main Authors Herron, Emily, Young, Steven R., Rose, Derek
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.12.2022
Subjects
AutoML
Deep Learning
Machine learning
Neural Architecture Search
Neural Networks
Protocols
Training
Online AccessGet full text

Cover

More Information
Summary:Cyclic DARTS (CDARTS) is a Differentiable Architecture Search (DARTS)-based approach to neural architecture search (NAS) that uses a cyclic feedback mechanism to train search and evaluation networks concurrently. This training protocol aims to optimize the search process and evaluate the deep evaluation network comprised of discretized candidate operations. However, this approach introduces a loss function for the evaluation network dependent on the search network. The dissimilarity between the evaluation network's loss function used during the search and retraining phases results in a search network that is a sub-optimal proxy for the final evaluation network accessed during retraining. We present a revised approach that removes the dependency of the evaluation network weights upon those of the search network. In addition, we introduce a modified process for relaxing the search network's zero operations that allows these operations to be retained in the final evaluation networks.
DOI:10.1109/ICMLA55696.2022.00175