AutoSpace: Neural Architecture Search with Less Human Interference

• Published in: 2021 IEEE/CVF International Conference on Computer Vision (ICCV) pp. 327 - 336
• Main Authors: Zhou, Daquan, Jin, Xiaojie, Lian, Xiaochen, Yang, Linjie, Xue, Yujing, Hou, Qibin, Feng, Jiashi
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.10.2021
• Subjects: Computational modeling; Computer architecture; Costs; Efficient training and inference methods; Evolutionary computation; Interference; Knowledge engineering; Machine learning architectures and formulations; Manuals; Recognition and classification
• ISSN: 2380-7504
• DOI: 10.1109/ICCV48922.2021.00039

Current neural architecture search (NAS) algorithms still require expert knowledge and effort to design a search space for network construction. In this paper, we consider automating the search space design to minimize human interference, which however faces two challenges: the ex-plosive complexity of the exploration space and the expensive computation cost to evaluate the quality of different search spaces. To solve them, we propose a novel differentiable evolutionary framework named AutoSpace, which evolves the search space to an optimal one with following novel techniques: a differentiable fitness scoring function to efficiently evaluate the performance of cells and a reference architecture to speedup the evolution procedure and avoid falling into sub-optimal solutions. The frame-work is generic and compatible with additional computational constraints, making it feasible to learn specialized search spaces that fit different computational bud-gets. With the learned search space, the performance of recent NAS algorithms can be improved significantly compared with using previously manually designed spaces. Remarkably, the models generated from the new search space achieve 77.8% top-1 accuracy on ImageNet under the mobile setting (MAdds 500M), outperforming previous SOTA EfficientNet-B0 by≤0.7%. https://github.com/zhoudaquan/AutoSpace.git.