基于无梯度进化的神经架构搜索算法研究综述

doi:10.19678/j.issn.1000-3428.0057520

计算机工程 ›› 2020, Vol. 46 ›› Issue (9): 16-26. doi: 10.19678/j.issn.1000-3428.0057520

基于无梯度进化的神经架构搜索算法研究综述

尚迪雅¹, 孙华¹, 洪振厚², 曾庆亮¹

1. 新疆大学软件学院软件工程技术重点实验室, 乌鲁木齐 830002;
2. 深圳大学光电工程学院光电子器件与系统教育部/广东省重点实验室, 广东深圳 518060

收稿日期:2020-02-27 修回日期:2020-04-23 发布日期:2020-04-29
作者简介:尚迪雅(1995-),女,硕士研究生,主研方向为图像处理、自动化机器学习;孙华(通信作者),副教授、博士;洪振厚,硕士;曾庆亮,硕士研究生。
基金资助:
新疆维吾尔自治区自然科学基金（2015211C263）。

Review of Research on Neural Architecture Search Algorithms Based on Non-Gradient Evolution

SHANG Diya¹, SUN Hua¹, HONG Zhenhou², ZENG Qingliang¹

1. Key Laboratory of Software Engineering Technology, School of Software, Xinjiang University, Urumqi 830002, China;
2. Key Laboratory of Optoelectronic Devices and Systems of Ministry of Education and Guangdong Province, College of Optoelectronic Engineering, Shenzhen University, Shenzhen, Guangdong 518060, China

Received:2020-02-27 Revised:2020-04-23 Published:2020-04-29

摘要/Abstract

摘要： 自动化深度学习是目前深度学习领域的研究热点，神经架构搜索算法是实现自动化深度学习的主要方法之一，该类算法可以通过对搜索空间、搜索策略或优化策略进行不同定义来自动设计神经网络结构。阐述进化算法和进化神经网络的发展历程，分类介绍以进化算法为搜索策略实现神经架构搜索的方法和过程，并比较基于进化算法的不同神经架构搜索算法的特点和现状，在此基础上，对神经架构搜索算法的搜索空间、搜索策略以及算法的未来发展方向进行探讨和展望。

关键词: 神经架构搜索, 自动化深度学习, 进化算法, 搜索策略, 进化神经网络

Abstract: Automated deep learning is one of the new research hotspots in the field of deep learning.Neural architecture search algorithms are frequently used for the implementation of automated deep learning,as they can automatically design neural network structure by defining different search space,search strategy or optimization strategy.This paper introduces the development history of evolutionary algorithms and evolutionary neural networks.Then it introduces different methods and processes of using evolutionary algorithms as the search strategy to implement neural architecture search,and compares the features and development status of these neural architecture search algorithms.On this basis,this paper discusses the search space,search strategy and future development direction of neural architecture search algorithms.

Key words: neural architecture search, automated deep learning, evolutionary algorithm, search strategy, evolutionary neural network

中图分类号:

TP391

尚迪雅, 孙华, 洪振厚, 曾庆亮. 基于无梯度进化的神经架构搜索算法研究综述[J]. 计算机工程, 2020, 46(9): 16-26.

SHANG Diya, SUN Hua, HONG Zhenhou, ZENG Qingliang. Review of Research on Neural Architecture Search Algorithms Based on Non-Gradient Evolution[J]. Computer Engineering, 2020, 46(9): 16-26.

https://www.ecice06.com/CN/Y2020/V46/I9/16

图/表 10

20200915134052

20200915134055

20200915134101

20200915134104

20200915134107

20200915134111

20200915134114

20200915134118

20200915134122

20200915134125

参考文献

[1] WANG Jianzong,QU Xiaoyang.Dive into AutoML and AutoDL:building automated platforms for machine learning and deep learning[M].Beijing:China Machine Press,2019.(in Chinese)王健宗,瞿晓阳.深入理解AutoML和AutoDL:构建自动化机器学习与深度学习平台[M].北京:机械工业出版社,2019.
[2] QUANMING Y,MENGSHUO W,HUGO J E,et al.Taking human out of learning applications:a survey on automated machine learning[EB/OL].[2020-01-15].https://arxiv.org/abs/1810.13306v1.
[3] ZHOU Zhihua.Machine learning[M].Beijing:Tsinghua University Press,2016.(in Chinese)周志华.机器学习[M].北京:清华大学出版社,2016.
[4] JENNINGS N R,WOOLDRIDGE M J.Foundations of machine learning[M].Cambridge,USA:MIT Press,2012.
[5] ALPAYDIN E.Introduction to machine learning[M].Cambridge,USA:MIT Press,2004.
[6] FORSYTH D A,PONCE J.Computer vision:a modern approach[M].[S.l.]:Prentice Hall Professional Technical Reference,2002.
[7] GARCÍA S,LUENGO J,HERRERA F.Data reduction[M]//BENGIO Y,COURVILLE A.Intelligent systems reference library.Berlin,Germany:Springer,2014.
[8] HE Jun,GE Hong,WANG Yufeng.Overview of image segmentation algorithms[J].Computer Engineering and Science,2009,31(12):58-61.(in Chinese)何俊,葛红,王玉峰.图像分割算法研究综述[J].计算机工程与科学,2009,31(12):58-61.
[9] COLLOBERT R,WESTON J,BOTTOU L,et al.Natural language processing(almost) from scratch[J].Journal of Machine Learning Research,2011,12:2493-2537.
[10] LECUN Y,BENGIO Y,HINTON G.Deep learning[J].Nature,2015,521(7553):436-444.
[11] GOODFELLOW I,BENGIO Y,COURVILLE A.Deep learning[M].Cambridge,USA:MIT press,2016.
[12] SEIDE F,LI G,CHEN X,et al.Feature engineering in context-dependent deep neural networks for conver-sational speech transcription[C]//Proceedings of 2011 IEEE Workshop on Automatic Speech Recognition & Understanding.Washington D.C.,USA:IEEE Press,2011:24-29.
[13] CLAESKENS G,HJORT N L.Model selection and model averaging[M].Cambridge,UK:Cambridge University Press,2001.
[14] KHURANA U,NARGESIAN F,SAMULOWITZ H,et al.Automating feature engineering[J].Transformation,2016,18(10):1-10.
[15] SHANG D Y,SUN H,ZENG Q L.A reinforcement-algorithm framework for automatic model selection[J].IOP Conference Series:Earth and Environmental Science,2020,440:022060.
[16] OLSON R S,BARTLEY N,URBANOWICZ R J,et al.Evaluation of a tree-based pipeline optimization tool for automating data science[C]//Proceedings of 2016 Genetic and Evolutionary Computation Conference.New York,USA:ACM Press,2016:485-492.
[17] LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[18] KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks[J].Communications of the ACM,2017,60(6):84-90.
[19] SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[EB/OL].[2020-01-25].https://arxiv.org/pdf/1409.1556.pdf.
[20] SZEGEDY C,LIU W,JIA Y Q,et al.Going deeper with convolutions[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:1-9.
[21] SZEGEDY C,IOFFE S,VANHOUCKE V,et al.Inception-v4,inception-ResNet and the impact of residual connections on learning[EB/OL].[2020-01-25].https://arxiv.org/abs/1602.07261.
[22] HE K M,ZHANG X Y,REN S Q,et al.Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:770-778.
[23] ELSKEN T,METZEN J H,HUTTER F.Neural architecture search[EB/OL].[2020-01-25].https://arxiv.org/abs/1808.05377.
[24] BAKER B,GUPTA O,NAIK N,et al.Designing neural network architectures using reinforcement learning[EB/OL].[2020-01-25].https://arxiv.org/abs/1611.02167.
[25] ZOPH B,LE Q.Neural architecture search with reinforcement learning[EB/OL].[2020-01-25].https://arxiv.org/abs/1611.01578.
[26] SHIN R,PACKER C,SONG D.Differentiable neural network architecture search[EB/OL].[2020-01-25].https://openreview.net/pdf?id=BJ-MRKkwG.
[27] LIU H X,KAREN S,YIMING Y.Darts:differentiable architecture search[EB/OL].[2020-01-25].https://arxiv.org/abs/1806.09055.
[28] HOLLAND J H.Genetic algorithms and adaptation[M].Berlin,Germany:Springer,1984.
[29] BEYER H G,SCHWEFEL H.Evolution strategies-a comprehensive introduction[J].Natural Computing,2002,1(1):3-52.
[30] FOGEL D B,FOGEL L J,ATMAR J W.Meta-evolutionary programming[EB/OL].[2020-01-25].https://ieeexplore.ieee.org/document/186507.
[31] KOZA J R,POLI R.Genetic programming[M].Berlin,Germany:Springer,2005.
[32] GOMEZ F J,MIIKKULAINEN R.Active guidance for a finless rocket using neuro evolution[M].Berlin,Germany:Springer,2003.
[33] STANLEY K O,MIIKKULAINEN R.Evolving neural networks through augmenting topologies[J].Evolutionary Computation,2002,10(2):99-127.
[34] MIIKKULAINEN R,LIANG J,MEYERSON E,et al.Evolving deep neural networks[M].[S.l.]:Elsevier,2019.
[35] XIE L X,YUILLE A.Genetic CNN[C]//Proceedings of 2017 IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2017:1379-1388.
[36] MILLER J F,HARDING S L.Cartesian genetic programming[C]//Proceedings of GECCO'09.New York,USA:ACM Press,2009:15-26.
[37] SUGANUMA M,SHIRAKAWA S,NAGAO T.A genetic programming approach to designing convolutional neural network architectures[C]//Proceedings of Genetic and Evolutionary Computation Conference.Berlin,Germany:Springer,2017:497-504.
[38] REAL E,MOORE S,SELLE A,et al.Large-scale evolution of image classifiers[C]//Proceedings of the 34th International Conference on Machine Learning.Washington D.C.,USA:IEEE Press,2017:56-89.
[39] ZHANG H L,KIRANYAZ S,GABBOUJ M.Finding better topologies for deep convolutional neural networks by evolution[EB/OL].[2020-01-25].https://arxiv.org/abs/1809.03242.
[40] LIU H X,SIMONYAN K,VINYALS O,et al.Hierarchical representations for efficient architecture search[EB/OL].[2020-01-25].https://arxiv.org/abs/1711.00436.

选择文件类型/文献管理软件名称

选择包含的内容

基于无梯度进化的神经架构搜索算法研究综述

Review of Research on Neural Architecture Search Algorithms Based on Non-Gradient Evolution

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	齐宏拓, 伍洲, 周绪红, 李盛, 许成然, 郑晓磊, 刘界鹏, 廖耘竹. 基于模型预测控制的多钢筋并行排布智能深化设计方法[J]. 计算机工程, 2024, 50(9): 333-343.
[2]	郑锦灿, 邵立珍, 雷雪梅. 基于改进MOEA/D的模糊柔性作业车间调度算法[J]. 计算机工程, 2024, 50(6): 336-345.
[3]	薛羽, 卢畅畅. 基于有偏采样的连续进化神经架构搜索[J]. 计算机工程, 2024, 50(2): 91-97.
[4]	万昊楠, 吴飞, 尹玲. 基于自适应无人机数量的节时部署优化算法[J]. 计算机工程, 2024, 50(10): 302-312.
[5]	蔡倩倩, 史旭华. 自适应迁移的分解多目标多任务进化算法[J]. 计算机工程, 2023, 49(7): 55-64.
[6]	郑丽萍, 赵玉娟, 费选. 基于改进MOEA/D的车联网通信资源分配算法[J]. 计算机工程, 2023, 49(5): 191-197.
[7]	向海昀, 李鸿鑫, 符晓, 苏小平. 基于多策略的改进蜜獾算法及其应用[J]. 计算机工程, 2023, 49(12): 78-87.
[8]	伍洲, 杨寒石, 邬俊俊, 张海军, 宋晴. 进化迁移优化算法综述[J]. 计算机工程, 2023, 49(1): 1-14.
[9]	邱鸿辉, 刘海林, 陈磊. 基于协方差矩阵调整的多目标多任务优化算法[J]. 计算机工程, 2022, 48(8): 306-312.
[10]	生龙, 袁丽娜, 武南南, 姬少培. 基于GSA与DE优化混合核ELM的网络异常检测模型[J]. 计算机工程, 2022, 48(6): 146-153.
[11]	于成龙, 付国霞, 孙超利, 张国晨. 全局与局部模型交替辅助的差分进化算法[J]. 计算机工程, 2022, 48(3): 115-123.
[12]	谢艺蓉, 马永杰. 基于进化ResNet的交通标志识别[J]. 计算机工程, 2022, 48(10): 262-269.
[13]	倪水平, 戚海涛, 李慧芳. 基于多种群遗传与思维进化的混合算法[J]. 计算机工程, 2021, 47(12): 62-70.
[14]	王烁, 谷正气, 韩征彤, 马晓骙. 基于改进离散差分进化算法的桁架优化[J]. 计算机工程, 2021, 47(1): 275-283.
[15]	王斌, 房新秀, 魏天佑. 基于差异节点集的加权频繁项集挖掘算法[J]. 计算机工程, 2020, 46(5): 150-156.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于无梯度进化的神经架构搜索算法研究综述

Review of Research on Neural Architecture Search Algorithms Based on Non-Gradient Evolution

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献

相关文章 15

编辑推荐

Metrics

本文评价