作者投稿和查稿 主编审稿 专家审稿 编委审稿 远程编辑

计算机工程 ›› 2021, Vol. 47 ›› Issue (8): 1-13. doi: 10.19678/j.issn.1000-3428.0060931

• 热点与综述 • 上一篇    下一篇

深度学习的轻量化神经网络结构研究综述

王军1,2,3, 冯孙铖1,2, 程勇1,3   

  1. 1. 南京信息工程大学 计算机与软件学院, 南京 210044;
    2. 南京信息工程大学 数字取证教育部工程研究中心, 南京 210044;
    3. 南京信息工程大学 科技产业处, 南京 210044
  • 收稿日期:2021-02-24 修回日期:2021-03-24 发布日期:2021-03-29
  • 作者简介:王军(1970-),男,教授,主研方向为深度学习、物联网、图像处理;冯孙铖,硕士研究生;程勇,高级工程师、博士。
  • 基金资助:
    国家自然科学基金(41875184);江苏省“六大人才高峰”创新人才团队项目(TD-XYDXX-004)。

Survey of Research on Lightweight Neural Network Structures for Deep Learning

WANG Jun1,2,3, FENG Suncheng1,2, CHENG Yong1,3   

  1. 1. School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing 210044, China;
    2. Engineering Research Center of Digital Forensics of Ministry of Education, Nanjing University of Information Science and Technology, Nanjing 210044, China;
    3. Science and Technology Industry Division, Nanjing University of Information Science and Technology, Nanjing 210044, China
  • Received:2021-02-24 Revised:2021-03-24 Published:2021-03-29

摘要: 随着深度神经网络和智能移动设备的快速发展,网络结构轻量化设计逐渐成为前沿且热门的研究方向,而轻量化的本质是在保持深度神经网络精度的前提下优化存储空间和提升运行速度。阐述深度学习的轻量化网络结构设计方法,对比与分析人工设计的轻量化方法、基于神经网络结构搜索的轻量化方法和基于自动模型压缩的轻量化方法的创新点与优劣势,总结与归纳上述3种主流轻量化方法中性能优异的网络结构并分析各自的优势和局限性。在此基础上,指出轻量化网络结构设计所面临的挑战,同时对其应用方向及未来发展趋势进行展望。

关键词: 深度学习, 轻量化设计, 深度可分离卷积, Octave卷积, 神经网络结构搜索, 模型压缩

Abstract: With the rapid development of deep neural networks and smart mobile devices, the research of lightweight neural network structure has gradually become a hotspot. The essence of lightweight design is to optimize the storage space and improve the running speed without causing any loss to the precision of deep neural networks. Then an introduction to the mainstream methods of lightweight network structure design for deep learning is given, and the innovative features, strengths and weaknesses between the manual design methods, neural network structure search-based design methods and automated model compression-based design methods are compared. The advantages and disadvantages of the high-performance network structures generated by the above methods are also summarized. On this basis, the challenges faced by lightweight network structure design, and its applications and development trends are discussed.

Key words: deep learning, lightweight design, Depthwise Separable Convolution(DSC), Octave convolution, neural network structure search, model compression

中图分类号: