面向移动平台的轻量级卷积神经网络架构

doi:10.19678/j.issn.1000-3428.0049905

计算机工程 ›› 2019, Vol. 45 ›› Issue (1): 17-22. doi: 10.19678/j.issn.1000-3428.0049905

面向移动平台的轻量级卷积神经网络架构

胡挺^1,2,祝永新¹,田犁¹,封松林¹,汪辉¹

1.中国科学院上海高等研究院,上海 201210; 2.中国科学院大学,北京 100049

收稿日期:2017-12-28 出版日期:2019-01-15 发布日期:2019-01-15
作者简介:胡挺(1992—),男,硕士研究生,主研方向为深度学习、目标检测、网络压缩;祝永新,研究员、博士;田犁,副研究员、博士;封松林、汪辉(通信作者),研究员、博士
基金资助:
国家重点研发计划(2017YFA0206104);上海市科学技术委员会科研计划项目(16511108701);上海市张江管委会公共服务平台项目(2016-14)

Lightweight Convolutional Neural Network Architecture for Mobile Platforms

HU Ting ^1,2,ZHU Yongxin ¹,TIAN Li ¹,FENG Songlin¹,WANG Hui ¹

1.Shanghai Advanced Research Institute,Chinese Academy of Sciences,Shanghai 201210,China; 2.University of Chinese Academy of Sciences,Beijing 100049,China

Received:2017-12-28 Online:2019-01-15 Published:2019-01-15

摘要/Abstract

摘要：

针对深度神经网络在移动平台上存在准确度低、过拟合等问题,提出一种轻量级的卷积神经网络架构。将3×3的深度可分离卷积替换SqueezeNet网络模型基本模块Fire中的标准3×3卷积核,并构建SparkNet的网络结构,替换模型卷积得到网络变形结构。实验结果表明,与SqueezeNet网络结构相比,该架构可以提高网络模型的计算速度,有效降低网络模型规模并减少参数数量。

关键词: 深度学习, 卷积神经网络, 深度可分离卷积, 神经网络压缩, 轻量级

Abstract:

For the problem that the deep neural network has low accuracy and over-fitting on the mobile platforms,a lightweight Convolutional Neural Network(CNN) architecture is proposed.The 3×3 depthwise separable convolution replaces the standard 3×3 convolution kernel in the SqueezeNet network model basic module Fire,constructs the SparkNet network structure,and replaces the model convolution to obtain the network deformation structure.Experimental results show that compared with the SqueezeNet network structure,the architecture can improve the calculation speed of the network model,effectively reduce the network model size and reduce the number of parameters.

Key words: deep learning, Convolutional Neural Network(CNN), depthwise separable convolution, neural network compression, lightweight

中图分类号:

TP399

胡挺,祝永新,田犁,封松林,汪辉. 面向移动平台的轻量级卷积神经网络架构[J]. 计算机工程, 2019, 45(1): 17-22.

HU Ting,ZHU Yongxin,TIAN Li,FENG Songlin,WANG Hui. Lightweight Convolutional Neural Network Architecture for Mobile Platforms[J]. Computer Engineering, 2019, 45(1): 17-22.

http://www.ecice06.com/CN/Y2019/V45/I1/17

参考文献

［1］KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks［C］//Proceedings of the 25th International Conference on Neural Information Processing Systems.［S.l.］:Curran Associates Inc.,2012:1097-1105.
［2］PRATT L Y.Comparing biases for minimal network construction with back-propagation［C］//Proceedings of International Conference on Neural Information Processing Systems.Cambridge,USA:MIT Press,1988:177-185.
［3］LE Y,DENKER,JOHN S,et al.Optimal brain damage［C］//Proceedings of the 2nd International Conference on Neural Information Processing Systems.Cambridge,USA:MIT Press,1989:598-605.
［4］HASSIBI B,STORK D G.Second order derivatives for network pruning:optimal brain surgeon［J］.Advances in Neural Information Processing Systems,1993,5:164-171.
［5］李小夏,李孝安.一种改进的神经网络相关性剪枝算法［J］.电子设计工程,2013,21(8):65-67.
［6］熊俊,王士同,潘永惠,等.基于惩罚函数泛化的神经网络剪枝算法研究［J］.计算机工程,2014,40(11):149-154.
［7］赵蓉,唐楚淇,刘伟林,等.一种新的基于灰色关联分析的BP神经网络剪枝算法［J］.科技创新与应用,2016(13):17-18.
［8］TRAN J,TRAN J,TRAN J,et al.Learning both weights and connections for efficient neural networks［EB/OL］.［2017-11-20］.https://arxiv.org/abs/1506.02626.
［9］VANHOUCKE V,SENIOR A,MAO M Z.Improving the speed of neural networks on CPUs［EB/OL］.［2017-11-20］.https://ai.google/research/pubs/pub37631.
［10］GONG Y,LIU L,YANG M,et al.Compressing deep convolutional networks using vector quantization［EB/OL］.［2017-11-20］.http://adsabs.harvard.edu/abs/2014arXiv1412.6115G.
［11］CHEN W,TYREE S,TYREE S,et al.Compressing neural networks with the hashing trick［C］//Proceedings of the 32nd International Conference on Machine Learning.Cambridge,USA:MIT Press,2015:2285-2294.
［12］杨志刚,吴俊敏,徐恒,等.基于虚拟化的多GPU深度神经网络训练框架［J］.计算机工程,2018,44(2):68-74,83.
［13］COURBARIAUX M,BENGIO Y,DAVID J P.Binaryconnect:training deep neural networks with binary weights during propagations［C］//Proceedings of International Conference on Neural Information Processing Systems.Cambridge,USA:MIT Press,2015:3123-3131.
［14］COURBARIAUX M,HUBARA I,SOUDRY D,et al.Binarized neural networks:training deep neural networks with weights and activations constrained to +1 or -1［EB/OL］.［2017-11-20］.https://arxiv.org/pdf/1602.02830v3.pdf.
［15］RASTEGARI M,ORDONEZ V,REDMON J,et al.Xnor-net:imagenet classification using binary convolutional neural networks［C］//Proceedings of the European Conference on Computer Vision.Berlin,Germany:Springer,2016:525-542.
［16］IANDOLA F N,HAN S,MOSKEWICZ M W,et al.SqueezeNet:AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size［EB/OL］.［2017-11-20］.https://arxiv.org/pdf/1602.07360v3.pdf.
［17］HOWARD A G,ZHU M,CHEN B,et al.Mobilenets:efficient convolutional neural networks for mobile vision applications［EB/OL］.［2017-11-20］.https://arxiv.org/abs/1704.04861.
［18］LIN M,CHENQ,YAN S.Network in network［EB/OL］.［2017-11-20］.https://arxiv.org/pdf/1312.4400.pdf.
［19］LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-based learning applied to document recognition［J］.Proceedings of the IEEE,1998,86(11):2278-324.
［20］KRIZHEVSKY A.Learning multiple layers of features from tiny images［EB/OL］.［2017-11-20］.https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf.
［21］NETZER Y,WANG T,COATES A,et al.Reading digits in natural images with unsupervised feature learning［EB/OL］.［2017-11-20］.http://101.110.118.29/ufldl.stanford.edu/housenumbers/nips2011_housenumbers.pdf.
［22］HAN S,MAO H,DALLY W J.Deep compression:compressing deep neural networks with pruning,trained quantization and Huffman coding［EB/OL］.［2017-11-20］.https://arxiv.org/abs/1510.00149.
［23］ZEILER M D,FERGUS R.Stochastic pooling for regularization of deep convolutional neural networks［EB/OL］.［2017-11-20］.https://arxiv.org/abs/1301.3557.
［24］SNOEK J,LAROCHELLE H,ADAMS R P.Practical bayesian optimization of machine learning algorithms［C］//Proceedings of the 25th International Conference on Neural Information Processing Systems.［S.l.］:Curran Associates Inc.,2012:2951-2959.
［25］GOODFELLOW I J,WARDE-FARLEY D,MIRZA M,et al.Maxout networks［J］ Machine Learning,2013,28(3):1319-1327.
［26］MALINOWSKI M,FRITZ M.Learnable pooling regions for image classification［EB/OL］.［2017-11-20］.https://arxiv.org/abs/1301.3516?context=cs.CV.
［27］SRIVASTAVA N.Improving neural networks with dropout［EB/OL］.［2017-11-20］.http://www.cs.toronto.edu/~nitish/msc_thesis.pdf.

[1]	江雨燕, 陶承凤, 李平. 数据增强和自适应自步学习的深度子空间聚类算法[J]. 计算机工程, 2023, 49(8): 96-103, 110.
[2]	张欣怡, 张飞, 郝斌, 高鹭, 任晓颖. 基于改进YOLOv5的口罩佩戴检测算法[J]. 计算机工程, 2023, 49(8): 265-274.
[3]	李泽水, 冀俊忠, 杨翠翠. 基于边权重信息深度网络嵌入的PPIN功能模块检测[J]. 计算机工程, 2023, 49(8): 69-76.
[4]	王可铮, 徐玉芬, 周尚波. 结合对比感知损失和融合注意力的图像去雾模型[J]. 计算机工程, 2023, 49(8): 207-214.
[5]	刘俊豪, 王美林, 谢兴, 宋烨兴, 许莉花. 基于改进YOLOv5的皮革瑕疵检测算法[J]. 计算机工程, 2023, 49(8): 240-249.
[6]	曹坪, 杨怀志, 薄一军, 尤嘉, 张淳杰, 李丹勇. 面向低质量裂缝图像的多知识蒸馏分类[J]. 计算机工程, 2023, 49(7): 204-213.
[7]	白明昌. 基于折叠路径聚合的属性网络节点嵌入方法[J]. 计算机工程, 2023, 49(7): 76-84.
[8]	闫兴亚, 匡娅茜, 白光睿, 李月. 基于深度学习的学生课堂行为识别方法[J]. 计算机工程, 2023, 49(7): 251-258.
[9]	李军侠, 王星驰, 殷梓, 石德硕. 边缘深度挖掘的弱监督显著性目标检测[J]. 计算机工程, 2023, 49(7): 169-178.
[10]	吴珊, 周凤. 基于改进SSD算法的小目标检测[J]. 计算机工程, 2023, 49(7): 179-188.
[11]	席建锐, 唐红梅, 梁春阳, 刘鑫. 基于改进隐函数的点云物体重建[J]. 计算机工程, 2023, 49(7): 214-222.
[12]	齐咏生, 杜晓旭, 朱俊峰, 高胜利, 刘利强. 基于增强型轻量深度网络的牧区牲畜高效检测[J]. 计算机工程, 2023, 49(7): 278-287.
[13]	谌雨章, 黄逸姿, 张钧涵. 基于多速率空洞卷积的多尺度水下小目标检测[J]. 计算机工程, 2023, 49(6): 257-264.
[14]	张博旭, 蒲智, 程曦. 基于提示学习的维吾尔语文本分类研究[J]. 计算机工程, 2023, 49(6): 292-299,313.
[15]	于海洋, 景鹏, 张文涛, 谢赛飞, 滑志华, 宋草原. 基于残差与注意力机制的道路裂缝检测U-Net改进模型[J]. 计算机工程, 2023, 49(6): 265-273.

选择文件类型/文献管理软件名称

选择包含的内容

面向移动平台的轻量级卷积神经网络架构

Lightweight Convolutional Neural Network Architecture for Mobile Platforms

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

面向移动平台的轻量级卷积神经网络架构

Lightweight Convolutional Neural Network Architecture for Mobile Platforms

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价