面向移动平台的轻量级卷积神经网络架构

doi:10.19678/j.issn.1000-3428.0049905

计算机工程 ›› 2019, Vol. 45 ›› Issue (1): 17-22. doi: 10.19678/j.issn.1000-3428.0049905

面向移动平台的轻量级卷积神经网络架构

胡挺^1,2,祝永新¹,田犁¹,封松林¹,汪辉¹

1.中国科学院上海高等研究院,上海 201210; 2.中国科学院大学,北京 100049

收稿日期:2017-12-28 出版日期:2019-01-15 发布日期:2019-01-15
作者简介:胡挺(1992—),男,硕士研究生,主研方向为深度学习、目标检测、网络压缩;祝永新,研究员、博士;田犁,副研究员、博士;封松林、汪辉(通信作者),研究员、博士
基金资助:
国家重点研发计划(2017YFA0206104);上海市科学技术委员会科研计划项目(16511108701);上海市张江管委会公共服务平台项目(2016-14)

Lightweight Convolutional Neural Network Architecture for Mobile Platforms

HU Ting ^1,2,ZHU Yongxin ¹,TIAN Li ¹,FENG Songlin¹,WANG Hui ¹

1.Shanghai Advanced Research Institute,Chinese Academy of Sciences,Shanghai 201210,China; 2.University of Chinese Academy of Sciences,Beijing 100049,China

Received:2017-12-28 Online:2019-01-15 Published:2019-01-15

摘要/Abstract

摘要：

针对深度神经网络在移动平台上存在准确度低、过拟合等问题,提出一种轻量级的卷积神经网络架构。将3×3的深度可分离卷积替换SqueezeNet网络模型基本模块Fire中的标准3×3卷积核,并构建SparkNet的网络结构,替换模型卷积得到网络变形结构。实验结果表明,与SqueezeNet网络结构相比,该架构可以提高网络模型的计算速度,有效降低网络模型规模并减少参数数量。

关键词: 深度学习, 卷积神经网络, 深度可分离卷积, 神经网络压缩, 轻量级

Abstract:

For the problem that the deep neural network has low accuracy and over-fitting on the mobile platforms,a lightweight Convolutional Neural Network(CNN) architecture is proposed.The 3×3 depthwise separable convolution replaces the standard 3×3 convolution kernel in the SqueezeNet network model basic module Fire,constructs the SparkNet network structure,and replaces the model convolution to obtain the network deformation structure.Experimental results show that compared with the SqueezeNet network structure,the architecture can improve the calculation speed of the network model,effectively reduce the network model size and reduce the number of parameters.

Key words: deep learning, Convolutional Neural Network(CNN), depthwise separable convolution, neural network compression, lightweight

中图分类号:

TP399

胡挺,祝永新,田犁,封松林,汪辉. 面向移动平台的轻量级卷积神经网络架构[J]. 计算机工程, 2019, 45(1): 17-22.

HU Ting,ZHU Yongxin,TIAN Li,FENG Songlin,WANG Hui. Lightweight Convolutional Neural Network Architecture for Mobile Platforms[J]. Computer Engineering, 2019, 45(1): 17-22.

https://www.ecice06.com/CN/Y2019/V45/I1/17

参考文献

［1］KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks［C］//Proceedings of the 25th International Conference on Neural Information Processing Systems.［S.l.］:Curran Associates Inc.,2012:1097-1105.
［2］PRATT L Y.Comparing biases for minimal network construction with back-propagation［C］//Proceedings of International Conference on Neural Information Processing Systems.Cambridge,USA:MIT Press,1988:177-185.
［3］LE Y,DENKER,JOHN S,et al.Optimal brain damage［C］//Proceedings of the 2nd International Conference on Neural Information Processing Systems.Cambridge,USA:MIT Press,1989:598-605.
［4］HASSIBI B,STORK D G.Second order derivatives for network pruning:optimal brain surgeon［J］.Advances in Neural Information Processing Systems,1993,5:164-171.
［5］李小夏,李孝安.一种改进的神经网络相关性剪枝算法［J］.电子设计工程,2013,21(8):65-67.
［6］熊俊,王士同,潘永惠,等.基于惩罚函数泛化的神经网络剪枝算法研究［J］.计算机工程,2014,40(11):149-154.
［7］赵蓉,唐楚淇,刘伟林,等.一种新的基于灰色关联分析的BP神经网络剪枝算法［J］.科技创新与应用,2016(13):17-18.
［8］TRAN J,TRAN J,TRAN J,et al.Learning both weights and connections for efficient neural networks［EB/OL］.［2017-11-20］.https://arxiv.org/abs/1506.02626.
［9］VANHOUCKE V,SENIOR A,MAO M Z.Improving the speed of neural networks on CPUs［EB/OL］.［2017-11-20］.https://ai.google/research/pubs/pub37631.
［10］GONG Y,LIU L,YANG M,et al.Compressing deep convolutional networks using vector quantization［EB/OL］.［2017-11-20］.http://adsabs.harvard.edu/abs/2014arXiv1412.6115G.
［11］CHEN W,TYREE S,TYREE S,et al.Compressing neural networks with the hashing trick［C］//Proceedings of the 32nd International Conference on Machine Learning.Cambridge,USA:MIT Press,2015:2285-2294.
［12］杨志刚,吴俊敏,徐恒,等.基于虚拟化的多GPU深度神经网络训练框架［J］.计算机工程,2018,44(2):68-74,83.
［13］COURBARIAUX M,BENGIO Y,DAVID J P.Binaryconnect:training deep neural networks with binary weights during propagations［C］//Proceedings of International Conference on Neural Information Processing Systems.Cambridge,USA:MIT Press,2015:3123-3131.
［14］COURBARIAUX M,HUBARA I,SOUDRY D,et al.Binarized neural networks:training deep neural networks with weights and activations constrained to +1 or -1［EB/OL］.［2017-11-20］.https://arxiv.org/pdf/1602.02830v3.pdf.
［15］RASTEGARI M,ORDONEZ V,REDMON J,et al.Xnor-net:imagenet classification using binary convolutional neural networks［C］//Proceedings of the European Conference on Computer Vision.Berlin,Germany:Springer,2016:525-542.
［16］IANDOLA F N,HAN S,MOSKEWICZ M W,et al.SqueezeNet:AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size［EB/OL］.［2017-11-20］.https://arxiv.org/pdf/1602.07360v3.pdf.
［17］HOWARD A G,ZHU M,CHEN B,et al.Mobilenets:efficient convolutional neural networks for mobile vision applications［EB/OL］.［2017-11-20］.https://arxiv.org/abs/1704.04861.
［18］LIN M,CHENQ,YAN S.Network in network［EB/OL］.［2017-11-20］.https://arxiv.org/pdf/1312.4400.pdf.
［19］LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-based learning applied to document recognition［J］.Proceedings of the IEEE,1998,86(11):2278-324.
［20］KRIZHEVSKY A.Learning multiple layers of features from tiny images［EB/OL］.［2017-11-20］.https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf.
［21］NETZER Y,WANG T,COATES A,et al.Reading digits in natural images with unsupervised feature learning［EB/OL］.［2017-11-20］.http://101.110.118.29/ufldl.stanford.edu/housenumbers/nips2011_housenumbers.pdf.
［22］HAN S,MAO H,DALLY W J.Deep compression:compressing deep neural networks with pruning,trained quantization and Huffman coding［EB/OL］.［2017-11-20］.https://arxiv.org/abs/1510.00149.
［23］ZEILER M D,FERGUS R.Stochastic pooling for regularization of deep convolutional neural networks［EB/OL］.［2017-11-20］.https://arxiv.org/abs/1301.3557.
［24］SNOEK J,LAROCHELLE H,ADAMS R P.Practical bayesian optimization of machine learning algorithms［C］//Proceedings of the 25th International Conference on Neural Information Processing Systems.［S.l.］:Curran Associates Inc.,2012:2951-2959.
［25］GOODFELLOW I J,WARDE-FARLEY D,MIRZA M,et al.Maxout networks［J］ Machine Learning,2013,28(3):1319-1327.
［26］MALINOWSKI M,FRITZ M.Learnable pooling regions for image classification［EB/OL］.［2017-11-20］.https://arxiv.org/abs/1301.3516?context=cs.CV.
［27］SRIVASTAVA N.Improving neural networks with dropout［EB/OL］.［2017-11-20］.http://www.cs.toronto.edu/~nitish/msc_thesis.pdf.

[1]	魏嵬, 丁香香, 郭梦星, 杨钊, 刘辉. 文本相似度计算方法综述[J]. 计算机工程, 2024, 50(9): 18-32.
[2]	王志浩, 钱沄涛. 基于Swin Transformer的双流遥感图像时空融合超分辨率重建[J]. 计算机工程, 2024, 50(9): 33-45.
[3]	李俊俊, 董建刚, 李坤. 基于Kubernetes的集群节能策略研究[J]. 计算机工程, 2024, 50(9): 82-91.
[4]	蔡俊民, 梁正友, 孙宇, 陈子奥. 基于可变形三维图卷积的轻量级点云分类研究[J]. 计算机工程, 2024, 50(9): 255-265.
[5]	张鲁, 田春伟, 宋焕生, 刘侍刚. 用于低剂量CT图像去噪的多级双树复小波网络[J]. 计算机工程, 2024, 50(9): 266-275.
[6]	朱凯, 李理, 张彤, 江晟, 别一鸣. 基于Transformer的多阶段运动模糊图像修复网络[J]. 计算机工程, 2024, 50(9): 276-285.
[7]	张天鹏, 韩晶, 吕学强. 基于多任务学习的超分辨率辅助小目标检测[J]. 计算机工程, 2024, 50(9): 304-312.
[8]	高煜宝, 文志诚. 基于注意力机制的双路解码器图像去噪方法[J]. 计算机工程, 2024, 50(9): 324-332.
[9]	张华青, 夏张涛, 陆晓庆, 童基均. 基于字形特征的血管外科命名实体识别[J]. 计算机工程, 2024, 50(8): 13-21.
[10]	王蕾, 党时鹏, 潘丰. 基于卷积神经网络的隐匿性旁路预测模型[J]. 计算机工程, 2024, 50(8): 40-49.
[11]	张亚洲, 和玉, 戎璐, 王祥凯. 基于上下文知识增强型Transformer网络的抑郁检测[J]. 计算机工程, 2024, 50(8): 75-85.
[12]	高伟, 李帅龙, 茆琳, 王磊, 李颖颖, 韩林. 一种基于TVM的算子生成加速策略[J]. 计算机工程, 2024, 50(8): 353-362.
[13]	王宇, 祁琦, 王纯, 许才. 储能变流器信号高精度故障诊断方法[J]. 计算机工程, 2024, 50(8): 389-396.
[14]	耿丽丽, 牛保宁. 基于通道相似度熵的卷积神经网络裁剪[J]. 计算机工程, 2024, 50(7): 133-143.
[15]	张洋, 刘畅, 李少青. 基于可控制性度量的图神经网络门级硬件木马检测方法[J]. 计算机工程, 2024, 50(7): 164-173.

选择文件类型/文献管理软件名称

选择包含的内容

面向移动平台的轻量级卷积神经网络架构

Lightweight Convolutional Neural Network Architecture for Mobile Platforms

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

面向移动平台的轻量级卷积神经网络架构

Lightweight Convolutional Neural Network Architecture for Mobile Platforms

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价