卷积神经网络层级分解研究

doi:10.19678/j.issn.1000-3428.0053044

计算机工程 ›› 2019, Vol. 45 ›› Issue (11): 191-197. doi: 10.19678/j.issn.1000-3428.0053044

卷积神经网络层级分解研究

柯岩^1,2, 林小竹¹, 廖蕊^1,2, 魏战红¹

1. 北京石油化工学院信息工程学院, 北京 102617;
2. 北京化工大学信息科学与技术学院, 北京 100029

收稿日期:2018-10-31 修回日期:2019-01-08 出版日期:2019-11-15 发布日期:2019-01-24
作者简介:柯岩(1994-),男,硕士研究生,主研方向为深度学习、模式识别;林小竹(通信作者),教授、博士;廖蕊,硕士研究生;魏战红,讲师、博士。
基金资助:
国家自然科学基金（61702040）；北京市自然科学基金（4174089）；北京市教委科研科技计划一般项目（KM201810017006）。

Research on Hierarchical Decomposition of Convolutional Neural Network

KE Yan^1,2, LIN Xiaozhu¹, LIAO Rui^1,2, WEI Zhanhong¹

1. College of Information Engineering, Beijing Institute of Petrochemical Technology, Beijing 102617, China;
2. College of Information Science and Technology, Beijing University of Chemical Technology, Beijing 100029, China

Received:2018-10-31 Revised:2019-01-08 Online:2019-11-15 Published:2019-01-24

摘要/Abstract

摘要： 随着深度学习的不断发展，卷积神经网络（CNN）在目标检测与图像分类中受到研究者的广泛关注。CNN从LeNet-5网络发展到深度残差网络，其层数不断增加。基于神经网络中"深度"的含义，在确保感受野相同的前提下，给定标准的输入图片和输出特征图，对不同层数的卷积神经网络进行训练，并将训练结果与标准输出图进行对比。在此基础上，对标准的3×3卷积核进行分解，构建由2×2大小卷积核组成的CNN。根据目标特征是否具有中心对称的性质，提出多层卷积网络初始权值的选取规则。

关键词: 卷积神经网络, 卷积核, 感受野, 网络深度, 中心极限定理

Abstract: With the continuous development of deep learning,Convolutional Neural Network(CNN) have received extensive attention from researchers in target detection and image classification.CNN have evolved from LeNet-5 networks to deep residual networks,and the number of layers are increasing.According to the definition of the depth of a neural network,this paper trains CNN of different layers with given standard input images and an output feature graph.The training is performed on the premise that the CNN shares the same receptive field,and training results are compared with standard output graphs.On this basis,the standard 3×3 convolution kernel is decomposed,and a convolutional neural network composed of 2×2 convolution kernels is constructed.According to the central symmetry property of the target feature,the selection rule of the initial weight of the multi-layer convolutional network is proposed.

Key words: Convolutional Neural Network(CNN), convolution kernel, receptive field, network depth, central limit theorem

中图分类号:

TP391

柯岩, 林小竹, 廖蕊, 魏战红. 卷积神经网络层级分解研究[J]. 计算机工程, 2019, 45(11): 191-197.

KE Yan, LIN Xiaozhu, LIAO Rui, WEI Zhanhong. Research on Hierarchical Decomposition of Convolutional Neural Network[J]. Computer Engineering, 2019, 45(11): 191-197.

http://www.ecice06.com/CN/Y2019/V45/I11/191

图/表 16

20191118170930

20191118170933

20191118170937

20191118170939

20191118170942

20191118170944

20191118170947

20191118170950

20191118170953

20191118170956

20191118170958

20191118171001

20191118171004

20191118171007

20191118171010

20191118171013

参考文献

[1] 周飞燕,金林鹏,董军.卷积神经网络研究综述[J].计算机学报,2017,40(6):1229-1251.
[2] SILVER D,HUANG A,MADDISON C J,et al.Mastering the game of Go with deep neural networks and tree search[J].Nature,2016,529(7587):484-489.
[3] LECUN Y,BOSER B,DENKER J S,et al.Backpropagation applied to handwritten zip code recognition[J].Neural Computation,2014,1(4):541-551.
[4] FUKUSHIMA K.Neocognitron:a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position[J].Biological Cybernetics,1980,36(4):193-202.
[5] LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[6] LAWRENCE S,GILES C L,TSOI A C,et al.Face recognition:a convolutional neural-network approach[J].IEEE Transactions on Neural Networks,1997,8(1):98-113.
[7] NEBAUER C.Evaluation of convolutional neural networks for visual recognition[J].IEEE Transactions on Neural Networks,1998,9(4):685-696.
[8] 余凯,贾磊,陈雨强,等.深度学习的昨天、今天和明天[J].计算机研究与发展,2013,50(9):1799-1804.
[9] 李勇,林小竹,蒋梦莹.基于跨连接LeNet-5网络的面部表情识别[J].自动化学报,2018,44(1):176-182.
[10] KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks[C]//Proceedings of International Conference on Neural Information Processing Systems.[S.l.]:Curran Associates Inc.,2012:1097-1105.
[11] SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[EB/OL].[2018-12-20].https://arxiv.org/pdf/1409.1556.pdf.
[12] SZEGEDY C,LIU Wei,JIA Yangqing,et al.Going deeper with convolutions[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:1-9.
[13] SZEGEDY C,VANHOUCKE V,IOFFE S,et al.Rethinking the inception architecture for computer vision[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:2818-2826.
[14] HE Kaiming,ZHANG Xiangyu,REN Shaoqing,et al.Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:770-778.
[15] HE Kaiming,ZHANG Xiangyu,REN Shaoqing,et al.Identity mappings in deep residual networks[C]//Proceedings of Conference on Computer Vision.Berlin,Germany:Springer,2016:630-645.
[16] HE Kaiming,SUN Jian.Convolutional neural networks at constrained time cost[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:5353-5360.
[17] LUO Wenjie,LI Yujia,URTASUN R,et al.Understanding the effective receptive field in deep convolutional neural networks[C]//Proceedings of International Conference on Neural Information Processing Systems.[S.l.]:Curran Associates Inc.,2016:4898-4906.
[18] HE Kaiming,GIRSHICK R,DOLLÁR P.Rethinking ImageNet pre-training[EB/OL].[2018-12-20].https://arxiv.org/abs/1811.08883v1.
[19] HUBEL D H,WIESEL T N.Receptive fields,binocular interaction and functional architecture in the cat's visual cortex[J].The Journal of Physiology,1962,160(1):106-154.
[20] GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets[C]//Proceedings of International Conference on Neural Information Processing Systems.[S.l.]:Curran Associates Inc.,2014:2672-2680.
[21] WU Jiajun,ZHANG Chengkai,XUE Tianfan,et al.Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling[EB/OL].[2018-12-20].https://arxiv.org/abs/1610.07584.
[22] ODENA A,OLAH C,SHLENS J.Conditional image synthesis with auxiliary classifier GANs[EB/OL].[2018-12-20].https://arxiv.org/pdf/1610.09585.pdf.

选择文件类型/文献管理软件名称

选择包含的内容

卷积神经网络层级分解研究

Research on Hierarchical Decomposition of Convolutional Neural Network

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 16

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	曹坪, 杨怀志, 薄一军, 尤嘉, 张淳杰, 李丹勇. 面向低质量裂缝图像的多知识蒸馏分类[J]. 计算机工程, 2023, 49(7): 204-213.
[2]	白明昌. 基于折叠路径聚合的属性网络节点嵌入方法[J]. 计算机工程, 2023, 49(7): 76-84.
[3]	代祖华, 刘园园, 狄世龙. 语义增强的图神经网络方面级文本情感分析[J]. 计算机工程, 2023, 49(6): 71-80.
[4]	沈学利, 田桂源, 姜彦吉, 马琳琳. 基于双阶段Conv-Transformer的时频域语音增强算法[J]. 计算机工程, 2023, 49(6): 123-130.
[5]	丁子轩, 俞雷, 张娟, 李想, 王新宇. 基于深度残差自适应注意力网络的图像超分辨率重建[J]. 计算机工程, 2023, 49(5): 231-238.
[6]	陈治旭, 靳雁霞, 芦烨, 杨晶, 刘亚变, 史志儒. 基于子图卷积神经网络的多精度服装建模方法[J]. 计算机工程, 2023, 49(4): 174-181.
[7]	曹书鑫, 冯藤藤, 葛凤培, 梁春燕. 基于尺度相关‐双向长短期记忆网络模型的说话人识别[J]. 计算机工程, 2023, 49(4): 289-296.
[8]	徐康, 李霏, 姬东鸿. 结合依存图卷积与文本片段搜索的方面情感三元组抽取[J]. 计算机工程, 2023, 49(4): 61-67.
[9]	衡红军, 苗菁. 语义与句法信息加强的二元标记实体关系联合抽取[J]. 计算机工程, 2023, 49(4): 77-84.
[10]	钟宝荣, 吴夏灵. 基于高分辨率网络的轻量型人体姿态估计研究[J]. 计算机工程, 2023, 49(4): 226-232,239.
[11]	杨晶晶, 谢海燕, 薛妮妮, 张傲明. 基于双通道残差网络的水下图像去噪研究[J]. 计算机工程, 2023, 49(4): 188-198.
[12]	刘晶晶, 黄浩. 引入非局部模块卷积神经网络的基频提取模型[J]. 计算机工程, 2023, 49(3): 128-133,160.
[13]	邹长龙, 安敬民, 李冠宇. 基于邻域聚合与CNN的知识图谱实体类型补全[J]. 计算机工程, 2023, 49(3): 134-141.
[14]	翟社平, 张宇航, 柏晓夏. 融合实体邻域信息的知识图谱嵌入负采样方法[J]. 计算机工程, 2023, 49(3): 95-104.
[15]	程小辉, 李钰, 康燕萍. 基于中间图特征提取的卷积网络双标准剪枝[J]. 计算机工程, 2023, 49(3): 105-112.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

卷积神经网络层级分解研究

Research on Hierarchical Decomposition of Convolutional Neural Network

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 16

参考文献

相关文章 15

编辑推荐

Metrics

本文评价