Convolutional Neural Network Compression Method Based on LZW Encoding

doi:10.19678/j.issn.1000-3428.0052195

Computer Engineering ›› 2019, Vol. 45 ›› Issue (9): 188-193. doi: 10.19678/j.issn.1000-3428.0052195

Previous Articles Next Articles

Convolutional Neural Network Compression Method Based on LZW Encoding

LIU Chongyang, LIU Qinrang

China National Digital Switching System Engineering and Technological R & D Center, Zhengzhou 450002, China

Received:2018-07-24 Revised:2018-09-14 Online:2019-09-15 Published:2019-09-03

基于LZW编码的卷积神经网络压缩方法

刘崇阳, 刘勤让

国家数字交换系统工程技术研究中心, 郑州 450002

作者简介:刘崇阳(1994-),男,硕士研究生,主研方向为人工智能、深度学习;刘勤让,研究员、博士。
基金资助:
国家科技重大专项（2016ZX01012101）；国家自然科学基金（61572520，61521003）。

Abstract

Abstract: Aiming at the problem that the Convolutional Neural Network(CNN) parameters are large and difficult to be transplanted to the embedded platform,a CNN compression method based on Lempel-Ziv-Welch(LZW) encoding is proposed.The model capacity is compressed by two ways:convert floating point to fixed point and pruning.The weights are quantized by k-means clustering,and LZW encoding is performed on this basis.Experimental results on the MNIST dataset show that the pruning effect is better than the compression effect of converting floating point to fixed point.After pruning and quantification,the compression ratio of LZW encoding can reach 25.338.

Key words: Convolutional Neural Network(CNN), Lempel-Ziv-Welch(LZW) encoding, convert floating point to fixed point, model pruning, k-means clustering quantification

摘要： 针对卷积神经网络（CNN）因参数量大难以移植到嵌入式平台的问题，提出基于LZW编码的CNN压缩方法。通过浮点转定点和剪枝2种方法来压缩模型容量。对权值进行k-means聚类量化，并在此基础上进行LZW编码。在MNIST数据集上进行实验，结果表明，剪枝效果优于浮点转定点的压缩效果，在进行剪枝、量化后使用LZW编码，其压缩比可达25.338。

关键词: 卷积神经网络, LZW编码, 浮点转定点, 模型剪枝, k-means聚类量化

CLC Number:

TP391

LIU Chongyang, LIU Qinrang. Convolutional Neural Network Compression Method Based on LZW Encoding[J]. Computer Engineering, 2019, 45(9): 188-193.

刘崇阳, 刘勤让. 基于LZW编码的卷积神经网络压缩方法[J]. 计算机工程, 2019, 45(9): 188-193.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0052195

http://www.ecice06.com/EN/Y2019/V45/I9/188

Figures/Tables 6

References

[1] 深度学习大讲堂.ILSVRC2016目标检测任务回顾:图像目标检测[EB/OL].[2018-07-01].https://www.leiphone.com/news/201701/u3D5QnJbS9khm0VT.html.
[2] HU Jie,LI Shen,SUN Gang.Squeeze-and-excitation networks[EB/OL].[2018-07-01].https://arxiv.org/pdf/1709.01507.pdf.
[3] BOJARSKI M,TESTA D D,DWORAKOWSKI D,et al.End to end learning for self-driving cars[EB/OL].[2018-07-01].https://arxiv.org/pdf/1604.07316.pdf.
[4] FANG Shihau,FEI Yuxaing,XU Zhezhuang,et al.Learning transportation modes from smartphone sensors based on deep neural network[J].IEEE Sensors Journal,2017,17(18):6111-6118.
[5] GANDHI D,PINTO L,GUPTA A.Learning to fly by crashing[C]//Proceedings of 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems.Washington D.C.,USA:IEEE Press,2017:3948-3955.
[6] DENTON E,ZAREMBA W,BRUNA J,et al.Exploiting linear structure within convolutional networks for efficient evaluation[C]//Proceedings of International Conference on Neural Information Processing Systems.Cambridge,USA:MIT Press 2014:1269-1277.
[7] NOVIKOV A,PODOPRIKHIN D,OSOKIN A,et al.Tensorizing neural networks[EB/OL].[2018-07-01].https://arxiv.org/pdf/1509.06569.pdf.
[8] CHEN Wenlin,WILSON J T,TYREE S,et al.Compressing neural networks with the hashing trick[EB/OL].[2018-07-01].https://arxiv.org/pdf/1504.04788.pdf.
[9] ANWAR S,HWANG K,SUNG Wonyong.Fixed point optimization of deep convolutional neural networks for object recognition[C]//Proceedings of IEEE International Conference on Acoustics,Speech and Signal Processing.Washington D.C.,USA:IEEE Press,2015:1131-1135.
[10] HAN Song.Efficient Methods and Hardware for Deep Learning[D].San Francisco,USA:Stanford University,2017.
[11] HAN Song,MAO Huizi,DALLY W J.Deep compression:compressing deep neural networks with pruning,trained quantization and huffman coding[EB/OL].[2018-07-01].https://arxiv.org/pdf/1510.00149.pdf.
[12] 韩云飞,蒋同海,马玉鹏,等.深度神经网络的压缩研究[J].计算机应用研究,2018,35(10):2894-2897.
[13] 蔡瑞初,钟椿荣,余洋,等.面向"边缘"应用的卷积神经网络量化与压缩方法[J].计算机应用,2018,39(9):2449-2454.
[14] 雷杰,高鑫,宋杰,等.深度网络模型压缩综述[J].软件学报,2018,29(2):251-266.
[15] DENIL M,SHAKIBI B,DINH L,et al.Predicting parameters in deep learning[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems.New York,USA:ACM Press,2013:2148-2156.
[16] JIA Yangping,SHELHAMER E,DONAHUE J,et al.Caffe:convolutional architecture for fast feature embedding[C]//Proceedings of ACM International Conference on Multimedia.New York,USA:ACM Press,2014:675-678.
[17] HAN Song,JEFF P,JOHN T,et al.Learning both weights and connections for efficient neural networks[C]//Proceedings of Annual Conference on Neural Information Processing Systems.Cambridge,USA:MIT Press,2015:1135-1143.

Please choose a citation manager

Content to export