作者投稿和查稿 主编审稿 专家审稿 编委审稿 远程编辑

计算机工程 ›› 2022, Vol. 48 ›› Issue (1): 112-118,126. doi: 10.19678/j.issn.1000-3428.0060042

• 人工智能与模式识别 • 上一篇    下一篇

基于自适应分层阈值判断的神经网络模型压缩

卢鹏, 万莹, 邹国良, 陈金宇, 郑宗生, 王振华   

  1. 上海海洋大学 信息学院, 上海 201306
  • 收稿日期:2020-11-18 修回日期:2021-01-05 发布日期:2021-01-11
  • 作者简介:卢鹏(1981-),男,讲师、博士,主研方向为深度学习、图像处理;万莹,硕士研究生;邹国良,教授;陈金宇,硕士研究生;郑宗生、王振华,副教授。
  • 基金资助:
    国家自然科学基金(41501419,41671431);上海市地方院校能力建设项目(19050502100)。

Neural Network Model Compression Based on Adaptive Hierarchical Threshold Judgment

LU Peng, WAN Ying, ZOU Guoliang, CHEN Jinyu, ZHENG Zongsheng, WANG Zhenhua   

  1. College of Information, Shanghai Ocean University, Shanghai 201306, China
  • Received:2020-11-18 Revised:2021-01-05 Published:2021-01-11

摘要: 面对多样化的应用环境,卷积神经网络(CNN)的架构深度不断增加以提升精度,但同时需要大量的计算参数和网络存储。针对CNN卷积层参数冗余和运算效率低的问题,提出一种基于分层阈值的自适应动态剪枝方法。设计自适应分层阈值判断算法,对批归一化层的尺度因子进行聚类分析,自适应地找到每层的分类断点并据此确定最终阈值,利用该阈值修剪正则化后的输入模型,从而避免根据经验人为定义固定阈值,减小模型尺寸和运行时占用的内存。分别采用该方法和LIU等提出的使用固定阈值且全局修剪的方法对VGGNet、ResNet、DenseNet和LeNet模型进行压缩,并在CIFAR、SVHN和MNIST数据集上测试模型性能。实验结果表明,该方法能够在模型精度与剪枝率之间找到最优平衡,剪枝后模型的测试错误率较对比方法降低0.02~1.52个百分点,同时自适应分层阈值判断算法也能避免对比方法在全局修剪时减去整个层的问题。

关键词: 深度学习, 图像识别, 卷积神经网络, 模型压缩, 网络剪枝

Abstract: Aiming at diversified application environments, the architecture depth of Convolutional Neural Network(CNN) is increasing to improve the accuracy, but at the same time, it needs a lot of computing parameters and network storage.An adaptive dynamic pruning method based on layered threshold is proposed to solve the problems of CNN convolution parameter redundancy and low operation efficiency.An adaptive hierarchical threshold judgment algorithm is designed to cluster the scale factors of the Batch Normalization(BN) layer, adaptively find the classification cluster points of each layer, and determine the final threshold accordingly.The regularized input model is trimmed by using the threshold, so as to avoid artificially defining a fixed threshold according to experience and reduce the model size and memory occupied during operation.This method and the method of using fixed threshold and global pruning proposed by LIU et al are used to compress VGGNet, ResNet, DenseNet and LeNet models respectively, and the model performance are tested on CIFAR, SVHN and MNIST data sets.Experimental results show that this method can find the optimal balance between model accuracy and pruning rate.The test error rate of the model after pruning is 0.02~1.52 percentage points lower than that of the comparison method.At the same time, the adaptive layered threshold judgment algorithm can also avoid the problem of reducing the whole layer in the global pruning of the comparison method.

Key words: deep learning, image recognition, Convolutional Neural Network(CNN), model compression, network pruning

中图分类号: