计算机工程

• •    

基于自适应分层阈值判断的神经网络模型压缩研究

  

  • 发布日期:2021-01-11

Neural network model compression Method based on adaptive hierarchical threshold judgment

  • Published:2021-01-11

摘要: 随着深度学习的日渐火热和越发成熟,神经网络在各行各业都有着广泛的应用。伴随着应用环境 的多样化,卷积神经网络的精度也越来越高,但随之而来的是大量计算参数与内存存储。针对卷积神经网 络中卷积层参数冗余,运算效率低等问题,提出了一种基于分层阈值的自适应动态剪枝方法,通过逐层对 通道权重进行聚类分析得到自适应阈值,并通过该阈值对正则化后的输入模型进行修剪,最终使得模型尺 寸减小,运行时的内存占用减少。在 pytorch 的框架上对当前流行的网络结构进行了实验,其中在 Cifar10 数据集上,VGG16 网络模型剪枝率 56%,测试错误率 5.21%,参数 2.50M;DenseNet 网络模型剪枝率 36%, 测试错误率 4.21%,参数 0.69M;ResNet 网络模型剪枝率 34%,测试错误率 4.93%,参数 1.47M。其他数据 集结果将在论文中继续详细展示。

Abstract: With the increasing popularity and development of deep learning, neural networks have been widely applied in all walks of life. As the applying environments is diversified, convolutional neural networks call for accuracy. The higher the accuracy is, the larger the calculation parameters and memory storage are required. Aiming at redundancy problems of convolutional layer parameters and low computational efficiency in convolutional neural networks.We proposes an adaptive dynamic pruning method based on the gradient threshold. The adaptive threshold is obtained by clustering the channel weights layer by layer, and the regularized input model is pruned through this threshold.Thus the model size is reduced, while the running-time’s memory occupied at runtime is lowered. The current popular network structures, such as VGGNet, DenseNet, and ResNet, have been tested on the pytorch framework. The average compression ratio reaches 42%, which can still be increased by about 1% based on the original accuracy. And the effectiveness of the method on different image classification data sets is verified. The experimental results show that the adaptive dynamic pruning algorithm can reduce inference costs for VGGNet by up to 56% ,DenseNet by up to 36% and ResNet by up to 34% on CIFAR10 while keeping the accuracy.