摘要： 深度卷积神经网络的存储和计算需求巨大，难以在一些资源受限的嵌入式设备上进行部署。为尽可能减少深度卷积神经网络模型在推理过程中的资源消耗，引入基于几何中值的卷积核重要性判断标准，提出一种融合弱层惩罚的结构化非均匀卷积神经网络模型剪枝方法。使用欧式距离计算各层卷积核间的信息距离，利用各卷积层信息距离的数据分布特征识别弱层，通过基于贡献度的归一化函数进行弱层惩罚，消除各层间的差异性。在全局层面评估卷积核重要性，利用全局掩码技术对所有卷积核实现动态剪枝。在CIFAR-10、CIFAR-100和SVHN数据集上的实验结果表明，与SFP、PFEC、FPGM和MIL剪枝方法相比，该方法剪枝得到的VGG16单分支、Resnet多分支、Mobilenet-v1轻量化网络模型在保证精度损失较小的情况下，有效地减少了模型参数量和浮点操作数。

关键词: 模型剪枝, 弱层惩罚, 全局掩码, 欧式距离, 核重要性评估

Abstract: The extensive demand of convolutional neural networks for memory and computation makes it difficult to deploy them in resource-constrained embedded devices.To minimize the resource consumption of the deep convolutional neural network model during the inference process, this study introduces a criterion for judging the importance of the convolution kernel, based on the geometric median and further proposes a structured, non-uniform pruning method for convolutional neural network models with weak layer penalty.First, using the Euclidean distance, the algorithm calculates the information distance for each layer of the convolution kernel.Then, the data distribution characteristics of the information distance of each convolutional layer are used to identify the weak layers, and a normalization function based on the contribution degree is proposed to eliminate the difference between layers while weakening the redundant layers.Second, the importance of the convolution kernel is evaluated at the global level, and the global mask technique is used to achieve dynamic pruning.The experimental results on the CIFAR-10, CIFAR-100, and SVHN datasets demonstrate that compared with SFP, PFEC, FPGM, and MIL pruning methods, the proposed method prunes the VGG16 single-branch, Resnet multi-branch, and Mobilenet-v1 lightweight network models, effectively reducing the number of model parameters and Floating Points of Operations(FLOPs) while ensuring that the loss of precision is small.

Key words: model pruning, weak layer penalty, global mask, Euclidean distance, kernel importance evaluation

中图分类号: 

• TP391