摘要：

卷积神经网络(CNN)的高计算和存储需求限制了其在资源有限的移动边缘设备上的应用推广。模型压缩技术能够在保持网络性能不变的同时显著降低CNN的计算量及参数量。通道剪枝已被证明在模型压缩方面的有效性, 然而现有的大多数通道剪枝方法的剪枝标准是基于评估通道的重要性或人工设定的评价标准, 此类方法的实现需要较多超参数的参与, 且剪枝方法的本身也缺乏自动性。基于上述通道剪枝方法的局限性, 提出一种新的基于斑马优化算法(ZOA)的通道自动剪枝方法。该方法首先使用k-medoids聚类剪枝以形成初步压缩的网络结构, 接着利用ZOA对初步压缩形成的网络结构进行迭代优化, 以搜索出最佳的紧凑网络结构。在两种图像数据集上的实验结果验证了该方法的高效性, 尤其在CIFAR-10数据集上, 该方法在ResNet-56上取得59.3%和56.7%的浮点运算数(FLOPs)和参数剪枝率的情况下, Top-1准确率提高了0.24百分点。

关键词: 通道剪枝, k-medoids聚类, 迭代搜索, 斑马优化算法, 自动剪枝

Abstract:

The high computational and storage requirements of Convolutional Neural Networks (CNNs) limit their application in resource-limited mobile edge devices. Model compression techniques can significantly reduce the computational effort and parameters of CNNs without degrading network performance. Channel pruning has been proven to be effective for model compression. However, the pruning criteria of most existing channel pruning methods are based on assessing the importance of the channels or manually setting the evaluation criteria. The implementation of such methods requires the inclusion of more hyperparameters, and the pruning methods themselves lack automaticity. To address these limitations, a novel automatic channel-pruning method based on the Zebra Optimization Algorithm (ZOA) is proposed. This method begins with cluster pruning using k-medoids to form an initial compressed network structure, which is then utilized to iteratively optimize the network structure formed by the initial compression to search for the best compact network structure. Experimental results show that on the CIFAR-10 dataset, the Top-1 accuracy of this method improves by 0.24 percentage points over the baseline, while achieving Floating-Point Operations (FLOPs) and parameter pruning rates of 59.3% and 56.7%, respectively, on ResNet-56.

Key words: channel pruning, k-medoids clustering, iterative search, Zebra Optimization Algorithm (ZOA), automatic pruning