摘要：

由于没有跳跃连接结构的深度神经网络在超过一定深度后难以训练, 因此现有的深度神经网络模型大都采用跳跃连接结构来解决优化问题并提高泛化性能。然而, 人们对于跳跃连接结构如何影响深度神经网络特征提取的研究还较少, 在大多数情况下, 这些模型仍然被认为是黑盒。为了分析跳跃连接结构对深度神经网络特征提取的影响, 从特征可视化的角度, 以基于扰动的方法为切入点, 提出一种在保持图像总体颜色分布和轮廓特征基本不变的前提下弱化图像细节特征的扰动方法, 并将其命名为网格乱序模糊(GSB)方法。同时, 研究结合特征可视化中的激活最大化(AM)方法和所提出的GSB扰动方法, 分析了拥有不同程度跳跃连接结构的经典图像分类深度神经网络模型VGG 19, ResNet 50和DenseNet 201。实验结果表明, 没有跳跃连接结构的深度神经网络只提取了图像中较强的特征, 提取的特征数量比较少, 而拥有跳跃连接结构的深度神经网络提取了图像中更多的特征, 但是这些特征相对较弱; 跳跃连接结构使模型更关注图像的局部颜色分布和全局总体轮廓, 而不过多依赖图像细节特征, 并且跳跃连接结构越密集, 这种趋势越强。

关键词: 深度神经网络, 跳跃连接结构, 特征可视化, 激活最大化, 扰动方法, 可解释性

Abstract:

Training deep neural networks without skip connection structures is challenging when the depth of the networks is high. Thus, to address optimization issues and enhance generalization performance, skip connection structures have been integrated into the most recent deep neural network models. However, the effect of skip connection structures on feature extraction in deep neural networks has not yet been clarified; in most cases, these models are considered black boxes. Toward the elucidation of this effect, this study focuses on perturbation-based methods and introduces a method called Grid-Shuffled Blurring (GSB). This method aims to reduce the fine-grained details within an image, while maintaining its overall color distribution and contour characteristics. This study employs the Activation Maximization (AM) method for feature visualization and the GSB perturbation method to analyze classic deep neural network models such as VGG 19, ResNet 50, and DenseNet 201 in image classification tasks, which have different levels of skip connection structures. Experimental results show that the neural networks without the skip connection structures extract only stronger features from images, resulting in fewer extracted features, whereas those with the skip connection structures extract more features from images, albeit weaker ones. Moreover, the skip connection structures cause the models to focus more on the local color distribution and global contours of images, rather than the detailed features of images. The more the skip connection structures, the stronger is the trend.

Key words: deep neural network, skip connection structures, feature visualization, Activation Maximization(AM), perturbation method, interpretability