面向分类网络的视觉语义解释模型

doi:10.19678/j.issn.1000-3428.0066122

摘要/Abstract

摘要：

深度学习的可解释性对推动其在军事场景中应用至关重要。当前主流方法使用类激活图的方式可视化最后一层卷积特征，然而对于网络根据该特征进行分类解释比较模糊。针对此问题，设计一种面向分类网络的视觉语义解释模型。综合考虑前向传播与反向传播，提出CGNIS算法获取对分类结果起重要作用的神经元，并将其映射到原图，得到更加细化的视觉特征。提出分类网络IRENet，在VGG16中间层添加SIRM和ECA对视觉特征进行识别，更加客观地提取视觉特征中包含的语义特征，并结合视觉特征、语义特征、重要神经元分数信息生成描述模型分类过程的解释性语句。在ImageNet2012数据集上提取10类图像进行实验，结果表明，删除CGNIS算法得到的某一类重要神经元后，对应类的分类准确率下降3%以上，在语义特征提取任务上，IRENet的F1值、准确率、精确率和召回率4项指标较ResNet101等分类网络提升2%以上。此外，利用CGNIS、IRENet对飞机类别进行实验，可生成模型对其分类过程的解释性语句。

关键词: 分类网络, 可解释性, 类激活图, 重要神经元, 语义信息

Abstract:

The interpretability of deep learning is critical for promoting its application in military scenarios. Currently used mainstream methods involve class activation maps to visualize the last layer of convolutional features, but it is not clear how the network classifies based on this feature. To solve this problem, a visual semantic interpretation model is proposed for classification network. First, considering forward propagation and backward propagation, the CGNIS algorithm is proposed to obtain neurons that are important for classification and map them to the original image to obtain more refined visual features. Second, the classification network Indian Rural Energy Network(IRENet) is proposed. SIRM and ECA are added to the middle layer of VGG16 to recognize visual features. The semantic information contained in these visual features can be extracted more objectively, and the obtained visual features, semantic features, and important neuron score information can be combined to generate the explanatory statements describing the classification process of the model. Finally, 10 classes are extracted from ImageNet2012 dataset for experiments. Experiments show that the classification accuracy of the corresponding class decreases by more than 3% after removing the important neurons of a certain class obtained by applying the CGNIS algorithm. In terms of semantic feature extraction task, the F1 value, accuracy, precision and recall of IRENet are improved by more than 2% compared with ResNet101 and other classification networks. In addition, the experiments are conducted on the aircraft class using CGNIS and IRENet to generate model statements explaining its classification process.

Key words: classification network, interpretability, class activation map, important neuron, semantic information

吕学强, 赵兴强, 贾智彬, 韩晶. 面向分类网络的视觉语义解释模型[J]. 计算机工程, 2023, 49(11): 220-230.

Xueqiang LÜ, Xingqiang ZHAO, Zhibin JIA, Jing HAN. Visual Semantic Interpretation Model for Classification Network[J]. Computer Engineering, 2023, 49(11): 220-230.

http://www.ecice06.com/CN/Y2023/V49/I11/220

图/表 18

图1 视觉语义解释模型

Fig.1 Visual semantic interpretation model

图2 VGG16 CGNIS网络推导过程

Fig.2 VGG16 CGNIS network derivation process

图3 神经元映射区域裁剪和生成流程

Fig.3 Procedure of neuron mapping region cropping and generation

图4 IRENet特征提取网络

Fig.4 IRENet feature extraction network

图5 SIRM模块

Fig.5 SIRM module

图6 ImageNet2012数据集样例

Fig.6 Sample of ImageNet2012 dataset

图7 数据集划分样例

Fig.7 Sample of dataset partition

图8 多类别重要神经元关注区域可视化展示

Fig.8 Visualization of important neuronal regions of interest in multiple categories

图9 神经元对各类别的重要程度

Fig.9 Importance of neurons for each category

图10 删除重要与不重要神经元对两类别准确率的影响

Fig.10 The effect of removing significant and insignificant neurons on the accuracy of two categories

图11 各模型生成的解释性语句对比

Fig.11 Comparison of explanatory statements generated by each model

参考文献 27

1	苏炯铭, 刘鸿福, 项凤涛, 等. 深度神经网络解释方法综述. 计算机工程, 2020, 46 (9): 1- 15. URL
	SU J M, LIU H F, XIANG F T, et al. Survey of interpretation methods for deep neural networks. Computer Engineering, 2020, 46 (9): 1- 15. URL
2	HUFF D T, WEISMAN A J, JERAJ R. Interpretation and visualization techniques for deep learning models in medical imaging. Physics in Medicine & Biology, 2021, 66 (4): 04TR01. URL
3	VAN DER LAAK J, LITJENS G, CIOMPI F. Deep learning in histopathology: the path to the clinic. Nature Medicine, 2021, 27 (5): 775- 784. doi: 10.1038/s41591-021-01343-4
4	何华灿. 重新找回人工智能的可解释性. 智能系统学报, 2019, 14 (3): 393- 412. URL
	HE H C. Refining the interpretability of artificial intelligence. CAAI Transactions on Intelligent Systems, 2019, 14 (3): 393- 412. URL
5	ZHOU B L, KHOSLA A, LAPEDRIZA A, et al. Learning deep features for discriminative localization[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 2921-2929.
6	FRISTON K, HEINS C, UELTZHÖFFER K, et al. Stochastic chaos and Markov blankets. Entropy, 2021, 23 (9): 1220. doi: 10.3390/e23091220
7	YU N, SONG H Y, SUN D Y, et al. Image annotation based on middle-layer convolution features of deep learning. Journal of Graphics, 2019, 40 (5): 872. URL
8	WANG Q L, WU B G, ZHU P F, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 11531-11539.
9	ZEILER M D, FERGUS R. Visualizing and understanding convolutional networks[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2014: 818-833.
10	HESSE R, SCHAUB-MEYER S, ROTH S. Fast axiomatic attribution for neural networks. Neural Information Processing Systems, 2021, 34, 19513- 19524. URL
11	SMILKOV D, THORAT N, KIM B, et al. SmoothGrad: removing noise by adding noise[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1706.03825.
12	HOLZINGER A, MALLE B, SARANTI A, et al. Towards multi-modal causability with graph neural networks enabling information fusion for explainable AI. Information Fusion, 2021, 71, 28- 37. doi: 10.1016/j.inffus.2021.01.008
13	SRINIVAS S, FLEURET F. Full-gradient representation for neural network visualization[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1905.00780.
14	DESAI S, RAMASWAMY H G. Ablation-CAM: visual explanations for deep convolutional network via gradient-free localization[C]//Proceedings of IEEE Winter Conference on Applications of Computer Vision. Washington D. C., USA: IEEE Press, 2020: 972-980.
15	ZHOU L J, MA C, SHI X C, et al. Salience-CAM: visual explanations from convolutional neural networks via salience score[C]//Proceedings of International Joint Conference on Neural Networks. Washington D. C., USA: IEEE Press, 2021: 1-8.
16	OQUAB M, BOTTOU L, LAPTEV I, et al. Learning and transferring mid-level image representations using convolutional neural networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2014: 1717-1724.
17	CHANG Y T, WANG Q S, HUNG W C, et al. Weakly-supervised semantic segmentation via sub-category exploration[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 8988-8997.
18	ZHANG Q S, WU Y N, ZHU S C. Interpretable convolutional neural networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 8827-8836.
19	BAU D, ZHU J Y, STROBELT H, et al. Understanding the role of individual units in a deep neural network. Proceedings of the National Academy of Sciences of the United States of America, 2020, 117 (48): 30071- 30078. URL
20	YIN H X, MALLYA A, VAHDAT A, et al. See through gradients: image batch recovery via GradInversion[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 16332-16341.
21	REN P Z, XIAO Y, CHANG X J, et al. A survey of deep active learning. ACM Computing Surveys, 2021, 54 (9): 1- 40. URL
22	CHILKURI N, ELIASMITH C. Parallelizing Legendre memory unit training[EB/OL]. [2022-09-10]. https://arxiv.org/abs/2102.11417.
23	THECKEDATH D, SEDAMKAR R R. Detecting affect states using VGG16, ResNet50 and SE-ResNet50 networks. SN Computer Science, 2020, 1 (2): 79. doi: 10.1007/s42979-020-0114-9
24	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 770-778.
25	TOUVRON H, CORD M, DOUZE M, et al. Training data-efficient image transformers & distillation through attention[EB/OL]. [2022-09-10]. https://arxiv.org/abs/2012.12877.
26	HOWARD A, SANDLER M, CHEN B, et al. Searching for MobileNetV3[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2020: 1314-1324.
27	TAN M X, LE Q V. EfficientNet: rethinking model scaling for convolutional neural networks[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1905.11946.

[1]	陈文轩, 曾碧, 郭植星. 融合多特征与语义图卷积网络的摔倒检测方法[J]. 计算机工程, 2023, 49(5): 277-285,294.
[2]	衡红军, 苗菁. 语义与句法信息加强的二元标记实体关系联合抽取[J]. 计算机工程, 2023, 49(4): 77-84.
[3]	侯占伟, 杨鑫, 申自浩, 王辉, 刘沛骞. 面向安全匿名集构建的多属性决策方法[J]. 计算机工程, 2023, 49(11): 169-177, 186.
[4]	张文豪, 廖列法, 王茹霞. 融合句法树多信息学习方面级情感分析[J]. 计算机工程, 2023, 49(10): 72-79.
[5]	普瑞丽, 王元龙, 李茹. 融合因果关系表征的阅读理解因果关系类选项判断[J]. 计算机工程, 2022, 48(7): 89-96.
[6]	罗嗣卿, 张志超, 岳琪. 基于改进SEGNET模型的图像语义分割[J]. 计算机工程, 2021, 47(4): 256-261.
[7]	文韬, 周稻祥, 李明. Mask R-CNN中特征不平衡问题的全局信息融合方法[J]. 计算机工程, 2021, 47(3): 256-260,268.
[8]	石彩霞, 李书琴, 刘斌. 多重检验加权融合的短文本相似度计算方法[J]. 计算机工程, 2021, 47(2): 95-102.
[9]	周培春, 吴兰岸. 多尺度多核高斯过程隐变量模型[J]. 计算机工程, 2021, 47(2): 285-292.
[10]	张腾飞, 周书仁, 彭建. 基于双孪生网络的自适应选择跟踪系统[J]. 计算机工程, 2020, 46(6): 103-107.
[11]	马喆康, 迪力亚尔·帕尔哈提, 早克热·卡德尔, 吐尔根·依布拉音, 西尔艾力·色提, 艾山·吾买尔. 一种集成深度学习模型的旅游问句文本分类算法[J]. 计算机工程, 2020, 46(11): 70-76.
[12]	肖锋,李茹娜. 语义信息引导下的显著目标检测算法[J]. 计算机工程, 2019, 45(4): 248-253.
[13]	费延佳,李福翠,邵枫. 基于深度学习的图像美观度评价[J]. 计算机工程, 2019, 45(3): 212-216,224.
[14]	明拓思宇,陈鸿昶,黄瑞阳,柳杨. 基于加权AMR图的语义子图预测摘要算法[J]. 计算机工程, 2018, 44(10): 292-297,302.
[15]	夏翠翠,刘梦赤,胡婕. 基于信息网模型的Web实体语义信息搜索平台[J]. 计算机工程, 2017, 43(3): 18-23,31.

选择文件类型/文献管理软件名称

选择包含的内容