Visual Semantic Interpretation Model for Classification Network

doi:10.19678/j.issn.1000-3428.0066122

Abstract

Abstract:

The interpretability of deep learning is critical for promoting its application in military scenarios. Currently used mainstream methods involve class activation maps to visualize the last layer of convolutional features, but it is not clear how the network classifies based on this feature. To solve this problem, a visual semantic interpretation model is proposed for classification network. First, considering forward propagation and backward propagation, the CGNIS algorithm is proposed to obtain neurons that are important for classification and map them to the original image to obtain more refined visual features. Second, the classification network Indian Rural Energy Network(IRENet) is proposed. SIRM and ECA are added to the middle layer of VGG16 to recognize visual features. The semantic information contained in these visual features can be extracted more objectively, and the obtained visual features, semantic features, and important neuron score information can be combined to generate the explanatory statements describing the classification process of the model. Finally, 10 classes are extracted from ImageNet2012 dataset for experiments. Experiments show that the classification accuracy of the corresponding class decreases by more than 3% after removing the important neurons of a certain class obtained by applying the CGNIS algorithm. In terms of semantic feature extraction task, the F1 value, accuracy, precision and recall of IRENet are improved by more than 2% compared with ResNet101 and other classification networks. In addition, the experiments are conducted on the aircraft class using CGNIS and IRENet to generate model statements explaining its classification process.

Key words: classification network, interpretability, class activation map, important neuron, semantic information

摘要：

深度学习的可解释性对推动其在军事场景中应用至关重要。当前主流方法使用类激活图的方式可视化最后一层卷积特征，然而对于网络根据该特征进行分类解释比较模糊。针对此问题，设计一种面向分类网络的视觉语义解释模型。综合考虑前向传播与反向传播，提出CGNIS算法获取对分类结果起重要作用的神经元，并将其映射到原图，得到更加细化的视觉特征。提出分类网络IRENet，在VGG16中间层添加SIRM和ECA对视觉特征进行识别，更加客观地提取视觉特征中包含的语义特征，并结合视觉特征、语义特征、重要神经元分数信息生成描述模型分类过程的解释性语句。在ImageNet2012数据集上提取10类图像进行实验，结果表明，删除CGNIS算法得到的某一类重要神经元后，对应类的分类准确率下降3%以上，在语义特征提取任务上，IRENet的F1值、准确率、精确率和召回率4项指标较ResNet101等分类网络提升2%以上。此外，利用CGNIS、IRENet对飞机类别进行实验，可生成模型对其分类过程的解释性语句。

关键词: 分类网络, 可解释性, 类激活图, 重要神经元, 语义信息

Xueqiang LÜ, Xingqiang ZHAO, Zhibin JIA, Jing HAN. Visual Semantic Interpretation Model for Classification Network[J]. Computer Engineering, 2023, 49(11): 220-230.

吕学强, 赵兴强, 贾智彬, 韩晶. 面向分类网络的视觉语义解释模型[J]. 计算机工程, 2023, 49(11): 220-230.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0066122

https://www.ecice06.com/EN/Y2023/V49/I11/220

Figures/Tables 18

Fig.1 Visual semantic interpretation model

Fig.2 VGG16 CGNIS network derivation process

Fig.3 Procedure of neuron mapping region cropping and generation

Fig.4 IRENet feature extraction network

Fig.5 SIRM module

Fig.6 Sample of ImageNet2012 dataset

Fig.7 Sample of dataset partition

Fig.8 Visualization of important neuronal regions of interest in multiple categories

Fig.9 Importance of neurons for each category

Fig.10 The effect of removing significant and insignificant neurons on the accuracy of two categories

Fig.11 Comparison of explanatory statements generated by each model

References 27

1	苏炯铭, 刘鸿福, 项凤涛, 等. 深度神经网络解释方法综述. 计算机工程, 2020, 46 (9): 1- 15. URL
	SU J M, LIU H F, XIANG F T, et al. Survey of interpretation methods for deep neural networks. Computer Engineering, 2020, 46 (9): 1- 15. URL
2	HUFF D T, WEISMAN A J, JERAJ R. Interpretation and visualization techniques for deep learning models in medical imaging. Physics in Medicine & Biology, 2021, 66 (4): 04TR01. URL
3	VAN DER LAAK J, LITJENS G, CIOMPI F. Deep learning in histopathology: the path to the clinic. Nature Medicine, 2021, 27 (5): 775- 784. doi: 10.1038/s41591-021-01343-4
4	何华灿. 重新找回人工智能的可解释性. 智能系统学报, 2019, 14 (3): 393- 412. URL
	HE H C. Refining the interpretability of artificial intelligence. CAAI Transactions on Intelligent Systems, 2019, 14 (3): 393- 412. URL
5	ZHOU B L, KHOSLA A, LAPEDRIZA A, et al. Learning deep features for discriminative localization[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 2921-2929.
6	FRISTON K, HEINS C, UELTZHÖFFER K, et al. Stochastic chaos and Markov blankets. Entropy, 2021, 23 (9): 1220. doi: 10.3390/e23091220
7	YU N, SONG H Y, SUN D Y, et al. Image annotation based on middle-layer convolution features of deep learning. Journal of Graphics, 2019, 40 (5): 872. URL
8	WANG Q L, WU B G, ZHU P F, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 11531-11539.
9	ZEILER M D, FERGUS R. Visualizing and understanding convolutional networks[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2014: 818-833.
10	HESSE R, SCHAUB-MEYER S, ROTH S. Fast axiomatic attribution for neural networks. Neural Information Processing Systems, 2021, 34, 19513- 19524. URL
11	SMILKOV D, THORAT N, KIM B, et al. SmoothGrad: removing noise by adding noise[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1706.03825.
12	HOLZINGER A, MALLE B, SARANTI A, et al. Towards multi-modal causability with graph neural networks enabling information fusion for explainable AI. Information Fusion, 2021, 71, 28- 37. doi: 10.1016/j.inffus.2021.01.008
13	SRINIVAS S, FLEURET F. Full-gradient representation for neural network visualization[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1905.00780.
14	DESAI S, RAMASWAMY H G. Ablation-CAM: visual explanations for deep convolutional network via gradient-free localization[C]//Proceedings of IEEE Winter Conference on Applications of Computer Vision. Washington D. C., USA: IEEE Press, 2020: 972-980.
15	ZHOU L J, MA C, SHI X C, et al. Salience-CAM: visual explanations from convolutional neural networks via salience score[C]//Proceedings of International Joint Conference on Neural Networks. Washington D. C., USA: IEEE Press, 2021: 1-8.
16	OQUAB M, BOTTOU L, LAPTEV I, et al. Learning and transferring mid-level image representations using convolutional neural networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2014: 1717-1724.
17	CHANG Y T, WANG Q S, HUNG W C, et al. Weakly-supervised semantic segmentation via sub-category exploration[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 8988-8997.
18	ZHANG Q S, WU Y N, ZHU S C. Interpretable convolutional neural networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 8827-8836.
19	BAU D, ZHU J Y, STROBELT H, et al. Understanding the role of individual units in a deep neural network. Proceedings of the National Academy of Sciences of the United States of America, 2020, 117 (48): 30071- 30078. URL
20	YIN H X, MALLYA A, VAHDAT A, et al. See through gradients: image batch recovery via GradInversion[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 16332-16341.
21	REN P Z, XIAO Y, CHANG X J, et al. A survey of deep active learning. ACM Computing Surveys, 2021, 54 (9): 1- 40. URL
22	CHILKURI N, ELIASMITH C. Parallelizing Legendre memory unit training[EB/OL]. [2022-09-10]. https://arxiv.org/abs/2102.11417.
23	THECKEDATH D, SEDAMKAR R R. Detecting affect states using VGG16, ResNet50 and SE-ResNet50 networks. SN Computer Science, 2020, 1 (2): 79. doi: 10.1007/s42979-020-0114-9
24	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 770-778.
25	TOUVRON H, CORD M, DOUZE M, et al. Training data-efficient image transformers & distillation through attention[EB/OL]. [2022-09-10]. https://arxiv.org/abs/2012.12877.
26	HOWARD A, SANDLER M, CHEN B, et al. Searching for MobileNetV3[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2020: 1314-1324.
27	TAN M X, LE Q V. EfficientNet: rethinking model scaling for convolutional neural networks[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1905.11946.

[1]	WU Xing, YIN Haoyu, YAO Junfeng, LI Weimin, QIAN Quan. Multimodal Sentiment Analysis for Video Data [J]. Computer Engineering, 2024, 50(6): 218-227.
[2]	Weijian LI, Huijun HU. Video Description Generation Method Based on Latent Feature Augmented Network [J]. Computer Engineering, 2024, 50(2): 266-272.
[3]	CHEN Wenxuan, ZENG Bi, GUO Zhixing. Fall Detection Method Integrating Multi-Feature and Semantic Graph Convolution Network [J]. Computer Engineering, 2023, 49(5): 277-285,294.
[4]	HENG Hongjun, MIAO Jing. Joint Extraction of Binary Tagging Entity Relation for Enhanced Semantic and Syntactic Information [J]. Computer Engineering, 2023, 49(4): 77-84.
[5]	Zhanwei HOU, Xin YANG, Zihao SHEN, Hui WANG, Peiqian LIU. Multi-attribute Decision-making Method for Secure Anonymous Set Construction [J]. Computer Engineering, 2023, 49(11): 169-177, 186.
[6]	Wenhao ZHANG, Liefa LIAO, Ruxia WANG. Aspect-Level Sentiment Analysis for Multi-Information Learning by Fusing Syntax Trees [J]. Computer Engineering, 2023, 49(10): 72-79.
[7]	PU Ruili, WANG Yuanlong, LI Ru. Judgement of Causality Options in Reading Comprehension Incorporating Representation of Causality [J]. Computer Engineering, 2022, 48(7): 89-96.
[8]	YANG Wenxue, WU Fei, GUO Tong, XIAO Limin. Adversarial Sample Defense Method Based on Noise Dissolution [J]. Computer Engineering, 2022, 48(4): 158-164.
[9]	LUO Siqing, ZHANG Zhichao, YUE Qi. Semantic Image Segmentation Based on Improved SEGNET Model [J]. Computer Engineering, 2021, 47(4): 256-261.
[10]	WEN Tao, ZHOU Daoxiang, LI Ming. Global Information Fusion Method for Feature Imbalance Problem in Mask R-CNN [J]. Computer Engineering, 2021, 47(3): 256-260,268.
[11]	SHI Caixia, LI Shuqin, LIU Bin. Method for Calculating Short Text Similarity Using Multi-Check Weighted Fusion [J]. Computer Engineering, 2021, 47(2): 95-102.
[12]	ZHOU Peichun, WU Lan'an. Multi-Scale Multi-Kernel Gaussian Process Latent Variable Model [J]. Computer Engineering, 2021, 47(2): 285-292.
[13]	ZHANG Tengfei, ZHOU Shuren, PENG Jian. Adaptive Selective Tracking System Based on Twofold Siamese Network [J]. Computer Engineering, 2020, 46(6): 103-107.
[14]	MA Zhekang, Diliyaer Paerhati, Zaokere Kadeer, Tuergen Yibulayin, Xerali Setti, Aishan Wumaier. A Classification Algorithm for Tourist Question Texts Integrated with Deep Learning Models [J]. Computer Engineering, 2020, 46(11): 70-76.
[15]	XIAO Feng,LI Runa. Salient Object Detection Algorithm Under Guidance of Semantic Information [J]. Computer Engineering, 2019, 45(4): 248-253.

Please choose a citation manager

Content to export