全局相关块级自注意力的食管癌前病变区域分割

doi:10.19678/j.issn.1000-3428.0065311

计算机工程 ›› 2023, Vol. 49 ›› Issue (7): 313-320. doi: 10.19678/j.issn.1000-3428.0065311

• 开发研究与工程应用 • 上一篇

全局相关块级自注意力的食管癌前病变区域分割

刘波¹, 李小霞¹^,²^,*, 秦佳敏³, 周颖玥¹^,²

1. 西南科技大学信息工程学院, 四川绵阳 621000
2. 特殊环境机器人技术四川省重点实验室, 四川绵阳 621000
3. 四川绵阳四0四医院消化内科, 四川绵阳 621000

收稿日期:2022-07-21 出版日期:2023-07-15 发布日期:2022-09-28
通讯作者: 李小霞
作者简介:
刘波（1999—），男，硕士研究生，主研方向为医学图像分割、深度学习
秦佳敏，硕士
周颖玥，副教授、博士
基金资助:
国家自然科学基金(62071399); 四川省科技计划(2021YFG0383)

Segmentation for Esophageal Precancerous Lesions Region Using Globally Correlated Block-Level Self-Attention

Bo LIU¹, Xiaoxia LI¹^,²^,*, Jiamin QIN³, Yingyue ZHOU¹^,²

1. School of Information Engineering, Southwest University of Science and Technology, Mianyang 621000, Sichuan, China
2. Robot Technology Used for Special Environment Key Laboratory of Sichuan Province, Mianyang 621000, Sichuan, China
3. Department of Gastroenterology, Sichuan Mianyang 404 Hospital, Mianyang 621000, Sichuan, China

Received:2022-07-21 Online:2023-07-15 Published:2022-09-28
Contact: Xiaoxia LI

摘要/Abstract

摘要：

食管内镜图像多类癌前病变区域的类间特征差异小且个体差异大，难以实现高精度分割，使用自注意力机制可提取远距离依赖信息以获取判别性特征，但是计算开销大。为此，提出一种全局相关块级自注意力（GC-BLSA）方法，用于食管癌前病变区域分割。利用块级自注意力对骨干网络的输出特征进行分块，并在多个特征块上引入自注意力机制，降低网络参数量和计算量，采用块相关机制建模每个特征块和整个特征图之间的关系，解决每个特征块在单独使用自注意力时无法提取与全局相关的远距离依赖信息的问题。在此基础上，在块级自注意力模块中引入相对位置偏移对位置信息进行补充，有效提高网络分割精度。实验结果表明，在四分类食管癌数据集上该方法的分割指标mIoU和F1值分别为50.213%和63.786%，相比传统自注意力Non-local模块分别提高3.744和4.297个百分点，参数量和计算量分别下降26.38%和10.62%。

Abstract:

The inter-class feature differences and individual differences between multiple types of precancerous lesions in esophageal endoscopic images are small, making it difficult to achieve high-precision segmentation.The use of self-attention mechanism can extract long-distance dependent information to obtain discriminative features, but the computational cost is high.This study proposes the Globally Correlated Block-Level Self-Attention(GC-BLSA) method for the segmentation of esophageal precancerous lesions.Block-Level Self-Attention(BLSA) is used to block the output features of the backbone network, and self-attention mechanism is introduced on multiple feature blocks to reduce the amount of network parameters and computation.The Block Correlation Mechanism(BCM) is used to model the relationship between each feature block and the entire feature map, solving the problem of each feature block being unable to extract long-distance dependency information related to the entire local area when using self-attention alone.Based on this, a relative position offset is introduced in the BLSA module to supplement position information, effectively improving the network segmentation accuracy.The experimental results show that the segmentation indicators mIoU and F1 values of the proposed method on the four-classification esophageal cancer dataset are 50.213% and 63.786%, which are 3.744 and 4.297 percentage points higher than the traditional self-attention Non-local module, respectively.The parameter quantity and computation of the proposed method are reduced by 26.38% and 10.62%, respectively.

Key words: Block Level Self-Attention(BLSA), Block Correlation Mechanism(BCM), relative position offset, block correlation matrix, multiple types of esophageal precancerous lesions

刘波, 李小霞, 秦佳敏, 周颖玥. 全局相关块级自注意力的食管癌前病变区域分割[J]. 计算机工程, 2023, 49(7): 313-320.

Bo LIU, Xiaoxia LI, Jiamin QIN, Yingyue ZHOU. Segmentation for Esophageal Precancerous Lesions Region Using Globally Correlated Block-Level Self-Attention[J]. Computer Engineering, 2023, 49(7): 313-320.

http://www.ecice06.com/CN/Y2023/V49/I7/313

图/表 8

图1 GC-BLSA网络整体结构

Fig.1 Overall structure of the GC-BLSA network

图2 Non-local模块和块级自注意力模块对比

Fig.2 Comparison of Non-local module and block-level self-attention module

图3 带有相对位置偏移的自注意力模块

Fig.3 Self-attention module with relative position offset

图4 4种病变类型实例图

Fig.4 Example diagram of four types of lesions

表1 损失函数权重在食管癌数据集上的消融实验结果

Table 1 Loss function weights in an ablation experiment on the esophageal cancer dataset

实验编号	$ \alpha $	$ 1-\alpha $	mIoU/%	F1值/%
1	1.0	0.0	48.753	61.663
2	0.9	0.1	47.766	61.050
3	0.8	0.2	50.213	63.786
4	0.7	0.3	48.871	62.017

表2 BLSA模块、相对位置偏移与BCM模块的消融实验结果

Table 2 Ablation experiment results of BLSA module, relative position offset and BCM module %

实验编号	BLSA	+POS	BCM	mIoU	F1值
1	—	—	—	46.469	59.489
2	√	—	—	45.693	58.280
3	√	√	—	47.223	60.727
4	√	—	√	46.709	59.738
5	√	√	√	50.213	63.786

图5 不同实验的参数量和计算量对比

Fig.5 Parameter quantity and calculation comparison among different experiments

表3 不同网络在食管癌数据集上的精度、参数量和计算量对比

Table 3 Precision, parameters quantity and computations comparison among different networks on esophageal cancer datasets

实验编号	网络类型	方法	参数量/10⁶	FLOPs/10⁹	mIoU/%	F1值/%
1	CNN	CCNet^[17]	66.55	54.04	47.664	60.928
2		DeepLabV3+^[14]	70.07	264.60	49.188	62.531
3		PSPNet^[16]	65.70	78.57	49.240	61.946
4		DANet^[24]	66.43	76.57	48.544	62.015
5		ResNet101+Non-local	88.00	59.53	46.469	59.489
6		GC-BLSA	64.79	53.21	50.213	63.786
7	全注意力	Axial-DeepLab	63.11	136.44	46.075	58.712
8	Transformer+CNN	TransUnet^[22]	95.55	63.61	48.515	61.760
9	Transformer+CNN	SETR^[25]	164.76	60.99	41.302	53.418

参考文献 25

1	罗衡荣. 基于胃镜图像的食管早癌病灶计算机辅助诊断方法研究[D]. 成都: 电子科技大学, 2015.
	LUO H R. Study on the computer-assisted diagnostic of early esophageal cancer based on endoscopy images[D]. Chengdu: University of Electronic Science and Technology of China, 2015. (in Chinese)
2	牛娜, 张文俊, 陆小锋, 等. 早期食管癌诊断的多色彩空间定量分析方法. 电视技术, 2012, 36 (11): 134-137, 147 URL
	NIU N, ZHANG W J, LU X F, et al. Quantitative analysis in multi-color spaces for early esophageal cancer diagnosis. Video Engineering, 2012, 36 (11): 134-137, 147 URL
3	WANG X, GIRSHICK R, GUPTA A, et al. Non-local neural networks[C]//Proceedings of the Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 7794-7803.
4	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2015: 3431-3440.
5	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2015: 234-241.
6	吉彬, 任建君, 郑秀娟, 等. 改进U-Net在喉白斑病灶分割中的应用. 计算机工程, 2020, 46 (9): 248- 253. URL
	JI B, REN J J, ZHENG X J, et al. Application of improved U-Net in segmentation of laryngeal leukoplakia lesion. Computer Engineering, 2020, 46 (9): 248- 253. URL
7	CHEN W L, ZHANG Y, HE J J, et al. Prostate segmentation using 2D bridged U-net[C]//Proceedings of International Joint Conference on Neural Networks. Washington D. C., USA: IEEE Press, 2019: 1-7.
8	ZHOU Z W, RAHMAN SIDDIQUEE M M, TAJBAKHSH N, et al. UNet++: a nested U-Net architecture for medical image segmentation[C]//Proceedings of International Workshop on Deep Learning in Medical Image Analysis. Berlin, Germany: Springer, 2018: 3-11.
9	ALOM M Z, HASAN M, YAKOPCIC C, et al. Recurrent residual convolutional neural network based on U-Net (R2U-Net) for medical image segmentation[EB/OL]. [2022-06-18]. https://arxiv.org/ftp/arxiv/papers/1802/1802.06955.pdf.
10	WANG W, YU K C, HUGONOT J, et al. Recurrent U-net for resource-constrained segmentation[C]//Proceedings of International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2020: 2142-2151.
11	CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40 (4): 834- 848. doi: 10.1109/TPAMI.2017.2699184
12	CHEN L C, PAPANDREOU G, KOKKINOS I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[EB/OL]. [2022-06-18]. https://arxiv.org/pdf/1412.7062.pdf.
13	CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[EB/OL]. [2022-06-18]. https://arxiv.org/pdf/1706.05587.pdf.
14	CHEN L C, ZHU Y K, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 833-851.
15	谭镭, 孙怀江. SKASNet: 用于语义分割的轻量级卷积神经网络. 计算机工程, 2020, 46 (9): 261- 267. URL
	TAN L, SUN H J. SKASNet: lightweight convolutional neural network for semantic segmentation. Computer Engineering, 2020, 46 (9): 261- 267. URL
16	ZHAO H S, SHI J P, QI X J, et al. Pyramid scene parsing network[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 6230-6239.
17	HUANG Z L, WANG X G, HUANG L C, et al. CCNet: criss-cross attention for semantic segmentation[C]//Proceedings of International Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 603-612.
18	景庄伟, 管海燕, 彭代峰, 等. 基于深度神经网络的图像语义分割研究综述. 计算机工程, 2020, 46 (10): 1- 17. URL
	JING Z W, GUAN H Y, PENG D F, et al. Survey of research in image semantic segmentation based on deep neural network. Computer Engineering, 2020, 46 (10): 1- 17. URL
19	WANG H Y, ZHU Y K, GREEN B, et al. Axial-DeepLab: stand-alone axial-attention for panoptic segmentation[C]//Proceedings of Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 108-126.
20	VALANARASU J M J, OZA P, HACIHALILOGLU I, et al. Medical transformer: gated axial-attention for medical image segmentation[C]//Proceedings of Medical Image Computing and Computer Assisted Intervention. Berlin, Germany: Springer, 2021: 36-46.
21	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2017: 5998-6008.
22	CHEN J N, LU Y Y, YU Q H, et al. TransUNet: Transformers make strong encoders for medical image segmentation[EB/OL]. [2022-06-18]. https://arxiv.org/abs/2102.04306.
23	GUO R H, NIU D T, QU L, et al. SOTR: segmenting objects with transformers[C]//Proceedings of International Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2022: 7137-7146.
24	FU J, LIU J, TIAN H J, et al. Dual attention network for scene segmentation[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 3141-3149.
25	ZHENG S X, LU J C, ZHAO H S, et al. Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 6877-6886.

选择文件类型/文献管理软件名称

选择包含的内容

全局相关块级自注意力的食管癌前病变区域分割

Segmentation for Esophageal Precancerous Lesions Region Using Globally Correlated Block-Level Self-Attention

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 25

相关文章 0

编辑推荐

Metrics

本文评价

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

全局相关块级自注意力的食管癌前病变区域分割

Segmentation for Esophageal Precancerous Lesions Region Using Globally Correlated Block-Level Self-Attention

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 25

相关文章 0

编辑推荐

Metrics

本文评价