基于特征融合与注意力机制的脑肿瘤分割算法

doi:10.19678/j.issn.1000-3428.0066311

摘要/Abstract

摘要：

脑肿瘤核磁共振成像(MRI)的准确分割对手术方案的制定和放疗计划具有重要意义。U-Net作为脑肿瘤分割领域应用最广泛的网络，具有较优的性能，但是存在跳跃连接中语义差距较大、MRI图像中跨通道信息利用不足的问题。为对脑肿瘤各区域进行准确分割，提出一种基于特征融合与注意力机制的改进U-Net模型FFCA-U-Net。在跳跃连接中设计特征融合模块代替U-Net中的直接拼接操作，以有效融合不同层次、不同尺度的特征信息，减小语义差距并调整感受野，增强网络对肿瘤特征的学习能力。在编码器中引入改进的三维坐标注意力机制，沿MRI图像的3个方向捕获跨通道信息，增强网络对脑肿瘤边界信息的感知能力，获得肿瘤子区域更精确的位置。此外，为快速获得肿瘤的相对位置、减少网络学习冗余，增加的掩码图像与MRI图像一起作为网络输入。在MSD数据集上的实验结果表明，FFCA-U-Net在增强肿瘤区域、非增强肿瘤区域和水肿区域的Dice系数分别为0.803 4、0.628 6和0.799 3，平均Dice为0.743 8，优于TransBTS、UNETR等其他先进网络。

关键词: 脑肿瘤, U-Net模型, 特征融合, 三维坐标注意力机制, 医学图像分割

Abstract:

The accurate segmentation of brain tumors in Magnetic Resonance Imaging(MRI)is of great significance for surgical planning and radiotherapy. U-Net, as the most widely used network in the field of brain tumor segmentation, has excellent performance; however, there are problems such as significant semantic differences in skip connections and insufficient utilization of cross channel information in MRI images. To accurately segment various regions of brain tumors, an improved U-Net model, Feature Fusion(FF)and attention mechanism FFCA-U-Net is proposed. The FF module is designed in skip connections to replace the direct splicing operation in U-Net, to effectively fuse feature information at different levels and scales, reduce semantic differences, adjust receptive fields, and enhance the network's learning ability for tumor features. An improved three-dimensional Coordinate Attention(CA)mechanism is introduced into the encoder to capture cross channel information along the three directions of the MRI image, enhancing the network's perception of brain tumor boundary information, thereby obtaining more accurate positions of tumor subregions. In addition, to readily obtain the relative position of tumors and reduce network learning redundancy, mask images are added alongside MRI images as network inputs. In the experimental results on the MSD dataset, the Dice coefficients of FFCA-U-Net in the enhanced tumor area, non-enhanced tumor area, and edema area are 0.803 4, 0.628 6, and 0.799 3, respectively, with an average Dice of 0.743 8, which is superior to those of other advanced networks such as TransBTS and UNETR.

Key words: brain tumor, U-Net model, Feature Fusion(FF), three-dimensional Coordinate Attention(CA) mechanism, medical image segmentation

褚张晴晴, 钟志强, 颜子夜, 战荫伟. 基于特征融合与注意力机制的脑肿瘤分割算法[J]. 计算机工程, 2023, 49(10): 154-161.

Zhangqingqing CHU, Zhiqiang ZHONG, Ziye YAN, Yinwei ZHAN. Brain Tumor Segmentation Algorithm Based on Feature Fusion and Attention Mechanism[J]. Computer Engineering, 2023, 49(10): 154-161.

https://www.ecice06.com/CN/Y2023/V49/I10/154

图/表 10

图1 FFCA-U-Net网络结构

Fig.1 Structure of FFCA-U-Net network

图2 肿瘤区域的掩码图像

Fig.2 Mask image of tumor region

图3 特征融合模块结构

Fig.3 Structure of feature fusion module

图4 改进的三维坐标注意力模块结构

Fig.4 Structure of improved three-dimensional coordinate attention mechanism

图5 UNETR和FFCA-U-Net的分割结果

Fig.5 Segmentation results of UNETR and FFCA-U-Net

图6 不同模型在矢状面、冠状面和水平面的分割结果对比

Fig.6 Comparison of segmentation results among different models in sagittal, coronal, and horizontal planes

参考文献 32

1	MENZE B H, JAKAB A, BAUER S, et al. The multimodal brain tumor image segmentation benchmark. IEEE Transactions on Medical Imaging, 2015, 34(10): 1993- 2024. doi: 10.1109/TMI.2014.2377694
2	LOUIS D N, PERRY A, WESSELING P, et al. The 2021 WHO classification of tumors of the central nervous system: a summary. Neuro-Oncology, 2021, 23(8): 1231- 1251. doi: 10.1093/neuonc/noab106
3	国家卫生健康委员会医政医管局. 脑胶质瘤诊疗规范(2018年版). 中华神经外科杂志, 2019, 35(3): 217- 239. doi: 10.3760/cma.j.issn.1001-2346.2019.03.001
	National Health Commission Medical Administration. Guidelines for the diagnosis and treatment of glioma (2018 edition). Chinese Journal of Neurosurgery, 2019, 35(3): 217- 239. doi: 10.3760/cma.j.issn.1001-2346.2019.03.001
4	WADHWA A, BHARDWAJ A, SINGH VERMA V. A review on brain tumor segmentation of MRI images. Magnetic Resonance Imaging, 2019, 61, 247- 259. doi: 10.1016/j.mri.2019.05.043
5	WANG G T, LI W Q, OURSELIN S, et al. Automatic brain tumor segmentation based on cascaded convolutional neural networks with uncertainty estimation. Frontiers in Computational Neuroscience, 2019, 13, 56. doi: 10.3389/fncom.2019.00056
6	BEN NACEUR M, SAOULI R, AKIL M, et al. Fully automatic brain tumor segmentation using end-to-end incremental deep neural networks in MRI images. Computer Methods and Programs in Biomedicine, 2018, 166, 39- 49. doi: 10.1016/j.cmpb.2018.09.007
7	LIU Z H, TONG L, CHEN L, et al. Deep learning based brain tumor segmentation: a survey. Complex and Intelligent Systems, 2023, 9, 1001- 1026. doi: 10.1007/s40747-022-00815-5
8	BAUER S, SEILER C, BARDYN T, et al. Atlas-based segmentation of brain tumor images using a Markov random field-based tumor growth model and non-rigid registration[C]//Proceedings of Annual International Conference of the IEEE Engineering in Medicine and Biology. Washington D. C., USA: IEEE Press, 2010: 4080-4083.
9	LEFKOVITS L, LEFKOVITS S, SZILÁGYI L. Brain tumor segmentation with optimized random forest[M]//GRIMI A, MENZE B, MAIER O, et al. Brainlesion: glioma, multiple sclerosis, stroke and traumatic brain injuries. Berlin, Germany: Springer, 2016: 88-99.
10	WU W, CHEN A Y C, ZHAO L, et al. Brain tumor detection and segmentation in a CRF (conditional random fields) framework with pixel-pairwise affinity and superpixel-level features. International Journal of Computer Assisted Radiology and Surgery, 2014, 9(2): 241- 253. doi: 10.1007/s11548-013-0922-7
11	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of the 18th International Conference Munich. Berlin, Germany: Springer, 2015: 234-241.
12	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2015: 3431-3440.
13	ZHOU Z W, SIDDIQUEE R M M, TAJBAKHSH N, et al. UNet++: a nested U-Net architecture for medical image segmentation[C]//Proceedings of International Workshop on Deep Learning in Medical Image Analysis. Berlin, Germany: Springer, 2018: 1-10.
14	LI X M, CHEN H, QI X J, et al. H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Transactions on Medical Imaging, 2018, 37(12): 2663- 2674. doi: 10.1109/TMI.2018.2845918
15	HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 2261-2269.
16	MILLETARI F, NAVAB N, AHMADI S A. V-Net: fully convolutional neural networks for volumetric medical image segmentation[C]//Proceedings of the 4th International Conference on 3D Vision (3DV). Washington D. C., USA: IEEE Press, 2016: 565-571.
17	王磐, 强彦, 杨晓棠, 等. 基于双注意力3D-UNet的肺结节分割网络模型. 计算机工程, 2021, 47(2): 307- 313. doi: 10.19678/j.issn.1000-3428.0057019
	WANG P, QIANG Y, YANG X T, et al. Network model for lung nodule segmentation based on double attention 3D-UNet. Computer Engineering, 2021, 47(2): 307- 313. doi: 10.19678/j.issn.1000-3428.0057019
18	TIAN Z, HE T, SHEN C H, et al. Decoders matter for semantic segmentation: data-dependent decoding enables flexible feature aggregation[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 3121-3130.
19	ZHONG Z Q, HE L A, CHEN C X, et al. Full-scale attention network for automated organ segmentation on head and neck CT and MR images. IET Image Processing, 2023, 17(3): 660- 673. doi: 10.1049/ipr2.12663
20	曹加旺, 田维维, 刘学玲, 等. 基于改进U-Net的人脑黑质致密部分割. 计算机工程, 2022, 48(11): 14-21, 29. URL
	CAO J W, TIAN W W, LIU X L, et al. Dense partial segmentation of substantia nigra of human brain based on improved U-Net. Computer Engineering, 2022, 48(11): 14-21, 29. URL
21	OKTAY O, SCHLEMPER J, FOLGOC L L, et al. Attention U-Net: learning where to look for the pancreas[EB/OL]. [2022-10-15]. https://arxiv.org/pdf/1804.03999.pdf.
22	郝华颖, 赵昆, 苏攀, 等. 一种基于改进ResU-Net的角膜神经分割算法. 计算机工程, 2021, 47(1): 217- 223. URL
	HAO H Y, ZHAO K, SU P, et al. A corneal nerve segmentation algorithm based on improved ResU-Net. Computer Engineering, 2021, 47(1): 217- 223. URL
23	KAUL C, MANANDHAR S, PEARS N. FocusNet: an attention-based fully convolutional network for medical image segmentation[C]//Proceedings of the 16th International Symposium on Biomedical Imaging. Washington D. C., USA: IEEE Press, 2019: 455-458.
24	WANG C Y, HE Y, LIU Y F, et al. ScleraSegNet: an improved U-Net model with attention for accurate sclera segmentation[C]//Proceedings of International Conference on Biometrics. Washington D. C., USA: IEEE Press, 2019: 1-8.
25	SAAD N M, ABU-BAKAR S, MUDA S, et al. Fully automated region growing segmentation of brain lesion in diffusion-weighted MRI. IAENG International Journal of Computer Science, 2012, 39(2): 155- 164.
26	QIN X F, WU C Z, CHANG H, et al. Match feature U-Net: dynamic receptive field networks for biomedical image segmentation. Symmetry, 2020, 12(8): 1230. doi: 10.3390/sym12081230
27	ÇIÇEK Ö, ABDULKADIR A, LIENKAMP S S, et al. 3D U-Net: learning dense volumetric segmentation from sparse annotation[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2016: 424-432.
28	HATAMIZADEH A, TANG Y C, NATH V, et al. UNETR: Transformers for 3D medical image segmentation[C]//Proceedings of the Winter Conference on Applications of Computer Vision. Washington D. C., USA: IEEE Press, 2022: 574-584.
29	WANG W X, CHEN C, DING M, et al. TransBTS: multimodal brain tumor segmentation using transformer[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2021: 109-119.
30	HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 7132-7141.
31	WOO S, PARK J C, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 3-19.
32	HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 13708-13717.

[1]	李俊仪, 李向阳, 龙朝勋, 李海燕, 李红松, 余鹏飞. 基于多级区域选择与跨层特征融合的野生菌分类[J]. 计算机工程, 2024, 50(9): 179-188.
[2]	郭敏, 张熙涵, 李阳. 融合注意力的教师互一致性半监督医学图像分割[J]. 计算机工程, 2024, 50(9): 313-323.
[3]	张华青, 夏张涛, 陆晓庆, 童基均. 基于字形特征的血管外科命名实体识别[J]. 计算机工程, 2024, 50(8): 13-21.
[4]	李华昱, 张智康, 闫阳, 岳阳. 基于知识图谱增强的领域多模态实体识别[J]. 计算机工程, 2024, 50(8): 31-39.
[5]	刘锁兰, 王炎, 王洪元, 朱生升. 基于多流语义图卷积网络的人体行为识别[J]. 计算机工程, 2024, 50(8): 64-74.
[6]	赵婉秋, 张俊虎, 李海涛. 用于建筑物分割的平行结构特征融合网络[J]. 计算机工程, 2024, 50(8): 239-248.
[7]	赵宏, 王枭. 基于Swin-Transformer的黑色素瘤图像病灶分割研究[J]. 计算机工程, 2024, 50(8): 249-258.
[8]	王富平, 刘鸿玮, 张锲石, 段冠庄. 基于深度特征抑制的遮挡人脸识别网络[J]. 计算机工程, 2024, 50(8): 259-269.
[9]	闵莉, 董冰洁, 安冬. 基于多注意力机制与跨特征融合的语义分割算法[J]. 计算机工程, 2024, 50(8): 282-289.
[10]	高爽, 史轶伦, 徐巧枝, 于磊. 基于对比学习的非对称编解码结构的心脏MRI分割研究[J]. 计算机工程, 2024, 50(8): 290-300.
[11]	陈宇航, 杨勇, 先木斯亚·买买提明, 帕力旦·吐尔逊, 樊小超, 任鸽, 刁宇峰. 基于主题感知和语义增强的作文自动评分方法[J]. 计算机工程, 2024, 50(8): 363-371.
[12]	谭巨全, 王然. 特征融合下田径录像3D人体动作DTW捕捉算法[J]. 计算机工程, 2024, 50(7): 71-78.
[13]	张溢文, 蔡满春, 陈咏豪, 朱懿, 姚利峰. 融合空间特征的多尺度深度伪造检测方法[J]. 计算机工程, 2024, 50(7): 240-250.
[14]	王晋涛, 秦昂, 张元, 陈一飞, 王廷凤, 谢承霖, 邹刚. 基于注意力增强与特征融合的中文医学实体识别[J]. 计算机工程, 2024, 50(7): 324-332.
[15]	李亚康, 陈刚. 小角中子散射物理模型自动化筛选[J]. 计算机工程, 2024, 50(6): 56-64.

选择文件类型/文献管理软件名称

选择包含的内容