基于特征融合与注意力机制的脑肿瘤分割算法

doi:10.19678/j.issn.1000-3428.0066311

摘要/Abstract

摘要：

脑肿瘤核磁共振成像(MRI)的准确分割对手术方案的制定和放疗计划具有重要意义。U-Net作为脑肿瘤分割领域应用最广泛的网络，具有较优的性能，但是存在跳跃连接中语义差距较大、MRI图像中跨通道信息利用不足的问题。为对脑肿瘤各区域进行准确分割，提出一种基于特征融合与注意力机制的改进U-Net模型FFCA-U-Net。在跳跃连接中设计特征融合模块代替U-Net中的直接拼接操作，以有效融合不同层次、不同尺度的特征信息，减小语义差距并调整感受野，增强网络对肿瘤特征的学习能力。在编码器中引入改进的三维坐标注意力机制，沿MRI图像的3个方向捕获跨通道信息，增强网络对脑肿瘤边界信息的感知能力，获得肿瘤子区域更精确的位置。此外，为快速获得肿瘤的相对位置、减少网络学习冗余，增加的掩码图像与MRI图像一起作为网络输入。在MSD数据集上的实验结果表明，FFCA-U-Net在增强肿瘤区域、非增强肿瘤区域和水肿区域的Dice系数分别为0.803 4、0.628 6和0.799 3，平均Dice为0.743 8，优于TransBTS、UNETR等其他先进网络。

关键词: 脑肿瘤, U-Net模型, 特征融合, 三维坐标注意力机制, 医学图像分割

Abstract:

The accurate segmentation of brain tumors in Magnetic Resonance Imaging(MRI)is of great significance for surgical planning and radiotherapy. U-Net, as the most widely used network in the field of brain tumor segmentation, has excellent performance; however, there are problems such as significant semantic differences in skip connections and insufficient utilization of cross channel information in MRI images. To accurately segment various regions of brain tumors, an improved U-Net model, Feature Fusion(FF)and attention mechanism FFCA-U-Net is proposed. The FF module is designed in skip connections to replace the direct splicing operation in U-Net, to effectively fuse feature information at different levels and scales, reduce semantic differences, adjust receptive fields, and enhance the network's learning ability for tumor features. An improved three-dimensional Coordinate Attention(CA)mechanism is introduced into the encoder to capture cross channel information along the three directions of the MRI image, enhancing the network's perception of brain tumor boundary information, thereby obtaining more accurate positions of tumor subregions. In addition, to readily obtain the relative position of tumors and reduce network learning redundancy, mask images are added alongside MRI images as network inputs. In the experimental results on the MSD dataset, the Dice coefficients of FFCA-U-Net in the enhanced tumor area, non-enhanced tumor area, and edema area are 0.803 4, 0.628 6, and 0.799 3, respectively, with an average Dice of 0.743 8, which is superior to those of other advanced networks such as TransBTS and UNETR.

Key words: brain tumor, U-Net model, Feature Fusion(FF), three-dimensional Coordinate Attention(CA) mechanism, medical image segmentation

褚张晴晴, 钟志强, 颜子夜, 战荫伟. 基于特征融合与注意力机制的脑肿瘤分割算法[J]. 计算机工程, 2023, 49(10): 154-161.

Zhangqingqing CHU, Zhiqiang ZHONG, Ziye YAN, Yinwei ZHAN. Brain Tumor Segmentation Algorithm Based on Feature Fusion and Attention Mechanism[J]. Computer Engineering, 2023, 49(10): 154-161.

http://www.ecice06.com/CN/Y2023/V49/I10/154

图/表 10

图1 FFCA-U-Net网络结构

Fig.1 Structure of FFCA-U-Net network

图2 肿瘤区域的掩码图像

Fig.2 Mask image of tumor region

图3 特征融合模块结构

Fig.3 Structure of feature fusion module

图4 改进的三维坐标注意力模块结构

Fig.4 Structure of improved three-dimensional coordinate attention mechanism

图5 UNETR和FFCA-U-Net的分割结果

Fig.5 Segmentation results of UNETR and FFCA-U-Net

图6 不同模型在矢状面、冠状面和水平面的分割结果对比

Fig.6 Comparison of segmentation results among different models in sagittal, coronal, and horizontal planes

参考文献 32

1	MENZE B H, JAKAB A, BAUER S, et al. The multimodal brain tumor image segmentation benchmark. IEEE Transactions on Medical Imaging, 2015, 34(10): 1993- 2024. doi: 10.1109/TMI.2014.2377694
2	LOUIS D N, PERRY A, WESSELING P, et al. The 2021 WHO classification of tumors of the central nervous system: a summary. Neuro-Oncology, 2021, 23(8): 1231- 1251. doi: 10.1093/neuonc/noab106
3	国家卫生健康委员会医政医管局. 脑胶质瘤诊疗规范(2018年版). 中华神经外科杂志, 2019, 35(3): 217- 239. doi: 10.3760/cma.j.issn.1001-2346.2019.03.001
	National Health Commission Medical Administration. Guidelines for the diagnosis and treatment of glioma (2018 edition). Chinese Journal of Neurosurgery, 2019, 35(3): 217- 239. doi: 10.3760/cma.j.issn.1001-2346.2019.03.001
4	WADHWA A, BHARDWAJ A, SINGH VERMA V. A review on brain tumor segmentation of MRI images. Magnetic Resonance Imaging, 2019, 61, 247- 259. doi: 10.1016/j.mri.2019.05.043
5	WANG G T, LI W Q, OURSELIN S, et al. Automatic brain tumor segmentation based on cascaded convolutional neural networks with uncertainty estimation. Frontiers in Computational Neuroscience, 2019, 13, 56. doi: 10.3389/fncom.2019.00056
6	BEN NACEUR M, SAOULI R, AKIL M, et al. Fully automatic brain tumor segmentation using end-to-end incremental deep neural networks in MRI images. Computer Methods and Programs in Biomedicine, 2018, 166, 39- 49. doi: 10.1016/j.cmpb.2018.09.007
7	LIU Z H, TONG L, CHEN L, et al. Deep learning based brain tumor segmentation: a survey. Complex and Intelligent Systems, 2023, 9, 1001- 1026. doi: 10.1007/s40747-022-00815-5
8	BAUER S, SEILER C, BARDYN T, et al. Atlas-based segmentation of brain tumor images using a Markov random field-based tumor growth model and non-rigid registration[C]//Proceedings of Annual International Conference of the IEEE Engineering in Medicine and Biology. Washington D. C., USA: IEEE Press, 2010: 4080-4083.
9	LEFKOVITS L, LEFKOVITS S, SZILÁGYI L. Brain tumor segmentation with optimized random forest[M]//GRIMI A, MENZE B, MAIER O, et al. Brainlesion: glioma, multiple sclerosis, stroke and traumatic brain injuries. Berlin, Germany: Springer, 2016: 88-99.
10	WU W, CHEN A Y C, ZHAO L, et al. Brain tumor detection and segmentation in a CRF (conditional random fields) framework with pixel-pairwise affinity and superpixel-level features. International Journal of Computer Assisted Radiology and Surgery, 2014, 9(2): 241- 253. doi: 10.1007/s11548-013-0922-7
11	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of the 18th International Conference Munich. Berlin, Germany: Springer, 2015: 234-241.
12	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2015: 3431-3440.
13	ZHOU Z W, SIDDIQUEE R M M, TAJBAKHSH N, et al. UNet++: a nested U-Net architecture for medical image segmentation[C]//Proceedings of International Workshop on Deep Learning in Medical Image Analysis. Berlin, Germany: Springer, 2018: 1-10.
14	LI X M, CHEN H, QI X J, et al. H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Transactions on Medical Imaging, 2018, 37(12): 2663- 2674. doi: 10.1109/TMI.2018.2845918
15	HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 2261-2269.
16	MILLETARI F, NAVAB N, AHMADI S A. V-Net: fully convolutional neural networks for volumetric medical image segmentation[C]//Proceedings of the 4th International Conference on 3D Vision (3DV). Washington D. C., USA: IEEE Press, 2016: 565-571.
17	王磐, 强彦, 杨晓棠, 等. 基于双注意力3D-UNet的肺结节分割网络模型. 计算机工程, 2021, 47(2): 307- 313. doi: 10.19678/j.issn.1000-3428.0057019
	WANG P, QIANG Y, YANG X T, et al. Network model for lung nodule segmentation based on double attention 3D-UNet. Computer Engineering, 2021, 47(2): 307- 313. doi: 10.19678/j.issn.1000-3428.0057019
18	TIAN Z, HE T, SHEN C H, et al. Decoders matter for semantic segmentation: data-dependent decoding enables flexible feature aggregation[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 3121-3130.
19	ZHONG Z Q, HE L A, CHEN C X, et al. Full-scale attention network for automated organ segmentation on head and neck CT and MR images. IET Image Processing, 2023, 17(3): 660- 673. doi: 10.1049/ipr2.12663
20	曹加旺, 田维维, 刘学玲, 等. 基于改进U-Net的人脑黑质致密部分割. 计算机工程, 2022, 48(11): 14-21, 29. URL
	CAO J W, TIAN W W, LIU X L, et al. Dense partial segmentation of substantia nigra of human brain based on improved U-Net. Computer Engineering, 2022, 48(11): 14-21, 29. URL
21	OKTAY O, SCHLEMPER J, FOLGOC L L, et al. Attention U-Net: learning where to look for the pancreas[EB/OL]. [2022-10-15]. https://arxiv.org/pdf/1804.03999.pdf.
22	郝华颖, 赵昆, 苏攀, 等. 一种基于改进ResU-Net的角膜神经分割算法. 计算机工程, 2021, 47(1): 217- 223. URL
	HAO H Y, ZHAO K, SU P, et al. A corneal nerve segmentation algorithm based on improved ResU-Net. Computer Engineering, 2021, 47(1): 217- 223. URL
23	KAUL C, MANANDHAR S, PEARS N. FocusNet: an attention-based fully convolutional network for medical image segmentation[C]//Proceedings of the 16th International Symposium on Biomedical Imaging. Washington D. C., USA: IEEE Press, 2019: 455-458.
24	WANG C Y, HE Y, LIU Y F, et al. ScleraSegNet: an improved U-Net model with attention for accurate sclera segmentation[C]//Proceedings of International Conference on Biometrics. Washington D. C., USA: IEEE Press, 2019: 1-8.
25	SAAD N M, ABU-BAKAR S, MUDA S, et al. Fully automated region growing segmentation of brain lesion in diffusion-weighted MRI. IAENG International Journal of Computer Science, 2012, 39(2): 155- 164.
26	QIN X F, WU C Z, CHANG H, et al. Match feature U-Net: dynamic receptive field networks for biomedical image segmentation. Symmetry, 2020, 12(8): 1230. doi: 10.3390/sym12081230
27	ÇIÇEK Ö, ABDULKADIR A, LIENKAMP S S, et al. 3D U-Net: learning dense volumetric segmentation from sparse annotation[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2016: 424-432.
28	HATAMIZADEH A, TANG Y C, NATH V, et al. UNETR: Transformers for 3D medical image segmentation[C]//Proceedings of the Winter Conference on Applications of Computer Vision. Washington D. C., USA: IEEE Press, 2022: 574-584.
29	WANG W X, CHEN C, DING M, et al. TransBTS: multimodal brain tumor segmentation using transformer[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2021: 109-119.
30	HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 7132-7141.
31	WOO S, PARK J C, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 3-19.
32	HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 13708-13717.

[1]	范文卓, 吴涛, 许俊平, 李庆庆, 张建林, 李美惠, 魏宇星. 基于多分辨率特征融合的任意尺度图像超分辨率重建[J]. 计算机工程, 2023, 49(9): 217-225.
[2]	李嘉新, 侯进, 盛博莹, 周宇航. 基于改进YOLOv5的遥感小目标检测网络[J]. 计算机工程, 2023, 49(9): 256-264.
[3]	韩璐, 霍纬纲, 张永会, 刘涛. 基于多尺度特征融合与双注意力机制的多元时间序列预测[J]. 计算机工程, 2023, 49(9): 99-108.
[4]	宋志娜, 李莎, 杨建明, 徐川. 基于特征与区域定位增强的遥感舰船目标检测[J]. 计算机工程, 2023, 49(8): 257-264.
[5]	张欣怡, 张飞, 郝斌, 高鹭, 任晓颖. 基于改进YOLOv5的口罩佩戴检测算法[J]. 计算机工程, 2023, 49(8): 265-274.
[6]	杨长沛, 廖列法. 基于门控空洞卷积特征融合的中文命名实体识别[J]. 计算机工程, 2023, 49(8): 85-95.
[7]	杨祖赫, 黎智辉, 唐云祁, 晏于文, 宋华青. 结合语义与图像信息的行人属性识别算法[J]. 计算机工程, 2023, 49(8): 215-222, 231.
[8]	陈露萌, 曹彦彦, 黄民, 谢鑫钢. 基于改进YOLOv5的火焰检测方法[J]. 计算机工程, 2023, 49(8): 291-301, 309.
[9]	刘豪, 吴红兰, 房宇轩. 结合全局上下文信息的高效人体姿态估计[J]. 计算机工程, 2023, 49(7): 102-109.
[10]	吴珊, 周凤. 基于改进SSD算法的小目标检测[J]. 计算机工程, 2023, 49(7): 179-188.
[11]	齐咏生, 杜晓旭, 朱俊峰, 高胜利, 刘利强. 基于增强型轻量深度网络的牧区牲畜高效检测[J]. 计算机工程, 2023, 49(7): 278-287.
[12]	殷文君, 黄建华, 纪元法. 基于改进密集卷积网络的皮肤肿瘤分类方法[J]. 计算机工程, 2023, 49(7): 288-294.
[13]	王同官, 赖惠成, 蔡玉玺, 高古学, 汪烈军. 基于注意力残差网络的人脸超分辨率重建[J]. 计算机工程, 2023, 49(6): 234-241.
[14]	沈秀轩, 吴春雷, 冯叶棋, 程铭, 张俊三, 朱杰. 基于双分支特征融合的医学报告生成方法[J]. 计算机工程, 2023, 49(6): 274-283,291.
[15]	廖涛, 孙皓洁, 张顺香. 基于跨度和特征融合的实体关系联合抽取模型[J]. 计算机工程, 2023, 49(6): 107-114.

选择文件类型/文献管理软件名称

选择包含的内容