基于倒金字塔深度学习网络的三维医学图像分割

doi:10.19678/j.issn.1000-3428.0063687

摘要/Abstract

摘要： 基于深度学习的医学图像分割对医学研究和临床疾病诊断具有重要意义。然而，现有三维脑图像分割网络仅依赖单一模态信息，且最后一层网络的特征表达不准确，导致分割精度降低。引入注意力机制，提出一种基于深度学习的多模态交叉重构的倒金字塔网络MCRAIP-Net。以多模态磁共振图像作为输入，通过三个独立的编码器结构提取各模态的特征信息，并将提取的特征信息在同一分辨率级进行初步融合。利用双通道交叉重构注意力模块实现多模态特征的细化与融合。在此基础上，采用倒金字塔解码器对解码器各阶段不同分辨率的特征进行整合，完成脑组织的分割任务。在MRBrainS13和IBSR18数据集上的实验结果表明，相比3D U-Net、MMAN、SW-3D-Unet等网络，MCRAIP-Net能够充分利用多模态图像的互补信息，获取更准确丰富的细节特征且具有较优的分割精度，白质、灰质、脑脊液的Dice系数分别达到91.67%、88.95%、84.79%。

关键词: 多模态融合, 交叉重构注意力, 倒金字塔解码器, 医学图像分割, 深度学习

Abstract: Segmentation of medical images based on deep learning is of great significance to both medical research and clinical disease diagnosis.However, existing 3D image segmentation networks only rely on single modality information, and the feature representation of the last layer of these existing networks is not accurate.As a result, segmentation accuracy is reduced.By introducing an attention mechanism, a inverted pyramid network with multi-modality cross reconstruction MCRAIP-Net, based on deep learning, is proposed.With multi-modality Magnetic Resonance Imaging(MRI) as input, the feature information of each modality is extracted by three independent encoder structures, and the extracted feature information is preliminarily fused at the same resolution level.The Dual-channel Cross Reconstruction Attention(DCRA) module is used to refine and fuse multi-modality features.An inverted pyramid decoder is used to integrate the features of different resolutions at each stage of the decoder, to complete the task of segmenting a 3D medical image.The experimental results on MRBrainS13 and IBSR18 datasets show that the proposed MCRAIP-Net can use complementary information from the multi-modality images to obtain more accurate and rich details, and has improved segmentation accuracy, compared with networks such as the 3D U-Net, MMAN, SW-3D-Unet.The Dice coefficients of white matter, gray matter, and cerebrospinal fluid are 91.67%, 88.95%, and 84.79%, respectively.

Key words: multi-modality fusion, cross reconstruction attention, inverted pyramid decoder, medical image segmentation, deep learning

中图分类号:

TP181

张相芬, 刘艳, 袁非牛. 基于倒金字塔深度学习网络的三维医学图像分割[J]. 计算机工程, 2022, 48(12): 304-311.

ZHANG Xiangfen, LIU Yan, YUAN Feiniu. 3D Medical Image Segmentation Based on Inverted Pyramid Deep Learning Network[J]. Computer Engineering, 2022, 48(12): 304-311.

https://www.ecice06.com/CN/Y2022/V48/I12/304

图/表 10

20230112184250

20230112184254

20230112184259

20230112184303

20230112184307

20230112184310

20230112184314

20230112184317

20230112184320

20230112184324

参考文献

[1] LIU X B, SONG L P, LIU S, et al.A review of deep-learning-based medical image segmentation methods[J].Sustainability, 2021, 13(3):1224.
[2] SOMASUNDARAM K, KALAVATHI P.Brain segmentation in magnetic resonance human head scans using multi-seeded region growing[J].The Imaging Science Journal, 2014, 62(5):273-284.
[3] LI M, ZHENG X L, LUO H Y, et al.Automated segmentation of brain tissue and white matter in cryosection images from Chinese visible human dataset[J].Journal of Medical and Biological Engineering, 2014, 34(2):178-187.
[4] SALMAN N H, GHAFOUR B M, HADI G M.Medical image segmentation based on edge detection techniques[J].Advances in Image and Video Processing, 2015, 3(2):16-27.
[5] 杨兵, 刘晓芳, 张纠.基于深度特征聚合网络的医学图像分割[J].计算机工程, 2021, 47(4):187-196. YANG B, LIU X F, ZHANG J.Medical image segmentation based on deep feature aggregation network[J].Computer Engineering, 2021, 47(4):187-196.(in Chinese)
[6] RONNEBERGER O, FISCHER P, BROX T.U-Net:Convolutional networks for biomedical image segmentation[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention.Berlin, Germany:Springer, 2015:234-241.
[7] CICEK Ö, ABDULKADIR A, LIENKAMP S S, et al.3D U-Net:learning dense volumetric segmentation from sparse annotation[C]//Proceedings of Conference on Medical Image Computing and Computer-Assisted Intervention.Berlin, Germany:Springer, 2016:424-432.
[8] LONG J S, MA G Z, SONG E M, et al.Learning U-net based multi-scale features in encoding-decoding for MR image brain tissue segmentation[J].Sensors (Basel, Switzerland), 2021, 21(9):3232.
[9] SUN L Y, MA W A, DING X H, et al.A 3D spatially weighted network for segmentation of brain tissue from MRI[J].IEEE Transactions on Medical Imaging, 2020, 39(4):898-909.
[10] HUANG Y J, DOU Q, WANG Z X, et al.3-D RoI-aware U-Net for accurate and efficient colorectal tumor segmentation[J].IEEE Transactions on Cybernetics, 2021, 51(11):5397-5408.
[11] YU J K, YANG D D, ZHAO H S.FFANet:feature fusion attention network to medical image segmentation[J].Biomedical Signal Processing and Control, 2021, 69:102912.
[12] HU J, SHEN L, ALBANIE L, et al.Squeeze-and-excitation networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8):2011-2023.
[13] WANG F, JIANG M Q, QIAN C, et al.Residual attention network for image classification[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:6450-6458.
[14] WOO S, PARK J, LEE J Y, et al.CBAM:convolutional block attention module[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2018:3-19.
[15] SUN X, GARG P, PLEIN S, et al.SAUN:stack attention U-Net for left ventricle segmentation from cardiac cine magnetic resonance imaging[J].Medical Physics, 2021, 48(4):1750-1763.
[16] YANG T X, YOSHIMURA Y, MORITA A, et al.Synergistic attention U-Net for sublingual vein segmentation[J].Artificial Life and Robotics, 2019, 24(4):550-559.
[17] KAUL C, MANANDHAR S, PEARS N.FocusNet:an attention-based fully convolutional network for medical image segmentation[C]//Proceedings of the 16th International Symposium on Biomedical Imaging.Washington D.C., USA:IEEE Press, 2019:455-458.
[18] 魏颖, 雷志浩, 齐林.基于注意力机制的3D U-Net婴幼儿脑组织MR图像分割[J].东北大学学报(自然科学版), 2021, 42(5):616-623. WEI Y, LEI Z H, QI L.3D U-Net infant brain tissue MR image segmentation based on attention mechanism[J].Journal of Northeastern University(Natural Science), 2021, 42(5):616-623.(in Chinese)
[19] MA C, LUO G N, WANG K Q.Concatenated and connected random forests with multiscale patch driven active contour model for automated brain tumor segmentation of MR images[J].IEEE Transactions on Medical Imaging, 2018, 37(8):1943-1954.
[20] ZHOU T X, RUAN S, CANU S.A review:deep learning for medical image segmentation using multi-modality fusion[J].Array, 2019(3/4):100004.
[21] DOLZ J, DESROSIERS C, BEN AYED I.IVD-Net:intervertebral disc localization and segmentation in MRI with a multi-modal UNet[C]//Proceedings of Computational Methods and Clinical Applications for Spine Imaging.Berlin, Germany:Springer, 2019:130-143.
[22] KAMNITSAS K, BAI W, FERRANTE E, et al.Ensembles of multiple models and architectures for robust brain tumour segmentation[C]//Proceedings of International MICCAI Brainlesion Workshop.Berlin, Germany:Springer, 2017:450-462.
[23] KAMNITSAS K, LEDIG C, NEWCOMBE V F J, et al.Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation[J].Medical Image Analysis, 2017, 36:61-78.
[24] DOLZ J, GOPINATH K, YUAN J, et al.HyperDense-Net:a hyper-densely connected CNN for multi-modal image segmentation[J].IEEE Transactions on Medical Imaging, 2019, 38(5):1116-1126.
[25] LI J C, YU Z L, GU Z H, et al.MMAN:multi-modality aggregation network for brain segmentation from MR images[J].Neurocomputing, 2019, 358:10-19.
[26] ZHANG Z X, LIU Q J, WANG Y H.Road extraction by deep residual U-net[J].IEEE Geoscience and Remote Sensing Letters, 2018, 15(5):749-753.
[27] CAHALL D E, RASOOL G, BOUAYNAYA N C, et al.Inception modules enhance brain tumor segmentation[J].Frontiers in Computational Neuroscience, 2019, 13:44.
[28] BADRINARAYANAN V, KENDALL A, CIPOLLA R.SegNet:a deep convolutional encoder-decoder architecture for image segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495.
[29] GHOSAL P, CHOWDHURY T, KUMAR A, et al.MhURI:a supervised segmentation approach to leverage salient brain tissues in magnetic resonance images[J].Computer Methods and Programs in Biomedicine, 2021, 200:1-15.

选择文件类型/文献管理软件名称

选择包含的内容