基于多期相注意力融合网络的肝脏病灶CT影像分类研究

doi:10.19678/j.issn.1000-3428.0064536

摘要/Abstract

摘要： 肝脏病灶是指肝脏疾病集中的部位或是综合病症、感染的主要部位。由于不同类型的多期相肝脏病灶计算机断层扫描（CT）影像存在异病同影或同病异影的情况，导致同一类型的CT影像结构变化较大，传统方法难以提取丰富的图像特征信息，肝脏病灶分类准确率有待提高。提出一种多期相注意力融合网络MAFNet，使用单期相分支表征单期相病灶图像特征，并在融合分支中采用中期融合的方式，融合单期相分支中提取出的特征映射，从而充分提取图像中不同层次的特征。利用多期相注意力模块提取单期相分支中肝脏病灶的加权特征，重新组织多期相肝脏病灶的特征映射，以保持不同单期相图像信息，表达3个期相影像的时序增强模式，得到更准确的分类结果。实验结果表明，基于该网络的分类方法能充分利用多期相肝脏CT影像的时序特征，有效捕捉同一患者不同期相的信息，肝脏病灶CT影像分类的平均准确率为90.99%。

关键词: 计算机辅助诊断系统, 残差神经网络, 肝脏病灶, 多期相计算机断层扫描影像, 注意力

Abstract: Liver lesions refer to the concentrated parts of liver diseases or main parts of comprehensive diseases and infections.Owing to different types of multi-phase liver lesions with different diseases or different Computed Tomography(CT) images of the same disease, the structure of the same CT images changes significantly.It becomes difficult to extract rich image feature information using traditional methods, and the accuracy to classify liver lesions needs to be improved.Therefore, in this study, a Multi-phase Attention Fusion Network(MAFNet) is proposed, it uses monophasic branches to represent the features of monophasic focus images.In the fusion branch, a medium-term fusion method is used to fuse the feature maps extracted from monophasic branches to fully extract the features of different levels from the images.The Multi-phase Attention Module(MAM) is used to extract the weighted features of liver lesions in monophasic branches, reorganize the feature mapping of the multi-phase liver lesions, store the information of the monophasic images, express the temporal enhancement mode of the three-phase images, and obtain more accurate classification results.The experimental results show that the classification method based on the proposed network can fully utilize the time sequence characteristics of multi-phase liver CT images and effectively store the information of different phases of the same patient;moreover, the accuracy of the CT image classification of liver lesions is 90.99%.

Key words: Computer-Aided Diagnosis (CAD) system, Residual Neural Network(ResNet), liver lesion, multi-phase Computed Tomography(CT) images, attention

中图分类号:

TP391.41

田炜, 雷志超, 王楚正. 基于多期相注意力融合网络的肝脏病灶CT影像分类研究[J]. 计算机工程, 2023, 49(2): 231-237.

TIAN Wei, LEI Zhichao, WANG Chuzheng. Research on CT Images Classification of Liver Lesions Based on Multi-phase Attention Fusion Network[J]. Computer Engineering, 2023, 49(2): 231-237.

http://www.ecice06.com/CN/Y2023/V49/I2/231

图/表 10

20230216181724

20230216181727

20230216181731

20230216181734

20230216181738

20230216181741

20230216181745

20230216181748

20230216181752

20230216181755

参考文献

[1] ISLAMI F, WARD E M, SUNG H, et al.Annual report to the nation on the status of cancer, part 1:national cancer statistics[J].Journal of the National Cancer Institute, 2021, 113(12):1648-1669.
[2] 张杰妹, 杨词慧.基于RV-FCN的CT肝脏影像自动分割算法[J].计算机工程, 2019, 45(7):258-263. ZHANG J M, YANG C H.Automatic segmentation algorithm of CT liver image based on RV-FCN[J].Computer Engineering, 2019, 45(7):258-263.(in Chinese)
[3] CHI Y L, ZHOU J Y, VENKATESH S K, et al.Content-based image retrieval of multiphase CT images for focal liver lesion characterization[J].Medical Physics, 2013, 40(10):103502-103513.
[4] CHEN E L, CHUNG P C, CHEN C L, et al.An automatic diagnostic system for CT liver image classification[J].IEEE Transactions on Biomedical Engineering, 1998, 45(6):783-794.
[5] NINO-MURCIA M, OLCOTT E W, JEFFREY R B, et al.Focal liver lesions:pattern-based classification scheme for enhancement at arterial phase CT[J].Radiology, 2000, 215(3):746-751.
[6] YANG W, LU Z, YU M, et al.Content-based retrieval of focal liver lesions using bag-of-visual-words representations of single and multiphase contrast-enhanced CT images[J].Digit Imaging, 2012, 25(6):708-719.
[7] WANG J, HAN X H, XU Y Y, et al.Sparse codebook model of local structures for retrieval of focal liver lesions using multiphase medical images[J].International Journal of Biomedical Imaging, 2017, 17(5):1-13.
[8] YU M, FENG Q J, YANG W, et al.Extraction of lesion-partitioned features and retrieval of contrast-enhanced liver images[J].Computational and Mathematical Methods in Medicine, 2012, 61(11):1-12.
[9] ROY S, CHI Y L, LIU J M, et al.Three-dimensional spatiotemporal features for fast content-based retrieval of focal liver lesions[J].IEEE Transactions on Biomedical Engineering, 2014, 61(11):2768-2778.
[10] DIAMANT I, HOOGI A, BEAULIEU C F, et al.Improved patch-based automated liver lesion classification by separate analysis of the interior and boundary regions[J].IEEE Journal of Biomedical and Health Informatics, 2016, 20(6):1585-1594.
[11] 吉志远.基于多期相三维CT图像的肝脏肿瘤分类算法研究[D].杭州:浙江大学, 2018. JI Z Y.A study on focal liver lesion classification based on multiphase 3D CT images[D].Hangzhou:Zhejiang University, 2018.(in Chinese)
[12] HE K M, ZHANG X Y, REN S Q, et al.Deep residual learning for image recognition[EB/OL].[2022-03-10].https://arxiv.org/abs/1512.03385.
[13] BELHAOUARI S B, ISLAM A.Deep learning in healthcare[EB/OL].[2022-03-10].2021.https://www.researchgate.net/publication/353727328_Deep_Learning_in_Healthcare.
[14] ESTEVA A, KUPREL B, NOVOA R A, et al.Dermatologist-level classification of skin cancer with deep neural networks[J].Nature, 2017, 542(7639):115-118.
[15] PENG L Y, LIN L F, HU H J, et al.Classification and quantification of emphysema using a multi-scale residual network[J].IEEE Journal of Biomedical and Health Informatics, 2019, 23(6):2526-2536.
[16] FRID-ADAR M, DIAMANT I, KLANG E, et al.Modeling the intra-class variability for liver lesion detection using a multi-class patch-based CNN[M]//WU G R, MUNSELL B C, ZHAN Y Q, et al.Patch-Based Techniques in Medical Imaging.Berlin, Germany:Springer, 2017:129-137.
[17] YASAKA K, AKAI H, ABE O, et al.Deep learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced CT:a preliminary study[J].Radiology, 2018, 286(3):887-896.
[18] LIANG D, LIN L F, HU H J, et al.Residual convolutional neural networks with global and local pathways for classification of focal liver lesions[M]//GENG X, KANG B H.PRICAI 2018:Trends in Artificial Intelligence.Berlin, Germany:Springer, 2018:617-628.
[19] TODOROKI Y, IWAMOTO Y, LIN L F, et al.Automatic detection of focal liver lesions in multi-phase CT images using a multi-channel & multi-scale CNN[C]//Proceedings of the 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society.Washington D.C., USA:IEEE Press, 2019:872-875.
[20] LIANG D, LIN L F, CHEN X, et al.Multi-stream scale-insensitive convolutional and recurrent neural networks for liver tumor detection in dynamic ct images[C]//Proceedings of IEEE International Conference on Image Processing.Washington D.C., USA:IEEE Press, 2019:794-798.
[21] HASEGAWA R, IWAMOTO Y, LIN L F, et al.Automatic segmentation of liver tumor in multiphase CT images by mask R-CNN[C]//Proceedings of the 2nd Global Conference on Life Sciences and Technologies.Washington D.C., USA:IEEE Press, 2020:231-234.
[22] VAEZI JOZE H R, SHABAN A, IUZZOLINO M L, et al.MMTM:multimodal transfer module for CNN fusion[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:13286-13296.
[23] HU J, SHEN L, ALBANIE S, et al.Squeeze-and-excitation networks[EB/OL].[2022-03-10].https://arxiv.org/abs/1709.01507.
[24] ZHANG Y L, LI K P, LI K, et al.Image super-resolution using very deep residual channel attention networks[M]//FERRARI V, HEBERT M, SMINCHISESCU C, et aL Computer Vision-ECCV 2018.Berlin, Germany:Springer, 2018:294-310.
[25] SUN Y, WANG X G, TANG X O.Deep learning face representation from predicting 10000 classes[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2014:1891-1898.
[26] XU Y Y, LIN L F, HU H J, et al.Texture-specific bag of visual words model and spatial cone matching-based method for the retrieval of focal liver lesions using multiphase contrast-enhanced CT images[J].International Journal of Computer Assisted Radiology and Surgery, 2018, 13(1):151-164.
[27] WANG J.Tensor-based sparse representations of multi-phase medical images for classification of focal liver lesions[J].Pattern Recognition Letters, 2020, 130:207-215.
[28] BROOKS T, MILDENHALL B, XUE T F, et al.Unprocessing images for learned raw denoising[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:11028-11037.
[29] RUDER S.An overview of gradient descent optimization algorithms[EB/OL].[2022-03-10].https://arxiv.org/abs/1609.04747.
[30] KRIZHEVSKY A, SUTSKEVER I, HINTON G E.ImageNet classification with deep convolutional neural networks[J].Communications of the ACM, 2017, 60(6):84-90.
[31] SIMONYAN K, ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[EB/OL].[2022-03-10].https://arxiv.org/abs/1409.1556.
[32] SZEGEDY C, LIU W, JIA Y Q, et al.Going deeper with convolutions[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2014:1-9.

选择文件类型/文献管理软件名称

选择包含的内容