基于深度学习的肝脏CT-MR图像无监督配准

doi:10.19678/j.issn.1000-3428.0063999

计算机工程 ›› 2023, Vol. 49 ›› Issue (1): 223-233. doi: 10.19678/j.issn.1000-3428.0063999

基于深度学习的肝脏CT-MR图像无监督配准

王帅坤^1,2, 周志勇², 胡冀苏^1,2, 钱旭升^1,2, 耿辰², 陈光强³, 纪建松⁴, 戴亚康^2,5

1. 中国科学技术大学 (苏州)生物医学工程学院生命科学与医学部, 江苏苏州 215163;
2. 中国科学院苏州生物医学工程技术研究所, 江苏苏州 215163;
3. 苏州大学附属第二医院, 江苏苏州 215000;
4. 丽水市中心医院, 浙江丽水 323000;
5. 济南国科医工科技发展有限公司, 济南 250000

收稿日期:2022-02-22 修回日期:2022-03-23 发布日期:2022-07-19
作者简介:王帅坤(1995-),男,硕士研究生,主研方向为医学图像处理;周志勇,研究员、博士;胡冀苏,博士;钱旭升,硕士;耿辰、陈光强,博士;纪建松,主任医师、博士;戴亚康,研究员、博士。
基金资助:
国家自然科学基金（81971685）；国家重点研发计划（2018YFA0703101）；中国科学院青年创新促进会会员基金（2021324）；江苏省重点研发计划（BE2021053）；苏州市科技计划（SS202054）。

Unsupervised Registration for Liver CT-MR Images Based on Deep Learning

WANG Shuaikun^1,2, ZHOU Zhiyong², HU Jisu^1,2, QIAN Xusheng^1,2, GENG Chen², CHEN Guangqiang³, JI Jiansong⁴, DAI Yakang^2,5

1. Division of Life Sciences and Medicine, School of Biomedical Engineering(Suzhou), University of Science and Technology of China, Suzhou, Jiangsu 215163, China;
2. Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou, Jiangsu 215163, China;
3. The Second Affiliated Hospital of Suzhou University, Suzhou, Jiangsu 215000, China;
4. The Lishui Central Hospital, Lishui, Zhejiang 323000, China;
5. Jinan Guoke Medical Engineering Technology Development Co., Ltd., Jinan 250000, China

Received:2022-02-22 Revised:2022-03-23 Published:2022-07-19

摘要/Abstract

摘要： 多模态配准是医学图像分析中的关键环节，在肝癌辅助诊断、图像引导的手术治疗中具有重要作用。针对传统的迭代式肝脏多模态配准计算量大、耗时长、配准精度低等问题，提出一种基于多尺度形变融合和双输入空间注意力的无监督深度学习配准算法。利用多尺度形变融合框架提取不同分辨率的图像特征，实现肝脏的逐阶配准，在提高配准精度的同时避免网络陷入局部最优。采用双输入空间注意力模块在编解码阶段融合不同水平的空间和文本信息提取图像间的差异特征，增强特征表达。引入基于邻域描述符的结构信息损失项进行网络迭代优化，不需要任何先验信息即可实现精确的无监督配准。在临床肝脏CT-MR数据集上的实验结果表明，与传统的Affine、Elastix、VoxelMorph等算法相比，该算法达到最优的DSC值和TRE值，分别为0.926 1±0.018 6和6.39±3.03 mm，其平均配准时间为0.35±0.018 s，相比Elastix算法提升了近380倍，能准确地提取特征及估计规则的形变场，具有较高的配准精度和较快的配准速度。

关键词: 深度学习, 无监督配准, 多模态配准, 形变融合, 结构信息损失, 空间注意力

Abstract: Multimodal registration is a key step in medical image analysis, which plays an important role in the assisted diagnosis and the image-guided surgical treatment of liver cancer.Aiming at the problems of large computation, long time consuming, and low registration accuracy of traditional iterative multimodal registration, this paper proposes an unsupervised deep learning-based image registration method based on multi-scale deformation fusion and dual-input spatial attention.Using the multi-scale deformation fusion architecture captures different resolution features of images to achieve liver registration in a coarse-to-fine pattern and avoids local optimization.The dual-input spatial attention module is used to extract the discrepant features between images by integrating spatial and text information at different levels in the codec stage and enhancing feature expression.Additionally, a structural information loss is introduced to globally optimize the registration network, which does not require any prior information and achieves an accurate unsupervised registration.Experimental results on liver Computed Tomography-Magnetic Resonance(CT-MR) datasets show that the proposed algorithm achieved an optimal global Dice Similarity Coefficient(DSC) and Target Registration Error(TRE) values of 0.926 1 ±0.018 6 and 6.39 ±3.03 mm, respectively, which is superior to Affine, Elastix, and VoxelMorph amongst other algorithms.In addition, the average registration time of the proposed algorithm is 0.35 ±0.018 s, which is nearly 380 times faster than the Elastix algorithm.Results show that the proposed algorithm demonstrates higher registration accuracy and faster registration speed by accurately extracting features and estimating the regular deformation field.

Key words: deep learning, unsupervised registration, multimodal registration, deformation fusion, structural information loss, spatial attention

中图分类号:

TP391

王帅坤, 周志勇, 胡冀苏, 钱旭升, 耿辰, 陈光强, 纪建松, 戴亚康. 基于深度学习的肝脏CT-MR图像无监督配准[J]. 计算机工程, 2023, 49(1): 223-233.

WANG Shuaikun, ZHOU Zhiyong, HU Jisu, QIAN Xusheng, GENG Chen, CHEN Guangqiang, JI Jiansong, DAI Yakang. Unsupervised Registration for Liver CT-MR Images Based on Deep Learning[J]. Computer Engineering, 2023, 49(1): 223-233.

https://www.ecice06.com/CN/Y2023/V49/I1/223

图/表 16

20230701180708

20230701180711

20230701180714

20230701180717

20230701180720

20230701180724

20230701180727

20230701180731

20230701180734

20230701180738

20230701180741

20230701180745

20230701180748

20230701180752

20230701180755

20230701180759

参考文献

[1] SUNG H, FERLAY J, SIEGEL R L, et al.Global cancer statistics 2020:GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries[J].CA:a Cancer Journal for Clinicians, 2021, 71(3):209-249.
[2] MAES F, COLLIGNON A, VANDERMEULEN D, et al.Multimodality image registration by maximization of mutual information[J].IEEE Transactions on Medical Imaging, 1997, 16(2):187-198.
[3] WACHINGER C, NAVAB N.Entropy and Laplacian images:structural representations for multi-modal registration[J].Medical Image Analysis, 2012, 16(1):1-17.
[4] HEINRICH M P, JENKINSON M, BHUSHAN M, et al.MIND:modality independent neighbourhood descriptor for multi-modal deformable registration[J].Medical Image Analysis, 2012, 16(7):1423-1435.
[5] ZHOU S K, GREENSPAN H, DAVATZIKOS C, et al.A review of deep learning in medical imaging:imaging traits, technology trends, case studies with progress highlights, and future promises[J].Proceedings of the IEEE, 2021, 109(5):820-838.
[6] SIMONOVSKY M, GUTIÉRREZ-BECKER B, MATEUS D, et al.A deep metric for multimodal registration[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention.Berlin, Germany:Springer, 2016:10-18.
[7] HASKINS G, KRUECKER J, KRUGER U, et al.Learning deep similarity metric for 3D MR-TRUS image registration[J].International Journal of Computer Assisted Radiology and Surgery, 2019, 14(3):417-425.
[8] GOODFELLOW I, POUGET ABADIE J, MIRZA M, et al.Generative adversarial networks[EB/OL].[2022-01-10].https://arxiv.org/abs/1406.2661?context=cs.LG.
[9] YAN P K, XU S, RASTINEHAD A R, et al.Adversarial image registration with application for MR and TRUS image fusion[C]//Proceedings of International Conference on Machine Learning in Medical Imaging.Berlin, Germany:Springer, 2018:197-204.
[10] MAHAPATRA D, ANTONY B, SEDAI S M, et al.Deformable medical image registration using generative adversarial networks[C]//Proceedings of the 15th International Symposium on Biomedical Imaging.Washington, USA:IEEE Press, 2018:1449-1453.
[11] FAN J F, CAO X H, WANG Q, et al.Adversarial learning for mono- or multi-modal registration[J].Medical Image Analysis, 2019, 58:101545-101556.
[12] DE VOS B D, BERENDSEN F F, VIERGEVER M A, et al.End-to-end unsupervised deformable image registration with a convolutional neural network[C]//Proceedings of Conference on Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support.Berlin, Germany:Springer, 2017:204-212.
[13] ROHÉ M M, DATAR M, HEIMANN T, et al.SVF-Net:learning deformable image registration using shape matching[C]//Proceedings of Conference on Medical Image Computing and Computer Assisted Intervention.Berlin, Germany:Springer, 2017:266-274.
[14] BALAKRISHNAN G, ZHAO A, SABUNCU M R, et al.VoxelMorph:a learning framework for deformable medical image registration[J].IEEE Transactions on Medical Imaging, 2019, 38(8):1788-1800.
[15] HU Y P, MODAT M, GIBSON E, et al.Weakly-supervised convolutional neural networks for multimodal image registration[J].Medical Image Analysis, 2018, 49:1-13.
[16] ZHOU B, AUGENFELD Z, CHAPIRO J, et al.Anatomy-guided multimodal registration by learning segmentation without ground truth:application to intraprocedural CBCT/MR liver segmentation and registration[J].Medical Image Analysis, 2021, 71:102041-102049.
[17] MOK T C W, CHUNG A C S.Large deformation diffeomorphic image registration with laplacian pyramid networks[C]//Proceedings of Conference on Medical Image Computing and Computer Assisted Intervention.Berlin, Germany:Springer, 2020:211-221.
[18] ZHOU Y, PANG S, CHENG J, et al.Unsupervised deformable medical image registration via pyramidal residual deformation fields estimation[EB/OL].[2022-01-08].https://doi.org/10.48550/arXiv.2004.07624.
[19] RONNEBERGER O, FISCHER P, BROX T.U-Net:Convolutional networks for biomedical image segmentation[C]//Proceedings of International Conference on Medical Image Computing and Computer-assisted Intervention.Berlin, Germany:Springer, 2015:234-241.
[20] ROMERA-PAREDES B, TORR P H S.Recurrent instance segmentation[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2016:312-329.
[21] JETLEY S, LORD N A, LEE N, et al.Learn to pay attention[EB/OL].[2022-01-08].https://doi.org/10.48550/arXiv.1804.02391.
[22] WANG X L, GIRSHICK R, GUPTA A, et al.Non-local neural networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:7794-7803.
[23] OKTAY O, SCHLEMPER J, FOLGOC L L, et al.Attention U-net:learning where to look for the pancreas[EB/OL].[2022-01-08].https://doi.org/10.48550/arXiv.1804.03999.
[24] ZAGORUYKO S, KOMODAKIS N.Paying more attention to attention:improving the performance of convolutional neural networks via attention transfer[EB/OL].[2022-01-08].https://www.semanticscholar.org/paper/Paying-More-Attention-to-Attention%3A-Improving-the-Zagoruyko-Komodakis/f7b032a4df721d4ed2bab97f6acd33d62477b7a5.
[25] YANG H R, SUN J, CARASS A, et al.Unsupervised MR-to-CT synthesis using structure-constrained CycleGAN[J].IEEE Transactions on Medical Imaging, 2020, 39(12):4249-4261.
[26] WEI D M, AHMAD S, HUO J Y, et al.SLIR:Synthesis, localization, inpainting, and registration for image-guided thermal ablation of liver tumors[J].Medical Image Analysis, 2020, 65:101763-101771.
[27] KLEIN S, STARING M, MURPHY K, et al.Elastix:a toolbox for intensity-based medical image registration[J].IEEE Transactions on Medical Imaging, 2009, 29(1):196-205.
[28] PLUIM J P W, MAINTZ J B A, VIERGEVER M A.Mutual-information-based registration of medical images:a survey.[J].IEEE Transactions on Medical Imaging, 2003, 22(8):986-1004.
[29] WANG Z, BOVIK A C, SHEIKH H R, et al.Image quality assessment:from error visibility to structural similarity[J].IEEE Transactions on Image Processing, 2004, 13(4):600-612.

选择文件类型/文献管理软件名称

选择包含的内容

基于深度学习的肝脏CT-MR图像无监督配准

Unsupervised Registration for Liver CT-MR Images Based on Deep Learning

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 16

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	牛瑞婷, 严天峰, 高锐, 王映植. 低信噪比下基于深度学习TCNN-MobileNet的调制识别[J]. 计算机工程, 2024, 50(7): 204-215.
[2]	肖慈, 徐杨, 张永丹, 冯明文, 黄易仟. 结合注意力和低光增强的夜间语义分割[J]. 计算机工程, 2024, 50(7): 271-281.
[3]	张诗婧, 莫绪涛, 赵行, 董杨林. 基于球面折反射成像和YOLOv7的内螺纹缺陷检测[J]. 计算机工程, 2024, 50(7): 282-292.
[4]	李子杰, 周菊香, 韩晓瑜, 甘健侯, 鹿泽光, 王俊. 序列特征与学习过程融合的知识追踪模型[J]. 计算机工程, 2024, 50(6): 77-85.
[5]	李永飞, 李铭洋, 常鑫, 曹可欣. 基于可解释性深度学习的物联网水质监测数据异常检测[J]. 计算机工程, 2024, 50(6): 179-187.
[6]	李致金, 汤佳辉, 闫金凤. 基于边缘计算的轻量化识别方法[J]. 计算机工程, 2024, 50(6): 287-295.
[7]	李雪, 王雅文, 张前进. 基于信息检索的源代码自动命名[J]. 计算机工程, 2024, 50(6): 304-310.
[8]	徐明亮, 李芳媛, 马浩然, 何飞. 大规模神经记录的峰电位聚类算法(特邀)[J]. 计算机工程, 2024, 50(6): 1-34.
[9]	魏琢艺, 罗迈, 李文兵, 曾远松, 余伟江, 杨跃东. 基于多源域适应的单细胞智能分类[J]. 计算机工程, 2024, 50(6): 48-55.
[10]	孙帮勇, 马铭, 于涛. 基于区域特征强化的多尺度伪装目标检测方法[J]. 计算机工程, 2024, 50(5): 209-219.
[11]	郑娟毅, 张庆珏, 董嘉豪, 郭梦月, 杨溥江. 一种深度学习的波束空间信道估计算法[J]. 计算机工程, 2024, 50(5): 298-305.
[12]	赵逸飞, 张俊华. 虚拟脊柱侧凸病例腰椎自动生成方法[J]. 计算机工程, 2024, 50(5): 323-329.
[13]	刘仲民, 严梁. 融合动态特征与注意力的敦煌壁画修复模型[J]. 计算机工程, 2024, 50(5): 342-353.
[14]	隗昊, 刁宏悦, 孔亮宸, 邓耀臣. 东北亚舆情文本细粒度命名实体识别方法研究[J]. 计算机工程, 2024, 50(5): 354-362.
[15]	隗昊, 刁宏悦, 孔亮宸, 邓耀臣. 东北亚舆情文本细粒度命名实体识别方法研究[J]. 计算机工程, 2024, 50(5): 354-362.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于深度学习的肝脏CT-MR图像无监督配准

Unsupervised Registration for Liver CT-MR Images Based on Deep Learning

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 16

参考文献

相关文章 15

编辑推荐

Metrics

本文评价