Dual-Domain Multi-Modal Magnetic Resonance Image Reconstruction Based on Attention Enhancement

doi:10.19678/j.issn.1000-3428.0252289

Abstract

Abstract: Magnetic Resonance Imaging (MRI) serves as an important tool for auxiliary clinical diagnosis and lesion detection. Convolutional Neural Network (CNN)-based magnetic resonance image reconstruction has made significant progress in terms of speed and accuracy. However, existing models mostly depend on single-domain feature extraction and remain limited by local receptive fields, resulting in slightly lower reconstruction quality for complex anatomical structures. To address these issues, this study proposes an Improved Hybrid Attention-Enhanced Dual-Domain Multi-modal Reconstruction Network (AMC-Net) based on a parallel framework. The network constructs a Multi-Modal Feature Fusion (MMFF) module that matches and fuses shared features from multi-modal inputs, thereby supplementing missing structural features in the initial input and effectively alleviating artifact interference. Based on the Iterative Shrinkage Thresholding Algorithm (ISTA), we build a Progressive Interactive Reconstruction Subnetwork (PIRN), which employs multi-layer iteration and dual-domain information interaction data streams for progressive reconstruction, and introduces an improved attention-based convolution mechanism with global receptive fields to achieve global feature learning. Furthermore, the approach synergistically integrates channel-spatial attention and Self-Attention (SA) mechanisms to refine reconstruction results, thereby highlighting detailed features, suppressing artifacts, and ultimately improving the restoration quality of fine structures and overall visual quality. The experimental results demonstrate that AMC-Net outperforms mainstream algorithms on the static brain MRI datasets IXI and BraTS2018, adapting well to various sampling conditions. Under 5×random sampling, it achieves an average Peak Signal-to-Noise Ratio (PSNR) of 42.66 dB and a Structural Similarity Index Measure (SSIM) exceeding 0.97, generating reconstructed images with high clarity and detailed restoration.

Key words: Magnetic Resonance Imaging (MRI), dual-domain reconstruction, parallel network, multi-modal, attention mechanism

摘要： 磁共振成像(MRI)是临床辅助诊断、病变检测的重要手段。基于卷积神经网络（CNN）的磁共振图像重建在速度和精准度上取得较大进展。但现有模型多依赖单一域的特征提取，且受限于局部感受野，导致复杂解剖结构的重建质量略低。为解决这些问题，基于并行框架提出一种改进的混合注意力增强的双域多模态重建网络（AMC-Net）。该网络构造多模态特征融合（MMFF）模块，匹配并融合多模态输入之间的共享特征，补充初始输入中缺失的结构特征，尽量减轻初始输入的伪影干扰。基于迭代收缩阈值算法（ISTA）构建渐进式交互重建子网络（PIRN），采用多层迭代和双域信息交互数据流实现渐进式重建，并引入注意力机制改进的卷积实现全局特征学习。此外，协同通道-空间注意力和单头自注意力（SA）机制来精细化重建结果，突出细节特征并抑制伪影，从而提升微小结构的恢复质量和整体视觉效果。实验结果表明，AMC-Net在静态脑部MRI数据集IXI和BraTS2018上的重建效果优于主流模型，在5倍随机采样下，平均峰值信噪比(PSNR)达到42.66 dB且结构相似性指数(SSIM)超过0.97，能够生成高清晰度且细节还原准确的重建图像，同时具备对多种采样条件的良好适应性。

关键词: 磁共振成像, 双域重建, 并行网络, 多模态, 注意力机制

CLC Number:

TP391

LI Yanqing, ZHU Hongqing. Dual-Domain Multi-Modal Magnetic Resonance Image Reconstruction Based on Attention Enhancement[J]. Computer Engineering, 2026, 52(11): 176-190.

李彦青, 朱宏擎. 基于注意力增强的双域多模态磁共振图像重建[J]. 计算机工程, 2026, 52(11): 176-190.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0252289

https://www.ecice06.com/EN/Y2026/V52/I11/176

References

[1] 邓戈文, 魏国辉, 马志庆. 基于深度学习的MRI重建方法综述[J]. 计算机工程与应用, 2023, 59(20): 67-76.DENG G W, WEI G H, MA Z Q. Review of deep learning methods for MRI reconstruction[J]. Computer Engineering and Applications, 2023, 59(20): 67-76. (in Chinese)
[2] GRISWOLD M A, JAKOB P M, NITTKA M, et al. Partially Parallel Imaging with Localized Sensitivities (PILS)[J]. Magnetic Resonance in Medicine, 2000, 44(4): 602-609.
[3] DONOHO D L. Compressed sensing[J]. IEEE Transactions on Information Theory, 2006, 52(4): 1289-1306.
[4] WANG S S, SU Z H, YING L, et al. Accelerating magnetic resonance imaging via deep learning[C]//Proceedings of the IEEE 13th International Symposium on Biomedical Imaging (ISBI). Washington D.C., USA: IEEE Press, 2016: 514-517.
[5] HUANG J H, WU Y Z, WU H J, et al. Fast MRI reconstruction: how powerful transformers are? [C]//Proceedings of the 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). Washington D.C., USA: IEEE Press, 2022: 2066-2070.
[6] ZHANG J, GHANEM B. ISTA-Net: interpretable optimization-inspired deep network for image compressive sensing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2018: 1828-1837.
[7] ZHOU B, SCHLEMPER J, DEY N, et al. Dual-domain self-supervised learning for accelerated non-Cartesian MRI reconstruction[J]. Medical Image Analysis, 2022, 81: 102538.
[8] HONG G Q, WEI Y T, MORLEY W A W, et al. Dual-domain accelerated MRI reconstruction using transformers with learning-based undersampling[J]. Computerized Medical Imaging and Graphics, 2023, 106: 102206.
[9] SUN H Z, LI Y Z, LI Z S, et al. Fourier convolution block with global receptive field for MRI reconstruction[J]. Medical Image Analysis, 2025, 99: 103349.
[10] ZHANG H Y, YANG T J, WANG H, et al. FDuDoCLNet: fully dual-domain contrastive learning network for parallel MRI reconstruction[J]. Magnetic Resonance Imaging, 2025, 117: 110336.
[11] YANG J W, LI X X, LIU F H, et al. Fast multi-contrast MRI acquisition by optimal sampling of information complementary to pre-acquired MRI contrast[J]. IEEE Transactions on Medical Imaging, 2023, 42(5): 1363-1373.
[12] LIU X, CHEN H Y, YAO C, et al. BTMF-GAN: a multi-modal MRI fusion generative adversarial network for brain tumors[J]. Computers in Biology and Medicine, 2023, 157: 106769.
[13] WEI J B, YANG G, WANG Z J, et al. Misalignment-resistant deep unfolding network for multi-modal MRI super-resolution and reconstruction[J]. Knowledge-Based Systems, 2024, 296: 111866.
[14] ZHOU X Y, ZHANG Z X, DU H W, et al. MLMFNet: a multi-level modality fusion network for multi-modal accelerated MRI reconstruction[J]. Magnetic Resonance Imaging, 2024, 111: 246-255.
[15] HU C, LI C, WANG H F, et al. Self-supervised learning for MRI reconstruction with a parallel network training framework[C]//Proceedings of the MICCAI 2021. Berlin, Germany: Springer, 2021: 382-391.
[16] ZHOU B, DEY N, SCHLEMPER J, et al. DSFormer: a dual-domain self-supervised transformer for accelerated multi-contrast MRI reconstruction[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Washington D.C., USA: IEEE Press, 2023: 4955-4964.
[17] CHEN Y P, FAN H Q, XU B, et al. Drop an octave: reducing spatial redundancy in convolutional neural networks with octave convolution[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Washington D.C., USA: IEEE Press, 2020: 3434-3443.
[18] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[M]. Berlin, Germany: Springer, 2018.
[19] BAKAS S, REYES M, JAKAB A, et al. Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge[EB/OL]. [2025-03-17]. https://arxiv.org/abs/1811.02629.
[20] RAN M S, XIA W J, HUANG Y Q, et al. MD-Recon-Net: a parallel dual-domain convolutional neural network for compressed sensing MRI[J]. IEEE Transactions on Radiation and Plasma Medical Sciences, 2021, 5(1): 120-135.
[21] FENG C M, YAN Y L, CHEN G, et al. Multimodal transformer for accelerated MR imaging[J]. IEEE Transactions on Medical Imaging, 2023, 42(10): 2804-2816.
[22] LIU Y, PANG Y W, LIU X H, et al. DIIK-Net: a full-resolution cross-domain deep interaction convolutional neural network for MR image reconstruction[J]. Neurocomputing, 2023, 517: 213-222.
[23] ZHAO X, YANG T J, LI B J, et al. SwinGAN: a dual-domain Swin Transformer-based generative adversarial network for MRI reconstruction[J]. Computers in Biology and Medicine, 2023, 153: 106513.
[24] LEI P C, FANG F M, ZHANG G X, et al. Deep unfolding convolutional dictionary model for multi-contrast MRI super-resolution and reconstruction[EB/OL]. [2025-03-17]. https://arxiv.org/abs/2309.01171.
[25] LI B W, WANG Z W, YANG Z Y, et al. Progressive dual-domain-transfer cycleGAN for unsupervised MRI reconstruction[J]. Neurocomputing, 2024, 563: 126934.
[26] WOOLSON R F. Wilcoxon signed-rank test[EB/OL]. [2025-03-17]. https://doi.org/10.1002/0470011815.b2a15177.
[27] FENG C M, YAN Y L, YU K, et al. Exploring separable attention for multi-contrast MR image super-resolution[J]. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(9): 12251-12262.
[28] ZHANG H, WANG Q, SHI J, et al. Deep unfolding network with spatial alignment for multi-modal MRI reconstruction[J]. Medical Image Analysis, 2025, 99: 103331.
[29] 杨青海, 杨敏. 基于低秩稀疏分解快速算法的动态MRI重建[J]. 软件工程, 2022, 25(7): 33-36, 32.YANG Q H, YANG M. Dynamic MRI reconstruction based on low-rank sparse decomposition fast algorithm[J]. Software Engineer, 2022, 25(7): 33-36, 32. (in Chinese)

Please choose a citation manager

Content to export