摘要： Transformer模型凭借其出色的全局信息捕获能力和强大的表示能力,被广泛应用于医学图像分割领域,并取得了显著成效。然而,这些方法在对图像进行序列化时,会将图像分割成固定大小的块,仅提取单一尺度的全局特征,在一定程度上割裂了图像的语义特征,最终导致分割精度不佳。为此,提出一种多尺度自注意力Transformer架构(MultiFormer)。首先,采用连续卷积和下采样模块对图像进行处理;然后,使用多尺度卷积投影模块来替换原来的1×1投影模块;最后,在自注意力模块生成的特征图中引入可变形卷积。与传统Transformer图像序列化过程相比,这种连续卷积在产生相同分辨率的特征时,有效增大了感受野,保留了二维(2D)图像的空间相关性,避免固定位置和大小的图像块带来的语义信息损失。同时,多尺度卷积投影模块利用4种不同大小的卷积核捕捉图像中的上下文信息,并通过通道拼接实现多尺度特征融合,反映了不同尺度下局部与局部间的注意力,而不仅限于单一尺度,实现了模型对不同尺度语义信息的聚合能力,进一步减轻了语义割裂问题。此外,可变形卷积通过引入一个额外的卷积层来学习并生成偏移场,使得卷积核能够灵活调整其形状,以适应图像中形态多变的病变或器官,提升模型对复杂医学图像的处理能力。将该模块分别插入到SETR、TransUNet、TransFuse这3个网络结构中,并在ACDC心脏数据集和ISIC 2018皮肤病变数据集上进行实验,结果表明,Dice系数分别提升了3.63、1.06、2.30百分点和1.22、2.31、3.01百分点。MultiFormer具有即插即用性,能够方便地集成到各种下游医学图像分析任务中。

关键词: 医学图像分割, Transformer, 注意力机制, 多尺度卷积投影, 可变性卷积

Abstract: The Transformer model, owing to its excellent global information capture and powerful representation abilities, has been widely used in medical image segmentation and has achieved remarkable results. However, when serializing images, these methods divide them into fixed-sized blocks and extract only global features of a single scale, which, to some extent, fragments the semantic features of the images, ultimately leading to poor segmentation accuracy. To address this issue, this paper proposes a Multi-scale self-attention Transformer architecture (MultiFormer). First, continuous convolution and downsampling modules are used to process images. Then, the original 1×1 projection module is replaced with a multi-scale convolution projection module. Finally, deformable convolution is introduced into the feature maps generated by the self-attention module. Compared to the traditional Transformer image serialization process, this continuous convolution effectively enlarges the receptive field while generating features of the same resolution, retains the spatial correlation of 2-Dimensional (2D) images, and avoids the loss of semantic information caused by fixed-position and fixed-size image blocks. Meanwhile, the multi-scale convolution projection module captures contextual information in the images using four convolution kernels of different sizes and achieves multi-scale feature fusion through channel concatenation, reflecting attention between local regions at different scales, rather than being limited to a single scale, which enables the model to aggregate semantic information at different scales and further alleviates the problem of semantic fragmentation. Deformable convolution introduces an additional convolution layer to learn and generate an offset field, allowing the convolution kernel to flexibly adjust its shape to adapt to morphologically diverse lesions or organs in the images and enhancing the ability of the model to process complex medical images. This module is inserted into three network structures—SETR, TransUNet, and TransFuse—and experiments are conducted on the ACDC heart and ISIC 2018 skin lesion datasets. The results show that the Dice coefficient improves by 3.63, 1.06, and 2.30 percentage points and by 1.22, 2.31, and 3.01 percentage points, respectively. MultiFormer is a plug-and-play tool and can be easily integrated into various downstream medical image analysis tasks.

Key words: medical image segmentation, Transformer, attention mechanism, multi-scale convolutional projection, deformable convolution

中图分类号: 

• TP391