用于建筑物分割的平行结构特征融合网络

doi:10.19678/j.issn.1000-3428.0068259

摘要/Abstract

摘要：

遥感建筑物分割是对遥感图像中的建筑物进行像素级别的分割, 从遥感图像中准确提取出建筑物区域, 包括建筑物轮廓和内部细节信息。由于遥感图像的特殊性, 在对建筑物分割时, 阴影与建筑物颜色相似易造成欠分割, 树木遮挡等因素易造成过分割。针对遥感图像中建筑物轮廓分割不完整、阴影干扰强以及分割边缘锯齿状明显等问题, 提出一种平行结构的多分支特征融合网络(MFF-Net)。该网络以ResNet-50作为主干网络, 解码器采用包含双通道掩码分支的多条平行结构, 分别恢复不同尺度的特征图。同时, 在每条分支结构中使用改进后的CBAM注意力以加强边缘重要特征, 通过双通道掩码结构调整通道交互性, 最后进行特征融合。在ISPRS Potsdam和ISPRS Vaihingen数据集上的实验结果表明, 与现有主流分割网络相比, MFF-Net的全局准确率、精确率、召回率、F1值、均交并比(mIoU)均有不同程度的提升, 在Vaihingen数据集上精确率达到96.22%, F1值达到95.55%, mIoU达到92.16%, 在Potsdam数据集上精确率达到96.95%, F1值达到96.32%, mIoU达到93.40%, 其提取的建筑物轮廓完整清晰, 抗干扰性更强。

关键词: 遥感图像, 特征融合, 建筑物分割, 双通道掩码, 注意力

Abstract:

Remote sensing building segmentation refers to pixel-level segmentation of buildings in remote sensing images, including precise outlines of buildings and detailed internal information. However, the unique characteristics of remote sensing images, with shadows similar in color to that of buildings, often result in under-segmentation, whereas factors such as tree occlusion can easily result in over-segmentation. A Multi-Feature Fusion Network (MFF-Net) based on a parallel structure is presented to address weak shadow interference, visible jagged edges of segmentation, and poor segmentation of building outlines in remote-sensing photos. The decoder uses ResNet-50 as its backbone and numerous parallel-structured dual-channel mask branches to reconstruct feature maps at various scales. To strengthen the critical edge features, an improved Convolutional Block Attention Module (CBAM) is further added to each branch structure. The bidirectional channel mask competition module is subsequently used to adjust channel interaction, thereby completing feature fusion. Experimental results on the ISPRS Potsdam and ISPRS Vaihingen datasets show that compared with existing mainstream segmentation networks, the global accuracy, precision, recall, F1 value, and mean Intersection over Union (mIoU) are improved to varying degrees. The precision of the Vaihingen dataset is 96.22%, the F1 value 95.55%, and mIoU 92.16%. On the Potsdam dataset, the precision is 96.95%, the F1 value 96.32%, and mIoU 93.40%. The extracted building contours are complete and clear, and resulted in strong interference resistance.

Key words: remote sensing image, feature fusion, building segmentation, bidirectional channel mask, attention

赵婉秋, 张俊虎, 李海涛. 用于建筑物分割的平行结构特征融合网络[J]. 计算机工程, 2024, 50(8): 239-248.

Wanqiu ZHAO, Junhu ZHANG, Haitao LI. Feature Fusion Network with Parallel Structure for Building Segmentation[J]. Computer Engineering, 2024, 50(8): 239-248.

https://www.ecice06.com/CN/Y2024/V50/I8/239

图/表 13

图1 MFF-Net网络结构

Fig.1 Structure of MFF-Net network

图2 双通道掩码模块

Fig.2 Dual channel masking module

图3 改进的CBAM通道注意力结构

Fig.3 Improved CBAM channel attention structure

图4 改进的CBAM空间注意力结构

Fig.4 Improved CBAM spatial attention structure

图5 深监督与原始监督的对比

Fig.5 Comparison between deep supervision and original supervision

图6 Vaihingen测试集上的预测结果

Fig.6 Prediction results on the Vaihingen test set

图7 Potsdam测试集上的预测结果

Fig.7 Prediction results on the Potsdam test set

参考文献 32

1	WANG Y H, GAO L R, HONG D F, et al. Mask DeepLab: end-to-end image segmentation for change detection in high-resolution remote sensing images. International Journal of Applied Earth Observation and Geoinformation, 2021, 104, 102582. doi: 10.1016/j.jag.2021.102582
2	SONG Y H, HAO Y. Image segmentation algorithms overview[EB/OL]. [2023-07-05]. https://arxiv.org/abs/1707.02051.
3	WANG B, CHEN L L, CHENG J. New result on maximum entropy threshold image segmentation based on P system. Optik, 2018, 163, 81- 85. doi: 10.1016/j.ijleo.2018.02.062
4	YAYLA R, SEN B. Region-based segmentation of terrain fields in SAR images[C]//Proceedings of the 28th Signal Processing and Communications Applications Conference. Washington D. C., USA: IEEE Press, 2020: 1-4.
5	胡高珍, 徐胜军, 孟月波, 等. 基于边缘约束局部区域MRF的图像分割方法. 计算机工程, 2021, 47(6): 253-261, 270. URL
	HU G Z, XU S J, MENG Y B, et al. Image segmentation method based on MRF with edge constrained local region. Computer Engineering, 2021, 47(6): 253-261, 270. URL
6	XIE X L, XIE G, XU X Y, et al. Automatic image segmentation with superpixels and image-level labels. IEEE Access, 2019, 7, 10999- 11009. doi: 10.1109/ACCESS.2019.2891941
7	YUAN X H, SHI J F, GU L C. A review of deep learning methods for semantic segmentation of remote sensing imagery. Expert Systems with Applications, 2021, 169, 114417. doi: 10.1016/j.eswa.2020.114417
8	林思玉, 王敬东, 顾东泽, 等. 一种基于FCOS神经网络的小建筑物目标检测方法. 半导体光电, 2022, 43(2): 369- 376. URL
	LIN S Y, WANG J D, GU D Z, et al. Small buildings detection method based on FCOS neural network. Semiconductor Optoelectronics, 2022, 43(2): 369- 376. URL
9	SUN Y, TIAN Y, XU Y P. Problems of encoder-decoder frameworks for high-resolution remote sensing image segmentation: structural stereotype and insufficient learning. Neurocomputing, 2019, 330, 297- 304. doi: 10.1016/j.neucom.2018.11.051
10	景庄伟, 管海燕, 彭代峰, 等. 基于深度神经网络的图像语义分割研究综述. 计算机工程, 2020, 46(10): 1- 17. doi: 10.3778/j.issn.1002-8331.2001-0320
	JING Z W, GUAN H Y, PENG D F, et al. Survey of research in image semantic segmentation based on deep neural network. Computer Engineering, 2020, 46(10): 1- 17. doi: 10.3778/j.issn.1002-8331.2001-0320
11	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[EB/OL]. [2023-07-05]. https://arxiv.org/abs/1706.03762.
12	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: Transformers for image recognition at scale[EB/OL]. [2023-07-05]. https://arxiv.org/abs/2010.11929.
13	XIE E Z, WANG W H, YU Z D, et al. SegFormer: simple and efficient design for semantic segmentation with transformers[EB/OL]. [2023-07-05]. https://arxiv.org/abs/2105.15203.
14	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2015: 3431-3440.
15	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[EB/OL]. [2023-07-05]. https://arxiv.org/abs/1505.04597.
16	褚张晴晴, 钟志强, 颜子夜, 等. 基于特征融合与注意力机制的脑肿瘤分割算法. 计算机工程, 2023, 49(10): 154- 161. URL
	CHUZHANG Q Q, ZHONG Z Q, YAN Z Y, et al. Brain tumor segmentation algorithm based on feature fusion and attention mechanism. Computer Engineering, 2023, 49(10): 154- 161. URL
17	CHEN L C, PAPANDREOU G, KOKKINOS I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[EB/OL]. [2023-07-05]. https://arxiv.org/abs/1412.7062.
18	CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834- 848. doi: 10.1109/TPAMI.2017.2699184
19	CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[EB/OL]. [2023-07-05]. https://arxiv.org/abs/1706.05587.
20	CHEN L C, ZHU Y K, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[EB/OL]. [2023-07-05]. https://link.springer.com/chapter/10.1007/978-3-030-01234-2_49.
21	苏晓东, 李世洲, 赵佳圆, 等. 基于多级叠加和注意力机制的图像语义分割. 计算机工程, 2023, 49(9): 265-271, 278. URL
	SU X D, LI S Z, ZHAO J Y, et al. Image semantic segmentation based on multi-level superposition and attention mechanism. Computer Engineering, 2023, 49(9): 265-271, 278. URL
22	张宸嘉, 朱磊, 俞璐. 卷积神经网络中的注意力机制综述. 计算机工程与应用, 2021, 57(20): 64- 72. URL
	ZHANG C J, ZHU L, YU L. Review of attention mechanism in convolutional neural networks. Computer Engineering and Applications, 2021, 57(20): 64- 72. URL
23	HU J, SHEN L, SUN G. Squeeze-and-Excitation networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 7132-7141.
24	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[EB/OL]. [2023-07-05]. https://arxiv.org/abs/1807.06521.
25	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 770-778.
26	LEE C Y, XIE S, GALLAGHER P, et al. Deeply-supervised nets[EB/OL]. [2023-07-05]. https://arxiv.org/abs/1409.5185.
27	ISPRS WG III/4. ISPRS 2D semantic labeling contest[EB/OL]. [2023-07-05]. http://www2.isprs.org/commissions/comm3/wg4/semantic-labeling.html.
28	BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481- 2495. doi: 10.1109/TPAMI.2016.2644615
29	ZHAO H S, SHI J P, QI X J, et al. Pyramid scene parsing network[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 2881-2890.
30	QIN X B, ZHANG Z C, HUANG C Y, et al. U²-Net: going deeper with nested U-structure for salient object detection. Pattern Recognition, 2020, 106, 107404. doi: 10.1016/j.patcog.2020.107404
31	SUN K, XIAO B, LIU D, et al. Deep high-resolution representation learning for human pose estimation[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 5693-5703.
32	GUO M H, LU C G, HOU Q B, et al. SegNeXt: rethinking convolutional attention design for semantic segmentation[EB/OL]. [2023-07-05]. https://arxiv.org/abs/2209.08575.

[1]	李俊俊, 董建刚, 李坤. 基于Kubernetes的集群节能策略研究[J]. 计算机工程, 2024, 50(9): 82-91.
[2]	杨明强, 卢健. 基于跨模态注意力的目标语音提取[J]. 计算机工程, 2024, 50(9): 121-129.
[3]	林畅, 郭伟, 任哲聪, 金海波. 基于Transformer的目标跟踪与分割统一算法[J]. 计算机工程, 2024, 50(9): 130-141.
[4]	李泽霖, 吕兆峰, 陈富强, 李克. 基于多跳信息融合的实体对齐模型[J]. 计算机工程, 2024, 50(9): 142-152.
[5]	王汝英, 马嘉骏, 董建强, 刘万龙, 张海涛, 尹凯, 赵博超. 基于MTS-BiGRU-DMHSA的工业负荷预测方法[J]. 计算机工程, 2024, 50(9): 169-178.
[6]	李俊仪, 李向阳, 龙朝勋, 李海燕, 李红松, 余鹏飞. 基于多级区域选择与跨层特征融合的野生菌分类[J]. 计算机工程, 2024, 50(9): 179-188.
[7]	蔡俊民, 梁正友, 孙宇, 陈子奥. 基于可变形三维图卷积的轻量级点云分类研究[J]. 计算机工程, 2024, 50(9): 255-265.
[8]	朱凯, 李理, 张彤, 江晟, 别一鸣. 基于Transformer的多阶段运动模糊图像修复网络[J]. 计算机工程, 2024, 50(9): 276-285.
[9]	张天鹏, 韩晶, 吕学强. 基于多任务学习的超分辨率辅助小目标检测[J]. 计算机工程, 2024, 50(9): 304-312.
[10]	郭敏, 张熙涵, 李阳. 融合注意力的教师互一致性半监督医学图像分割[J]. 计算机工程, 2024, 50(9): 313-323.
[11]	曾钰琦, 刘博, 钟柏昌, 钟瑾. 智慧教育下基于改进YOLOv8的学生课堂行为检测算法[J]. 计算机工程, 2024, 50(9): 344-355.
[12]	张华青, 夏张涛, 陆晓庆, 童基均. 基于字形特征的血管外科命名实体识别[J]. 计算机工程, 2024, 50(8): 13-21.
[13]	饶日昕, 王怡文, 曾砺志, 童心恬, 赵海涛. 面向废旧电缆检测的轻量化网络模型[J]. 计算机工程, 2024, 50(8): 22-30.
[14]	李华昱, 张智康, 闫阳, 岳阳. 基于知识图谱增强的领域多模态实体识别[J]. 计算机工程, 2024, 50(8): 31-39.
[15]	王蕾, 党时鹏, 潘丰. 基于卷积神经网络的隐匿性旁路预测模型[J]. 计算机工程, 2024, 50(8): 40-49.

选择文件类型/文献管理软件名称

选择包含的内容