基于感知增强Swin Transformer的遥感图像检测

doi:10.19678/j.issn.1000-3428.0066941

摘要/Abstract

摘要：

随着遥感技术的快速发展，遥感图像检测被广泛应用于农业、军事、国防安全等众多领域。遥感图像相较于传统图像检测存在诸多难点，如何实现高效精准的遥感图像检测成为该领域的研究热点。针对遥感图像检测中存在的计算复杂度高、正负样本不平衡、目标尺度小等问题，提出一种基于感知增强Swin Transformer的遥感图像检测网络，以提升遥感图像检测性能。在主干网络设计过程中，利用Swin Transformer分层设计和移动窗口的优点有效减小计算复杂度，同时插入空间局部感知块，加强网络对局部相关性和结构信息的提取能力。设计区域分布回归损失为小目标分配更大的权重，解决了正负样本不平衡的问题，同时结合改进的IoU-aware分类损失消除不同分支之间的差距，降低分类和回归损失。在公共遥感数据集DOTA上的多组实验结果表明，该网络获得了78.47%的平均精度均值和10.8帧/s的检测速度，检测性能优于经典的目标检测网络Faster R-CNN、Mask R-CNN以及现有优秀的遥感图像检测网络，并且在各类不同尺度的目标上均具有较好的性能表现。

关键词: 遥感图像, 目标检测, Swin Transformer, 多尺度特征, 深度学习

Abstract:

Owing to the rapid development of remote sensing technology, remote sensing image detection technology is being used extensively in agriculture, military, national defense security, and other fields. Compared with conventional images, remote sensing images are more difficult to detect; therefore, researchers have endeavored to detect remote sensing images efficiently and accurately. To address the high calculation complexity, large-scale range variation, and scale imbalance of remote sensing images, this study proposes a perceptually enhanced Swin Transformer network, which improves the detection of remote sensing images. Exploiting the hierarchical design and shift windows of the basic Swin Transformer, the network inserts spatial local perceptually blocks into each stage, thus enhancing local feature extraction while negligibly increasing the calculation amount. An area-distributed regression loss is introduced to assign larger weights to small objects for solving scale imbalance; additionally, the network is combined with an improved IoU-aware classification loss to eliminate the discrepancy between different branches and reduce the loss of classification and regression. Experimental results on the public dataset DOTA show that the proposed network yields a mean Average Precision(mAP) of 78.47% and a detection speed of 10.8 frame/s, thus demonstrating its superiority over classical object detection networks(i.e., Faster R-CNN and Mask R-CNN) and existing excellent remote sensing image detection networks. Additionally, the network performs well on all types of objects at different scales.

Key words: remote sensing image, object detection, Swin Transformer, multi-scale feature, deep learning

祝冰艳, 陈志华, 盛斌. 基于感知增强Swin Transformer的遥感图像检测[J]. 计算机工程, 2024, 50(1): 216-223.

Bingyan ZHU, Zhihua CHEN, Bin SHENG. Remote Sensing Image Detection Based on Perceptually Enhanced Swin Transformer[J]. Computer Engineering, 2024, 50(1): 216-223.

http://www.ecice06.com/CN/Y2024/V50/I1/216

图/表 12

图1 基于感知增强Swin Transformer的遥感图像检测网络框架

Fig.1 Framework of remote sensing image detection network based on the perceptually enhanced Swin Transformer

图2 Swin Transformer网络结构

Fig.2 Network structure of the Swin Transformer

图3 Swin Transformer块结构

Fig.3 Block structure of the Swin Transformer

图4 局部增强Swin Transformer主干网络结构

Fig.4 Structure of local enhanced Swin Transformer backbone

图5 空间局部感知模块结构

Fig.5 Structure of spatial local perceptually module

图6 使用不同主干网络的检测结果可视化

Fig.6 Visualization of detection results by using different backbones

图7 DOTA数据集上的检测结果

Fig.7 Detection results on the DOTA dataset

图8 Swin-T和PESTN各项检测性能比较

Fig.8 Comparison of the detection performance of Swin-T and PESTN

图9 Swin-T和PESTN的ROC曲线比较

Fig.9 Comparison of the ROC curves of Swin-T and PESTN

参考文献 30

1	YANG X E, YANG J R, YAN J C, et al. SCRDet: towards more robust detection for small, cluttered and rotated objects[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 8232-8241.
2	YANG X E, YAN J C, LIAO W L, et al. SCRDet++: detecting small, cluttered and rotated objects via instance-level feature denoising and rotation loss smoothing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(2): 2384- 2399. doi: 10.1109/TPAMI.2022.3166956
3	DING J A, XUE N, LONG Y, et al. Learning RoI Transformer for oriented object detection in aerial images[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 2849-2858.
4	GUAN H Y, YU Y T, LI D L, et al. RoadCapsFPN: capsule feature pyramid network for road extraction from VHR optical remote sensing imagery. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(8): 11041- 11051. doi: 10.1109/TITS.2021.3098855
5	谢星星, 程塨, 姚艳清, 等. 动态特征融合的遥感图像目标检测. 计算机学报, 2022, 45(4): 735- 747.
	XIE X X, CHENG G, YAO Y Q, et al. Dynamic feature fusion for object detection in remote sensing images. Chinese Journal of Computers, 2022, 45(4): 735- 747.
6	王道累, 杜文斌, 刘易腾, 等. 基于密集连接与特征增强的遥感图像检测. 计算机工程, 2022, 48(6): 251-256, 262. doi: 10.19678/j.issn.1000-3428.0061482
	WANG D L, DU W B, LIU Y T, et al. Remote sensing images detection based on dense connection and feature enhancement. Computer Engineering, 2022, 48(6): 251-256, 262. doi: 10.19678/j.issn.1000-3428.0061482
7	ZHANG C, LAM K M, WANG Q. CoF-Net: a progressive coarse-to-fine framework for object detection in remote-sensing imagery. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61, 1- 17.
8	ZHU X Z, SU W J, LU L W, et al. Deformable DETR: deformable Transformers for end-to-end object detection[EB/OL]. [2023-01-11]. https://arxiv.org/abs/2010.04159.pdf.
9	LI W T, CHEN Y J, HU K X, et al. Oriented RepPoints for aerial object detection[C]//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2022: 1829-1838.
10	CHEN H, QI Z P, SHI Z W. Remote sensing image change detection with Transformers. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60, 1- 14.
11	LIU X L, MA S P, HE L Y, et al. Hybrid network model: TransConvNet for oriented object detection in remote sensing images. Remote Sensing, 2022, 14(9): 2090. doi: 10.3390/rs14092090
12	DAI L H, LIU H, TANG H, et al. AO2-DETR: arbitrary-oriented object detection Transformer[EB/OL]. [2023-01-11]. https://arxiv.org/abs/2205.12785.pdf.
13	DAI Y N, YU J Y, ZHANG D A, et al. RODFormer: high-precision design for rotating object detection with Transformers. Sensors, 2022, 22(7): 2633. doi: 10.3390/s22072633
14	LI Q Y, CHEN Y S, ZENG Y. Transformer with transfer CNN for remote-sensing-image object detection. Remote Sensing, 2022, 14(4): 984. doi: 10.3390/rs14040984
15	ZHENG Y B, SUN P, ZHOU Z T, et al. ADT-det: adaptive dynamic refined single-stage Transformer detector for arbitrary-oriented object detection in satellite optical imagery. Remote Sensing, 2021, 13(13): 2623. doi: 10.3390/rs13132623
16	LIU Z, LIN Y T, CAO Y E, et al. Swin Transformer: hierarchical vision Transformer using shifted windows[C]//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2021: 10012-10022.
17	YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[EB/OL]. [2023-01-11]. https://arxiv.org/abs/1511.07122.pdf.
18	ZHANG H Y, WANG Y, DAYOUB F, et al. VarifocalNet: an IoU-aware dense object detector[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 8514-8523.
19	HOU L P, LU K, XUE J A. Refined one-stage oriented object detection method for remote sensing images. IEEE Transactions on Image Processing, 2022, 31, 1545- 1558. doi: 10.1109/TIP.2022.3143690
20	ZHENG W, TANG W L, CHEN S J, et al. CIA-SSD: confident IoU-aware single-stage object detector from point cloud[C]//Proceedings of AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2021: 3555-3562.
21	XIA G S, BAI X A, DING J A, et al. DOTA: a large-scale dataset for object detection in aerial images[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 3974-3983.
22	PADILLA R, NETTO S L, DA SILVA E A B. A survey on performance metrics for object-detection algorithms[C]//Proceedings of 2020 International Conference on Systems, Signals and Image Processing. Washington D. C., USA: IEEE Press, 2020: 237-242.
23	CHICCO D, JURMAN G. The advantages of the Matthews Correlation Coefficient(MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics, 2020, 21(1): 6. doi: 10.1186/s12864-019-6413-7
24	TOUVRON H, CORD M, DOUZE M, et al. Training data-efficient image Transformers & distillation through attention[EB/OL]. [2023-01-11]. https://arxiv.org/abs/2012.12877.pdf.
25	HE K M, GKIOXARI G, DOLLAR P, et al. Mask R-CNN[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 2980-2988.
26	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137- 1149. doi: 10.1109/TPAMI.2016.2577031
27	YANG X E, YAN J C, FENG Z M, et al. R3Det: refined single-stage detector with feature refinement for rotating object[C]//Proceedings of AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2021: 3163-3171.
28	XU Y C, FU M T, WANG Q M, et al. Gliding vertex on the horizontal bounding box for multi-oriented object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(4): 1452- 1459. doi: 10.1109/TPAMI.2020.2974745
29	ZHANG H, LI F, LIU S L, et al. DINO: DETR with improved denoising anchor boxes for end-to-end object detection[EB/OL]. [2023-01-11]. https://arxiv.org/abs/2203.03605.pdf.
30	WANG J W, DING J A, GUO H W, et al. Mask OBB: a semantic attention-based mask oriented bounding box representation for multi-category object detection in aerial images. Remote Sensing, 2019, 11(24): 2930. doi: 10.3390/rs11242930

[1]	圣文顺, 余熊峰, 林佳燕, 陈欣. 融合注意力与特征金字塔的小尺度目标检测算法[J]. 计算机工程, 2024, 50(1): 242-250.
[2]	蒋心璐, 陈天恩, 王聪, 赵春江. 大田环境下的农业害虫图像小目标检测算法[J]. 计算机工程, 2024, 50(1): 232-241.
[3]	白尚旺, 王梦瑶, 胡静, 陈志泊. 多区域注意力的细粒度图像分类网络[J]. 计算机工程, 2024, 50(1): 271-278.
[4]	曹广硕, 黄瑞章, 陈艳平, 秦永彬. 基于多模态学习的乳腺癌生存预测研究[J]. 计算机工程, 2024, 50(1): 296-305.
[5]	李嘉新, 侯进, 盛博莹, 周宇航. 基于改进YOLOv5的遥感小目标检测网络[J]. 计算机工程, 2023, 49(9): 256-264.
[6]	池亚平, 岳梓岩, 林雨衡. 基于Transformer的SM4算法工作模式识别[J]. 计算机工程, 2023, 49(9): 109-117.
[7]	林中霖, 时金桥, 王美琪, 王学宾, 王雨燕. 基于应用行为划分的Android恶意应用检测技术[J]. 计算机工程, 2023, 49(9): 125-136.
[8]	韩璐, 霍纬纲, 张永会, 刘涛. 基于多尺度特征融合与双注意力机制的多元时间序列预测[J]. 计算机工程, 2023, 49(9): 99-108.
[9]	龙玉江, 卫薇, 舒彧, 张正刚, 王道累, 李峰. 基于自适应关键点的破损旋转绝缘子检测方法[J]. 计算机工程, 2023, 49(9): 272-278.
[10]	徐春波, 闫娟, 杨慧斌, 王博, 吴晗. 基于目标检测和语义分割的视觉SLAM算法[J]. 计算机工程, 2023, 49(8): 199-206, 214.
[11]	江雨燕, 陶承凤, 李平. 数据增强和自适应自步学习的深度子空间聚类算法[J]. 计算机工程, 2023, 49(8): 96-103, 110.
[12]	宋志娜, 李莎, 杨建明, 徐川. 基于特征与区域定位增强的遥感舰船目标检测[J]. 计算机工程, 2023, 49(8): 257-264.
[13]	李泽水, 冀俊忠, 杨翠翠. 基于边权重信息深度网络嵌入的PPIN功能模块检测[J]. 计算机工程, 2023, 49(8): 69-76.
[14]	王可铮, 徐玉芬, 周尚波. 结合对比感知损失和融合注意力的图像去雾模型[J]. 计算机工程, 2023, 49(8): 207-214.
[15]	刘俊豪, 王美林, 谢兴, 宋烨兴, 许莉花. 基于改进YOLOv5的皮革瑕疵检测算法[J]. 计算机工程, 2023, 49(8): 240-249.

选择文件类型/文献管理软件名称

选择包含的内容