基于感知增强Swin Transformer的遥感图像检测

doi:10.19678/j.issn.1000-3428.0066941

摘要/Abstract

摘要：

随着遥感技术的快速发展，遥感图像检测被广泛应用于农业、军事、国防安全等众多领域。遥感图像相较于传统图像检测存在诸多难点，如何实现高效精准的遥感图像检测成为该领域的研究热点。针对遥感图像检测中存在的计算复杂度高、正负样本不平衡、目标尺度小等问题，提出一种基于感知增强Swin Transformer的遥感图像检测网络，以提升遥感图像检测性能。在主干网络设计过程中，利用Swin Transformer分层设计和移动窗口的优点有效减小计算复杂度，同时插入空间局部感知块，加强网络对局部相关性和结构信息的提取能力。设计区域分布回归损失为小目标分配更大的权重，解决了正负样本不平衡的问题，同时结合改进的IoU-aware分类损失消除不同分支之间的差距，降低分类和回归损失。在公共遥感数据集DOTA上的多组实验结果表明，该网络获得了78.47%的平均精度均值和10.8帧/s的检测速度，检测性能优于经典的目标检测网络Faster R-CNN、Mask R-CNN以及现有优秀的遥感图像检测网络，并且在各类不同尺度的目标上均具有较好的性能表现。

关键词: 遥感图像, 目标检测, Swin Transformer, 多尺度特征, 深度学习

Abstract:

Owing to the rapid development of remote sensing technology, remote sensing image detection technology is being used extensively in agriculture, military, national defense security, and other fields. Compared with conventional images, remote sensing images are more difficult to detect; therefore, researchers have endeavored to detect remote sensing images efficiently and accurately. To address the high calculation complexity, large-scale range variation, and scale imbalance of remote sensing images, this study proposes a perceptually enhanced Swin Transformer network, which improves the detection of remote sensing images. Exploiting the hierarchical design and shift windows of the basic Swin Transformer, the network inserts spatial local perceptually blocks into each stage, thus enhancing local feature extraction while negligibly increasing the calculation amount. An area-distributed regression loss is introduced to assign larger weights to small objects for solving scale imbalance; additionally, the network is combined with an improved IoU-aware classification loss to eliminate the discrepancy between different branches and reduce the loss of classification and regression. Experimental results on the public dataset DOTA show that the proposed network yields a mean Average Precision(mAP) of 78.47% and a detection speed of 10.8 frame/s, thus demonstrating its superiority over classical object detection networks(i.e., Faster R-CNN and Mask R-CNN) and existing excellent remote sensing image detection networks. Additionally, the network performs well on all types of objects at different scales.

Key words: remote sensing image, object detection, Swin Transformer, multi-scale feature, deep learning

祝冰艳, 陈志华, 盛斌. 基于感知增强Swin Transformer的遥感图像检测[J]. 计算机工程, 2024, 50(1): 216-223.

Bingyan ZHU, Zhihua CHEN, Bin SHENG. Remote Sensing Image Detection Based on Perceptually Enhanced Swin Transformer[J]. Computer Engineering, 2024, 50(1): 216-223.

https://www.ecice06.com/CN/Y2024/V50/I1/216

图/表 12

图1 基于感知增强Swin Transformer的遥感图像检测网络框架

Fig.1 Framework of remote sensing image detection network based on the perceptually enhanced Swin Transformer

图2 Swin Transformer网络结构

Fig.2 Network structure of the Swin Transformer

图3 Swin Transformer块结构

Fig.3 Block structure of the Swin Transformer

图4 局部增强Swin Transformer主干网络结构

Fig.4 Structure of local enhanced Swin Transformer backbone

图5 空间局部感知模块结构

Fig.5 Structure of spatial local perceptually module

图6 使用不同主干网络的检测结果可视化

Fig.6 Visualization of detection results by using different backbones

图7 DOTA数据集上的检测结果

Fig.7 Detection results on the DOTA dataset

图8 Swin-T和PESTN各项检测性能比较

Fig.8 Comparison of the detection performance of Swin-T and PESTN

图9 Swin-T和PESTN的ROC曲线比较

Fig.9 Comparison of the ROC curves of Swin-T and PESTN

参考文献 30

1	YANG X E, YANG J R, YAN J C, et al. SCRDet: towards more robust detection for small, cluttered and rotated objects[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 8232-8241.
2	YANG X E, YAN J C, LIAO W L, et al. SCRDet++: detecting small, cluttered and rotated objects via instance-level feature denoising and rotation loss smoothing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(2): 2384- 2399. doi: 10.1109/TPAMI.2022.3166956
3	DING J A, XUE N, LONG Y, et al. Learning RoI Transformer for oriented object detection in aerial images[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 2849-2858.
4	GUAN H Y, YU Y T, LI D L, et al. RoadCapsFPN: capsule feature pyramid network for road extraction from VHR optical remote sensing imagery. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(8): 11041- 11051. doi: 10.1109/TITS.2021.3098855
5	谢星星, 程塨, 姚艳清, 等. 动态特征融合的遥感图像目标检测. 计算机学报, 2022, 45(4): 735- 747.
	XIE X X, CHENG G, YAO Y Q, et al. Dynamic feature fusion for object detection in remote sensing images. Chinese Journal of Computers, 2022, 45(4): 735- 747.
6	王道累, 杜文斌, 刘易腾, 等. 基于密集连接与特征增强的遥感图像检测. 计算机工程, 2022, 48(6): 251-256, 262. doi: 10.19678/j.issn.1000-3428.0061482
	WANG D L, DU W B, LIU Y T, et al. Remote sensing images detection based on dense connection and feature enhancement. Computer Engineering, 2022, 48(6): 251-256, 262. doi: 10.19678/j.issn.1000-3428.0061482
7	ZHANG C, LAM K M, WANG Q. CoF-Net: a progressive coarse-to-fine framework for object detection in remote-sensing imagery. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61, 1- 17.
8	ZHU X Z, SU W J, LU L W, et al. Deformable DETR: deformable Transformers for end-to-end object detection[EB/OL]. [2023-01-11]. https://arxiv.org/abs/2010.04159.pdf.
9	LI W T, CHEN Y J, HU K X, et al. Oriented RepPoints for aerial object detection[C]//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2022: 1829-1838.
10	CHEN H, QI Z P, SHI Z W. Remote sensing image change detection with Transformers. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60, 1- 14.
11	LIU X L, MA S P, HE L Y, et al. Hybrid network model: TransConvNet for oriented object detection in remote sensing images. Remote Sensing, 2022, 14(9): 2090. doi: 10.3390/rs14092090
12	DAI L H, LIU H, TANG H, et al. AO2-DETR: arbitrary-oriented object detection Transformer[EB/OL]. [2023-01-11]. https://arxiv.org/abs/2205.12785.pdf.
13	DAI Y N, YU J Y, ZHANG D A, et al. RODFormer: high-precision design for rotating object detection with Transformers. Sensors, 2022, 22(7): 2633. doi: 10.3390/s22072633
14	LI Q Y, CHEN Y S, ZENG Y. Transformer with transfer CNN for remote-sensing-image object detection. Remote Sensing, 2022, 14(4): 984. doi: 10.3390/rs14040984
15	ZHENG Y B, SUN P, ZHOU Z T, et al. ADT-det: adaptive dynamic refined single-stage Transformer detector for arbitrary-oriented object detection in satellite optical imagery. Remote Sensing, 2021, 13(13): 2623. doi: 10.3390/rs13132623
16	LIU Z, LIN Y T, CAO Y E, et al. Swin Transformer: hierarchical vision Transformer using shifted windows[C]//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2021: 10012-10022.
17	YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[EB/OL]. [2023-01-11]. https://arxiv.org/abs/1511.07122.pdf.
18	ZHANG H Y, WANG Y, DAYOUB F, et al. VarifocalNet: an IoU-aware dense object detector[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 8514-8523.
19	HOU L P, LU K, XUE J A. Refined one-stage oriented object detection method for remote sensing images. IEEE Transactions on Image Processing, 2022, 31, 1545- 1558. doi: 10.1109/TIP.2022.3143690
20	ZHENG W, TANG W L, CHEN S J, et al. CIA-SSD: confident IoU-aware single-stage object detector from point cloud[C]//Proceedings of AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2021: 3555-3562.
21	XIA G S, BAI X A, DING J A, et al. DOTA: a large-scale dataset for object detection in aerial images[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 3974-3983.
22	PADILLA R, NETTO S L, DA SILVA E A B. A survey on performance metrics for object-detection algorithms[C]//Proceedings of 2020 International Conference on Systems, Signals and Image Processing. Washington D. C., USA: IEEE Press, 2020: 237-242.
23	CHICCO D, JURMAN G. The advantages of the Matthews Correlation Coefficient(MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics, 2020, 21(1): 6. doi: 10.1186/s12864-019-6413-7
24	TOUVRON H, CORD M, DOUZE M, et al. Training data-efficient image Transformers & distillation through attention[EB/OL]. [2023-01-11]. https://arxiv.org/abs/2012.12877.pdf.
25	HE K M, GKIOXARI G, DOLLAR P, et al. Mask R-CNN[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 2980-2988.
26	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137- 1149. doi: 10.1109/TPAMI.2016.2577031
27	YANG X E, YAN J C, FENG Z M, et al. R3Det: refined single-stage detector with feature refinement for rotating object[C]//Proceedings of AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2021: 3163-3171.
28	XU Y C, FU M T, WANG Q M, et al. Gliding vertex on the horizontal bounding box for multi-oriented object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(4): 1452- 1459. doi: 10.1109/TPAMI.2020.2974745
29	ZHANG H, LI F, LIU S L, et al. DINO: DETR with improved denoising anchor boxes for end-to-end object detection[EB/OL]. [2023-01-11]. https://arxiv.org/abs/2203.03605.pdf.
30	WANG J W, DING J A, GUO H W, et al. Mask OBB: a semantic attention-based mask oriented bounding box representation for multi-category object detection in aerial images. Remote Sensing, 2019, 11(24): 2930. doi: 10.3390/rs11242930

[1]	牛瑞婷, 严天峰, 高锐, 王映植. 低信噪比下基于深度学习TCNN-MobileNet的调制识别[J]. 计算机工程, 2024, 50(7): 204-215.
[2]	逯焕宇, 张永宏, 马光义, 谢东林, 田伟. 基于半监督对抗学习的遥感图像水体提取[J]. 计算机工程, 2024, 50(7): 251-263.
[3]	肖慈, 徐杨, 张永丹, 冯明文, 黄易仟. 结合注意力和低光增强的夜间语义分割[J]. 计算机工程, 2024, 50(7): 271-281.
[4]	张诗婧, 莫绪涛, 赵行, 董杨林. 基于球面折反射成像和YOLOv7的内螺纹缺陷检测[J]. 计算机工程, 2024, 50(7): 282-292.
[5]	贵向泉, 刘世清, 李立, 秦庆松, 李唐艳. 基于改进YOLOv8的景区行人检测算法[J]. 计算机工程, 2024, 50(7): 342-351.
[6]	徐明亮, 李芳媛, 马浩然, 何飞. 大规模神经记录的峰电位聚类算法(特邀)[J]. 计算机工程, 2024, 50(6): 1-34.
[7]	魏琢艺, 罗迈, 李文兵, 曾远松, 余伟江, 杨跃东. 基于多源域适应的单细胞智能分类[J]. 计算机工程, 2024, 50(6): 48-55.
[8]	李子杰, 周菊香, 韩晓瑜, 甘健侯, 鹿泽光, 王俊. 序列特征与学习过程融合的知识追踪模型[J]. 计算机工程, 2024, 50(6): 77-85.
[9]	更藏措毛, 黄鹤鸣, 杨毅杰. 融合多尺度特征与上下文信息的语音增强方法[J]. 计算机工程, 2024, 50(6): 138-147.
[10]	李永飞, 李铭洋, 常鑫, 曹可欣. 基于可解释性深度学习的物联网水质监测数据异常检测[J]. 计算机工程, 2024, 50(6): 179-187.
[11]	于洋, 孙芳芳, 吕华, 李扬, 王晓民. 基于多尺度时空注意力网络的微表情检测方法[J]. 计算机工程, 2024, 50(6): 228-235.
[12]	杨硕, 王一丁. 基于改进薄板样条运动模型的人脸动画算法[J]. 计算机工程, 2024, 50(6): 255-265.
[13]	李致金, 汤佳辉, 闫金凤. 基于边缘计算的轻量化识别方法[J]. 计算机工程, 2024, 50(6): 287-295.
[14]	李雪, 王雅文, 张前进. 基于信息检索的源代码自动命名[J]. 计算机工程, 2024, 50(6): 304-310.
[15]	刘仕兵, 周诗涵. 高铁接触网绝缘子检测算法研究[J]. 计算机工程, 2024, 50(5): 200-208.

选择文件类型/文献管理软件名称

选择包含的内容