融合超分辨率和特征增强的轻量化遥感图像小目标检测

doi:10.19678/j.issn.1000-3428.0068647

摘要/Abstract

摘要：

为了应对遥感图像目标检测中小目标像素低、背景复杂、硬件资源有限等问题, 提出一种融合超分辨率(SR)和特征增强的小目标检测模型。采用GhostNet网络中的Ghost卷积层替换YOLOv8网络中的传统卷积层Conv, 在不影响检测精度的情况下降低网络模型的参数量和计算量。在主干网络中, 构建超分辨率辅助增强(SRAE)模块提升图像的分辨率和特征提取能力。利用三层特征融合(TFF)模块, 获取主干网络较低层的空间特征, 改善快速空间金字塔池化(SPPF)层特征空间提取不足的问题, 提高小目标空间定位能力。设计自注意力信息转移(SAT)模块, 在保证模型轻量化的同时增强小目标的语义信息和全局信息。实验结果表明, 改进模型在DIOR数据集上实现了90.5%的mAP@0.5、15.1×10⁶的参数量和30.3×10⁹的每秒浮点运算次数(FLOPs), 相比于其他模型在实现网络轻量化的同时提升了小目标检测精度。

关键词: 目标检测, 超分辨率, 遥感图像, YOLOv8网络, 注意力机制, 特征融合

Abstract:

To address the problems of low pixel size, complex background, and limited hardware resources in remote sensing image object detection, a small-object detection algorithm that combines Super-Resolution(SR) and feature enhancement is proposed. The Ghost convolution layers in the GhostNet are used to replace the conventional convolution layers, Conv, in the You Only Look Once v8 (YOLOv8) network, reducing the number of parameters and calculations of the network model without compromising detection accuracy. A Super-Resolution Assisted Enhancement (SRAE) is built in the backbone network to improve image resolution and feature extraction capabilities. A Three-layer Feature Fusion (TFF) module is proposed to obtain the spatial features of the lower layer of the backbone network, improve the insufficient feature space extraction in the Spatial Pyramid Pooling Fast(SPPF) layer, and enhance the spatial positioning ability of small targets. A Self-Attention information Transfer (SAT) module is designed to enhance the semantic and global information of small targets while ensuring a lightweight model. The improved model achieves 90.5% Mean Average Precision (mAP)@0.5, 15.1×10⁶ parameter quantity, and 30.3×10⁹ Floating Point Operations Per Second (FLOPs) on the DIOR dataset; additionally, it achieves lightweight while improving detection accuracy compared to other models.

Key words: object detection, Super-Resolution (SR), remote sensing image, YOLOv8 network, attention mechanism, feature fusion

杨雨迪, 葛海波, 辛世澳, 薛紫涵, 袁昊. 融合超分辨率和特征增强的轻量化遥感图像小目标检测[J]. 计算机工程, 2024, 50(11): 284-296.

YANG Yudi, GE Haibo, XIN Shiao, XUE Zihan, YUAN Hao. Lightweight Small-Object Detection for Remote Sensing Images Integrating Super-Resolution and Feature Enhancement[J]. Computer Engineering, 2024, 50(11): 284-296.

https://www.ecice06.com/CN/Y2024/V50/I11/284

图/表 14

图1 改进后的YOLOv8结构

Fig.1 Structure of improved YOLOv8

图2 传统卷积层

Fig.2 Traditional convolutional layer

图3 Ghost模块

Fig.3 Ghost module

图4 SRAE模块

Fig.4 SRAE module

图5 TFF模块

Fig.5 TFF module

图6 SAT模块

Fig.6 SAT module

图7 精确率-召回率曲线

Fig.7 Precision-Recall curves

图8 混淆矩阵

Fig.8 Confusion matrix

图9 DIOR数据集上不同模型的目标检测结果比较

Fig.9 Comparison of object detection results using different models on the DIOR datasets

参考文献 32

1	ZHENG X T, WANG B Q, DU X Q, et al. Mutual attention inception network for remote sensing visual question answering. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60, 5606514. doi: 10.1109/TGRS.2021.3079918
2	霍光, 林大为, 刘元宁, 等. 基于轻量级卷积神经网络的小样本虹膜图像分割. 吉林大学学报(理学版), 2023, 61 (3): 583- 591. URL
	HUO G, LIN D W, LIU Y N, et al. Small-sample iris image segmentation based on lightweight convolutional neural networks. Journal of Jilin University (Science Edition), 2023, 61 (3): 583- 591. URL
3	BONANNELLA C, CHIRICI G, TRAVAGLINI D, et al. Characterization of wildfires and harvesting forest disturbances and recovery using Landsat time series: a case study in mediterranean forests in central Italy. Fire, 2022, 5 (3): 68. doi: 10.3390/fire5030068
4	WU D, SONG H B, FAN C Z. Object tracking in satellite videos based on improved kernel correlation filter assisted by road information. Remote Sensing, 2022, 14 (17): 4215. doi: 10.3390/rs14174215
5	崔丽群, 曹华维. 基于改进YOLOv5的遥感图像目标检测. 计算机工程, 2024, 50 (4): 228- 236. URL
	CUI L Q, CAO H W. Target detection of remote-sensing images based on improved YOLOv5. Computer Engineering, 2024, 50 (4): 228- 236. URL
6	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2016: 779-788.
7	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector[M]. Berlin, Germany: Springer International Publishing, 2016: 21-37.
8	CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with Transformers[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2020: 213-229.
9	GUO G M, FANG L Y, YUE J. Oriented spatial correlative aligned feature for remote sensing object detection[C]//Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Washington D. C., USA: IEEE Press, 2021: 5319-5322.
10	MA W P, LI N, ZHU H, et al. Feature split-merge-enhancement network for remote sensing object detection. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60, 5616217.
11	GONG H, MU T K, LI Q X, et al. Swin-Transformer-enabled YOLOv5 with attention mechanism for small object detection on satellite images. Remote Sensing, 2022, 14 (12): 2861. doi: 10.3390/rs14122861
12	LIU K, HUANG J, LI X L. Eagle-eye-inspired attention for object detection in remote sensing. Remote Sensing, 2022, 14 (7): 1743. doi: 10.3390/rs14071743
13	LI P, CHE C. SeMo-YOLO: a multiscale object detection network in satellite remote sensing images[C]//Proceedings of the International Joint Conference on Neural Networks (IJCNN). Washington D. C., USA: IEEE Press, 2021: 1-8.
14	LIU Z G, GAO Y, DU Q Q, et al. YOLO-extract: improved YOLOv5 for aircraft object detection in remote sensing images. IEEE Access, 2023, 11, 1742- 1751. doi: 10.1109/ACCESS.2023.3233964
15	WANG C Y, BOCHKOVSKIY A, LIAO H Y M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2023: 7464-7475.
16	REDMON J, FARHADI A. YOLOv3: an incremental improvement[EB/OL]. [2023-09-18]. http://arxiv.org/abs/1804.02767.
17	LIU S, QI L, QIN H F, et al. Path aggregation network for instance segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 8759-8768.
18	LI X, WANG W H, WU L J, et al. Generalized focal loss: learning qualified and distributed bounding boxes for dense object detection[EB/OL]. [2023-09-18]. http://arxiv.org/abs/2006.04388.
19	ZHENG Z H, WANG P, LIU W, et al. Distance-IoU loss: faster and better learning for bounding box regression[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2020: 12993-13000.
20	WANG P J, BAYRAM B, SERTEL E. A comprehensive review on deep learning based remote sensing image super-resolution methods. Earth Science Reviews, 2022, 232, 104110. doi: 10.1016/j.earscirev.2022.104110
21	RAZZAK M T, MATEO-GARCÍA G, LECUYER G, et al. Multi-spectral multi-image super-resolution of Sentinel-2 with radiometric consistency losses and its effect on building delineation. ISPRS Journal of Photogrammetry and Remote Sensing, 2023, 195, 1- 13. doi: 10.1016/j.isprsjprs.2022.10.019
22	端木春江, 左德遥. 锚点领域回归与稀疏表示的图像超分辨率方法. 计算机工程, 2019, 45 (5): 194- 198. URL
	DUANMU C J, ZUO D Y. Image super-resolution method via anchored neighborhood regression and sparse representation. Computer Engineering, 2019, 45 (5): 194- 198. URL
23	MNIH V, HEESS N, GRAVES A, et al. Recurrent models of visual attention[EB/OL]. [2023-09-18]. http://arxiv.org/abs/1406.6247.
24	祝冰艳, 陈志华, 盛斌. 基于感知增强Swin Transformer的遥感图像检测. 计算机工程, 2024, 50 (1): 216- 223. URL
	ZHU B Y, CHEN Z H, SHENG B. Remote sensing image detection based on perceptually enhanced Swin Transformer. Computer Engineering, 2024, 50 (1): 216- 223. URL
25	HAN K, WANG Y H, TIAN Q, et al. GhostNet: more features from cheap operations[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2020: 1577-1586.
26	HOWARD A G, ZHU M L, CHEN B. MobileNets: effificient convolutional neural networks for mobile vision applications [EB/OL]. [2023-09-18]. https://arxiv.org/abs/1704.04861.
27	ZHANG X Y, ZHOU X Y, LIN M X, et al. ShuffleNet: an extremely efficient convolutional neural network for mobile devices[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 6848-6856.
28	HAN Q, FAN Z J, DAI Q, et al. On the connection between local attention and dynamic depth-wise convolution[EB/OL]. [2023-09-18]. http://arxiv.org/abs/2106.04263.
29	ZHOU Y P, LI Z, GUO C L, et al. SRFormerV2: taking a closer look at permuted self-attention for image super-resolution[EB/OL]. [2023-09-18]. http://arxiv.org/abs/2303.09735.
30	XIA G S, BAI X, DING J, et al. DOTA: a large-scale dataset for object detection in aerial images[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 3974-3983.
31	CHENG G, ZHOU P C, HAN J W. Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54 (12): 7405- 7415. doi: 10.1109/TGRS.2016.2601622
32	LI K, WAN G, CHENG G, et al. Object detection in optical remote sensing images: a survey and a new benchmark. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 159, 296- 307. doi: 10.1016/j.isprsjprs.2019.11.023

[1]	朱凯, 李理, 张彤, 江晟, 别一鸣. 基于Transformer的多阶段运动模糊图像修复网络[J]. 计算机工程, 2024, 50(9): 276-285.
[2]	李大海, 吕春桂, 王振东. 基于双分支序列残差注意力的场景文本图像超分辨率重建[J]. 计算机工程, 2024, 50(9): 286-295.
[3]	张天鹏, 韩晶, 吕学强. 基于多任务学习的超分辨率辅助小目标检测[J]. 计算机工程, 2024, 50(9): 304-312.
[4]	郭敏, 张熙涵, 李阳. 融合注意力的教师互一致性半监督医学图像分割[J]. 计算机工程, 2024, 50(9): 313-323.
[5]	曾钰琦, 刘博, 钟柏昌, 钟瑾. 智慧教育下基于改进YOLOv8的学生课堂行为检测算法[J]. 计算机工程, 2024, 50(9): 344-355.
[6]	王志浩, 钱沄涛. 基于Swin Transformer的双流遥感图像时空融合超分辨率重建[J]. 计算机工程, 2024, 50(9): 33-45.
[7]	李俊俊, 董建刚, 李坤. 基于Kubernetes的集群节能策略研究[J]. 计算机工程, 2024, 50(9): 82-91.
[8]	林畅, 郭伟, 任哲聪, 金海波. 基于Transformer的目标跟踪与分割统一算法[J]. 计算机工程, 2024, 50(9): 130-141.
[9]	李泽霖, 吕兆峰, 陈富强, 李克. 基于多跳信息融合的实体对齐模型[J]. 计算机工程, 2024, 50(9): 142-152.
[10]	王汝英, 马嘉骏, 董建强, 刘万龙, 张海涛, 尹凯, 赵博超. 基于MTS-BiGRU-DMHSA的工业负荷预测方法[J]. 计算机工程, 2024, 50(9): 169-178.
[11]	李俊仪, 李向阳, 龙朝勋, 李海燕, 李红松, 余鹏飞. 基于多级区域选择与跨层特征融合的野生菌分类[J]. 计算机工程, 2024, 50(9): 179-188.
[12]	陈宇航, 杨勇, 先木斯亚·买买提明, 帕力旦·吐尔逊, 樊小超, 任鸽, 刁宇峰. 基于主题感知和语义增强的作文自动评分方法[J]. 计算机工程, 2024, 50(8): 363-371.
[13]	王宇, 祁琦, 王纯, 许才. 储能变流器信号高精度故障诊断方法[J]. 计算机工程, 2024, 50(8): 389-396.
[14]	张华青, 夏张涛, 陆晓庆, 童基均. 基于字形特征的血管外科命名实体识别[J]. 计算机工程, 2024, 50(8): 13-21.
[15]	饶日昕, 王怡文, 曾砺志, 童心恬, 赵海涛. 面向废旧电缆检测的轻量化网络模型[J]. 计算机工程, 2024, 50(8): 22-30.

选择文件类型/文献管理软件名称

选择包含的内容