Remote Sensing Image Retrieval Based on Class Center Optimization Added Triplet Loss

doi:10.19678/j.issn.1000-3428.0068894

Abstract

Abstract:

The key to remote sensing image retrieval is to efficiently and accurately retrieve target samples from massive images. Intraclass samples in remote sensing images are dispersed and exhibit large variance. Traditional remote sensing image retrieval based on limited samples cannot effectively learn the differences between intraclass samples. The existing Cross-Batch Memory (XBM) method has triplet pairing redundancy and complex computations. A remote sensing image retrieval method based on Class Center Optimization added for Triplet Loss (CCO-TL) is proposed to address these problems. CCO-TL uses class center features to limit the distance between positive samples within a class, assisting in optimizing the triplet loss and achieving interclass separation. Simultaneously, samples within a class are clustered and compacted, generating optimized sample features. By improving the XBM module, a Batch Feature Queue (BFQ) is obtained to store the feature vectors of previous training batches, and by changing the triplet pairing method, sample information is mined fully, data redundancy problems are solved, and the training time is reduced. Simultaneously, the BFQ module is used for the real-time calculation of class center point features, replacing the estimated values of traditional methods with calculated values. Experimental results show that the network model trained with the triplet loss function based on real class center feature assisted optimization has a stronger learning ability between samples, more intraclass clustering, and more obvious interclass differentiation. The proposed method is evaluated in terms of the Recall@K metric on four remote sensing datasets. The proposed method achieves accuracies of 93.1%, 87.2%, 97.1%, and 82.2%, on the UCMD, AID, PN, and OP datasets, respectively, outperforming other methods.

Key words: image retrieval, deep metric learning, triplet loss, class center, batch

摘要：

遥感图像检索的关键是从海量图像中高效、准确地检索出目标样本。遥感图像类内样本分散、方差大, 依靠有限样本的传统遥感图像检索不能很好地学习类内样本差异特征, 现有的跨批处理内存(XBM)方法的三元组配对冗余、计算复杂。针对这些问题, 提出一种基于类中心优化辅助的三元组损失(CCO-TL)的遥感图像检索方法。CCO-TL使用类中心特征限制类内正样本之间的距离以辅助优化三元组损失, 实现类间相互分离, 同时类内的样本更加聚集紧凑, 得到优化的样本特征; 通过改进XBM模块得到批次特征队列(BFQ), 用于存储先前训练批次的特征向量, 通过改变三元组配对方式, 充分挖掘样本信息并解决数据冗余问题, 减少训练时间。同时使用BFQ模块进行类中心点特征的实时计算, 用计算值取代传统方法的估计值。实验结果表明, 基于真实类中心特征辅助优化的三元组损失函数训练的网络模型学习样本间的能力更强, 类内更加聚集, 类间区分也更明显。最后结合Recall@K等指标进行评估, 在UCMD、AID、PN、OP 4个遥感数据集上进行实验, 所提算法的精度分别达到93.1%、87.2%、97.1%、82.2%, 优于其他研究方法。

关键词: 图像检索, 深度度量学习, 三元组损失, 类中心, 批次

ZHENG Zongsheng, HUO Zhijun, GAO Meng, WANG Zhenghan, ZHOU Wenhuan, ZHANG Yuewei. Remote Sensing Image Retrieval Based on Class Center Optimization Added Triplet Loss[J]. Computer Engineering, 2025, 51(5): 305-313.

郑宗生, 霍志俊, 高萌, 王政翰, 周文睆, 张月维. 基于类中心优化辅助三元组损失的遥感图像检索[J]. 计算机工程, 2025, 51(5): 305-313.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0068894

https://www.ecice06.com/EN/Y2025/V51/I5/305

Figures/Tables 11

Fig.1 CCO-TL method flow chart

Fig.2 BFQ module

Fig.3 Examples of a query obtained by CCO-TL

Fig.4 2D clustering grayscale plots of feature t-SNE

References 29

1	冯孝鑫, 王子健, 吴奇. 基于三元采样图卷积网络的半监督遥感图像检索. 电子与信息学报, 2023, 45 (2): 644- 653.
	FENG X X , WANG Z J , WU Q . Semi-supervised learning remote sensing image retrieval method based on triplet sampling graph convolutional network. Journal of Electronics & Information Technology, 2023, 45 (2): 644- 653.
2	WANG Y, ALBRECHT C M, BRAHAM N A A, et al. Self-supervised learning in remote sensing: a review[EB/OL]. (2022-09-02)[2023-10-22]. https://arxiv.org/pdf/2206.13188.
3	XIAO Y , YUAN Q , JIANG K , et al. From degrade to upgrade: learning a self-supervised degradation guided adaptive network for blind remote sensing image super-resolution. Information Fusion, 2023, 96, 297- 311. doi: 10.1016/j.inffus.2023.03.021
4	STOJNIC V, RISOJEVIC V. Self-supervised learning of remote sensing scene representations using contrastive multiview coding[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 1182-1191.
5	CHEN X, PAN J, JIANG K, et al. Unpaired deep image deraining using dual contrastive learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2022: 2017-2026.
6	CHENG Q , GAN D , FU P , et al. A novel ensemble architecture of residual attention-based deep metric learning for remote sensing image retrieval. Remote Sensing, 2021, 13 (17): 3445. doi: 10.3390/rs13173445
7	ZHANG B , CHEN Z , PENG D , et al. Remotely sensed big data: evolution in model development for information extraction[point of view]. Proceedings of the IEEE, 2019, 107 (12): 2294- 2301. doi: 10.1109/JPROC.2019.2948454
8	XIAO Y , YUAN Q , JIANG K , et al. TTST: a top-k token selective transformer for remote sensing image super-resolution. IEEE Transactions on Image Processing, 2024, 33, 738- 752. doi: 10.1109/TIP.2023.3349004
9	梁天佑, 孟敏, 武继刚. 基于特征融合的无监督跨模态哈希. 计算机工程, 2023, 49 (2): 90- 97. doi: 10.19678/j.issn.1000-3428.0063841
	LIANG T Y , MENG M , WU J G . Unsupervised cross-modal hashing based on feature fusion. Computer Engineering, 2023, 49 (2): 90- 97. doi: 10.19678/j.issn.1000-3428.0063841
10	彭晏飞, 梅金业, 王恺欣, 等. 基于区域注意力机制的遥感图像检索. 激光与光电子学进展, 2020, 57 (10): 101017.
	PENG Y F , MEI J Y , WANG K X , et al. Remote sensing image retrieval based on regional attention mechanism. Laser & Optoelectronics Progress, 2020, 57 (10): 101017.
11	黄娜, 何泾沙. 基于深度特征与局部特征融合的图像检索. 北京工业大学学报, 2020, 46 (12): 1345- 1354. doi: 10.11936/bjutxb2019070005
	HUANG N , HE J S . Image retrieval based on fusion of deep feature and local feature. Journal of Beijing University of Technology, 2020, 46 (12): 1345- 1354. doi: 10.11936/bjutxb2019070005
12	吴刚, 葛芸, 储珺, 等. 面向遥感图像检索的级联池化自注意力研究. 光电工程, 2022, 49 (12): 220029.
	WU G , GE Y , CHU J , et al. Cascade pooling self-attention research for remote sensing image retrieval. Opto-Electronic Engineering, 2022, 49 (12): 220029.
13	金柱璋, 方旭源, 黄彦慧, 等. 基于深度度量学习的卫星云图检索. 光电工程, 2022, 49 (4): 210307.
	JIN Z Z , FANG X Y , HUANG Y H , et al. Satellite cloud image retrieval based on deep metric learning. Opto-Electronic Engineering, 2022, 49 (4): 210307.
14	WEN Y, ZHANG K, LI Z, et al. A discriminative feature learning approach for deep face recognition[C]//Proceedings of Computer Vision-ECCV 2016: 14th European Conference. Berlin, Germany: Springer, 2016: 499-515.
15	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[EB/OL]. (2023-08-02)[2023-10-22]. https://arxiv.org/abs/1706.03762.
16	HADSELL R, CHOPRA S, LECUN Y. Dimensionality reduction by learning an invariant mapping[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2006: 1735-1742.
17	SCHROFF F, KALENICHENKO D, PHILBIN J. Facenet: a unified embedding for face recognition and clustering[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2015: 815-823.
18	SOHN K. Improved deep metric learning with multi-class N-pair loss objective[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. New York, USA: Curran Associates Inc., 2016: 1857-1865.
19	WANG X, ZHANG H, HUANG W, et al. Cross-batch memory for embedding learning[EB/OL]. (2020-04-21)[2023-10-22]. https://arxiv.org/abs/1912.06798.
20	LIU C, YU H, LI B, et al. Noise-resistant deep metric learning with ranking-based instance selection[EB/OL]. (2021-04-12)[2023-10-22]. https://arxiv.org/abs/2103.16047.
21	LI X , WEI S , WANG J , et al. Adaptive multi-proxy for remote sensing image retrieval. Remote Sensing, 2022, 14 (21): 5615. doi: 10.3390/rs14215615
22	GU G, KO B. Symmetrical synthesis for deep metric learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI, 2020: 10853-10860.
23	WANG X, HAN X, HUANG W, et al. Multi-similarity loss with general pair weighting for deep metric learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 5022-5030.
24	QIAN Q, SHANG L, SUN B, et al. Softtriple loss: deep metric learning without triplet sampling[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 6450-6458.
25	MUSGRAVE K, BELONGIE S, LIM S N. A metric learning reality check[EB/OL]. (2020-09-16)[2023-10-22]. https://arxiv.org/abs/2003.08505.
26	YANG Y, NEWSAM S. Bag-of-visual-words and spatial extensions for land-use classification[C]//Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems. New York, USA: ACM, 2010: 270-279.
27	XIA G S , HU J , HU F , et al. AID: a benchmark data set for performance evaluation of aerial scene classification. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55 (7): 3965- 3981. doi: 10.1109/TGRS.2017.2685945
28	ZHOU W , NEWSAM S , LI C , et al. PatternNet: a benchmark dataset for performance evaluation of remote sensing image retrieval. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 145, 197- 209. doi: 10.1016/j.isprsjprs.2018.01.004
29	WANG Q , LIU S , CHANUSSOT J , et al. Scene classification with recurrent attention of VHR remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 2018, 57 (2): 1155- 1167.

[1]	ZHENG Yazhou, LIU Wanping, HUANG Dong. A BERT-CNN-GRU Detection Method Based on Attention Mechanism [J]. Computer Engineering, 2025, 51(1): 258-268.
[2]	DONG Feng, WANG Yongxin, MA Yuling, WANG Kuikui. Prototype-Aligned and Domain-Aware Zero-Shot Hashing [J]. Computer Engineering, 2024, 50(5): 260-271.
[3]	WU Zhengjiang, LÜ Chenggong, WANG Mengsong. Calculation Method for Semi-Monolayer Covering Approximation Sets Fushing GPU [J]. Computer Engineering, 2024, 50(5): 71-82.
[4]	GUO Rui, HU Guoliang, WANG Junming. Anonymous Certificateless Aggregate Signature Scheme in VANETs [J]. Computer Engineering, 2024, 50(11): 207-222.
[5]	Qianglong LI, Xinwen ZHOU, Meng'en WEI, Yangzhou GAN. Infrared Target Detection Algorithm Based on Strip Pooling and Attention Mechanism in Street Scene [J]. Computer Engineering, 2023, 49(8): 310-320.
[6]	WANG Yicheng, GUO Rui, MENG Tong, LIU Yingfei. Privacy Protection Scheme Based on Proxy Blind Signcryption in Smart Grid [J]. Computer Engineering, 2023, 49(5): 150-164.
[7]	HE Yue, CHEN Guangsheng, JING Weipeng, XU Zekun. Remote Sensing Image Retrieval Based on Deep Multi-Similarity Hashing Method [J]. Computer Engineering, 2023, 49(2): 206-212.
[8]	HAO Axiang, JIA Guojun. Person Re-identification Model Combining Attention and Batch Feature Erasure [J]. Computer Engineering, 2022, 48(7): 270-276,306.
[9]	PENG Hongyan, LI Jie, SHI Zhenkui, LI Xianxian. A Blockchain-based Verifiable Encrypted Image Retrieval Scheme [J]. Computer Engineering, 2022, 48(2): 25-33,39.
[10]	WANG Qingrong, WEI Yimeng, ZHU Changfeng, TIAN Keke. Research on Traffic Accident Risk Prediction Based on Spatio-Temporal Graph Convolutional Network [J]. Computer Engineering, 2022, 48(11): 22-29.
[11]	WU Tiantian, YANG Yafang, ZHAO Yunlei. An Authentication Protocol with Conditional Privacy Protection for IoV Communication [J]. Computer Engineering, 2021, 47(6): 14-22.
[12]	LUO Shuo, HOU Jin, TAN Guanghong, HAN Yanpeng. A Low Parameter Real-Time Target Tracking Algorithm Based on Siamese Convolutional Network [J]. Computer Engineering, 2021, 47(2): 84-89.
[13]	ZHU Song, WANG Huaqun. Paillier-Based Data Aggregation and Stimulation Scheme in the Smart Grid [J]. Computer Engineering, 2021, 47(11): 166-174.
[14]	CAO Yukun, WEI Jianqiang, SUN Tao, XU Yue. Deep Image Description Model Based on IndRNN and BN [J]. Computer Engineering, 2021, 47(10): 194-200.
[15]	GU Yan, ZHAO Chongyu, HUANG Ping. Deep Hash Learning Model Based on High-Order Statistical Information [J]. Computer Engineering, 2020, 46(7): 260-267,276.

Please choose a citation manager

Content to export