End-to-End Person Search Method Based on Prototype Separation Network

doi:10.19678/j.issn.1000-3428.0068462

Abstract

Abstract:

Person search aims to detect and identify target pedestrians in panoramic images and can be viewed as a combination of target detection and person re-identification tasks. However, the similarity in attire among different pedestrians and the appearance variations of the same person in diverse environmental conditions contribute to the increased difficulty of pedestrian identity recognition. To address this issue, a prototype separation network is proposed to enhance the network's discrimination ability by adjusting the distribution of prototypes. First, a prototype enhancement module is designed to guide the learning of the attention network through prototype features and retain more important pedestrian features using the K-max pooling method. This allows the network to focus on important local regions and learn finer-grained pedestrian features with the guidance of the prototypes, which in turn improves the network's ability to discriminate between similar pedestrians. Second, an adaptive updating prototype learning strategy is proposed to ensure that the detection of accurate candidate boxes makes a greater contribution when the prototype features are updated. Finally, distributed sparse loss ensures that the stored prototypes are well dispersed, thus enabling the network to recognize the distinctive features of pedestrians. Finally, relevant experiments are conducted on the public pedestrian search datasets CUHK-SYSU and PRW, achieving mAP (mean average precision) values of 95.1% and 49.8% and Top-1 values of 95.9% and 88.5%, respectively. The proposed method effectively improves the accuracy of person search.

Key words: person search, person re-identification, prototype, adaptive updating, distributed sparse loss

摘要：

行人搜索旨在全景图像中对目标行人进行定位和识别, 可以看作目标检测和行人重识别任务的结合。然而, 不同行人的着装相似性和同一行人在多变环境条件下的外观差异, 增加了行人身份辨别的难度。为了解决这一问题, 提出了一个原型分散网络, 通过调整原型的分布情况, 增强网络的辨别能力。首先, 设计了一个原型增强模块, 通过原型特征指导注意力网络的学习并利用K最大池化方法保留更多重要的行人特征, 借助原型的指导使网络关注更重要的区域, 学习细粒度的行人特征, 进而提高网络对相似行人的辨别能力。其次, 提出一种自适应更新的原型学习策略, 以在原型特征更新时保留更多检测精准的候选框信息。最后, 通过分布稀疏损失保证所存储的原型尽可能分散, 从而确保网络能识别到行人的可区分性特征。最终在公共的行人搜索数据集CUHK-SYSU和PRW上进行了实验, 该方法在平均精度均值(mAP)上分别达到了95.1%和49.8%, 在Top-1准确率上分别达到了95.9%和88.5%, 有效地提高了行人搜索的准确率。

关键词: 行人搜索, 行人重识别, 原型, 自适应更新, 分布稀疏损失

ZHANG Wenxin, LIU Yujie, WANG Zhaoyong, SUN Haomiao, LI Zongmin. End-to-End Person Search Method Based on Prototype Separation Network[J]. Computer Engineering, 2025, 51(1): 269-276.

张雯欣, 刘玉杰, 王兆勇, 孙浩淼, 李宗民. 基于原型分散网络的端到端行人搜索方法[J]. 计算机工程, 2025, 51(1): 269-276.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0068462

https://www.ecice06.com/EN/Y2025/V51/I1/269

Figures/Tables 8

References 29

1	ISLAM K . Person search: new paradigm of person re-identification: a survey and outlook of recent works. Image and Vision Computing, 2020, 101, 103970. doi: 10.1016/j.imavis.2020.103970
2	REN S , HE K , GIRSHICK R , et al. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis[WT《Times New Roman》] & Machine Intelligence, 2017, 39 (6): 1137- 1149.
3	杨玉婷, 苗夺谦. 基于多粒度匹配的行人搜索算法. 智能系统学报, 2022, 17 (2): 420- 426.
	YANG Y T , MIAO D Q . Person search algorithm based on multi-granularity matching. CAAI Transactions on Intelligent Systems, 2022, 17 (2): 420- 426.
4	CHEN D, ZHANG S, YANG J, et al. Norm-aware embedding for efficient person search[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2020: 12615-12624.
5	XIAO T, LI S, WANG B, et al. Joint detection and identification feature learning for person search[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2017: 3415-3424.
6	LI Z, MIAO D. Sequential end-to-end network for efficient person search[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Washington D.C., USA: AAAI Press, 2021, 35(3): 2011-2019.
7	HANSELMANN H, NEY H. Elope: fine-grained visual classification with efficient localization, pooling and embedding[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Washington D.C., USA: IEEE Press, 2020: 1247-1256.
8	ZHENG L, ZHANG H, SUN S, et al. Person re-identification in the wild[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2017: 1367-1376.
9	CHEN D, ZHANG S, OUYANG W, et al. Person search via a mask-guided two-stream CNN model[C]//Proceedings of the European Conference on Computer Vision (ECCV). Berlin, Germany: Springer, 2018: 734-750.
10	DONG W, ZHANG Z, SONG C, et al. Instance guided proposal network for person search[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2020: 2585-2594.
11	LAN X, ZHU X, GONG S. Person search by multi-scale matching[C]//Proceedings of the European Conference on Computer Vision (ECCV). Berlin, Germany: Springer, 2018: 536-552.
12	WANG C, MA B, CHANG H, et al. TCTS: a task-consistent two-stage framework for person search[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2020: 11952-11961.
13	CHEN D, ZHANG S, OUYANG W, et al. Hierarchical online instance matching for person search[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Washington D.C., USA: AAAI Press, 2020: 10518-10525.
14	ZHANG Y , WANG S , KAN S , et al. End-to-end feature diversity person search with rank constraint of cross-class matrix. Neurocomputing, 2023, 518, 453- 465. doi: 10.1016/j.neucom.2022.10.080
15	KIM H, JOUNG S, KIM I J, et al. Prototype-guided saliency feature learning for person search[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2021: 4865-4874.
16	YAN Y, ZHANG Q, NI B, et al. Learning context graph for person search[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2019: 2158-2167.
17	HUANG W , JIA X , ZHONG X , et al. Beyond the parts: learning coarse-to-fine adaptive alignment representation for person search. ACM Transactions on Multimedia Computing, Communications and Applications, 2023, 19 (3): 105.
18	MUNJAL B, AMIN S, TOMBARI F, et al. Query-guided end-to-end person search[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2019: 811-820.
19	MUNJAL B , FLABOREA A , AMIN S , et al. Query-guided networks for few-shot fine-grained classification and person search. Pattern Recognition, 2023, 133, 109049. doi: 10.1016/j.patcog.2022.109049
20	YANG J , WANG X . Multi-attention-guided cascading network for end-to-end person search. Applied Sciences, 2023, 13 (9): 5576. doi: 10.3390/app13095576
21	YANG Y, ZHANG X, GENG Q, et al. Multi-task feature decomposition based marginal distribution for person search[C]//Proceedings of 2022 IEEE International Conference on Multimedia and Expo (ICME). Washington D.C., USA: IEEE Press, 2022: 1-6.
22	ZHANG P , YU X , BAI X , et al. Joint discriminative representation learning for end-to-end person search. Pattern Recognition, 2024, 147, 110053. doi: 10.1016/j.patcog.2023.110053
23	GUERRIERO S, CAPUTO B, MENSINK T. DeepNCM: deep nearest class mean classifiers[EB/OL]. (2018-02-12)[2023-09-26]. https://openreview.net/forum?id=rkPLZ4JPM.
24	杨燕燕, 谢明轩, 曹江峡, 等. 基于原型网络的中文分类模型对抗样本生成. 计算机工程, 2023, 49 (8): 54- 62. URL
	YANG Y Y , XIE M X , CAO J X , et al. Adversarial sample generation for Chinese classification model based on prototypical network. Computer Engineering, 2023, 49 (8): 54- 62. URL
25	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2016: 770-778.
26	HE K, GKIOXARI G, Dollár P, et al. Mask R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2017: 2961-2969.
27	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision (ECCV). Berlin, Germany: Springer, 2018: 3-19.
28	ZHENG Z, WANG P, LIU W, et al. Distance-IoU loss: faster and better learning for bounding box regression[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Washington, USA: AAAI Press, 2020: 12993-13000.
29	DENG J, DONG W, SOCHER R, et al. ImageNet: a large-scale hierarchical image database[C]//Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2009: 248-255.

[1]	HU Yongtao, HUANG Hongqiong. Multi-Branch Clothes-Changing Person Re-Identification with Feature Fusion and Channel Attention [J]. Computer Engineering, 2025, 51(1): 225-234.
[2]	HU Qing. Multi-Scale Fusion and Dual Output U-Net Network for Person Re-Identification [J]. Computer Engineering, 2024, 50(6): 102-109.
[3]	CAI Yixiang, QIN Pinle, ZENG Jianchao, JIN Zanxia, QIN Jia, ZHAI Shuangjiao. Research on Person Re-Identification Method for Large-Angle Viewpoint Differences [J]. Computer Engineering, 2024, 50(5): 330-341.
[4]	DONG Feng, WANG Yongxin, MA Yuling, WANG Kuikui. Prototype-Aligned and Domain-Aware Zero-Shot Hashing [J]. Computer Engineering, 2024, 50(5): 260-271.
[5]	Daoqing LIU, Hongchao HU, Shumin HUO. Research on Persistent Storage-Oriented Mimic Defense Technology in Container Clouds [J]. Computer Engineering, 2024, 50(2): 165-179.
[6]	Hua HAN, Li HUANG, Jin TIAN, Chunyuan WANG. Cross-Modality Person Re-identification Using Four-Stream Network Based on Dual-Intermediate Modalities [J]. Computer Engineering, 2023, 49(8): 302-309.
[7]	HAO Axiang, JIA Guojun. Person Re-identification Model Combining Attention and Batch Feature Erasure [J]. Computer Engineering, 2022, 48(7): 270-276,306.
[8]	WEI Ziwei, QU Dan, LIU Cong. Method for Person Re-identification Feature Extraction Based on Connected Attention [J]. Computer Engineering, 2022, 48(7): 220-226.
[9]	LIU Xiaorong, LI Xiaoxia, QIN Changhui. Person Re-Identification Method with Multi-Scale Contrast Pooling Feature [J]. Computer Engineering, 2022, 48(4): 292-298.
[10]	FU Jinwu, FAN Zizhu, SHI Linrui, GUO Xinyue, HUANG Yijing. Person Re-Identification Method Based on Multi-Scale and Multi-Granularity Fusion [J]. Computer Engineering, 2022, 48(3): 271-279.
[11]	ZHOU Dongming, ZHANG Canlong, TANG Yanping, LI Zhixin. Pedestrian Re-Identification Model Combining Semantic Segmentation and Attention Mechanism [J]. Computer Engineering, 2022, 48(2): 201-206.
[12]	ZENG Tao, XUE Feng, YANG Tian. Channel and Spatial Dual-Attention Network for Person Re-Identification [J]. Computer Engineering, 2022, 48(12): 281-287,295.
[13]	XU Zhiming, QI Yong. Person Re-Identification Based on UV Map for Optimizing Human Features [J]. Computer Engineering, 2022, 48(11): 83-88,95.
[14]	WANG Fuyin, HAN Hua, HUANG Li, CHEN Yiping. Unsupervised Video Person Re-identification with Complementary Temporal Features [J]. Computer Engineering, 2022, 48(10): 313-320.
[15]	LI Jiabin, LI Xuewei, LIU Hongzhe, XU Cheng. Person Re-Identification Based on Local Feature Relation and Global Attention Mechanism [J]. Computer Engineering, 2022, 48(1): 245-252.

Please choose a citation manager

Content to export