针对大角度下视角差异的行人重识别方法研究

doi:10.19678/j.issn.1000-3428.0067532

摘要/Abstract

摘要： 行人重识别(Re-ID)也称为行人再识别,旨在给定一个目标行人,确定该行人是否出现在不同的摄像机下,或者是在不同的时间出现在相同的摄像机下。通常由于不同摄像机拍摄到的行人视角不同,在视角差异过大的情况下会对行人重识别准确率造成严重影响。因此,针对目标行人相对摄像机的视角不同而带来的识别率下降问题,提出一种基于外观-步态特征融合的行人重识别算法,使用视角信息对RGB图像与步态能量图(GEI)进行重要性权重估计后再加权融合,以此来克服视角不同而带来的影响。具体来讲,首先利用ResNet-50提取图像序列中每张图像的特征,采用时间池化的方式将其聚合为外观特征。其次使用另一个ResNet-50对GEI图像提取步态特征。然后对行人进行视角估计之后,映射函数将估计的角度映射为两种特征的重要性权重。最后基于自编码器结构将两种特征在重要性权重的指导下进行加权融合,生成对视角鲁棒的融合特征。在CASIA-B数据集上的实验结果表明,对于具有大角度视角差异的行人Re-ID,所提出方法在mAP和Rank-1评估指标上都表现出了显著的改进。在大角度差异情况下进行测试,准确率最高提升了2.7%。

关键词: 行人重识别, 视角差异, 外观特征, 步态特征, 特征融合

Abstract: Person Re-Identification(Re-ID) aims to determine whether a person of interest appeared under different cameras or under the same camera at different times. Owing to the different viewpoints of people captured by different cameras, the accuracy of person Re-ID can be adversely affected. Therefore, in this study, a person Re-ID method based on the fusion of appearance and gait features is proposed to solve the reduced recognition rate caused by different viewpoints of people relative to a camera. Here, viewpoint information is utilized to estimate the importance weights of an RGB image and Gait Energy Image(GEI); subsequently, weighted fusion is performed to overcome the effects of different viewpoints. Specifically, ResNet-50 is first used to extract the features of each image in the image sequence, which are then further aggregated into appearance features via temporal pooling. Second, another ResNet-50 model is used to extract gait features from the GEI images. Third, after estimating the viewpoint of a person, a mapping function is designed to map the viewpoints to the importance weights of the two features. Finally, based on the auto-encoder structure, the two features are weighted and fused under the guidance of importance weights to generate fusion features that are robust to the perspective. Experimental results on the CASIA-B dataset show that the proposed method exhibits significant improvements in terms of the mAP and Rank-1 evaluation metrics for person Re-ID with large-angle viewpoint differences. When tested under large-angle differences, the highest accuracy improvement is 2.7%.

Key words: person Re-Identification(Re-ID), viewpoint differences, appearance feature, gait feature, feature fusion

中图分类号:

TP391

蔡毅翔, 秦品乐, 曾建潮, 晋赞霞, 秦佳, 翟双姣. 针对大角度下视角差异的行人重识别方法研究[J]. 计算机工程, 2024, 50(5): 330-341.

CAI Yixiang, QIN Pinle, ZENG Jianchao, JIN Zanxia, QIN Jia, ZHAI Shuangjiao. Research on Person Re-Identification Method for Large-Angle Viewpoint Differences[J]. Computer Engineering, 2024, 50(5): 330-341.

https://www.ecice06.com/CN/Y2024/V50/I5/330

参考文献

[1] YE M, SHEN J, LIN G, et al. Deep learning for person re-identification:a survey and outlook[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(6):2872-2893.
[2] QIAN X L, FU Y W, XIANG T, et al. Pose-normalized image generation for person re-identification[C]//Proceedings of the 15th European Conference on Computer Vision. Berlin, Germany:Springer, 2018:661-678.
[3] LIU F Y, ZHANG L. View confusion feature learning for person re-identification[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C.,USA:IEEE Press, 2019:6639-6648.
[4] ZHU Z H, JIANG X Y, ZHENG F, et al. Viewpoint-aware loss with angular regularization for person re-identification[J]. Artificial Intelligence, 2020, 34(7):13114-13121.
[5] LU X Y, LI X D, SHENG W J, et al. Long-term person re-identification based on appearance and gait feature fusion under covariate changes[J]. Processes, 2022, 10(4):770.
[6] HAN J, BHANU B. Individual recognition using gait energy image[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(2):316-322.
[7] YU S Q, TAN D L, TAN T N. A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition[C]//Proceedings of the 18th International Conference on Pattern Recognition. New York, USA:ACM Press, 2006:441-444.
[8] GRAY D, TAO H. Viewpoint invariant pedestrian recognition with an ensemble of localized features[C]//Proceedings of the 10th European Conference on Computer. Berlin, Germany:Springer, 2008:262-275.
[9] FARENZENA M, BAZZANI L, PERINA A, et al. Person re-identification by symmetry-driven accumulation of local features[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington D. C., USA:IEEE Press, 2010:2360-2367.
[10] KARANAM S, LI Y, RADKE R J. Person re-identification with discriminatively trained viewpoint invariant dictionaries[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Washington D. C.,USA:IEEE Press, 2015:4516-4524.
[11] CHO Y J, YOON K J. Improving person re-identification via pose-aware multi-shot matching[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C.,USA:IEEE Press, 2016:1354-1362.
[12] 杨永胜, 邓淼磊, 李磊, 等. 基于深度学习的行人重识别综述[J]. 计算机工程与应用, 2022, 58(9):51-66. YANG Y S, DENG M L, LI L, et al. Overview of pedestrian re-identification based on deep learning[J]. Computer Engineering and Applications, 2022, 58(9):51-66. (in Chinese)
[13] ZHENG L, YANG Y, HAUPTMANN A G, et al. Person re-identification:past, present and future[EB/OL].[2023-04-01]. http://arxiv.org/abs/1610.02984v1.
[14] YI D, LEI Z, LIAO S C, et al. Deep metric learning for person re-identification[C]//Proceedings of the 22nd International Conference on Pattern Recognition. Washington D. C.,USA:IEEE Press, 2014:34-39.
[15] LI W, ZHAO R, XIAO T, et al. DeepReID:deep filter pairing neural network for person re-identification[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C.,USA:IEEE Press, 2014:152-159.
[16] ZHANG Z Z, LAN C L, ZENG W J, et al. Relation-aware global attention for person re-identification[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA:IEEE Press, 2020:3186-3195.
[17] 董亚超, 刘宏哲, 徐成. 基于显著性多尺度特征协作融合的行人重识别方法[J]. 计算机工程, 2021, 47(6):234-244, 252. DONG Y C, LIU H Z, XU C. Person re-identification method based on joint fusion of saliency multi-scale features[J]. Computer Engineering, 2021, 47(6):234-244, 252.(in Chinese)
[18] 曾涛, 薛峰, 杨添. 面向行人重识别的通道与空间双重注意力网络[J]. 计算机工程, 2022, 48(12):281-287, 295. ZENG T, XUE F, YANG T. Channel and spatial dual-attention network for person re-identification[J]. Computer Engineering, 2022, 48(12):281-287, 295.(in Chinese)
[19] MCLAUGHLIN N, MARTINEZ DEL RINCON J, MILLER P. Recurrent convolutional network for video-based person re-identification[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C.,USA:IEEE Press, 2016:1325-1334.
[20] MENG J, ZHENG W S, LAI J H, et al. Deep graph metric learning for weakly supervised person re-identification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(10):6074-6093.
[21] SEPAS-MOGHADDAM A, ETEMAD A. Deep gait recognition:a survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(1):264-284.
[22] YU S Q, LIAO R J, AN W Z, et al. GaitGANv2:invariant gait feature extraction using generative adversarial networks[J]. Pattern Recognition, 2019, 87:179-189.
[23] WU Z F, HUANG Y Z, WANG L, et al. A comprehensive study on cross-view gait based human identification with deep CNNs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(2):209-226.
[24] YU S Q, CHEN H F, REYES E B G, et al. GaitGAN:invariant gait feature extraction using generative adversarial networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops. Washington D. C., USA:IEEE Press, 2017:30-37.
[25] WANG Y Y, SONG C F, HUANG Y, et al. Learning view invariant gait features with two-stream GAN[J]. Neurocomputing, 2019, 339:245-254.
[26] WU A C, ZHENG W S, YU H X, et al. RGB-infrared cross-modality person re-identification[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C.,USA:IEEE Press, 2017:5390-5399.
[27] WAN L, SUN Z Y, JING Q Y, et al. G²DA:geometry-guided dual-alignment learning for RGB-infrared person re-identification[J]. Pattern Recognition, 2023, 135:109150.
[28] WU A C, ZHENG W S, LAI J H. Robust depth-based person re-identification[J]. IEEE Transactions on Image Processing, 2017, 26(6):2588-2603.
[29] LI S, XIAO T, LI H S, et al. Person search with natural language description[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA:IEEE Press, 2017:5187-5196.
[30] CHEN D P, LI H S, LIU X H, et al. Improving deep visual representation for person re-identification by global and local image-language association[M]. Berlin, Germany:Springer, 2018:56-73.
[31] LIU J W, ZHA Z J, HONG R C, et al. Deep adversarial graph attention convolution network for text-based person search[C]//Proceedings of the 27th ACM International Conference on Multimedia. New York, USA:ACM Press, 2019:665-673.
[32] LI S Q, ZHANG M, LIU W, et al. Appearance and gait-based progressive person re-identification for surveillance systems[C]//Proceedings of the 4th IEEE International Conference on Multimedia Big Data. Washington D. C., USA:IEEE Press, 2018:1-7.
[33] GAO J Y, NEVATIA R, LIAO X Y, et al. Revisiting temporal modeling for video-based person ReID[EB/OL].[2023-04-01]. http://arxiv.org/abs/1805.02104v2.
[34] KIPF T N, WELLING M. Variational graph auto-encoders[EB/OL].[2023-04-01]. http://arxiv.org/abs/1611.07308v1.
[35] CHAO H Q, HE Y W, ZHANG J P, et al. GaitSet:regarding gait as a set for cross-view gait recognition[J]. Artificial Intelligence, 2019, 33(1):8126-8133.
[36] DAI Y M, GIESEKE F, OEHMCKE S, et al. Attentional feature fusion[C]//Proceedings of IEEE Winter Conference on Applications of Computer Vision. Waikoloa, USA:IEEE Press, 2021:3560-3569.
[37] MOGAN J N, LEE C P, LIM K M, et al. VGG16-MLP:gait recognition with fine-tuned VGG-16 and multilayer perceptron[J]. Applied Sciences, 2022, 12(15):7639.
[38] LIAO R J, LI Z, BHATTACHARYYA S S, et al. PoseMapGait:a model-based gait recognition method with pose estimation maps and graph convolutional networks[J]. Neurocomputing, 2022, 501:514-528.
[39] WANG L K, CHEN J Y, LIU Y X. Frame-level refinement networks for skeleton-based gait recognition[J]. Computer Vision and Image Understanding, 2022, 222:103500.

选择文件类型/文献管理软件名称

选择包含的内容