基于CLIP增强细粒度特征的换装行人重识别方法

doi:10.19678/j.issn.1000-3428.0068921

摘要/Abstract

摘要：

换装行人重识别旨在检索穿着不同服装的目标行人。现有方法通过引入额外信息(如轮廓、步态、3D信息)辅助学习服装无关特征。但受光照、姿态变化等因素的影响, 提取的生物特征可能存在误差。为提高准确性, 探索对比语言-图像预训练(CLIP)在该任务的应用, 提出CLIP驱动的细粒度特征增强方法(CFFE)。首先建模CLIP提取的类文本特征和图像特征的潜在内在联系, 然后引入显著性特征保留模块和显著性特征引导模块。显著性特征保留模块利用注意力掩码定位服装相关的前景区域, 进而擦除该部分特征, 使网络关注有效的非服装特征, 显著性特征引导模块通过注意力机制进一步关注行人的重要局部和全局特征。实验结果表明, 该方法在LTCC、PRCC和VC-Clothes数据集上的检测精度分别达到42.1%、71.1%和89.9%, 与AIM、CAL等算法相比, 能够提取到更细粒度的特征, 在多项指标上有明显提升。

关键词: 换装行人重识别, 对比语言-图像预训练, 特征保留策略, 注意力机制, 语义解析

Abstract:

Cloth-Changing person Re-Identification (CC-ReID) aims to identify target pedestrians wearing different outfits. Existing methods incorporate additional information (such as contours, gait, and 3D information) to assist the model in learning the clothing-agnostic features of pedestrians. However, owing to factors such as lighting and pose variations, the extracted biometric features may contain errors. To enhance accuracy, this paper explores the application of Contrastive Language-Image Pre-training (CLIP) and proposes CLIP-driven Fine-grained Feature Enhancement (CFFE) for CC-ReID. This method first models the potential intrinsic relationship between the class text features and image features extracted by CLIP. Subsequently, it uses a salient feature retention module and a saliency feature guiding module. The saliency feature retention module utilizes attention masks to locate foreground regions relevant to clothing and erases these features to ensure that the network focus on effective non-clothing features. Next, the saliency feature guidance module focuses on the important local and global features of pedestrians through attention mechanisms. The CFFE method achieves detection accuracies of 42.1%, 71.1%, and 89.9% on the LTCC, PRCC, and VC-Clothes datasets, respectively. Compared with algorithms such as AIM and CAL, CFFE extracts more robust features, showing significant improvements across multiple metrics.

Key words: Cloth-Changing person Re-Identification(CC-ReID), Contrastive Language-Image Pre-training(CLIP), feature retention strategy, attention mechanisms, semantic parsing

耿霞, 汪尧. 基于CLIP增强细粒度特征的换装行人重识别方法[J]. 计算机工程, 2025, 51(4): 293-302.

GENG Xia, WANG Yao. Cloth-Changing Person Re-Identification Method Based on CLIP Enhanced Fine-Grained Features[J]. Computer Engineering, 2025, 51(4): 293-302.

https://www.ecice06.com/CN/Y2025/V51/I4/293

图/表 12

图1 CFFE的整体网络结构

Fig.1 The overall network structure of CFFE

图2 SFG模块的整体结构

Fig.2 The overall structure of SFG module

图3 t-SNE的可视化结果

Fig.3 The visualization results of t-SNE

图4 不同骨干网络的对比结果

Fig.4 Comparison results of different backbone networks

图5 热力图结果的可视化

Fig.5 Visualization of heat-map results

图6 排序结果的可视化

Fig.6 Visualization of sorting results

参考文献 28

1	FAN L J, LI T H, FANG R Y, et al. Learning longterm representations for person re-identification using radio signals[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE Press, 2020: 10699-10709.
2	GU X Q , CHANG H , MA B P , et al. Appearance-preserving 3D convolution for video-based person re-identification. Berlin, Germany: Springer, 2020.
3	SUN Y F, ZHENG L, YANG Y, et al. Beyond part models: person retrieval with refined part pooling (and a strong convolutional baseline)[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 501-518.
4	JIN X, HE T Y, ZHENG K C, et al. Cloth-changing person re-identification from a single image with gait prediction and regularization[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans, USA: IEEE Press, 2022: 14278-14287.
5	YU S J, LI S H, CHEN D P, et al. COCAS: a large-scale clothes changing person dataset for re-identification[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE Press, 2020: 3400-3409.
6	YANG Q , WU A , ZHENG W S . Person re-identification by contour sketch under moderate clothing change. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43 (6): 2029- 2046. doi: 10.1109/TPAMI.2019.2960509
7	QIAN X L, WANG W X, ZHANG L, et al. Long-term cloth-changing person re-identification[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2021: 71-88.
8	CHEN J X, JIANG X Y, WANG F D, et al. Learning 3D shape feature for texture-insensitive person re-identification[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, USA: IEEE Press, 2021: 8146-8155.
9	LI Y J, LUO Z Y, WENG X S, et al. Learning shape representations for clothing variations in person re-identification[EB/OL]. [2023-10-20]. https://arxiv.org/abs/2003.07340v1.
10	SHU X J , LI G , WANG X , et al. Semantic-guided pixel sampling for cloth-changing person re-identification. IEEE Signal Processing Letters, 2021, 28, 1365- 1369. doi: 10.1109/LSP.2021.3091924
11	HAN K, GONG S, HUANG Y, et al. Clothing-change feature augmentation for person re-identi\ufb01cation[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2023: 22066-22075.
12	YANG Z W, LIN M, ZHONG X, et al. Good is bad: causality inspired cloth-debiasing for cloth-changing person re-identification[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2023: 1472-1481.
13	HUANG Y, WU Q, XU J S, et al. Clothing status awareness for long-term person re-identification[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2021: 11895-11904.
14	YAN Y M, YU H M, LI S Z, et al. Weakening the influence of clothing: universal clothing attribute disentanglement for person re-identification[C]//Proceedings of the 31st International Joint Conference on Artificial Intelligence. Washington D.C., USA: IEEE Press, 2022: 1523-1529.
15	GU X Q, CHANG H, MA B P, et al. Clothes-changing person re-identification with RGB modality only[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans, USA: IEEE Press, 2022: 1060-1069.
16	CUI Z Y , ZHOU J H , PENG Y X , et al. DCR-ReID: deep component reconstruction for cloth-changing person re-identification. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33 (8): 4415- 4428. doi: 10.1109/TCSVT.2023.3241988
17	MU J Y, LI Y, LI J Y, et al. Learning clothes-irrelevant cues for clothes-changing person re-identification[C]//Proceedings of British Machine Vision Conference. London, UK: [s. n. ], 2023: 253-268.
18	HAN K, HUANG Y, GONG S G, et al. 3D shape temporal aggregation forVideo-based clothing-change person re-identification[C]//Proceedings of ACCV'22. Washington D.C., USA: IEEE Press, 2023: 71-88.
19	WAN F B, WU Y, QIAN X L, et al. When person re-identification meets changing clothes[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE Press, 2020: 830-831.
20	ZHU K, GUO H Y, LIU Z W, et al. Identity-guided human semantic parsing for person re-identification[C]//Proceedings of ECCV'20. Berlin, Germany: Springer, 2020: 346-363.
21	ADFORD A, KIM J W, HALLACY C, et al. Learning transferable visual models from natural language supervision[C]//Proceedings of International Conference on Machine Learning. Washington D.C., USA: IEEE Press, 2021: 8748-8763.
22	CAO M, YANG T Y, WENG J W, et al. LocVTP: video-text pre-training for temporal localization[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2022: 38-56.
23	LI S Y , SUN L , LI Q L . CLIP-ReID: exploiting vision-language model for image re-identification without concrete text labels. Artificial Intelligence, 2023, 37 (1): 1405- 1413.
24	ZHAO S, ZHU L C, WANG X H, et al. CenterCLIP[C]//Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM Press, 2022: 970-981.
25	TANG M K, WANG Z Y, LIU Z H, et al. CLIP4Caption: CLIP for video caption[C]//Proceedings of the 29th ACM International Conference on Multimedia. New York, USA: ACM Press, 2021: 4858-4862.
26	WANG Z Q, LU Y, LI Q, et al. CRIS: CLIP-driven referring image segmentation[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans, USA: IEEE Press, 2022: 11686-11695.
27	RAO Y M, ZHAO W L, CHEN G Y, et al. DenseCLIP: language-guided dense prediction with context-aware prompting[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans, USA: IEEE Press, 2022: 18082-18091.
28	ZHANG R R, GUO Z Y, ZHANG W, et al. PointCLIP: point cloud understanding by CLIP[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans, USA: IEEE Press, 2022: 8552-8562.

[1]	杜晨阳, 张雪英, 黄丽霞, 李娟. 基于改进高效通道注意力机制的多特征语音情感识别[J]. 计算机工程, 2025, 51(4): 97-106.
[2]	孙子文, 钱立志, 袁广林, 杨传栋, 凌冲. 基于实时动态模板更新的Transformer目标跟踪方法[J]. 计算机工程, 2025, 51(4): 158-168.
[3]	董红亮, 钮焱, 孙杨, 李军. 基于记忆胶囊与注意力的语音情感识别[J]. 计算机工程, 2025, 51(4): 169-177.
[4]	解庆, 张凌峰, 马艳春, 刘永坚. 基于反射分类与梯度恢复的单幅图像去反射模型[J]. 计算机工程, 2025, 51(4): 227-238.
[5]	杨萍, 张汐. 改进DeepLabv3+的道路表面裂缝检测方法[J]. 计算机工程, 2025, 51(4): 261-270.
[6]	徐永刚, 孙琦烜, 李凡甲, 程健维, 戴佳俊. 基于扩展时间和时空特征融合图卷积网络的骨架行为识别[J]. 计算机工程, 2025, 51(4): 281-292.
[7]	刘云翔, 梁智超. 一种高效的连续时序图注意力网络的交通预测模型[J]. 计算机工程, 2025, 51(4): 350-359.
[8]	胡书林, 张华军, 邓小涛, 王征华. 结合依存图卷积的中文文本相似度计算研究[J]. 计算机工程, 2025, 51(3): 76-85.
[9]	卢鹏, 仲闯. 改进CycleGAN的半监督建筑物提取算法[J]. 计算机工程, 2025, 51(3): 241-251.
[10]	王新良, 王璐莹. 特征增强的低照度爆破现场安全帽检测算法[J]. 计算机工程, 2025, 51(3): 252-260.
[11]	孙亭, 杨洁, 李家璇, 王耀宗. 面向弱光交通场景的YOLOv7道路标志检测算法优化[J]. 计算机工程, 2025, 51(3): 342-351.
[12]	栾方军, 龚琪, 袁帅. 基于注意力机制和多尺度融合的人群计数网络[J]. 计算机工程, 2025, 51(3): 352-361.
[13]	张欢, 王晨, 单景东, 仇润鹤. 基于领域自适应与注意力机制的电梯安全风险预测[J]. 计算机工程, 2025, 51(2): 86-93.
[14]	张元, 吕德芳, 孟建军, 祁文哲. 基于双注意力和GSSN轻量化的钢轨扣件缺陷检测[J]. 计算机工程, 2025, 51(2): 289-299.
[15]	许明, 屈泰澎, 姜彦吉. 改进YOLOv7在复杂场景下的交通标志检测算法[J]. 计算机工程, 2025, 51(2): 335-343.

选择文件类型/文献管理软件名称

选择包含的内容