复杂光照条件下自适应的车脸重识别模型

doi:10.19678/j.issn.1000-3428.0065453

摘要/Abstract

摘要：

为提高车脸特征提取对于复杂光照条件的自适应性，降低图像采集过程中光照变化对车脸重识别带来的影响，提出一种对光照强度变化具有较好鲁棒性的基于自适应特征提取的车脸重识别模型。利用YOLOv3模型对采集的图像进行预处理后，采用改进的MobileNetV3-Small模型提取车脸图像的原始特征。由于光照条件变化时不同类型的车脸特征受影响程度不同，因此通过训练获得2种特征转换矩阵，将原始特征划分为不受光照条件影响的稳定特征和易受光照条件影响的易变特征。在训练网络模型时，对鉴别网络的输出结果进行信息熵约束，保证样本间稳定特征分布的一致性，同时通过融合稳定特征和基于时间注意力机制调整的易变特征，实现对车脸样本的有监督学习。实验结果表明，在3种车脸图像数据集中，该模型的识别准确率分别达到0.866、0.872、0.923，较对比模型中的最优值提升了0.033、0.026、0.080，并且对光照差异较大的车脸图像对也能获得较好的识别效果。

关键词: 车脸重识别, 鉴别网络, 有监督学习, 自适应特征提取, 时间注意力机制

Abstract:

To improve the adaptability of vehicle face feature extraction to complex illumination conditions and reduce the impact of illumination variation on vehicle face re-identification during image acquisition, this study proposes an adaptive feature extraction-based vehicle face re-identification model with better robustness to illumination intensity variations. First, the captured image undergoes preprocessing based on the YOLOv3 model, following which an improved MobileNetV3-Small model extracts the original features of the vehicle face image.Then, because different types of vehicle face features are affected variably by alterations in illumination conditions, two feature transformation matrices obtained through training allow for the division of original features into stable features that are not influenced by illumination variation and volatile features that are easily influenced by illumination variation.Finally, when training the network model, the proposed discrimination network ensures distribution consistency of stable features among samples by constraining the information entropy of output results.Concurrently, supervised learning of the model is achieved by fusing the volatile features that are adjusted based on the time attention mechanism and stable features. The experimental results show that for three vehicle face image datasets, the recognition accuracies of the proposed model are 0.866, 0.872, and 0.923, which are improvements of 0.033, 0.026, and 0.080, respectively, over the best values of the comparison models.Particularly, for pairs of vehicle images with large differences in illumination, the proposed model still achieves superior recognition performance.

Key words: vehicle face re-identification, discriminant network, supervised learning, adaptive feature extraction, time attention mechanism

马娜, 温廷新, 贾旭, 李晓会. 复杂光照条件下自适应的车脸重识别模型[J]. 计算机工程, 2023, 49(8): 275-282, 290.

Na MA, Tingxin WEN, Xu JIA, Xiaohui LI. Adaptive Vehicle Face Re-identification Model Under Complex Illumination Conditions[J]. Computer Engineering, 2023, 49(8): 275-282, 290.

http://www.ecice06.com/CN/Y2023/V49/I8/275

图/表 17

图1 摄像头抓拍图像

Fig.1 The images captured by camera

图2 模型结构

Fig.2 The model structure

表1 改进MobileNetV3-Small模型结构

Table 1 The improved MobileNetV3-Small model structure

输入	卷积核	通道膨胀	输出通道	使用SE	激活函数	步长
224×224×3	conv2d, 3×3	—	16	×	HS	2
112×112×16	bneck, 3×3	16	16	√	RE	2
56×56×16	bneck, 3×3	72	24	×	RE	2
28×28×24	bneck, 3×3	88	24	×	RE	1
28×28×24	bneck, 3×3	96	40	√	HS	2
14×14×40	bneck, 3×3	240	40	√	HS	1
14×14×40	bneck, 3×3	120	48	√	HS	1
14×14×48	bneck, 3×3	144	48	√	HS	1
14×14×48	bneck, 3×3	288	96	√	HS	2
7×7×96	bneck, 3×3	576	96	√	HS	1
7×7×96	conv2d, 1×1	—	576	√	HS	1

图3 稳定特征与易变特征

Fig.3 Stable and volatile features

图4 19：15与11：02采集的车脸图像

Fig.4 The vehicle face images captured at 19:15 and 11:02

图5 Veri776数据集部分样本

Fig.5 Some samples in Veri776 dataset

图6 VERI-Wild数据集部分样本

Fig.6 Some samples in VERI-Wild dataset

图7 自建数据集部分样本

Fig.7 Some samples in self-established dataset

图8 不同

$ \boldsymbol{\lambda } $

参数值下的迭代次数-准确率曲线

Fig.8 The epoch-accuracy curves under different

$ \boldsymbol{\lambda } $

图9 Veri776数据集下添加与删除鉴别网络时的实验结果

Fig.9 The experimental results under Veri776 dataset when adding and removing discrimination network

图10 VERI-Wild数据集下添加与删除鉴别网络时的实验结果

Fig.10 The experimental results under VERI-Wild dataset when adding and removing discrimination network

图11 自建数据集下添加与删除鉴别网络时的实验结果

Fig.11 The experimental results under self-established dataset when adding and removing discrimination network

图12 VERI-Wild数据集下添加与删除时间注意力矩阵时的实验结果

Fig.12 The experimental results under VERI-Wild dataset when adding and removing time attention matrice

图13 自建数据集下添加与删除时间注意力矩阵时的实验结果

Fig.13 The experimental results under self-established dataset when adding and removing time attention matrice

表2 不同图像采集时段下的识别结果比较

Table 2 Comparison of the recognition results under different image capturing times

数据集	模型	准确率
数据集	模型	DD	NN	DN
VERI-Wild 数据集	Model B	0.945	0.877	0.787
VERI-Wild 数据集	本文模型	0.944	0.881	0.822
自建数据集	Model B	0.962	0.896	0.806
自建数据集	本文模型	0.964	0.902	0.846

表3 不同模型的识别结果对比

Table 3 Comparison of the recognition results of different models

模型	准确率
模型	Veri776数据集	VERI-Wild数据集	自建数据集
TN	0.766	0.743	0.784
MAM	0.785	0.792	0.798
PAMLN	0.821	0.833	0.882
DBM	0.813	0.846	0.842
NMF-TD	0.833	0.836	0.843
本文模型	0.866	0.872	0.923

表4 不同模型的部分测试样本识别结果对比

Table 4 Comparison of the recognition results of some test samples by different models

测试图像对	样本属性	模型	识别结果
	正样本	TN MAM PAMLN DBM NMF-TD 本文模型	准确准确准确准确准确准确
	正样本	TN MAM PAMLN DBM NMF-TD 本文模型	不准确不准确准确不准确不准确准确
	负样本	TN MAM PAMLN DBM NMF-TD 本文模型	不准确准确不准确不准确准确准确
	正样本	TN MAM PAMLN DBM NMF-TD 本文模型	不准确不准确不准确不准确不准确准确

参考文献 41

1	WU Y, LIN Y T, DONG X Y, et al. Progressive learning for person re-identification with one example. IEEE Transactions on Image Processing, 2019, 28(6): 2872- 2881. doi: 10.1109/TIP.2019.2891895
2	ZAKRIA J, DENG J H, HAO Y, et al. Trends in vehicle re-identification past, present, and future: a comprehensive review. Mathematics, 2021, 9(24): 3162. doi: 10.3390/math9243162
3	ZHENG L, SHEN L Y, TIAN L, et al. Scalable person re-identification: a benchmark[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2016: 1116-1124.
4	LIAO S C, HU Y, ZHU X Y, et al. Person re-identification by local maximal occurrence representation and metric learning[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2015: 2197-2206.
5	LIU X C, LIU W, MEI T, et al. PROVID: progressive and multimodal vehicle reidentification for large-scale urban surveillance. IEEE Transactions on Multimedia, 2018, 20(3): 645- 658. doi: 10.1109/TMM.2017.2751966
6	LIU H Y, TIAN Y H, WANG Y W, et al. Deep relative distance learning: tell the difference between similar vehicles[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 2167-2175.
7	ZHANG Y H, LIU D, ZHA Z J. Improving triplet-wise training of convolutional neural network for vehicle re-identification[C]//Proceedings of IEEE International Conference on Multimedia and Expo. Washington D. C., USA: IEEE Press, 2017: 1386-1391.
8	ZHANG Q, ZHUO L, LI J F, et al. Vehicle color recognition using multiple-layer feature representations of lightweight convolutional neural network. Signal Processing, 2018, 147, 146- 153. doi: 10.1016/j.sigpro.2018.01.021
9	SHEN F, ZHU J Q, ZHU X B, et al. Exploring spatial significance via hybrid pyramidal graph network for vehicle re-identification. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(7): 8793- 8804. doi: 10.1109/TITS.2021.3086142
10	ZHANG L, XIANG T, GONG S G. Learning a discriminative null space for person re-identification[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 1239-1248.
11	BAI Y, LOU Y H, GAO F, et al. Group-sensitive triplet embedding for vehicle reidentification. IEEE Transactions on Multimedia, 2018, 20(9): 2385- 2399. doi: 10.1109/TMM.2018.2796240
12	MARÍN-REYES P A, BERGAMINI L, LORENZO-NAVARRO J, et al. Unsupervised vehicle re-identification using triplet networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 166-171.
13	ZHAO J J, ZHAO Y F, LI J, et al. Heterogeneous relational complement for vehicle re-identification[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2022: 205-214.
14	WANG Z D, TANG L M, LIU X H, et al. Orientation invariant feature embedding and spatial temporal regularization for vehicle re-identification[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 379-387.
15	ZHOU Y, SHAO L. Vehicle re-identification by adversarial bi-directional LSTM network[C]//Proceedings of IEEE Winter Conference on Applications of Computer Vision. Washington D. C., USA: IEEE Press, 2018: 653-662.
16	ZHOU Y, LIU L, SHAO L. Vehicle re-identification by deep hidden multi-view inference. IEEE Transactions on Image Processing, 2018, 27(7): 3275- 3287. doi: 10.1109/TIP.2018.2819820
17	MENG D C, LI L, LIU X J, et al. Parsing-based view-aware embedding network for vehicle re-identification[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 7101-7110.
18	CHEN Y C, MA B P, CHANG H. Part alignment network for vehicle re-identification. Neurocomputing, 2020, 418, 114- 125. doi: 10.1016/j.neucom.2020.08.016
19	LOU Y H, BAI Y, LIU J, et al. Embedding adversarial learning for vehicle re-identification. IEEE Transactions on Image Processing, 2019, 28(8): 3794- 3807. doi: 10.1109/TIP.2019.2902112
20	ZHOU Y, SHAO L. Cross-view GAN based vehicle generation for re-identification[C]//Proceedings of 2017 British Machine Vision Conference. London, UK: British Machine Vision Association, 2017: 1-12.
21	LI H C, LIN X M, ZHENG A H, et al. Attributes guided feature learning for vehicle re-identification. IEEE Transactions on Emerging Topics in Computational Intelligence, 2022, 6(5): 1211- 1221. doi: 10.1109/TETCI.2021.3127906
22	GUO H Y, ZHU K, TANG M, et al. Two-level attention network with multi-grain ranking loss for vehicle re-identification. IEEE Transactions on Image Processing, 2019, 28(9): 4328- 4338. doi: 10.1109/TIP.2019.2910408
23	ZHOU Y Y, SHAO L. Viewpoint-aware attentive multi-view inference for vehicle re-identification[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 6489-6498.
24	ZHENG A H, LIN X M, DONG J C, et al. Multi-scale attention vehicle re-identification. Neural Computing and Applications, 2020, 32(23): 17489- 17503. doi: 10.1007/s00521-020-05108-x
25	HUYNH S V, NGUYEN N H, NGUYEN N T, et al. A strong baseline for vehicle re-identification[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 4142-4149.
26	ZHU Y C, ZHA Z J, ZHANG T Z, et al. A structured graph attention network for vehicle re-identification[C]//Proceedings of the 28th ACM International Conference on Multimedia. New York, USA: ACM Press, 2020: 646-654.
27	TUMRANI S, DENG Z Y, LIN H Y, et al. Partial attention and multi-attribute learning for vehicle re-identification. Pattern Recognition Letters, 2020, 138, 290- 297. doi: 10.1016/j.patrec.2020.07.034
28	WANG J T, ZHENG H, HUANG Y, et al. Vehicle type recognition in surveillance images from labeled Web-nature data using deep transfer learning. IEEE Transactions on Intelligent Transportation Systems, 2018, 19(9): 2913- 2922. doi: 10.1109/TITS.2017.2765676
29	CHEN Y, YANG C L, YANG S Y. A method for special vehicle recognition based on deep-transfer model[C]//Proceedings of the 6th International Conference on Instrumentation & Measurement, Computer, Communication and Control. Washington D. C., USA: IEEE Press, 2016: 167-170.
30	HE S T, LUO H, CHEN W H, et al. Multi-domain learning and identity mining for vehicle re-identification[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 2485-2493.
31	WANG Q, MIN W D, HAN Q, et al. Inter-domain adaptation label for data augmentation in vehicle re-identification. IEEE Transactions on Multimedia, 2022, 24, 1031- 1041. doi: 10.1109/TMM.2021.3104141
32	HU A Q, LI H, ZHANG F, et al. Deep Boltzmann machines based vehicle recognition[C]//Proceedings of the 26th Chinese Control and Decision Conference. Washington D. C., USA: IEEE Press, 2014: 3033-3038.
33	SANTOS D F S, DE SOUZA G B, MARANA A N. A 2D deep Boltzmann machine for robust and fast vehicle classification[C]//Proceedings of the 30th Conference on Graphics, Patterns and Images. Washington D. C., USA: IEEE Press, 2017: 155-162.
34	GOU C, WANG K F, YAO Y J, et al. Vehicle license plate recognition based on extremal regions and restricted Boltzmann machines. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(4): 1096- 1107. doi: 10.1109/TITS.2015.2496545
35	贾旭, 孙福明. 基于孪生非负矩阵分解的车脸重识别算法. 仪器仪表学报, 2020, 41(6): 132- 139. URL
	JIA X, SUN F M. Vehicle face re-identification algorithm based on Siamese nonnegative matrix factorization. Chinese Journal of Scientific Instrument, 2020, 41(6): 132- 139. URL
36	SHI C H, WU C D. Vehicle face recognition algorithm based on weighted nonnegative matrix factorization with double regularization terms. KSII Transactions on Internet and Information Systems, 2020, 14(5): 2171- 2185.
37	MA N, WEN T X. Vehicle face re-identification based on nonnegative matrix factorization with time difference constraint. KSII Transactions on Internet and Information Systems, 2021, 15(6): 2098- 2114.
38	李国进, 胡洁, 艾矫燕. 基于改进SSD算法的车辆检测. 计算机工程, 2022, 48(1): 266- 274. URL
	LI G J, HU J, AI J Y. Vehicle detection based on improved SSD algorithm. Computer Engineering, 2022, 48(1): 266- 274. URL
39	HOWARD A, SANDLER M, CHEN B, et al. Searching for MobileNetV3[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2020: 1314-1324.
40	LIU X C, LIU W, MA H D, et al. Large-scale vehicle re-identification in urban surveillance videos[C]//Proceedings of IEEE International Conference on Multimedia and Expo. Washington D. C., USA: IEEE Press, 2016: 1-6.
41	LOU Y H, BAI Y, LIU J, et al. VERI-Wild: a large dataset and a new method for vehicle re-identification in the wild[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 3230-3238.

[1]	陈扬洋, 钱鹏江, 赵开发, 苏冠豪. 基于mDixon序列下腹部MRI数据的sCT生成方法[J]. 计算机工程, 2019, 45(7): 273-281.
[2]	郭丽，郑忠龙，贾炯，张海新，付芳梅. 一种有监督的线性降维人脸识别算法[J]. 计算机工程, 2013, 39(11): 169-173.
[3]	钟明, 薛惠锋, 梅觅. 基于局部线性嵌入的最大散度矩阵算法[J]. 计算机工程, 2011, 37(12): 176-178.

选择文件类型/文献管理软件名称

选择包含的内容