标签引导的生成对抗网络人脸表情识别域适应方法

doi:10.19678/j.issn.1000-3428.0054134

摘要/Abstract

摘要： 传统的人脸表情识别方法主要针对实验室环境下的基本表情，难以应对现实场景中人类微妙和复杂的表情变化，并且目前自然环境人脸表情识别数据集普遍缺乏足够的训练数据。针对该问题，利用实验室环境下的数据库样本，提出以标签引导的生成对抗网络表情识别域适应方法。将情感标签作为辅助条件，训练生成对抗网络的生成模型，把实验室环境的数据库样本转化为类似自然环境数据库的样本，以扩充自然环境数据库，同时基于扩充的数据库样本训练基本分类器VGG、Resnet等，从而学习自然环境的数据库的情感特征。在RAF_DB等自然环境人脸表情数据库上的实验结果表明，与Boosting-POOF和PixelDA方法相比，该方法扩充得到的数据库可使人脸表情识别率取得6%~9%的提升。

关键词: 生成对抗网络, 情感标签, 人脸表情识别, 域适应, 自然环境, 数据库样本

Abstract: In Facial Expression Recognition(FER),sufficient training samples have significant influence on recognition results.To address insufficient database samples in natural environment,this paper proposes a Label-guided Domain Adaption method in Generative Adversarial Network(LDAGAN) for FER by using laboratory environment database samples.This method adopts the generation model of GAN and takes emotional labels as auxiliary condition.Then the method uses the laboratory environment database samples to generate samples similar to those of natural environment database,so as to build a bridge between laboratory environment database and natural environment database,enlarging the natural environment database.The samples assist in learning the emotional features of the natural environment database.Experimental results on the facial expression database of natural environment such as RAF_DB show that the proposed method achieves an improvement of 6% to 9% in FER accuracy compared with Boosting-POOF and PixelDA methods.

Key words: Generative Adversarial Network(GAN), emotional label, Facial Expression Recognition(FER), domain adaptation, natural environment, database sample

中图分类号:

TP18

孙冬梅, 张飞飞, 毛启容. 标签引导的生成对抗网络人脸表情识别域适应方法[J]. 计算机工程, 2020, 46(5): 267-273,281.

SUN Dongmei, ZHANG Feifei, MAO Qirong. Label-Guided Domain Adaptation Method in Generative Adversarial Network for Facial Expression Recognition[J]. Computer Engineering, 2020, 46(5): 267-273,281.

https://www.ecice06.com/CN/Y2020/V46/I5/267

图/表 7

20200513205041

20200513205046

20200513205049

20200513205054

20200513205058

20200513205102

20200513205106

参考文献

[1] XUE Yuli,MAO Xia,GUO Ye,et al.The research advance of facial expression recognition in human computer interaction[J].Journal of Image and Graphics,2009,14(5):764-772.(in Chinese)薛雨丽,毛峡,郭叶,等.人机交互中的人脸表情识别研究进展[J].中国图象图形学报,2009,14(5):764-772.
[2] CORNEANU C A,SIMÓN M O,COHN J F,et al.Survey on RGB,3D,thermal,and multimodal approaches for facial expression recognition:history,trends,and affect-related applications[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(8):1548-1568.
[3] JIANG Dapeng,YANG Biao,ZOU Ling.Facial expression recognition based on local binary mode convolution neural network[J].Computer Engineering and Design,2018,39(7):179-185.(in Chinese)江大鹏,杨彪,邹凌.基于LBP卷积神经网络的面部表情识别[J].计算机工程与设计,2018,39(7):179-185.
[4] KONG Yinghui,YIN Ziwei,CHE Linlin.Research on face recognition method based on sparse learning and significance[J].Computer Engineering,2018,44(2):271-276.(in Chinese)孔英会,尹紫薇,车辚辚.基于稀疏学习与显著性的人脸识别方法研究[J].计算机工程,2018,44(2):271-276.
[5] ZHAI Yikui,LIU Jian.Facial expression recognition based on transferring convolutional neural network[J].Journal of Signal Processing,2018,34(6):103-112.(in Chinese)翟懿奎,刘健.面向人脸表情识别的迁移卷积神经网络研究[J].信号处理,2018,34(6):103-112.
[6] MOLLAHOSSEINI A,CHAN D,MAHOOR M H.Going deeper in facial expression recognition using deep neural networks[C]//Proceedings of IEEE Winter Conference on Applications of Computer Vision.Washington D.C.,USA:IEEE Computer Society,2016:1-10.
[7] AN Xu,ZHANG Shudong.Research on fuzzy feature classification algorithm based on support vector machine[J].Computer Engineering,2017,43(1):237-240,246.(in Chinese)安旭,张树东.基于支持向量机的模糊特征分类算法研究[J].计算机工程,2017,43(1):237-240,246.
[8] GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems.New York,USA:ACM Press,2014:2672-2680.
[9] YAO Naiming,GUO Qingpei,QIAO Fengchun,et al.Robust facial expression recognition with generative adversarial networks[J].Acta Automatica Sinica,2018,44(5):100-112.(in Chinese)姚乃明,郭清沛,乔逢春,等.基于生成式对抗网络的鲁棒人脸表情识别[J].自动化学报,2018,44(5):100-112.
[10] SHRIVASTAVA A,PFISTER T,TUZEL O,et al.Learning from simulated and unsupervised images through adversarial training[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:2107-2116.
[11] CHOI Y,CHOI M,KIM M,et al.StarGAN:unified generative adversarial networks for multi-domain image-to-image translation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:8789-8797.
[12] ZHANG Feifei,ZHANG Tianzhu,MAO Qirong,et al.Facial expression recognition in the wild:a cycle-consistent adversarial attention transfer approach[C]//Proceedings of 2018 ACM Multimedia Conference on Multimedia Conference.New York,USA:ACM Press,2018:126-135.
[13] BOUSMALIS K,SILBERMAN N,DOHAN D,et al.Unsupervised pixel-level domain adaptation with generative adversarial networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:3722-3731.
[14] FANG Xianghong,BAI Haoli,GUO Ziyi,et al.DART:domain-adversarial residual-transfer networks for unsupervised cross-domain image classification[EB/OL].[2019-03-10].https://arxiv.org/pdf/1812.11478.pdf.
[15] WU Jia,XU Dong,WU Shengyin.Lib face detection[EB/OL].[2019-03-10].https://github.com/ShiqiYu/libface-detection.
[16] YIN Lijun,WEI Xiaozhou,SUN Yi.A 3D facial expression database for facial behavior research[C]//Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition.Washington D.C.,USA:IEEE Press,2006:211-216.
[17] LI Shan,DENG Weihong,DU Junping.Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:2852-2861.
[18] DHALL A,GOECKE R,LUCEY S,et al.Static facial expression analysis in tough conditions:data,evaluation protocol and benchmark[C]//Proceedings of IEEE International Conference on Computer Vision Workshops.Washington D.C.,USA:IEEE Press,2011:2106-2112.
[19] LUCEY P,COHN J F,KANADE T,et al.The extended Cohn-Kanade Dataset(CK+):a complete dataset for action unit and emotion-specified expression[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2010:94-101.
[20] LIU Zhiwen,LI Shan,DENG Weihong.Boosting-poof:boosting part based one vs one feature for facial expression recognition in the wild[C]//Proceedings of the 12th IEEE International Conference on Automatic Face and Gesture Recognition.Washington D.C.,USA:IEEE Press,2017:967-972.
[21] SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[EB/OL].[2019-03-10].https://arxiv.org/pdf/1409.1556.pdf.
[22] HE Kaiming,ZHANG Xiangyu,REN Shaoqing,et al.Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:770-778.

选择文件类型/文献管理软件名称

选择包含的内容