选择性传输与铰链对抗的多图像域人脸属性迁移

doi:10.19678/j.issn.1000-3428.0060631

摘要/Abstract

摘要： 在基于生成对抗网络的人脸属性迁移过程中，存在图像域表达形式单一、图像域迁移细节失真的问题。提出一种结合选择性传输单元与铰链对抗损失的多图像域人脸属性迁移方法。在生成器中，利用自适应实例归一化融合图像的内容信息与图像域控制器生成的样式信息，增加图像域表达方式的多样性，同时通过选择性传输单元将下采样提取的内容特征根据相对属性标签选择性地传输到上采样，形成融合特征以增强图像的细节信息。在判别器中，通过增加双尺度判别，协同鉴定人脸图像的真伪及类别，从而提高判定的准确度。在此基础上，设计融合相对鉴别和铰链损失的对抗损失函数，增强真伪图像域之间的联系。在CelebA数据集上的实验结果表明，与StarGAN、STGAN等主流的多图像域人脸属性迁移方法相比，该方法能够建立更准确的多图像域映射关系，提高迁移图像的质量同时增加迁移图像表达的多样性。

关键词: 生成对抗网络, 多图像域属性迁移, 自适应实例归一化, 选择性传输单元, 相对属性标签, 域控制器, 双尺度判别

Abstract: In facial attribute migration based on a Generative Adversarial Network(GAN), the expression form of the image domain is single and the details of the image domain migration are distorted.This paper presents facial attribute migration across multiple image domains method, which combines a selective transfer units and a hinge against loss.In the generator, the adaptive instance normalization and fuse the content information of the image and the style information that is generated by the image domain controller to increase the diversity of the expression methods in the image domain.The content features that are extracted from the downsampling are selectively transmitted to the upsampling according to the relative attribute label through the selective transmission unit to form fusion features for enhancing the detail information of the image.In the discriminator, dual-scale discrimination is included to identify the authenticity and category of face images jointly, which improves the judgment accuracy.On this basis, an anti-loss function that integrates relative discrimination and hinge loss is designed to enhance the relationship between the true and false image domains.The experimental results on the CelebA dataset demonstrate that, compared to mainstream facial attribute migration across multiple image domains methods such as StarGAN and STGAN, this method can establish a more accurate multiple image domains mapping relationship, improve the quality of the migrated image, and increase the diversity of the expression of the migrated image.

Key words: Generative Adversarial Network(GAN), attribute migration across multiple image domains, adaptive instance normalization, Selective Transfer Units(STU), relative attributes label, domain controller, dual-scale discrimination

中图分类号:

TP391

林泓, 陈壮源, 任硕, 李琳, 李玉强. 选择性传输与铰链对抗的多图像域人脸属性迁移[J]. 计算机工程, 2022, 48(4): 179-190.

LIN Hong, CHEN Zhuangyuan, REN Shuo, LI Lin, LI Yuqiang. Facial Attribute Migration Across Multiple Image Domains Using Selective Transfer and Hinge Adversary[J]. Computer Engineering, 2022, 48(4): 179-190.

http://www.ecice06.com/CN/Y2022/V48/I4/179

图/表 18

20230131201547

20230131201550

20230131201553

20230131201557

20230131201600

20230131201604

20230131201607

20230131201610

20230131201613

20230131201616

20230131201620

20230131201623

20230131201626

20230131201629

20230131201634

20230131201638

20230131201642

20230131201645

参考文献

[1] XIAO T, HONG J, MA J.Elegant:exchanging latent encodings with GAN for transferring multiple face attributes[C]//Proceedings of the European Conference on Computer Vision.Berlin, Germany:Springer, 2018:172-187.
[2] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al.Generative adversarial networks[J].Communications of the ACM, 2020, 63(11):139-144.
[3] ISOLA P, ZHU J Y, ZHOU T, et al.Image-to-image translation with conditional adversarial networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:5967-5976.
[4] 曹仰杰, 贾丽丽, 陈永霞, 等.生成式对抗网络及其计算机视觉应用研究综述[J].中国图象图形学报, 2018, 23(10):1433-1449. CAO Y J, JIA L L, CHEN Y X, et al.Review of computer vision based on generative adversarial networks[J].Journal of Image and Graphics, 2018, 23(10):1433-1449.(in Chinese)
[5] HUANG H, YU P S, WANG C.An introduction to image synthesis with generative adversarial nets[EB/OL].[2020-12-16].https://arxiv.org/pdf/1803.04469.pdf.
[6] UPCHURCH P, GARDNER J, PLEISS G, et al.Deep feature interpolation for image content changes[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:6090-6099.
[7] ZHU J Y, PARK T, ISOLA P, et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2017:2242-2251.
[8] 林泓, 任硕, 杨益, 等.融合自注意力机制和相对鉴别的无监督图像翻译[J].自动化学报, 2021, 47(9):2226-2237. LIN H, REN S, YANG Y, et al.Unsupervised image-to-image translation with self-attention and relativistic discriminator adversarial networks[J].Acta Automatica Sinice, 2021, 47(9):2226-2237(in Chinese).
[9] PERARNAU G, VAN DE WEIJER J, RADUCANU B, et al.Invertible conditional gans for image editing[EB/OL].[2021-01-16].https://arxiv.org/pdf/1611.06355.pdf.
[10] MIRZA M, OSINDERO S.Conditional generative adversarial nets[EB/OL].[2021-12-16].https://arxiv.org/pdf/1411.1784.pdf.
[11] HUANG X, LIU M Y, BELONGIE S, et al.Multimodal unsupervised image-to-image translation[C]//Proceedings of European Conference on Computer Vision.Munich, Germany:Springer, 2018:179-196.
[12] LIU M Y, BREUEL T, KAUTZ J.Unsupervised image-to-image translation networks[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.New York, USA:ACM Press, 2017:700-708.
[13] CHOI Y, CHOI M, KIM M, et al.StarGAN:unified generative adversarial networks for multi-domain image-to-image translation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:8789-8797.
[14] CHOI Y, UH Y, YOO J, et al.StarGAN v2:diverse image synthesis for multiple domains[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:8188-8197.
[15] LIU M, DING Y, XIA M, et al.STGAN:a unified selective transfer network for arbitrary image attribute editing[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:3668-3677.
[16] HE Z, ZUO W, KAN M, et al.AttGAN:facial attribute editing by only changing what you want[J].IEEE Transactions on Image Processing, 2019, 28(11):5464-5478.
[17] CHUNG J, GULCEHRE C, CHO K H, et al.Empirical evaluation of gated recurrent neural networks on sequence modeling[EB/OL].[2021-12-16].https://arxiv.org/pdf/1412.3555.pdf.
[18] WU P W, LIN Y J, CHANG C H, et al.Relgan:multi-domain image-to-image translation via relative attributes[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2019:5914-5922.
[19] HUANG X, BELONGIE S.Arbitrary style transfer in real-time with adaptive instance normalization[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2017:1510-1519.
[20] JOLICOEUR-MARTINEAU A.The relativistic discriminator:a key element missing from standard GAN[EB/OL].[2020-12-16].https://arxiv.org/pdf/1807.00734.pdf.
[21] JOLICOEUR-MARTINEAU A, MITLIAGKAS I.Connections between support vector machines, wasserstein distance and gradient-penalty GANs[EB/OL].[2020-12-16].https://arxiv.org/pdf/1910.06922.pdf.
[22] LIU Z, LUO P, WANG X, et al.Deep learning face attributes in the wild[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2015:3730-3738.
[23] HEUSEL M, RAMSAUER H, UNTERTHINER T, et al.GANs trained by a two time-scale update rule converge to a local nash equilibrium[EB/OL].[2020-12-15].https://arxiv.org/pdf/1706.08500.pdf.
[24] CHOLLET F.Xception:deep learning with depthwise separable convolutions[C]//Proceedings of IEEE conference on computer vision and pattern recognition.Washington D.C., USA:IEEE Press, 2017:1800-1807.
[25] SZEGEDY C, VANHOUCKE V, IOFFE S, et al.Rethinking the inception architecture for computer vision[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:2818-2826.

选择文件类型/文献管理软件名称

选择包含的内容