基于多通道注意力机制的人脸替换鉴别

doi:10.19678/j.issn.1000-3428.0060739

摘要/Abstract

摘要： 基于深度学习的人脸替换技术取得快速发展，但由DeepFake自动生成的人脸替换图片有可能危害人们的隐私安全。针对DeepFake图片鉴别问题，建立一种基于多通道注意力机制的深度学习鉴别网络模型。将Xception网络作为基础特征提取器，在多通道注意力模块中通过矩阵相乘的思想融合全局和局部的注意力表示，以减少重要信息损失。设计损失函数时添加中心损失，从而提高特征区分度。在训练过程中利用注意力图来引导训练图像的裁剪和去除，以达到数据增强的目的。实验结果表明，相比Xception、B4Att方法，在FaceForensics++数据集上该网络模型对DeepFake的检测精度分别提高0.77和0.45个百分点，在Celeb-DF数据集上分别提高5.30和4.68个百分点。

关键词: 人脸替换, 多通道注意力机制, 图片鉴别, Xception网络, 深度学习

Abstract: In recent years, the face swapping technology based on deep learning has developed rapidly, but the images of face swapping automatically generated by DeepFake may endanger people's privacy and security.To detection the DeepFake images, a deep learning-based network model using the multi-channel attention mechanism is designed.The model employs Xception network as the basic feature extractor, and the idea of matrix multiplication is used to combine global and local attention representations in the multi-channel attention module to reduce information loss.Then a center loss is introduced into the design of the loss function, so the feature discrimination can be improved.At the same time, in the training process, the attention map is used to guide the cropping and erasing of training images to achieve data enhancement.The experimental results show that the detection accuracy of the network model for DeepFake is 0.77 percentage points higher than that of Xception, and 0.45 percentage points higher than that of B4Att on the FaceForensics++ dataset.The accuracy of the proposed model is 5.30 percentage points higher than that of Xception, and 4.68 percentage points higher than that of B4Att on Celeb-DF dataset.

Key words: face swapping, multi-channel attention mechanism, image detection, Xception network, deep learning

中图分类号:

TP391

武茜, 贾世杰. 基于多通道注意力机制的人脸替换鉴别[J]. 计算机工程, 2022, 48(2): 180-185,193.

WU Qian, JIA Shijie. Face Swapping Detection Based on Multi-Channel Attention Mechanism[J]. Computer Engineering, 2022, 48(2): 180-185,193.

http://www.ecice06.com/CN/Y2022/V48/I2/180

图/表 7

20220301124012

20220301124016

20220301124020

20220301124024

20220301124028

20220301124032

20220301124036

参考文献

[1] 暴雨轩, 芦天亮, 杜彦辉.深度伪造视频检测技术综述[J].计算机科学, 2020, 47(9):289-298. BAO Y X, LU T L, DU Y H.Overview of DeepFake video detection technology[J].Computer Science, 2020, 47(9):289-298.(in Chinese)
[2] MIRSKY Y, LEE W.The creation and detection of DeepFakes:a survey[J].ACM Computing Surveys, 2021, 54(1):1-41.
[3] GÜERA D, DELP E J.DeepFake video detection using recurrent neural networks[C]//Proceedings of 2018 IEEE International Conference on Advanced Video and Signal Based Surveillance.Washington D.C., USA:IEEE Press, 2018:1-6.
[4] LI Y, CHANG M C, LÜ S.In ictu oculi:exposing AI created fake videos by detecting eye blinking[C]//Proceedings of 2018 IEEE International Workshop on Information Forensics and Security.Washington D.C., USA:IEEE Press, 2018:1-7.
[5] DONAHUE J, ANNE HENDRICKS L, GUADARRAMA S, et al.Long-term recurrent convolutional networks for visual recognition and description[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2015:2625-2634.
[6] 张怡暄, 李根, 曹纭, 等.基于帧间差异的人脸篡改视频检测方法[J].信息安全学报, 2020, 5(2):49-72. ZHANG Y X, LI G, CAO Y, et al.A method for detecting human-face-tampered videos based on interframe difference[J].Journal of Cyber Security, 2020, 5(2):49-72.(in Chinese)
[7] 陈鹏, 梁涛, 刘锦, 等.融合全局时序和局部空间特征的伪造人脸视频检测方法[J].信息安全学报, 2020, 5(2):73-83. CHEN P, LIANG T, LIU J, et al.Forged facial video detection based on global temporal and local spatial feature[J].Journal of Cyber Security, 2020, 5(2):73-83.(in Chinese)
[8] LI X, LANG Y, CHEN Y, et al.Sharp multiple instance learning for DeepFake video detection[C]//Proceedings of the 28th ACM International Conference on Multimedia.New York, USA:ACM Press, 2020:1864-1872.
[9] TOLOSANA R, VERA-RODRIGUEZ R, FIERREZ J, et al.DeepFakes and beyond:a survey of face manipulation and fake detection[J].Information Fusion, 2020, 64:143-184.
[10] YANG X, LI Y, LÜ S.Exposing deep fakes using inconsistent head poses[C]//Proceedings of 2019 IEEE International Conference on Acoustics, Speech and Signal Processing.Washington D.C., USA:IEEE Press, 2019:8261-8265.
[11] AFCHAR D, NOZICK V, YAMAGISHI J, et al.Mesonet:a compact facial video forgery detection network[C]//Proceedings of 2018 IEEE International Workshop on Information Forensics and Security.Washington D.C., USA:IEEE Press, 2018:1-7.
[12] LI Y, LÜ S.Exposing DeepFake videos by detecting face warping artifacts[EB/OL].[2020-12-26].https://arxiv.org/pdf/1811.00656.pdf.
[13] NGUYEN H H, YAMAGISHI J, ECHIZEN I.Capsule-forensics:using capsule networks to detect forged images and videos[C]//Proceedings of 2019 IEEE International Conference on Acoustics, Speech and Signal Processing.Washington D.C., USA:IEEE Press, 2019:2307-2311.
[14] ROSSLER A, COZZOLINO D, VERDOLIVA L, et al.FaceForensics++:learning to detect manipulated facial images[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2019:1-11.
[15] BONETTINI N, CANNAS E D, MANDELLI S, et al.Video face manipulation detection through ensemble of CNNs[EB/OL].[2020-12-26].https://arxiv.org/pdf/2004. 07676.pdf.
[16] HE K, ZHANG X, REN S, et al.Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:770-778.
[17] CHOLLET F.Xception:deep learning with depthwise separable convolutions[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:1251-1258.
[18] WOO S, PARK J, LEE J Y, et al.CBAM:convolutional block attention module[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2018:3-19.
[19] WEN Y, ZHANG K, LI Z, et al.A discriminative feature learning approach for deep face recognition[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2016:499-515.
[20] HU T, QI H, HUANG Q, et al.See better before looking closer:weakly supervised data augmentation network for fine-grained visual classification[EB/OL].[2020-12-26].https://arxiv.org/pdf/1901.09891v2.pdf.
[21] LI Y, YANG X, SUN P, et al.Celeb-DF:a large-scale challenging dataset for deepfake forensics[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:3207-3216.
[22] ZHANG K, ZHANG Z, LI Z, et al.Joint face detection and alignment using multitask cascaded convolutional networks[J].IEEE Signal Processing Letters, 2016, 23(10):1499-1503.
[23] DOLHANSKY B, BITTON J, PFLAUM B, et al.The deepfake detection challenge(DFDC) dataset[EB/OL].[2020-12-26].https://arxiv.org/pdf/2006.07397.pdf.
[24] SIMONYAN K, ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[EB/OL].[2020-12-26].https://arxiv.org/pdf/1409.1556.pdf.
[25] SZEGEDY C, VANHOUCKE V, IOFFE S, et al.Rethinking the Inception architecture for computer vision[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:2818-2826.

选择文件类型/文献管理软件名称

选择包含的内容