基于生成对抗网络的深度伪造跨模型防御方法

doi:10.19678/j.issn.1000-3428.0068106

摘要/Abstract

摘要：

为了降低深度伪造技术滥用带来的社会风险, 提出一种基于生成对抗网络的主动防御深度伪造方法, 通过在原始图像上增加微弱扰动制作对抗样本, 使多个伪造模型输出产生明显失真。提出模型由对抗样本生成模块和对抗样本优化模块组成。对抗样本生成模块包括生成器和鉴别器, 生成器在接收原始图像生成扰动后, 通过对抗训练约束扰动的空间分布, 降低扰动的视觉感知, 提高对抗样本的真实性; 对抗样本优化模块由基础对抗水印、深度伪造模型和鉴别器等组成, 通过模拟黑盒场景攻击多个深度伪造模型, 提高对抗样本的攻击性和迁移性。在常用深度伪造数据集CelebA和LFW上进行训练和测试, 实验结果表明, 相比现有主动防御方法, 提出方法在实现跨模型主动防御的基础上, 防御成功率达到85%以上, 并且对抗样本生成效率比传统算法提高20~30倍。

关键词: 深度伪造, 对抗样本, 主动防御, 生成对抗网络, 迁移性

Abstract:

To reduce social risks caused by the abuse of deepfake technology, an active defense method against deep forgery based on a Generative Adversarial Network (GAN) is proposed. Adversarial samples are created by adding imperceptible perturbation to original images, which significantly distorts the output of multiple forgery models. The proposed model comprises an adversarial sample generation module and an adversarial sample optimization module. The adversarial-sample generation module includes a generator and discriminator. After the generator receives an original image to generate a perturbation, the spatial distribution of the perturbation is constrained through adversarial training. By reducing the visual perception of the perturbation, the authenticity of the adversarial sample is improved. The adversarial sample optimization module comprises basic adversarial watermarking, deep forgery models, and discriminators. This module simulates black-box scenarios to attack multiple deep forgery models, thereby improving the attack and migration of adversarial samples. Training and testing are conducted on commonly used deepfake datasets Celebfaces Attributes (CelebA) and Labeled Faces in the Wild (LFW). Experimental results show that compared with existing active defense methods, the proposed method achieves a defense success rate exceeding 85% based on the cross-model active defense method and generates adversarial samples. Additionally, the method improves efficiency by 20-30 times compared with those of conventional algorithms.

Key words: deepfake, adversarial samples, active defense, Generative Adversarial Network(GAN), generalization

戴磊, 曹林, 郭亚男, 张帆, 杜康宁. 基于生成对抗网络的深度伪造跨模型防御方法[J]. 计算机工程, 2024, 50(10): 100-109.

DAI Lei, CAO Lin, GUO Yanan, ZHANG Fan, DU Kangning. Deepfake Cross-Model Defense Method Based on Generative Adversarial Network[J]. Computer Engineering, 2024, 50(10): 100-109.

https://www.ecice06.com/CN/Y2024/V50/I10/100

图/表 12

图1 跨模型攻击深度伪造示意图

Fig.1 　Schematic diagram of cross-model attacks on deepfake

图2 　DDGAN模型框架

Fig.2 　The architecture of DDGAN model

图3 攻击深度伪造模型的可视化结果

Fig.3 　Visualized results of attacking deepfake models

图4 　LFW数据集对抗攻击结果

Fig.4 　Results of adversarial attack on LFW dataset

图5 扰动边界值对攻击性能的影响

Fig.5 　Influence of perturbation boundary values on attack performance

图6 不同扰动边界值对图像质量的影响

Fig.6 　Influence of different perturbation boundary values on image quality

参考文献 35

1	ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 2242-2251.
2	CHOI Y, CHOI M, KIM M, et al. StarGAN: unified generative adversarial networks for multi-domain image-to-image translation[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 8789-8797.
3	MCCLOSKEY S, CHEN C, YU J Y. Focus manipulation detection via photometric histogram analysis[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 1674-1682.
4	耿鹏志, 樊红兴, 张翌阳, 等. 基于篡改伪影的深度伪造检测方法. 计算机工程, 2021, 47 (12): 156- 162.
	GENG P Z, FAN H X, ZHANG Y Y, et al. Deepfake detection method based on tampering artifacts. Computer Engineering, 2021, 47 (12): 156- 162.
5	YANG X, LI Y Z, LYU S W. Exposing deep fakes using inconsistent head poses[C]//Proceedings of 2019 IEEE International Conference on Acoustics, Speech and Signal Processing. Washington D. C., USA: IEEE Press, 2019: 8261-8265.
6	LI Y, LYU S. Exposing deepfake videos by detecting face warping artifacts[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 46-52.
7	YU N, SKRIPNIUK V, ABDELNABI S, et al. Artificial fingerprinting for generative models: rooting deepfake attribution in training data[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2021: 14428-14437.
8	NEEKHARA P, HUSSAIN H, ZHANG X Q, et al. FaceSigns: semi-fragile neural watermarks for media authentication and countering deepfakes[EB/OL]. [2023-06-10]. https://arxiv.org/pdf/2204.01960v1.
9	RUIZ N, BARGAL S A, SCLAROFF S. Disrupting deepfakes: adversarial attacks against conditional image translation networks and facial manipulation systems[M]. Berlin, Germany: Springer, 2020.
10	DONG J H, XIE X H. Visually maintained image disturbance against deepfake face swapping[C]//Proceedings of IEEE International Conference on Multimedia and Expo. Washington D. C., USA: IEEE Press, 2021: 1-6.
11	QIU H X, DU Y H, LU T L. The framework of cross-domain and model adversarial attack against deepfake. Future Internet, 2022, 14 (2): 46. doi: 10.3390/fi14020046
12	HUANG H, WANG Y T, CHEN Z Y, et al. CMUA-watermark: a cross-model universal adversarial watermark for combating deepfakes. Artificial Intelligence, 2022, 36 (1): 989- 997.
13	KORSHUNOVA I, SHI W Z, DAMBRE J, et al. Fast face-swap using convolutional neural networks[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 3677-3685.
14	NIRKIN Y, KELLER Y, HASSNER T. FSGAN: subject agnostic face swapping and reenactment[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 7184-7193.
15	HUANG R, ZHANG S, LI T Y, et al. Beyond face rotation: global and local perception GAN for photorealistic and identity preserving frontal view synthesis[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 2439-2448.
16	ZHU Y H, LI Q, WANG J, et al. One shot face swapping on megapixels[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 4832-4842.
17	GOODFELLOW I J, SHLENS J, SZEGEDY C. Explaining and harnessing adversarial examples[C]//Proceedings of the 3rd IEEE International Conference on Learning Representations. Washington D. C., USA: IEEE Press, 2015: 2931-2944.
18	MADRY A, MAKELOV A, SCHMIDT L, et al. Towards deep learning models resistant to adversarial attacks[C]// Proceedings of IEEE International Conference on Learning Representations. Washington D. C., USA: IEEE Press, 2018: 316-327.
19	DONG Y P, LIAO F Z, PANG T Y, et al. Boosting adversarial attacks with momentum[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 185-9193.
20	陈晓楠, 胡建敏, 张本俊, 等. 基于模型间迁移性的黑盒对抗攻击起点提升方法. 计算机工程, 2021, 47 (8): 162- 169.
	CHEN X N, HU J M, ZHANG B J, et al. Black box adversarial attack starting point promotion method based on mobility between models. Computer Engineering, 2021, 47 (8): 162- 169.
21	SEGALIS E. Disrupting deepfakes with an adversarial attack that survives training[EB/OL]. [2023-06-10].https://arxiv.org/pdf/2006.12247v1.
22	BALUJA S, FISCHER I. Learning to attack: adversarial transformation networks. Artificial Intelligence, 2018, 32 (1): 3719- 3721.
23	BOSE A J, AARABI P. Adversarial attacks on face detectors using neural net based constrained optimization[C]//Proceedings of the 20th IEEE International Workshop on Multimedia Signal Processing. Washington D. C., USA: IEEE Press, 2018: 1-6.
24	XIAO C W, LI B, ZHU J Y, et al. Generating adversarial examples with adversarial networks[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence. Washington D. C., USA: IEEE Press, 2018: 1207-1219.
25	JANDIAL S, MANGLA P, VARSHNEY S, et al. AdvGAN++: harnessing latent layers for adversary generation[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 2045-2048.
26	WANG X, HE K, HOPCROFT J E. AT-GAN: a generative attack model for adversarial transferring on generative adversarial nets[EB/OL]. [2023-06-10]. https://arxiv.org/pdf/1904.07793v1.
27	ZHAO G P, ZHANG M Y, LIU J J, et al. Unsupervised adversarial attacks on deep feature-based retrieval with GAN[EB/OL]. [2023-06-10]. https://arxiv.org/pdf/1907.05793v1.
28	LIN Z L, SHI Y, XUE Z. IDSGAN: generative adversarial networks for attack generation against intrusion detection[M]. Berlin, Germany: Springer, 2022.
29	TANG H, XU D, SEBE N, et al. Attention-guided generative adversarial networks for unsupervised image-to-image translation[C]//Proceedings of IEEE International Joint Conference on Neural Networks. Washington D. C., USA: IEEE Press, 2019: 1-8.
30	LI X Y, ZHANG S C, HU J, et al. Image-to-image translation via hierarchical style disentanglement[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 8639-8648.
31	SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 2818-2826.
32	KURAKIN A, GOODFELLOW I J, BENGIO S. Adversarial examples in the physical world[M]. [S. 1. ]: Chapman and Hall/CRC, 2018.
33	XIE C H, ZHANG Z S, ZHOU Y Y, et al. Improving transferability of adversarial examples with input diversity[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 2730-2739.
34	CROCE F, HEIN M. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks[C]//Proceedings of the 37th International Conference on Machine Learning. Washington D. C., USA: IEEE Press, 2022: 2206-2216.
35	WANG X Y, HUANG J J, MA S Q, et al. DeepFake disrupter: the detector of Deepfake is my friend[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2022: 14900-14909.

[1]	王夙喆, 张雪英, 陈晓玉, 李凤莲, 吴泽林. 基于有效注意力和GAN结合的脑卒中EEG增强算法[J]. 计算机工程, 2024, 50(8): 336-344.
[2]	张溢文, 蔡满春, 陈咏豪, 朱懿, 姚利峰. 融合空间特征的多尺度深度伪造检测方法[J]. 计算机工程, 2024, 50(7): 240-250.
[3]	胡庆. 多尺度融合与双输出U-Net网络的行人重识别[J]. 计算机工程, 2024, 50(6): 102-109.
[4]	张慧妍, 梁勇, 兰景宏, 赵强. 基于记忆模块与过滤式生成对抗网络的入侵检测方法[J]. 计算机工程, 2024, 50(6): 197-207.
[5]	李田芳, 普园媛, 赵征鹏, 徐丹, 钱文华. 基于CLIP和双空间自适应归一化的图像翻译[J]. 计算机工程, 2024, 50(5): 229-240.
[6]	刘帅威, 李智, 王国美, 张丽. 基于Transformer和GAN的对抗样本生成算法[J]. 计算机工程, 2024, 50(2): 180-187.
[7]	何银银, 胡静, 陈志泊, 张荣国. 融合门控变换机制和GAN的低光照图像增强方法[J]. 计算机工程, 2024, 50(2): 247-255.
[8]	李倩, 向海昀, 张玉婷, 甘昀, 廖浩德. 结合高斯滤波与MASK的G-MASK人脸对抗攻击[J]. 计算机工程, 2024, 50(2): 308-316.
[9]	张美美, 秦品乐, 柴锐, 曾建潮, 翟双姣, 闫俊义, 冯二燕. 面向急性缺血性脑卒中的CT生成MRI算法[J]. 计算机工程, 2024, 50(2): 317-326.
[10]	张学军, 席阿友, 加小红, 张斌, 李梅, 杜晓刚, 黄海燕. 基于深度学习的指纹室内定位对抗样本攻击研究[J]. 计算机工程, 2024, 50(10): 228-239.
[11]	罗偲, 李凯扬, 吴吉花, 任鹏. 基于对抗注意力机制的水下遮挡目标检测算法[J]. 计算机工程, 2024, 50(10): 313-321.
[12]	张玉婷, 向海昀, 李倩, 廖浩德. 基于稳定Adam和空间域变换的对抗样本生成算法[J]. 计算机工程, 2024, 50(1): 251-258.
[13]	李哲铭, 王晋东, 侯建中, 李伟, 张世华, 张恒巍. 基于显著区域优化的对抗样本攻击方法[J]. 计算机工程, 2023, 49(9): 246-255, 264.
[14]	杨燕燕, 谢明轩, 曹江峡, 王学宾, 柳厅文, 杜彦辉. 基于原型网络的中文分类模型对抗样本生成[J]. 计算机工程, 2023, 49(8): 54-62.
[15]	沈梦强, 于文年, 易黎, 宋南. 基于GAN的全时间尺度语音增强方法[J]. 计算机工程, 2023, 49(6): 115-122,130.

选择文件类型/文献管理软件名称

选择包含的内容