基于多尺度双流网络的深度伪造检测方法

doi:10.19678/j.issn.1000-3428.0069948

摘要/Abstract

摘要：

人脸深度伪造技术的滥用给社会和个人带来了极大的安全隐患, 因此深度伪造检测技术已成为当今研究的热点。目前基于深度学习的伪造检测技术在高质量(HQ)数据集上效果较好, 但在低质量(LQ)数据集和跨数据集上的检测效果不佳。为提升深度伪造检测的泛化性, 提出一种基于多尺度双流网络(MSDSnet)的深度伪造检测方法。MSDSnet输入分为空域特征流和高频噪声特征流, 首先采用多尺度融合(MSF)模块捕获不同情况下图像在空域被篡改的粗粒度人脸特征和伪造图像的细粒度高频噪声特征信息, 然后通过MSF模块将空域流和高频噪声流的双流特征充分融合, 由多模态交互注意力(MIA)模块进一步交互以充分学习双流特征信息, 最后利用FcaNet(Frequency Channel Attention Network)获取伪造人脸特征的全局信息并完成检测分类。实验结果表明, 该方法在HQ数据集Celeb-DF v2上的准确率为98.54%, 在LQ数据集FaceForensics++上的准确率为93.11%, 同时在跨数据集上的实验效果也优于其他同类方法。

关键词: 深度伪造检测, 双流网络, 多尺度融合, 多模态交互注意力, 高频噪声

Abstract:

DeepFake-enabled abuse of face forgery technology has given rise to considerable security risks to society and individuals; therefore, DeepFake detection has become a hot topic of research. Current deep learning-based forgery detection techniques exhibit good results on High-Quality (HQ) datasets but show poor performance on Low-Quality (LQ) datasets and across different datasets. To improve the generalization of DeepFake detection performance, this paper proposes a Multi-Scale Dual-Stream Network (MSDSnet) for DeepFake detection. The network input is divided into a spatial-domain feature stream and a high-frequency noise feature flow. First, the Multi-Scale Fusion (MSF) module is used to capture the tampered coarse-grain facial features from images and fine-grained high-frequency noise information from forged images in different situations. The network fully integrates the dual-stream features of the spatial-domain feature stream and high-frequency noise feature flow through the MSF module. The Multi-modal Interaction Attention (MIA) module further interacts to learn the dual-stream information. Finally, a Frequency Channel Attention Network (FcaNet) is used to obtain the global information of the forged face features for complete detection and classification. Experimental results show that the proposed method achieves 98.54% accuracy on the HQ dataset Celeb-DF v2 and 93.11% on the LQ dataset FaceForensics++. Simultaneously, the experimental results are better than those obtained using other methods in cross-dataset experiments.

Key words: DeepFake detection, dual-stream network, Multi-Scale Fusion (MSF), Multi-modal Interactive Attention (MIA), high-frequency noise

蒋翠玲, 程梓源, 俞新贵, 万永菁. 基于多尺度双流网络的深度伪造检测方法[J]. 计算机工程, 2026, 52(1): 242-253.

JIANG Cuiling, CHENG Ziyuan, YU Xingui, WAN Yongjing. DeepFake Detection Method Based on Multi-Scale Dual-Stream Network[J]. Computer Engineering, 2026, 52(1): 242-253.

https://www.ecice06.com/CN/Y2026/V52/I1/242

图/表 16

图1 MSDSnet结构

Fig.1 Structure of MSDSnet

图2 MSF模块结构

Fig.2 Structure of MSF module

图3 Bayar可视化

Fig.3 Bayar visualization

图4 MIA模块

Fig.4 MIA Module

图5 MIA功能的可视化

Fig.5 MIA function visualization

图6 FF++数据集的样本

Fig.6 Samples of FF++ dataset

图7 Celeb-DF v2数据集的样本

Fig.7 Samples of Celeb-DF v2 dataset

图8 Celeb-DF v2数据集上消融实验的AUC值

Fig.8 AUC value of ablation experiment on Celeb-DF v2 dataset

参考文献 32

1	杨睿, 胡心如, 黄卓超, 等. 深度网络生成式伪造人脸检测方法研究综述. 计算机辅助设计与图形学学报, 2024, 30 (10): 1491- 1510.
	YANG R , HU X R , HUANG Z C , et al. Review of deep network generative fake face detection methods. Journal of Computer-Aided Design & Computer Graphics, 2024, 36 (10): 1491- 1510.
2	祝恺蔓, 徐文博, 卢伟, 等. 多关键帧特征交互的人脸篡改视频检测. 中国图象图形学报, 2022, 27 (1): 188- 202.
	ZHU K M , XU W B , LU W , et al. DeepFake video detection with feature interaction amongst key frames. Journal of Image and Graphics, 2022, 27 (1): 188- 202.
3	TOLOSANA R , VERA-RODRIGUEZ R , FIERREZ J , et al. DeepFakes and beyond: a survey of face manipulation and fake detection. Information Fusion, 2020, 64, 131- 148. doi: 10.1016/j.inffus.2020.06.014
4	GUPTA P, CHUGH K, DHALL A, et al. The eyes know it: FakeET—an eye-tracking database to understand DeepFake perception[C]//Proceedings of the 2020 International Conference on Multimodal Interaction. New York, USA: ACM Press, 2020: 519-527.
5	QI H, GUO Q, XU J F, et al. DeepRhythm: exposing DeepFakes with attentional visual heartbeat rhythms[C]//Proceedings of the 28th ACM International Conference on Multimedia. New York, USA: ACM Press, 2020: 4318-4327.
6	ROSSLER A, COZZOLINO D, VERDOLIVA L, et al. FaceForensics++: learning to detect manipulated facial images[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Washington D.C., USA: IEEE Press, 2019: 1-11.
7	ZHAO H Q, WEI T Y, ZHOU W B, et al. Multi-attentional DeepFake detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2021: 2185-2194.
8	DURALL R, KEUPER M, PFREUNDT F J, et al. Unmasking DeepFakes with simple features[EB/OL]. [2024-05-11]. https://arxiv.org/abs/1911.00686.
9	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of ECCV'18. Berlin, Germany: Springer, 2018: 3-19.
10	QIN Z Q, ZHANG P Y, WU F, et al. FcaNet: frequency channel attention networks[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Washington D.C., USA: IEEE Press, 2021: 763-772.
11	QIAN Y Y , YIN G J , SHENG L , et al. Thinking in frequency: face forgery detection by mining frequency-aware clues. Berlin, Germany: Springer, 2020.
12	LIANG Y F , WANG M M , JIN Y N , et al. Hierarchical supervisions with two-stream network for DeepFake detection. Pattern Recognition Letters, 2023, 172, 121- 127. doi: 10.1016/j.patrec.2023.05.029
13	NGUYEN H H, YAMAGISHI J, ECHIZEN I. Capsule-forensics: using capsule networks to detect forged images and videos[C]//Proceedings of the 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Washington D.C., USA: IEEE Press, 2019: 2307-2311.
14	DING X L, ZHU W J, ZHANG D Y. DeepFake videos detection via spatiotemporal inconsistency learning and interactive fusion[C]//Proceedings of the 19th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON). Washington D.C., USA: IEEE Press, 2022: 425-433.
15	GAO J , XIA Z Q , MARCIALIS G L , et al. DeepFake detection based on high-frequency enhancement network for highly compressed content. Expert Systems with Applications, 2024, 249, 123732. doi: 10.1016/j.eswa.2024.123732
16	WANG B , WU X H , WANG F , et al. Spatial-frequency feature fusion based DeepFake detection through knowledge distillation. Engineering Applications of Artificial Intelligence, 2024, 133, 108341. doi: 10.1016/j.engappai.2024.108341
17	WANG B , HUANG L Q , HUANG T Q , et al. Two-stream Xception structure based on feature fusion for DeepFake detection. International Journal of Computational Intelligence Systems, 2023, 16 (1): 134. doi: 10.1007/s44196-023-00312-8
18	AMIN M A , HU Y J , LI C T , et al. DeepFake detection based on cross-domain local characteristic analysis with multi-domain Transformer. Alexandria Engineering Journal, 2024, 91, 592- 609. doi: 10.1016/j.aej.2024.02.035
19	JIN X , WU N , JIANG Q , et al. A dual descriptor combined with frequency domain reconstruction learning for face forgery detection in DeepFake videos. Forensic Science International: Digital Investigation, 2024, 49, 301747. doi: 10.1016/j.fsidi.2024.301747
20	BAYAR B , STAMM M C . Constrained convolutional neural networks: a new approach towards general purpose image manipulation detection. IEEE Transactions on Information Forensics and Security, 2018, 13 (11): 2691- 2706. doi: 10.1109/TIFS.2018.2825953
21	GUO Z Q , WANG L J , YANG W Z , et al. LDFnet: lightweight dynamic fusion network for face forgery detection by integrating local artifacts and global texture information. IEEE Transactions on Circuits and Systems for Video Technology, 2024, 34 (2): 1255- 1265. doi: 10.1109/TCSVT.2023.3289147
22	DONG C B , CHEN X R , HU R H , et al. MVSS-Net: multi-view multi-scale supervised networks for image manipulation detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45 (3): 3539- 3553. doi: 10.1109/TPAMI.2022.3180556
23	GUO Z Q , JIA Z H , WANG L J , et al. Constructing new backbone networks via space-frequency interactive convolution for DeepFake detection. IEEE Transactions on Information Forensics and Security, 2023, 19, 401- 413.
24	COCCOMINI D A , MESSINA N , GENNARO C , et al. Combining EfficientNet and vision Transformers for video DeepFake detection. Berlin, Germany: Springer, 2022.
25	DONG F K , ZOU X Q , WANG J H , et al. Contrastive learning-based general DeepFake detection with multi-scale RGB frequency clues. Journal of King Saud University (Computer and Information Sciences), 2023, 35 (4): 90- 99. doi: 10.1016/j.jksuci.2023.03.005
26	HALIASSOS A, VOUGIOUKAS K, PETRIDIS S, et al. Lips don't lie: a generalisable and robust approach to face forgery detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2021: 5039-5049.
27	GU Z H, CHEN Y, YAO T P, et al. Spatiotemporal inconsistency learning for DeepFake video detection[C]//Proceedings of the 29th ACM International Conference on Multimedia. New York, USA: ACM Press, 2021: 3473-3481.
28	AFCHAR D, NOZICK V, YAMAGISHI J, et al. MesoNet: a compact facial video forgery detection network[C]//Proceedings of the IEEE International Workshop on Information Forensics and Security (WIFS). Washington D.C., USA: IEEE Press, 2018: 1-7.
29	CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2017: 1800-1807.
30	PANG G L , ZHANG B P , TENG Z , et al. MRE-Net: multi-rate excitation network for DeepFake video detection. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33 (8): 3663- 3676. doi: 10.1109/TCSVT.2023.3239607
31	YANG J C , XIAO S , LI A Y , et al. MSTA-Net: forgery detection by generating manipulation trace based on multi-scale self-texture attention. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32 (7): 4854- 4866. doi: 10.1109/TCSVT.2021.3133859
32	LI Y, LV S. Exposing DeepFake videos by detecting face warping artifacts[EB/OL]. [2024-05-11]. https://arxiv.org/abs/1811.00656v2.

[1]	杨郅树, 梁佳楠, 曹永军, 钟震宇, 何永伦. 基于局部分离与多尺度融合的图像超分辨率重建[J]. 计算机工程, 2024, 50(7): 314-323.
[2]	胡庆. 多尺度融合与双输出U-Net网络的行人重识别[J]. 计算机工程, 2024, 50(6): 102-109.
[3]	孙帮勇, 马铭, 于涛. 基于区域特征强化的多尺度伪装目标检测方法[J]. 计算机工程, 2024, 50(5): 209-219.
[4]	谢新林, 尹东旭, 张涛源, 谢刚. 基于注意力机制的多尺度融合人群计数算法[J]. 计算机工程, 2024, 50(3): 290-297.
[5]	董丰恺, 邹晓强, 王佳慧, 马利民, 杨文元, 刘熙尧. 基于帧内-帧间自融合的双流泛化人脸伪造检测方法[J]. 计算机工程, 2024, 50(10): 185-195.
[6]	兰红, 王惠钊. 结合轻量化与多尺度融合的交通标志检测算法[J]. 计算机工程, 2024, 50(10): 381-392.
[7]	周静, 钟原, 李平, 杨毅, 马立泰, 张涛. 用于颈椎MRI分割的多尺度特征融合注意力网络模型[J]. 计算机工程, 2023, 49(10): 298-304.
[8]	吴旭, 刘翔, 赵静文. 一种轻量级多尺度融合的图像篡改检测算法[J]. 计算机工程, 2022, 48(2): 224-229,236.
[9]	古平, 邱嘉涛, 罗长江, 张志鹏. 基于目标时空上下文融合的视频异常检测算法[J]. 计算机工程, 2022, 48(10): 169-175.
[10]	耿鹏志, 樊红兴, 张翌阳, 唐云祁. 基于篡改伪影的深度伪造检测方法[J]. 计算机工程, 2021, 47(12): 156-162.
[11]	张聪聪, 何宁, 孙琪翔, 尹晓杰. 基于注意力机制的3D DenseNet人体动作识别方法[J]. 计算机工程, 2021, 47(11): 313-320.
[12]	李祥兵, 陈炼. 基于改进Faster?RCNN的自然场景人脸检测[J]. 计算机工程, 2021, 47(1): 210-216.
[13]	王金鹤, 苏翠丽, 孟凡云, 车志龙, 谭浩, 张楠. 基于非对称空间金字塔池化的立体匹配网络[J]. 计算机工程, 2020, 46(7): 228-234,242.
[14]	郑秋梅, 王璐璐, 王风华. 基于改进卷积神经网络的交通场景小目标检测[J]. 计算机工程, 2020, 46(6): 26-33.
[15]	赵亚男, 吴黎明, 陈琦. 基于多尺度融合SSD的小目标检测算法[J]. 计算机工程, 2020, 46(1): 247-254.

选择文件类型/文献管理软件名称

选择包含的内容