Author Login Chief Editor Login Reviewer Login Editor Login Remote Office

Computer Engineering

   

Temporal Forgery Localization in Audio-Visual Deepfake Detection: A Survey

  

  • Published:2026-10-21

音视频深度伪造检测时序定位综述

Abstract: Advances in generative artificial intelligence and localized content editing have extended deepfakes from whole-sequence synthesis to targeted modifications of speech and visual content. Video-level detection can obscure brief anomalies within predominantly authentic content, whereas frame-level detection often produces fragmented predictions. Neither paradigm directly identifies the onset, offset, or duration of manipulated segments. Deepfake temporal localization addresses this limitation by detecting suspicious segments in untrimmed audio, video, or audio-visual streams and returning verifiable temporal intervals. We searched Google Scholar, Web of Science, IEEE Xplore, the ACM Digital Library, CVF Open Access, and arXiv, and included studies on temporal localization methods and the associated datasets, evaluation protocols, and enabling techniques. Taking changes in detection evidence and localization objectives as the organizing thread, we trace the development of the field from local artifact detection and physiological or behavioral cue analysis, through joint audio-visual detection and forged-segment proposal generation, to boundary refinement. Methods are further organized by input modality into audio, visual, and audio-visual approaches. For multimodal methods, we establish a layered framework comprising forgery evidence, cross-modal interaction, and temporal segment decoding. Our analysis shows that current research is largely concerned with the reliability of forgery evidence and the precision of temporal boundaries. Contextual modeling and self-supervised learning can reduce dependence on specific generators and known manipulation types, but excessive feature fusion may suppress short-lived anomalies, while naturally occurring audio-visual asynchrony can trigger false alarms. Multimodal interaction exploits complementary acoustic and visual information, yet remains sensitive to modality quality, cross-modal correspondence, and missing inputs. Among peer-reviewed studies published to date, the best reported AP@0.50 scores on the official LAV-DF and AV-Deepfake1M test sets exceed 99% and 94%, respectively, whereas the corresponding AP@0.95 scores reach only approximately 70% and 51%. Existing methods can therefore identify the approximate extent of manipulated content, but accurate boundary recovery under strict overlap criteria remains difficult. Cross-dataset and cross-lingual evaluations further indicate that performance is affected by generators, languages, editing operations, segment duration, and biases introduced during dataset construction. Strong in-domain results therefore do not necessarily imply stable generalization to real-world media. Future research should strengthen dataset auditing and evaluation under realistic media dissemination conditions, establish more rigorous cross-dataset, cross-generator, and cross-lingual protocols, and incorporate modality-quality assessment, uncertainty estimation, and modality-specific evidence preservation to improve the reliability, interpretability, and practical deployability of temporal localization systems.

摘要: 随着生成式人工智能和局部内容编辑技术的发展,深度伪造已由整段合成扩展至局部语音和画面修改。视频级检测容易使短时异常被真实内容稀释,逐帧检测则常产生缺乏连续性的零散结果,二者均难以直接给出篡改片段的起止位置和持续时间。深度伪造时序定位因此需要从未裁剪的音频、视频或音视频序列中识别可疑片段,并输出可复核的时间区间。本文检索Google Scholar、Web of Science、IEEE Xplore、ACM Digital Library、CVF Open Access和arXiv,纳入时序定位方法及相关数据集、评价协议和支撑技术研究。本文以检测证据和定位目标的变化为主线,将技术演进概括为“局部伪影检测—生理与行为特征分析—音视频联合检测—伪造区间生成—伪造边界修正”,并按输入模态梳理音频、视觉和音视频多模态方法。针对多模态研究,进一步建立“伪造证据—模态交互—区间解码”的分层框架。综合分析表明,现有研究主要围绕证据可靠性和边界精度展开。上下文建模与自监督学习能够降低对特定生成器和已知伪造类型的依赖,但过度融合可能平滑短时异常,真实场景中的正常声画偏移也可能触发误报。多模态交互有助于利用音频与视觉信息的互补性,但其效果容易受到模态质量、对应关系及模态缺失的影响。在已正式发表的研究中,代表性多模态方法在LAV-DF和AV-Deepfake1M官方测试集上的AP@0.50最高分别超过99%和94%,而AP@0.95最高约为70%和51%,表明现有方法已能较好地检出伪造区间,但在严格重叠条件下准确恢复片段边界仍面临较大挑战。跨数据集和跨语言结果进一步表明,生成器、语言、编辑方式、片段长度及数据构造偏差均会影响模型性能,域内结果不能直接代表真实场景下的稳定泛化。未来研究应加强数据集审计和真实传播条件评测,完善跨数据集、跨生成器与跨语言评价,并结合模态质量评估、不确定性估计和分模态证据保留,提高定位结果的可靠性、可解释性与实际部署能力。