Author Login Chief Editor Login Reviewer Login Editor Login Remote Office

Computer Engineering

   

Robustness-Enhanced Proactive Defense Against Face Swapping in Black-Box Scenarios

  

  • Published:2026-07-15

面向黑盒场景的面部换脸鲁棒增强主动防御研究

Abstract: Generative face-swapping technology poses a severe threat to personal privacy. Existing proactive defense methods suffer from poor generalization in black-box scenarios and insufficient robustness under image processing distortions. To address these issues, this paper proposes a Robust Black-box Proactive Defense (RBPD) framework. The framework aims to inject imperceptible adversarial perturbations into the source image to disrupt the identity feature extraction process of unknown face-swapping models, thereby achieving reliable identity protection.The framework employs a two-stage generation mechanism. In the first stage, a Semantic-Aware Encoder (SAE) and a Texture-Guided Decoder (TGD) are utilized to generate initial semantic perturbations. The SAE uses facial semantic masks for guidance and incorporates a Convolutional Block Attention Module to accurately focus on identity-critical regions. The TGD fuses shallow texture features through skip connections, constraining the perturbations to adapt to the local texture distribution and gradient intensity of the source image. This ensures effective attack performance while significantly reducing visual artifacts and improving visual quality.In the second stage, a Dual-Stream Fusion Encoder (DFE) and a Multi-Scale Aggregation Decoder (MAD) are introduced. The DFE extracts deep features from both the source image and initial perturbations and performs nonlinear fusion, deeply embedding adversarial information into the image’s semantic feature space. The MAD employs three parallel dilated convolutions to capture multi-scale contextual information and integrates a Squeeze-and-Excitation module for adaptive channel recalibration, thereby enhancing the robustness of the perturbations against image distortions. Additionally, a Meta-learning Adaptive Attack (MAA) strategy is designed, which integrates gradient feedback from four heterogeneous identity feature extractors (ArcFace, FaceNet, MagFace, and AdaFace), dynamically adjusts optimization weights, and achieves precise breakthroughs against strongly robust extractors, thereby improving the cross-model generalization of the perturbations on unknown black-box models.Evaluations on the CelebA-HQ and RaFD datasets, targeting three mainstream face-swapping models (SimSwap, E4S, and DiffSwap) as well as commercial facial recognition APIs from Baidu and Tencent, yield the following results: the protected images achieve average Top-1 and Top-5 identity matching rates of 0.311 and 0.396, respectively, representing decreases of 63.28% and 54.79% compared to unprotected images, and further reductions of 16.17% and 19.02% compared to the best baseline methods. On the unseen RaFD dataset, the method maintains stable performance with average Top-1 and Top-5 matching rates of 0.349 and 0.394. In black-box face-swapping tests, the cosine similarity between swapped images and source images mostly drops below 0.3, with the lowest reaching 0.185, achieving reliable identity mismatch. When facing common social network distortions such as JPEG compression, Gaussian blur, noise, and resizing, the Distortion Defense Volatility (DDV) averages only 3.4%, significantly outperforming the baseline methods Saliency (55.5%), DF-RAP (27.1%), NullSwap (19.70%), and ID-Eraser (13.90%). In terms of visual quality, the protected images attain a PSNR of 37.38 dB, SSIM of 0.976, and LPIPS of 0.0065, demonstrating good perceptual naturalness. In commercial API tests, the face matching pass rate on Baidu API decreases from 89.92% to 0.40%, and on Tencent API from 90.37% to 4.25%. The total processing time for a single image is only 8.23 ms, indicating strong practical deployment potential.The proposed RBPD framework effectively addresses the deficiencies of existing methods in black-box generalization and distortion robustness. Through the synergistic design of semantic-texture guidance and multi-scale deep fusion, it achieves a strong balance among attack effectiveness, visual quality, and robustness, providing an efficient and practical proactive defense solution for personal privacy protection in complex social network environments. This work holds significant theoretical importance and application value.

摘要: 针对生成式面部换脸技术对个人隐私构成严重威胁,且现有主动防御方法在黑盒场景下泛化能力较差、对图像处理失真鲁棒性不足的问题,本文提出一种面向黑盒场景的鲁棒增强主动防御框架(Robust Black-box Proactive Defense, RBPD)。该框架旨在从源图像端注入不可见对抗扰动,干扰未知换脸模型的身份特征提取过程,实现可靠的身份保护。该框架采用两阶段生成机制。第一阶段利用语义感知编码器(Semantic-Aware Encoder, SAE)与纹理引导解码器(Texture-Guided Decoder, TGD)生成初始语义扰动。语义感知编码器SAE通过面部语义掩码引导并结合卷积块注意力模块精准聚焦身份关键区域,纹理引导解码器TGD借助跳跃连接融合浅层纹理特征,强制扰动适配源图像的局部纹理分布与梯度强度,在保证攻击有效性的同时显著减少视觉伪影并提升视觉质量。第二阶段引入双流融合编码器(Dual-Stream Fusion Encoder, DFE)与多尺度聚合解码器(Multi-Scale Aggregation Decoder, MAD)。双流融合编码器DFE将源图像与初始扰动进行深层特征提取与非线性融合,使对抗信息深度嵌入图像语义特征空间;多尺度聚合解码器MAD采用三路并行空洞卷积捕获多尺度上下文信息,并集成挤压激励模块自适应重校准通道响应,增强扰动对图像失真的鲁棒性。同时设计元学习自适应攻击策略(Meta-learning Adaptive Attack, MAA),集成ArcFace、FaceNet、MagFace和AdaFace四种异构身份特征提取器的梯度反馈,动态调整优化权重,实现对强鲁棒性提取器的精准突破,提升扰动在未知黑盒模型上的跨模型泛化能力。在CelebA-HQ数据集上,针对SimSwap、E4S、DiffSwap三种主流换脸模型以及百度、腾讯商业人脸识别API的评估结果显示:受保护图像平均Top-1和Top-5身份匹配率分别降至0.311和0.396,较未保护图像下降63.28%和54.79%,相较最佳基线进一步下降16.17%和19.02%;在未见数据集RaFD上仍保持稳定,平均Top-1和Top-5身份匹配率分别降至0.349与0.394。在黑盒换脸测试中,换脸后图像与源图像的身份余弦相似度大多降至0.3以下,最低达0.185,实现可靠的身份错配。面对JPEG压缩、高斯模糊、噪声和缩放等常见社交网络失真时,失真防御波动率(Distortion Defense Volatility, DDV)平均仅为3.4%,显著优于Saliency(55.5%)、DF-RAP(27.1%)、NullSwap(19.70%)和ID-Eraser(13.90%)四类基准方法。视觉质量方面,受保护图像PSNR达37.38dB,SSIM为0.976,LPIPS为0.0065,展现出良好的图像自然度。在商业API测试中,百度API身份匹配通过率从89.92%降至0.40%,腾讯API从90.37%降至4.25%。单张图像总处理时间仅8.23ms,展现部署潜力。本文提出的主动防御框架有效解决了现有方法在黑盒泛化性和失真鲁棒性方面的不足,通过语义纹理引导与多尺度深度融合的协同设计,实现了攻击效能、视觉质量和鲁棒性的良好平衡,为复杂社交网络环境下个人隐私保护提供了高效实用的主动防御方案,具有重要的理论意义和应用价值。