基于反射分类与梯度恢复的单幅图像去反射模型

doi:10.19678/j.issn.1000-3428.0069240

摘要/Abstract

摘要：

单幅图像反射去除是计算机视觉领域的一项重要任务。然而, 现有的图像反射去除模型都基于反射污染区域属于模糊型反射这一前提, 即反射区域仍然保留原始的图像内容信息。当污染图像中存在光斑反射时, 图像原始内容信息完全丢失, 导致现有模型无法从光斑区域中提取原始图像的透射层信息, 从而使模型失效。针对这一问题, 提出一种能够同时去除光斑与模糊反射的新模型, 通过自定义的反射分类器和结构恢复器引导模型预测图像透射层的梯度图, 并以此作为辅助条件, 最终生成纯净的透射层图像。实验结果表明, 该模型对不同类别的反射图像均具有较好的泛化性能, 在艺术图像唐卡上, 模型在结构相似度(SSIM)与峰值信噪比(PSNR)指标上均优于当前最优的反射去除模型, 其中SSIM与最优模型相比提升了1.6%, PSNR提升了5.5%。在公共的自然场景数据集上的实验结果也表明该模型与当前最优模型性能相当。

关键词: 单幅图像反射去除, 反射分类, 图像梯度恢复, 生成对抗网络, 注意力机制

Abstract:

Removing single image reflections is an important task in computer vision. However, existing image reflection removal models are based on the premise that reflection pollution areas are fuzzy types, which means the reflection areas retain the original image content information. In the case of spot reflection in a contaminated image, the original content information of the image is completely lost, leading to the failure of existing models in extracting the original image transmission layer information from the spot region. To address this problem, this study proposes a new model that can simultaneously remove spots and fuzzy reflections. By utilizing a self-defined reflection classifier and structure restorer, the model predicts the gradient map of the image transmission layer and uses it as an auxiliary condition to generate an ultimately pure transmission-layer image. Experiments show that our model has a good generalization performance on different categories of reflected images. Experiments on art images, specifically Tangka, demonstrate that our model outperforms the state-of-the-art removal model in terms of Structure Similarity Index Measure (SSIM) and Peak Signal-to-Noise Ratio (PSNR), which increase by 1.6% and 5.5%, respectively. Experiments on public natural scene datasets also indicate that our model is comparable to state-of-the-art models.

Key words: single image reflection removing, reflection classifier, image gradient restoration, generative adversarial network, attention mechanism

解庆, 张凌峰, 马艳春, 刘永坚. 基于反射分类与梯度恢复的单幅图像去反射模型[J]. 计算机工程, 2025, 51(4): 227-238.

XIE Qing, ZHANG Lingfeng, MA Yanchun, LIU Yongjian. Single Image Reflection Removal Model Based on Reflection Classifier and Gradient Restorer[J]. Computer Engineering, 2025, 51(4): 227-238.

https://www.ecice06.com/CN/Y2025/V51/I4/227

图/表 17

图1 反射污染图像与纯净图像

Fig.1 Reflection contaminated image and pure image

图2 第一类反射情况模拟图

Fig.2 Simulation diagram of the first type of reflection

图3 第二类反射情况模拟图

Fig.3 Simulation diagram of the second type of reflection

图4 第三类反射情况模拟图

Fig.4 Simulation diagram of the third type of reflection

图5 模型整体结构

Fig.5 Overall structure of the model

图6 光斑与模糊反射对比

Fig.6 Contrast between facula and blurred reflection

图7 反射分类器结构

Fig.7 Structure of reflection classifier

图8 梯度恢复器中G₂的结构

Fig.8 Structure of G₂ in gradient restorer

图9 GCBlock对梯度恢复器的影响

Fig.9 The influence of GCBlock on gradient restorer

图10 子网络的输出

Fig.10 Output of sub network

图11 各模型对唐卡图像的去反射效果对比

Fig.11 Comparison of the effect of each model on Tangka image de reflection

图12 光斑去除效果对比

Fig.12 Comparison of spot removal effects

图13 基于SIR²数据集的实验结果

Fig.13 Experimental results based on SIR² dataset

参考文献 32

1	FAN Q N, YANG J L, HUA G, et al. A generic deep architecture for single image reflection removal and image smoothing[C]//Proceedings of the IEEE International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2017: 3258-3267.
2	WEI K X, YANG J L, FU Y, et al. Single image reflection removal exploiting misaligned training data and network enhancements[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2019: 8178-8187.
3	WAN R J, SHI B X, DUAN L Y, et al. Benchmarking single-image reflection removal algorithms[C]//Proceedings of the IEEE International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2017: 3942-3950.
4	ZHANG X E, NG R, CHEN Q F. Single image reflection separation with perceptual losses[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2018: 4786-4794.
5	FERGUS R, SINGH B, HERTZMANN A, et al. Removing camera shake from a single photograph[C]//Proceedings of the ACM SIGGRAPH 2006 Papers. New York, USA: ACM Press, 2006: 787-794.
6	LEVIN A, ZOMET A, WEISS Y. Separating reflections from a single image using local features[C]//Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2004: 306-313.
7	LEVIN A, WEISS Y. User assisted separation of reflections from a single image using a sparsity prior[EB/OL]. [2023-08-05]. https://people.csail.mit.edu/alevin/papers/Assisted-Reflections-Levin-Weiss-PAMI.pdf.
8	CHUNG Y C, CHANG S L, WANG J M, et al. Interference reflection separation from a single image[C]//Proceedings of the Workshop on Applications of Computer Vision. Washington D.C., USA: IEEE Press, 2009: 1-6.
9	YAN Q , XU Y , YANG X K , et al. Separation of weak reflection from a single superimposed image. IEEE Signal Processing Letters, 2014, 21 (10): 1173- 1176. doi: 10.1109/LSP.2014.2327071
10	WAN R J, SHI B X, HWEE T A, et al. Depth of field guided reflection removal[C]//Proceedings of the IEEE International Conference on Image Processing. Washington D.C., USA: IEEE Press, 2016: 21-25.
11	SHIH Y, KRISHNAN D, DURAND F, et al. Reflection removal using ghosting cues[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2015: 3193-3201.
12	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[EB/OL]. [2023-08-05]. https://dl.acm.org/doi/10.5555/2969033.2969125.
13	WAN R , SHI B , LI H , et al. CoRRN: cooperative reflection removal network. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42 (12): 2969- 2982. doi: 10.1109/TPAMI.2019.2921574
14	胡宇航, 胡海洋, 李忠金. 基于条件生成对抗网络的梯级表面高光去除方法. 计算机应用研究, 2022, 39 (9): 2867-2872, 2880.
	HU Y H , HU H Y , LI Z J . Conditional generative adversarial network-based method for stepped surface highlight removal. Application Research of Computers, 2022, 39 (9): 2867-2872, 2880.
15	LI C, YANG Y X, HE K, et al. Single image reflection removal through cascaded refinement[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2020: 3562-3571.
16	YANG J, GONG D, LIU L Q, et al. Seeing deeply and bidirectionally: a deep learning approach for single image reflection removal[EB/OL]. [2023-08-05]. https://openaccess.thecvf.com/content_ECCV_2018/papers/Jie_Yang_Seeing_Deeply_and_ECCV_2018_paper.pdf.
17	HOCHREITER S , SCHMIDHUBER J . Long short-term memory. Neural Computation, 1997, 9 (8): 1735- 1780. doi: 10.1162/neco.1997.9.8.1735
18	WANG Y, CHEN Y C, TAO X, et al. VCNet: a robust approach to blind image inpainting[EB/OL]. [2023-08-05]. https://arxiv.org/abs/2003.06816.
19	SUN C, LIU S C, YANG T T, et al. Automatic reflection removal using gradient intensity and motion cues[C]//Proceedings of the 24th ACM International Conference on Multimedia. New York, USA: ACM Press, 2016: 466-470.
20	WAN R J, SHI B X, DUAN L Y, et al. CRRN: multi-scale guided concurrent reflection removal network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2018: 4777-4785.
21	NAZERI K, NG E, JOSEPH T, et al. EdgeConnect: generative image inpainting with adversarial edge learning[EB/OL]. [2023-08-05]. https://arxiv.org/abs/1901.00212v3.
22	姜艺, 胥加洁, 柳絮, 等. 边缘指导图像修复算法研究. 计算机科学与探索, 2022, 16 (3): 669- 682.
	JIANG Y , XU J J , LIU X , et al. Research on edge-guided image repair algorithm. Journal of Frontiers of Computer Science and Technology, 2022, 16 (3): 669- 682.
23	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. [2023-08-05]. https://arxiv.org/abs/1409.1556.
24	CAO Y, XU J R, LIN S, et al. GCNet: non-local networks meet squeeze-excitation networks and beyond[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision Workshop. Washington D.C., USA: IEEE Press, 2019: 1971-1980.
25	ZHANG D W , ZHENG Z L , LI M L , et al. CSART: channel and spatial attention-guided residual learning for real-time object tracking. Neurocomputing, 2021, 436, 260- 272. doi: 10.1016/j.neucom.2020.11.046
26	卞鹏程, 郑忠龙, 李明禄, 等. 基于注意力融合网络的视频超分辨率重建. 计算机应用, 2021, 41 (4): 1012- 1019.
	BIAN P C , ZHENG Z L , LI M L , et al. Attention fusion network based video super-resolution reconstruction. Journal of Computer Applications, 2021, 41 (4): 1012- 1019.
27	WANG Z , BOVIK A C , SHEIKH H R , et al. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13 (4): 600- 612. doi: 10.1109/TIP.2003.819861
28	郑元丰, 张威, 江昊, 等. 一种基于弱监督学习的图像镜面高光去除算法. 信号处理, 2023, 39 (6): 1016- 1024.
	ZHENG Y F , ZHANG W , JIANG H , et al. Weakly supervised specular highlight removal with only highlight images. Journal of Signal Processing, 2023, 39 (6): 1016- 1024.
29	CHANG Y C, LU C N, CHENG C C, et al. Single image reflection removal with edge guidance, reflection classifier, and recurrent decomposition[C]//Proceedings of the IEEE Winter Conference on Applications of Computer Vision. Washington D.C., USA: IEEE Press, 2021: 2032-2041.
30	DON J . Signal-to-noise ratio. Scholarpedia, 2006, 1 (12): 2088. doi: 10.4249/scholarpedia.2088
31	DONG Z, XU K, YANG Y, et al. Location-aware single image reflection removal[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2021: 4997-5006.
32	HU Q M, GUO X J. Single image reflection separation via component synergy[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2023: 13092-13101.

[1]	杜晨阳, 张雪英, 黄丽霞, 李娟. 基于改进高效通道注意力机制的多特征语音情感识别[J]. 计算机工程, 2025, 51(4): 97-106.
[2]	孙子文, 钱立志, 袁广林, 杨传栋, 凌冲. 基于实时动态模板更新的Transformer目标跟踪方法[J]. 计算机工程, 2025, 51(4): 158-168.
[3]	董红亮, 钮焱, 孙杨, 李军. 基于记忆胶囊与注意力的语音情感识别[J]. 计算机工程, 2025, 51(4): 169-177.
[4]	杨萍, 张汐. 改进DeepLabv3+的道路表面裂缝检测方法[J]. 计算机工程, 2025, 51(4): 261-270.
[5]	徐永刚, 孙琦烜, 李凡甲, 程健维, 戴佳俊. 基于扩展时间和时空特征融合图卷积网络的骨架行为识别[J]. 计算机工程, 2025, 51(4): 281-292.
[6]	耿霞, 汪尧. 基于CLIP增强细粒度特征的换装行人重识别方法[J]. 计算机工程, 2025, 51(4): 293-302.
[7]	刘云翔, 梁智超. 一种高效的连续时序图注意力网络的交通预测模型[J]. 计算机工程, 2025, 51(4): 350-359.
[8]	胡书林, 张华军, 邓小涛, 王征华. 结合依存图卷积的中文文本相似度计算研究[J]. 计算机工程, 2025, 51(3): 76-85.
[9]	卢鹏, 仲闯. 改进CycleGAN的半监督建筑物提取算法[J]. 计算机工程, 2025, 51(3): 241-251.
[10]	王新良, 王璐莹. 特征增强的低照度爆破现场安全帽检测算法[J]. 计算机工程, 2025, 51(3): 252-260.
[11]	孙亭, 杨洁, 李家璇, 王耀宗. 面向弱光交通场景的YOLOv7道路标志检测算法优化[J]. 计算机工程, 2025, 51(3): 342-351.
[12]	栾方军, 龚琪, 袁帅. 基于注意力机制和多尺度融合的人群计数网络[J]. 计算机工程, 2025, 51(3): 352-361.
[13]	张欢, 王晨, 单景东, 仇润鹤. 基于领域自适应与注意力机制的电梯安全风险预测[J]. 计算机工程, 2025, 51(2): 86-93.
[14]	张元, 吕德芳, 孟建军, 祁文哲. 基于双注意力和GSSN轻量化的钢轨扣件缺陷检测[J]. 计算机工程, 2025, 51(2): 289-299.
[15]	赵宏, 宋馥荣, 李文改. 基于SE-AdvGAN的图像对抗样本生成方法研究[J]. 计算机工程, 2025, 51(2): 300-311.

选择文件类型/文献管理软件名称

选择包含的内容