Single Image Reflection Removal Model Based on Reflection Classifier and Gradient Restorer

doi:10.19678/j.issn.1000-3428.0069240

Abstract

Abstract:

Removing single image reflections is an important task in computer vision. However, existing image reflection removal models are based on the premise that reflection pollution areas are fuzzy types, which means the reflection areas retain the original image content information. In the case of spot reflection in a contaminated image, the original content information of the image is completely lost, leading to the failure of existing models in extracting the original image transmission layer information from the spot region. To address this problem, this study proposes a new model that can simultaneously remove spots and fuzzy reflections. By utilizing a self-defined reflection classifier and structure restorer, the model predicts the gradient map of the image transmission layer and uses it as an auxiliary condition to generate an ultimately pure transmission-layer image. Experiments show that our model has a good generalization performance on different categories of reflected images. Experiments on art images, specifically Tangka, demonstrate that our model outperforms the state-of-the-art removal model in terms of Structure Similarity Index Measure (SSIM) and Peak Signal-to-Noise Ratio (PSNR), which increase by 1.6% and 5.5%, respectively. Experiments on public natural scene datasets also indicate that our model is comparable to state-of-the-art models.

Key words: single image reflection removing, reflection classifier, image gradient restoration, generative adversarial network, attention mechanism

摘要：

单幅图像反射去除是计算机视觉领域的一项重要任务。然而, 现有的图像反射去除模型都基于反射污染区域属于模糊型反射这一前提, 即反射区域仍然保留原始的图像内容信息。当污染图像中存在光斑反射时, 图像原始内容信息完全丢失, 导致现有模型无法从光斑区域中提取原始图像的透射层信息, 从而使模型失效。针对这一问题, 提出一种能够同时去除光斑与模糊反射的新模型, 通过自定义的反射分类器和结构恢复器引导模型预测图像透射层的梯度图, 并以此作为辅助条件, 最终生成纯净的透射层图像。实验结果表明, 该模型对不同类别的反射图像均具有较好的泛化性能, 在艺术图像唐卡上, 模型在结构相似度(SSIM)与峰值信噪比(PSNR)指标上均优于当前最优的反射去除模型, 其中SSIM与最优模型相比提升了1.6%, PSNR提升了5.5%。在公共的自然场景数据集上的实验结果也表明该模型与当前最优模型性能相当。

关键词: 单幅图像反射去除, 反射分类, 图像梯度恢复, 生成对抗网络, 注意力机制

XIE Qing, ZHANG Lingfeng, MA Yanchun, LIU Yongjian. Single Image Reflection Removal Model Based on Reflection Classifier and Gradient Restorer[J]. Computer Engineering, 2025, 51(4): 227-238.

解庆, 张凌峰, 马艳春, 刘永坚. 基于反射分类与梯度恢复的单幅图像去反射模型[J]. 计算机工程, 2025, 51(4): 227-238.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0069240

https://www.ecice06.com/EN/Y2025/V51/I4/227

Figures/Tables 17

Fig.1 Reflection contaminated image and pure image

Fig.2 Simulation diagram of the first type of reflection

Fig.3 Simulation diagram of the second type of reflection

Fig.4 Simulation diagram of the third type of reflection

Fig.5 Overall structure of the model

Fig.6 Contrast between facula and blurred reflection

Fig.7 Structure of reflection classifier

Fig.8 Structure of G₂ in gradient restorer

Fig.9 The influence of GCBlock on gradient restorer

Fig.10 Output of sub network

Fig.11 Comparison of the effect of each model on Tangka image de reflection

Fig.12 Comparison of spot removal effects

Fig.13 Experimental results based on SIR² dataset

References 32

1	FAN Q N, YANG J L, HUA G, et al. A generic deep architecture for single image reflection removal and image smoothing[C]//Proceedings of the IEEE International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2017: 3258-3267.
2	WEI K X, YANG J L, FU Y, et al. Single image reflection removal exploiting misaligned training data and network enhancements[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2019: 8178-8187.
3	WAN R J, SHI B X, DUAN L Y, et al. Benchmarking single-image reflection removal algorithms[C]//Proceedings of the IEEE International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2017: 3942-3950.
4	ZHANG X E, NG R, CHEN Q F. Single image reflection separation with perceptual losses[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2018: 4786-4794.
5	FERGUS R, SINGH B, HERTZMANN A, et al. Removing camera shake from a single photograph[C]//Proceedings of the ACM SIGGRAPH 2006 Papers. New York, USA: ACM Press, 2006: 787-794.
6	LEVIN A, ZOMET A, WEISS Y. Separating reflections from a single image using local features[C]//Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2004: 306-313.
7	LEVIN A, WEISS Y. User assisted separation of reflections from a single image using a sparsity prior[EB/OL]. [2023-08-05]. https://people.csail.mit.edu/alevin/papers/Assisted-Reflections-Levin-Weiss-PAMI.pdf.
8	CHUNG Y C, CHANG S L, WANG J M, et al. Interference reflection separation from a single image[C]//Proceedings of the Workshop on Applications of Computer Vision. Washington D.C., USA: IEEE Press, 2009: 1-6.
9	YAN Q , XU Y , YANG X K , et al. Separation of weak reflection from a single superimposed image. IEEE Signal Processing Letters, 2014, 21 (10): 1173- 1176. doi: 10.1109/LSP.2014.2327071
10	WAN R J, SHI B X, HWEE T A, et al. Depth of field guided reflection removal[C]//Proceedings of the IEEE International Conference on Image Processing. Washington D.C., USA: IEEE Press, 2016: 21-25.
11	SHIH Y, KRISHNAN D, DURAND F, et al. Reflection removal using ghosting cues[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2015: 3193-3201.
12	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[EB/OL]. [2023-08-05]. https://dl.acm.org/doi/10.5555/2969033.2969125.
13	WAN R , SHI B , LI H , et al. CoRRN: cooperative reflection removal network. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42 (12): 2969- 2982. doi: 10.1109/TPAMI.2019.2921574
14	胡宇航, 胡海洋, 李忠金. 基于条件生成对抗网络的梯级表面高光去除方法. 计算机应用研究, 2022, 39 (9): 2867-2872, 2880.
	HU Y H , HU H Y , LI Z J . Conditional generative adversarial network-based method for stepped surface highlight removal. Application Research of Computers, 2022, 39 (9): 2867-2872, 2880.
15	LI C, YANG Y X, HE K, et al. Single image reflection removal through cascaded refinement[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2020: 3562-3571.
16	YANG J, GONG D, LIU L Q, et al. Seeing deeply and bidirectionally: a deep learning approach for single image reflection removal[EB/OL]. [2023-08-05]. https://openaccess.thecvf.com/content_ECCV_2018/papers/Jie_Yang_Seeing_Deeply_and_ECCV_2018_paper.pdf.
17	HOCHREITER S , SCHMIDHUBER J . Long short-term memory. Neural Computation, 1997, 9 (8): 1735- 1780. doi: 10.1162/neco.1997.9.8.1735
18	WANG Y, CHEN Y C, TAO X, et al. VCNet: a robust approach to blind image inpainting[EB/OL]. [2023-08-05]. https://arxiv.org/abs/2003.06816.
19	SUN C, LIU S C, YANG T T, et al. Automatic reflection removal using gradient intensity and motion cues[C]//Proceedings of the 24th ACM International Conference on Multimedia. New York, USA: ACM Press, 2016: 466-470.
20	WAN R J, SHI B X, DUAN L Y, et al. CRRN: multi-scale guided concurrent reflection removal network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2018: 4777-4785.
21	NAZERI K, NG E, JOSEPH T, et al. EdgeConnect: generative image inpainting with adversarial edge learning[EB/OL]. [2023-08-05]. https://arxiv.org/abs/1901.00212v3.
22	姜艺, 胥加洁, 柳絮, 等. 边缘指导图像修复算法研究. 计算机科学与探索, 2022, 16 (3): 669- 682.
	JIANG Y , XU J J , LIU X , et al. Research on edge-guided image repair algorithm. Journal of Frontiers of Computer Science and Technology, 2022, 16 (3): 669- 682.
23	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. [2023-08-05]. https://arxiv.org/abs/1409.1556.
24	CAO Y, XU J R, LIN S, et al. GCNet: non-local networks meet squeeze-excitation networks and beyond[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision Workshop. Washington D.C., USA: IEEE Press, 2019: 1971-1980.
25	ZHANG D W , ZHENG Z L , LI M L , et al. CSART: channel and spatial attention-guided residual learning for real-time object tracking. Neurocomputing, 2021, 436, 260- 272. doi: 10.1016/j.neucom.2020.11.046
26	卞鹏程, 郑忠龙, 李明禄, 等. 基于注意力融合网络的视频超分辨率重建. 计算机应用, 2021, 41 (4): 1012- 1019.
	BIAN P C , ZHENG Z L , LI M L , et al. Attention fusion network based video super-resolution reconstruction. Journal of Computer Applications, 2021, 41 (4): 1012- 1019.
27	WANG Z , BOVIK A C , SHEIKH H R , et al. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13 (4): 600- 612. doi: 10.1109/TIP.2003.819861
28	郑元丰, 张威, 江昊, 等. 一种基于弱监督学习的图像镜面高光去除算法. 信号处理, 2023, 39 (6): 1016- 1024.
	ZHENG Y F , ZHANG W , JIANG H , et al. Weakly supervised specular highlight removal with only highlight images. Journal of Signal Processing, 2023, 39 (6): 1016- 1024.
29	CHANG Y C, LU C N, CHENG C C, et al. Single image reflection removal with edge guidance, reflection classifier, and recurrent decomposition[C]//Proceedings of the IEEE Winter Conference on Applications of Computer Vision. Washington D.C., USA: IEEE Press, 2021: 2032-2041.
30	DON J . Signal-to-noise ratio. Scholarpedia, 2006, 1 (12): 2088. doi: 10.4249/scholarpedia.2088
31	DONG Z, XU K, YANG Y, et al. Location-aware single image reflection removal[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2021: 4997-5006.
32	HU Q M, GUO X J. Single image reflection separation via component synergy[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2023: 13092-13101.

[1]	YANG Ping, ZHANG Xi. Improved DeepLabv3+ Road Surface Crack Detection Method [J]. Computer Engineering, 2025, 51(4): 261-270.
[2]	XU Yonggang, SUN Qixuan, LI Fanjia, CHENG Jianwei, DAI Jiajun. Skeleton Behavior Recognition Based on Extended Temporal and Spatiotemporal Feature Fusion Graph Convolutional Network [J]. Computer Engineering, 2025, 51(4): 281-292.
[3]	GENG Xia, WANG Yao. Cloth-Changing Person Re-Identification Method Based on CLIP Enhanced Fine-Grained Features [J]. Computer Engineering, 2025, 51(4): 293-302.
[4]	LIU Yunxiang, LIANG Zhichao. A Highly Efficient Traffic Prediction Model for Continuous Time-series Graph Attention Networks [J]. Computer Engineering, 2025, 51(4): 350-359.
[5]	DU Chenyang, ZHANG Xueying, HUANG Lixia, LI Juan. Multi-Feature Speech Emotion Recognition Based on Improved Efficient Channel Attention Mechanism [J]. Computer Engineering, 2025, 51(4): 97-106.
[6]	SUN Ziwen, QIAN Lizhi, YUAN Guanglin, YANG Chuandong, LING Chong. Transformer Object Tracking Method Based on Real-Time Dynamic Template Update [J]. Computer Engineering, 2025, 51(4): 158-168.
[7]	DONG Hongliang, NIU Yan, SUN Yang, LI Jun. Speech Emotion Recognition Based on Memory Capsules and Attention [J]. Computer Engineering, 2025, 51(4): 169-177.
[8]	SUN Ting, YANG Jie, LI Jiaxuan, WANG Yaozong. Optimization of YOLOv7 Road Sign Detection Algorithm for Low-Light Traffic Scenes [J]. Computer Engineering, 2025, 51(3): 342-351.
[9]	LUAN Fangjun, GONG Qi, YUAN Shuai. Crowd Counting Network Based on Attention Mechanism and Multiscale Fusion [J]. Computer Engineering, 2025, 51(3): 352-361.
[10]	HU Shulin, ZHANG Huajun, DENG Xiaotao, WANG Zhenghua. Similarity Calculation for Chinese Text Based on Dependency Graph Convolution [J]. Computer Engineering, 2025, 51(3): 76-85.
[11]	LU Peng, ZHONG Chuang. Improved CycleGAN Algorithm for Semi-Supervised Building Extraction [J]. Computer Engineering, 2025, 51(3): 241-251.
[12]	WANG Xinliang, WANG Luying. Safety Helmet Detection Algorithm with Feature Enhancement in Low Light Blasting Scenes [J]. Computer Engineering, 2025, 51(3): 252-260.
[13]	ZHANG Huan, WANG Chen, SHAN Jingdong, QIU Runhe. Elevator Safety Risk Prediction Based on Domain Adaptation and Attention Mechanism [J]. Computer Engineering, 2025, 51(2): 86-93.
[14]	ZHANG Yuan, LÜ Defang, MENG Jianjun, QI Wenzhe. Defect Detection of Rail Fasteners Based on Double Attention and GSSN Lightweight [J]. Computer Engineering, 2025, 51(2): 289-299.
[15]	ZHAO Hong, SONG Furong, LI Wengai. Research on Image Adversarial Example Generation Method Based on SE-AdvGAN [J]. Computer Engineering, 2025, 51(2): 300-311.

Please choose a citation manager

Content to export