Mural Image Color Restoration Based on Dual Reference Optimization

doi:10.19678/j.issn.1000-3428.0066835

Abstract

Abstract:

Research on color restoration of faded mural images can promote the protection and display of murals. Color restoration of mural images aims to restore the fading color areas of degraded mural images to their original colors. With the conventional color restoration method based on a single reference mural image, selecting reference mural images similar to degraded mural images is difficult, which in turn affects the quality of color restoration. To this end, a color restoration method based on dual reference optimization is proposed for mural images. The dual reference strategy involves using two reference mural images to restore the color of degraded mural images. The image optimization module is used to suppress the common multiple degradation such as noise and scratches in faded mural images. The encoder-decoder network is used to encode and extract multi-scale features of mural images, whereby a feature fusion module is constructed to optimize the multi-scale features of mural images. A dual reference guidance module is used to calculate the semantic correspondence confidence between the reference and degraded mural images, to achieve similarity matching between image regions for the style fusion of two reference mural images. On this basis, fusion features are utilized to achieve color restoration of degraded mural images.The experimental results show that this method can accurately restore the color of degraded mural images while maintaining the original edge structure information. Non-reference image quality evaluation indicators are used to objectively evaluate the restored mural images of each method. This method reduced the objective evaluation indicators up to 12.2%, on the comparison methods considered.

Key words: faded image, mural image, color restoration, encoder-decoder network, dual reference optimization method

摘要：

褪变色壁画图像的色彩还原研究可以促进壁画的保护和展示。壁画图像色彩还原旨在将退化壁画图像的色彩褪变区域还原为原有色彩。常规的基于单幅参考壁画图像的色彩还原方法难以选取与退化壁画图像相似的参考壁画图像，进而影响色彩还原质量。为此，提出一种基于双参考优化的壁画图像色彩还原方法。采用双参考策略，即使用2幅参考壁画图像对退化壁画图像进行色彩还原，利用图像优化模块抑制褪变色壁画图像中普遍存在的噪声、划痕等多重退化，通过编码器-解码器网络编码提取壁画图像多尺度特征，并构建特征融合模块优化壁画图像的多尺度特征。采用双参考指导模块计算参考壁画图像与退化壁画图像的语义对应置信度，以实现图像区域间的相似性匹配，并实现2幅参考壁画图像的风格融合。在此基础上，利用融合特征实现退化壁画图像的色彩还原。实验结果表明，该方法可以较准确地还原退化壁画图像色彩，同时能较好保持壁画图像原有的边缘结构信息，并且使用无参考图像质量评估指标对各个方法的还原壁画图像进行客观评估，与对比方法相比，该方法在客观评估指标上最多可降低12.2%。

关键词: 褪变色图像, 壁画图像, 色彩还原, 编码器-解码器网络, 双参考优化方法

Zhigang XU, Cong ZHANG. Mural Image Color Restoration Based on Dual Reference Optimization[J]. Computer Engineering, 2024, 50(2): 345-352.

徐志刚, 张聪. 基于双参考优化的壁画图像色彩还原[J]. 计算机工程, 2024, 50(2): 345-352.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0066835

https://www.ecice06.com/EN/Y2024/V50/I2/345

Figures/Tables 10

Fig.1 Structure of dual reference optimization network

Fig.2 Structure of feature fusion module

Fig.3 Restoration results 1 by adding different modules

Fig.4 Restoration results 2 by adding different modules

Fig.5 Comparison 1 of color restoration results for degraded mural image

Fig.6 Comparison 2 of color restoration results for degraded mural images

Fig.7 Comparison 1 of color restoration results for real degraded mural images

Fig.8 Comparison 2 of color restoration results for real degraded mural images

References 25

1	魏宝刚, 潘云鹤, 华忠. 基于类比的壁画色彩虚拟复原. 计算机研究与发展, 1999, 36 (11): 1364- 1368. URL
	WEI B G, PAN Y H, HUA Z. An analogy-based virtual approach for color restoration of wall painting. Journal of Computer Research and Development, 1999, 36 (11): 1364- 1368. URL
2	杨筱平, 王书文, 贾建芳, 等. 基于GrabCut分割和自动采样的敦煌壁画色彩修复. 兰州理工大学学报, 2010, 36 (3): 114- 117. URL
	YANG X P, WANG S W, JIA J F, et al. Color recovery of Dunhuang murals with GrabCut segmentation and automatic data acquisition. Journal of Lanzhou University of Technology, 2010, 36 (3): 114- 117. URL
3	REINHARD E, ADHIKHMIN M, GOOCH B, et al. Color transfer between images. IEEE Computer Graphics and Applications, 2001, 21 (4): 34- 41.
4	WELSH T, ASHIKHMIN M, MUELLER K. Transferring color to greyscale images[C]//Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques. New York, USA: ACM Press, 2002: 277-280.
5	李娜, 耿国华, 龚星宇, 等. 采用纹理图像对兵马俑褪色的复原方法. 西安电子科技大学学报, 2015, 42 (4): 127-132, 170. doi: 10.3969/j.issn.1001-2400.2015.04.021
	LI N, GENG G H, GONG X Y, et al. Recolor method for the fading terracotta warrior using the texture image. Journal of Xidian University, 2015, 42 (4): 127-132, 170. doi: 10.3969/j.issn.1001-2400.2015.04.021
6	HE M M, LIAO J, CHEN D D, et al. Progressive color transfer with dense semantic correspondences. ACM Transactions on Graphics, 2019, 38 (2): 1- 18. doi: 10.48550/arXiv.1710.00756
7	GATYS L A, ECKER A S, BETHGE M. Image style transfer using convolutional neural networks[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 2414-2423.
8	GATYS L A, ECKER A S, BETHGE M, et al. Controlling perceptual factors in neural style transfer[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 3985-3993.
9	HUANG X, BELONGIE S. Arbitrary style transfer in real-time with adaptive instance normalization[C]//Proceedings of International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 1501-1510.
10	SU J W, CHU H K, HUANG J B. Instance-aware image colorization[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 7968-7977.
11	PARK D Y, LEE K H. Arbitrary style transfer with style-attentional networks[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 5880-5888.
12	徐志刚, 陈士成, 朱红蕾. 基于风格注意力的壁画图像色彩还原. 华中科技大学学报(自然科学版), 2023, 51 (9): 88- 95. URL
	XU Z G, CHEN S C, ZHU H L. Color restoration of mural image based on style attention. Journal of Huazhong University of Science and Technology (Natural Science Edition), 2023, 51 (9): 88- 95. URL
13	HUANG Z X, ZHANG J H, LIAO J. Style mixer: semantic-aware multi-style transfer network. Computer Graphics Forum, 2019, 38 (7): 469- 480. doi: 10.48550/arXiv.1910.13093
14	李超, 石剑平, 姜麟. 深度学习中的图像多风格迁移算法. 软件导刊, 2021, 20 (7): 171- 176. doi: 10.11907/rjdk.202598
	LI C, SHI J P, JIANG L. Image multi-style transfer algorithm in deep learning. Software Guide, 2021, 20 (7): 171- 176. doi: 10.11907/rjdk.202598
15	LIU S H, LIN T W, HE D L, et al. AdaAttN: revisit attention mechanism in arbitrary neural style transfer[C]//Proceedings of International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2021: 6649-6658.
16	LIN T W, MA Z Q, LI F, et al. Drafting and revision: Laplacian pyramid network for fast high-quality artistic style transfer[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 5141-5150.
17	WANG H A, LI Y J, WANG Y H, et al. Collaborative distillation for ultra-resolution universal style transfer[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 1860-1869.
18	何静, 周伟强. 甘肃泾川罗汉洞石窟壁画及制作工艺研究. 文物保护与考古科学, 2021, 33 (2): 13- 22. URL
	HE J, ZHOU W Q. Research on the workmanship and materials of Luohan grottoes in Jingchuan, Gansu province. Sciences of Conservation and Archaeology, 2021, 33 (2): 13- 22. URL
19	WAN Z Y, ZHANG B, CHEN D D, et al. Bringing old photos back to life[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 2747-2757.
20	HU J E, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 7132-7141.
21	PIRRONE R, CANNELLA V, GAMBINO O, et al. WikiArt: an ontology-based information retrieval system for arts[C]//Proceedings of the 9th International Conference on Intelligent Systems Design and Applications. Washington D. C., USA: IEEE Press, 2009: 913-918.
22	TIAN Y T, SUZUKI C, CLANUWAT T, et al. KaoKore: a pre-modern Japanese art facial expression dataset[EB/OL]. [2022-12-28]. https://arxiv.org/abs/2002. 08595.
23	中国敦煌壁画全集编辑委员会. 中国敦煌壁画全集. 天津: 天津人民美术出版社, 2006.
	Editorial Committee of the Complete Works of China Dunhuang Frescoes. The complete collection of Dunhuang frescoes in China. Tianjin: Tianjin People's Fine Arts Publishing House, 2006.
24	KINGMA D P, BA J L. Adam: a method for stochastic optimization[EB/OL]. [2022-12-28]. https://arxiv.org/pdf/1412.6980.pdf.
25	MITTAL A, MOORTHY A K, BOVIK A C. No-reference image quality assessment in the spatial domain. IEEE Transactions on Image Processing, 2012, 21 (12): 4695- 4708. doi: 10.1109/TIP.2012.2214050

[1]	Ang LU, Jun CHU, Lu LENG. Image Dehazing Based on High-Frequency and Low-Frequency Feature Enhancement [J]. Computer Engineering, 2023, 49(8): 174-181.
[2]	ZHANG Jiong, WANG Lifang, LIN Suzhen, QIN Pinle, MI Jia, LIU Yang. Medical Image Fusion with Local-Global Feature Coupling and Cross-Scale Attention [J]. Computer Engineering, 2023, 49(3): 238-247.
[3]	LU Dongsheng, ZHANG Yujin, DANG Lianghui. Multi-Feature Fusion U-structure Deep Network for Image Tempering Forensics [J]. Computer Engineering, 2022, 48(4): 213-222.
[4]	WANG Jing,WANG Rong-gui,YANG Juan,LI Xiang. Local Adaptive ACE Algorithm of Double Selective Pixels [J]. Computer Engineering, 2014, 40(10): 186-191.

Please choose a citation manager

Content to export