Virtual Viewpoint Rendering Algorithm Based on Hierarchical Image Fusion

doi:10.19678/j.issn.1000-3428.0058057

Abstract

Abstract: To address the overlaps and holes in virtual viewpoint rendering,this paper proposes a new virtual viewpoint rendering algorithm based on hierarchical image fusion.The algorithm performs morphological operation on the holes generated by 3D image transformation to expand them,thus eliminating artifact defects.Then according to the depth information,the foreground and background of the left and right virtual viewpoint images are segmented.The linear weighting method is used to implement hierarchical fusion for segmented foreground and background images to solve pixel overlap.Holes filled,the background image is fused with the foreground image to obtain the virtual viewpoint image.Experimental results show that the PSNR of the proposed algorithm is 1.75 dB higher than that of the Criminisi algorithm.

Key words: virtual viewpoint, depth information, foreground and background segmentation, hierarchical fusion, three-dimensional image transformation

摘要： 针对虚拟视点绘制过程中出现的重叠和空洞问题，提出一种新的虚拟视点绘制算法。通过形态学操作对三维图像变换后出现的空洞进行膨胀来消除伪影瑕疵，根据深度信息对左、右虚拟视点图像进行前景和背景分割，利用线性加权法对分割后的前景图像和背景图像进行分层融合解决像素重叠问题，对分层融合后的背景图像进行空洞填充并与前景图像融合得到虚拟视点图像。实验结果表明，与经典的Criminisi算法相比，该算法PSNR值提高了1.75 dB，具有较高的绘图质量。

关键词: 虚拟视点, 深度信息, 前景和背景分割, 分层融合, 三维图像变换

CLC Number:

TP391

CAI Limei, LI Xinfu, TIAN Xuedong. Virtual Viewpoint Rendering Algorithm Based on Hierarchical Image Fusion[J]. Computer Engineering, 2021, 47(4): 204-210.

蔡李美, 李新福, 田学东. 基于分层图像融合的虚拟视点绘制算法[J]. 计算机工程, 2021, 47(4): 204-210.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0058057

http://www.ecice06.com/EN/Y2021/V47/I4/204

Figures/Tables 15

References

[1] TANIMO M.FTV:free-viewpoint television[J].Signal Processing:Image Communication,2012,27(6):555-570.
[2] LU Xiaohan,WEI Fang,CHEN Fangmin.Foreground-object-protected depth map smoothing for DIBR[C]//Proceedings of IEEE International Conference on Multi-media and Exposition.Los Alamitos,USA:IEEE Computer Society Press,2012:339-343.
[3] XU X Y,PO L M,NG K H.Depth map misalignment correction and dilation for DIBR view synthesis[J].Signal Processing:Image Communication,2013,28(9):1023-1045.
[4] LEI Jianjun,ZHANG Cuicui,WU Min,et al.A divide-and-conquer hole-filling method for handling disocclusion in single-view rendering[J].Multimedia Tools and Applica-tions,2017,76(6):7661-7676.
[5] LIANG Haitao,CHEN Xiaodong,XU Huaiyuan,et al.Virtual view rendering based on depth map preprocessing and image inpainting[J].Journal of Computer-Aided Design & Computer Graphics,2019,31(8):1278-1285.(in Chinese)梁海涛,陈晓冬,徐怀远,等.基于深度图预处理和图像修复的虚拟视点绘制[J].计算机辅助设计与图形学学报,2019,31(8):1278-1285.
[6] LASANG P.Optimal depth recovery using image guided TGV with depth confidence for high-quality view synthesis[J].Journal of Visual Communication & Image Representation,2016,39:24-39.
[7] LOU Daping,WANG Xiaodong,FU Xianzu,et al.Virtual view point rendering based on range image segmentation[J].Computer Engineering,2016,42(10):12-19.(in Chinese)娄达平,王晓东,富显祖,等.基于深度图像分割的虚拟视点绘制[J].计算机工程,2016,42(10):12-19.
[8] WANG Qi,CAO Ning.The hole-filling method for view synthesis in free viewpoint television and 3D video[C]//Proceedings of IEEE International Conference on Electric Information and Control Engineering.Washington D.C.,USA:IEEE Press,2011:1805-1807.
[9] WANG Hui,CHEN Fen.Virtual view rendering algorithm based on spatial weighting[J].Computer Engineering and Applications,2016,52(8):174-179,195.(in Chinese)汪辉,陈芬.基于空间加权的虚拟视点绘制算法[J].计算机工程与应用,2016,52(8):174-179,195.
[10] CRIMINISI A,PÉREZ P,TOYAMA K.Region filling and object removal by exemplar-based image inpainting[J].IEEE Transactions on Image Processing,2004,13(9):1200-1212.
[11] MULLER K,SMOLIC A,DIX K,et al.View synthesis for advanced 3D video systems[J].EURASIP Journal on Image and Video Processing,2009(1):438-448.
[12] WANG Dan,ZHAO Yan,WANG Zheng,et al.Hole-filling for DIBR based on depthand gradient information[J].International Journal of Advanced Robotic Systems,2015,12(1):39-42.
[13] LI Ying.Research on virtual viewpoint rendering algorithm based on depth image[D].Taiyuan:North University of China,2017.(in Chinese)李英.基于深度图像的虚拟视点绘制算法研究[D].太原:中北大学,2017.
[14] WANG S M.Recursive line-based algorithm forhole-filling in DIBR images[J].Journal of Computer-Aided Design & Computer Graphics,2017,29(8):1536-1541.
[15] YANG Jie,LI Zhaoxin,WANG Suqin,et al.Virtual viewpoint rendering based on hole filling[J].Computer Engineering,2017,43(10):234-240,245.(in Chinese)杨婕,李兆歆,王素琴,等.基于空洞填补的虚拟视点绘制[J].计算机工程,2017,43(10):234-240,245.
[16] YU Li,GUO Li,YUAN Hongxing.A novel method of depth-image-based view synthesis[J].Journal of the Graduate School of the Chinese Academy of Sciences,2010,27(5):638-644.
[17] MEI Ning.Reserch on layered virtual view synthesis technology[D].Nanjing:Southeast University,2016.(in Chinese)梅宁.基于分层的虚拟视点合成技术的研究[D].南京:东南大学,2016.
[18] ZINGER S,DO L,DE WITH P H N.Free-viewpoint depth image based rendering[J].Journal of Visual Communication and Image Representation,2010,21(5):533-541.
[19] WANG Lu,LAI Chunlu.A modifiedtw o-viewpoint DIBR based virtual view synthesis method[J].Journal of Shandong University(Natural Science),2016,51(3):122-131,142.(in Chinese)王路,赖春露.一种基于两视点DIBR的改进虚拟视点合成方法[J].山东大学学报(理学版),2016,51(3):122-131,142.
[20] LOU Daping,WANG Xiaodong,FU Xianzu,et al.A virtual view rendering method based on depth-guided hole filling[J].Computer Applications and Software,2017,34(6):162-168.(in Chinese)娄达平,王晓东,富显祖,等.基于深度引导空洞填补的虚拟视点绘制方法[J].计算机应用与软件,2017,34(6):162-168.

Please choose a citation manager

Content to export