基于视觉注意力分析的3D内容生成方法

doi:10.19678/j.issn.1000-3428.0056246

摘要/Abstract

摘要： 由于在某些特殊场景中获取深度线索的难度较高，使得已有3D内容生成方法的应用受到限制。为此，以显著图代替深度图进行2D-3D转换，提出一种3D内容生成方法。使用全卷积网络（FCN）生成粗糙的显著图，通过条件随机场对FCN的输出结果进行优化。实验结果表明，该方法可以解决现有方法中因使用低等级特征进行视觉注意力分析而导致显著图质量不高的问题，且能够生成具有良好视觉效果的3D内容。

关键词: 3D内容生成, 显著性检测, 显著图, 全卷积网络, 条件随机场

Abstract: Due to the difficulty of obtaining depth cues in some special scenarios,the application of existing 3D content generation methods is limited.Therefore,by replacing depth map with saliency map for 2D to 3D conversion,this paper proposes a 3D content generation method.The rough saliency map is generated by using Fully Convolutional Network(FCN) and the output results are optimized by Conditional Random Field (CRF).Experimental results show that the proposed method can solve the problem of low quality of saliency map in existing methods caused by using low-level features for visual attention analysis,and it can generate 3D content with satisfying visual effect.

Key words: 3D content generation, saliency detection, saliency map, Fully Convolutional Network(FCN), Conditional Random Field(CRF)

中图分类号:

TP751.1

蔡凯, 李新福, 田学东. 基于视觉注意力分析的3D内容生成方法[J]. 计算机工程, 2020, 46(4): 266-272.

CAI Kai, LI Xinfu, TIAN Xuedong. 3D Content Generation Method Based on Visual Attention Analysis[J]. Computer Engineering, 2020, 46(4): 266-272.

https://www.ecice06.com/CN/Y2020/V46/I4/266

图/表 8

20200414130728

20200414130732

20200414130735

20200414130739

20200414130742

20200414130745

20200414130748

20200414130752

参考文献 36

[1]	ZILLY F,MÜLLER M,EISERT P,et al.The stereoscopic analyzer——an image-based assistance tool for stereo shooting and 3D production[C]//Proceedings of 2010 IEEE International Conference on Image Processing.Washington D.C.,USA:IEEE Press,2010:4029-4032.
[2]	FENG Haozhe,ZHANG Peng,XU Xinnan,et al.An unsupervised suggestive annotation algorithm for 3D CT image processing[J].Journal of Computer-Aided Design and Computer Graphics,2019,31(2):3-9.(in Chinese)冯浩哲,张鹏,徐欣楠,等.面向3D CT影像处理的无监督推荐标注算法[J].计算机辅助设计与图形学学报,2019,31(2):3-9.
[3]	SONG Hongning,GUO Ruiqiang.Application and research progresses of three-dimensional printing based on medical imaging in diagnosis and treatment of cardiovascular diseases[J].Chinese Journal of Medical Imaging Technology,2017,33(3):375-380.(in Chinese)宋宏宁,郭瑞强.基于医学影像学的3D打印技术在心血管疾病诊疗中的应用现状及研究进展[J].中国医学影像技术,2017,33(3):375-380.
[4]	LU Chihao,LI Haifeng,GAO Tao,et al.Virtual reality head-mounted display with large field of view based on stitching[J].Acta Optica Sinica,2019,39(6):150-157.(in Chinese)陆驰豪,李海峰,高涛,等.基于拼接的大视场虚拟现实头戴显示装置[J].光学学报,2019,39(6):150-157.
[5]	HARMAN P V,FLACK J,FOX S,et al.Rapid 2D-to-3D conversion[EB/OL].[2019-09-20].https://www.ixueshu.com/document/fb7ae8da636b56a2318947a18e7 f9386.html.
[6]	HARMAN P.Home based 3D entertainment-an overview[C]//Proceedings of 2000 International Conference on Image Processing.Washington D.C.,USA:IEEE Press,2000:1-4.
[7]	YUAN Hongxing,AN Peng,WU Shaoqun,et al.Error-tolerant semi-automatic 2D-to-3D conversion via L_1 optimization[J].Acta Electronica Sinica,2018,46(2):447-455.(in Chinese)袁红星,安鹏,吴少群,等.具有容错能力的L_1最优化半自动2D转3D[J].电子学报,2018,46(2):447-455.
[8]	LIE W N,HU C H,CHEN Y K,et al.Multi-layer background sprite model for 2D-to-3D video conversion[C]//Proceedings of 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference.Washington D.C.,USA:IEEE Press,2017:270-274.
[9]	ZHANG R,TSAI P S,CRYER J E,et al.Shape-from-shading:a survey[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1999,21(8):690-706.
[10]	KIM J,BAIK A,JUNG Y J,et al.2D-to-3D conversion by using visual attention analysis[EB/OL].[2019-09-20].https://www.researchgate.net/publication/252733584_2D-to-3D_Conversion_by_Using_Visual_Attention_Analysis.
[11]	LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:3431-3440.
[12]	KRÄHENBÜHL P,KOLTUN V.Efficient inference in fully connected CRFS with Gaussian edge potentials[C]//Proceedings of the 24th International Conference on Neural Information Processing Systems.New York,USA:ACM Press,2011:109-117.
[13]	ITTI L,KOCH C,NIEBUR E.A model of saliency-based visual attention for rapid scene analysis[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,20(11):1254-1259.
[14]	KRUTHIVENTI S S S,AYUSH K,VENKATESH BABU R.DeepFix:a fully convolutional neural network for predicting human eye fixations[J].IEEE Transactions on Image Processing,2017,26(9):4446-4456.
[15]	WANG W,SHEN J.Deep visual attention prediction[J].IEEE Transactions on Image Processing,2017,27(5):2368-2378.
[16]	LI J,RAJAN D,YANG J.Locality and context-aware top-down saliency[J].IET Image Processing,2017,12(3):400-407.
[17]	CHENG M M,ZHANG G X,MITRA N J,et al.Global contrast based salient region detection[C]//Proceedings of 2011 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2011:569-582.
[18]	ZHANG P,WANG D,LU H,et al.Amulet:aggregating multi-level convolutional features for salient object detection[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2017:202-211.
[19]	WANG W,SHEN J,DONG X,et al.Salient object detection driven by fixation prediction[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:1711-1720.
[20]	JIANG H,WANG J,YUAN Z,et al.Salient object detection:a discriminative regional feature integration approach[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2013:2083-2090.
[21]	ROSIN P L.A simple method for detecting salient regions[J].Pattern Recognition,2009,42(11):2363-2371.
[22]	BORJI A,CHENG M M,JIANG H,et al.Salient object detection:a benchmark[J].IEEE Transactions on Image Processing,2015,24(12):5706-5722.
[23]	FAVARO P.Shape from focus and defocus:convexity,quasiconvexity and defocus-invariant textures[C]//Proceedings of 2007 IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2007:1-7.
[24]	TSAI T H,HUANG T W,WANG R Z.A novel method for 2D-to-3D video conversion based on boundary information[J].EURASIP Journal on Image and Video Processing,2018(1):2-5.
[25]	GUO G,ZHANG N,HUO L,et al.2D to 3D convertion based on edge defocus and segmentation[C]//Proceedings of 2008 IEEE International Conference on Acoustics,Speech and Signal Processing.Washington D.C.,USA:IEEE Press,2008:2181-2184.
[26]	ZHANG L,VAZQUEZ C,KNORR S.3D-TV content creation:automatic 2D-to-3D video conversion[J].IEEE Transactions on Broadcasting,2011,57(2):372-383.
[27]	KRIZHEVSKY A,SUTSKEVER I,HINTON G E.Imagenet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems.Washington D.C.,USA:IEEE Press,2012:1097-1105.
[28]	SIMONYAN K,ZISSERMAN A.Very deep convolu-tional networks for large-scale image recognition[EB/OL].[2019-09-20].http://www.arxiv.org/pdf/1409.1556.pdf.
[29]	SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:1-9.
[30]	SZEGEDY C,VANHOUCKE V,IOFFE S,et al.Rethinking the inception architecture for computer vision[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:2818-2826.
[31]	YAN Q,XU L,SHI J,et al.Hierarchical saliency detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2013:1155-1162.
[32]	HARIHARAN B,ARBELÁEZ P,GIRSHICK R,et al.Hypercolumns for object segmentation and fine-grained localization[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:447-456.
[33]	YANG C,ZHANG L,LU H,et al.Saliency detection via graph-based manifold ranking[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2013:3166-3173.
[34]	QUAN Wei,ZHAO Yunxiu,HAN Cheng,et al.Comfort evaluation model based on region of interest and contrast of stereo images[J].Acta Photonica Sinica,2018,47(12):174-180.(in Chinese)权巍,赵云秀,韩成,等.基于立体图像感兴趣区域及对比度的舒适度评价模型[J].光子学报,2018,47(12):174-180.
[35]	JU R,GE L,GENG W,et al.Depth saliency based on anisotropic center-surround difference[C]//Proceedings of 2014 IEEE International Conference on Image Processing.Washington D.C.,USA:IEEE Press,2014:1115-1119.
[36]	WANG W,SHEN J,YU Y,et al.Stereoscopic thumbnail creation via efficient stereo saliency detection[J].IEEE Transactions on Visualization and Computer Graphics,2016,23(8):2014-2027.

选择文件类型/文献管理软件名称

选择包含的内容