基于单目和双目视觉信息的全参考立体图像质量评价模型

doi:10.19678/j.issn.1000-3428.0060191

摘要/Abstract

摘要： 在立体图像质量评价领域，有效地模拟人类视觉系统对图像质量进行评价具有重要意义，考虑到人眼的视觉感知特性，基于单目和双目视觉信息构建一种立体图像质量评价模型MB-FR-SIQA。采用基于结构相似性的立体视差算法得到参考和失真立体图像的视差矩阵，结合Gabor能量响应图、显著性图和视差矩阵生成中间视图，并优化左右眼加权系数计算方法，以提高生成中间视图的准确性。分别利用单目图像和中间视图提取单目和双目视觉信息，计算单目质量分数和双目质量分数，并融合得到立体图像的质量分数，达到评价立体图像质量的目的。实验结果表明，MB-FR-SIQA模型在LIVE-I数据库上具有较高的预测精度，其斯皮尔曼等级相关系数、皮尔森线性相关系数、均方根误差分别为0.945、0.951、5.318，且预测的质量分数符合人类主观评估。

关键词: 立体图像质量评价, 人类视觉系统, 单目视觉, 双目视觉, 中间视图

Abstract: In the field of Stereoscopic Image Quality Assessment (SIQA), effectively simulate the Human Visual System (HVS) to evaluate the image quality remains an important problem.Considering the visual perception characteristics of human eyes, this paper proposes a SIQA model based on monocular and binocular visual information. The model uses stereo disparity algorithm to obtain disparity matrix of reference and distorted stereo images, and employs the saliency map, Gabor energy and disparity matrix response image to optimize the weight coefficients of the left eye and right eye, and thus improves the accuracy of the generated cyclopean image.Then the monocular and binocular visual information features are extracted from monocular images and cyclopean images respectively.The features are combined with the quality score to evaluate the quality of stereoscopic images.The experimental results show that the Spearman rank correlation coefficient, Pearson linear correlaion coefficient and Root mean square error of the model on LIVE-I reach 0.945, 0.951 and 5.318 respectively.The proposed model displays a higher prediction accuracy, and its results are more consistent with human perception.

Key words: Stereoscopic Image Quality Assessment(SIQA), Human Visual System(HVS), monocular vision, binocular vision, cyclopean image

中图分类号:

TP391.4

王宽, 杨环, 潘振宽, 司建伟. 基于单目和双目视觉信息的全参考立体图像质量评价模型[J]. 计算机工程, 2022, 48(2): 207-214,223.

WANG Kuan, YANG Huan, PAN Zhenkuan, SI Jianwei. Full-Reference Model for Stereoscopic Image Quality Assessment Based on Monocular and Binocular Visual Information[J]. Computer Engineering, 2022, 48(2): 207-214,223.

https://www.ecice06.com/CN/Y2022/V48/I2/207

图/表 8

20220301124300

20220301124303

20220301124307

20220301124312

20220301124317

20220301124321

20220301124326

20220301124329

参考文献

[1] WANG Z, BOVIK A C, SHEIKH H R, et al.Image quality assessment:from error visibility to structural similarity[J].IEEE Transactions on Image Processing, 2004, 13(4):600-612.
[2] WANG Z, SIMONCELLI E P, BOVIK A C.Multiscale structural similarity for image quality assessment[C]//Proceedings of the 37th Asilomar Conference on Signals, Systems & Computers.Washington D.C., USA:IEEE Press, 2003:1398-1402.
[3] 王威, 刘婧, 杨蔚蔚, 等.基于DCT域纹理结构相似度的模糊图像质量评价[J].计算机工程, 2015, 41(11):253-256. WANG W, LIU J, YANG W W, et al.Blur image quality assessment based on DCT domain texture structure similarity[J].Computer Engineering, 2015, 41(11):253-256.(in Chinese)
[4] BRADDICK O J.Binocular single vision and perceptual processing[J].Proceedings of the Royal Society B-Biological Sciences, 1979, 204(1157):503-512.
[5] LEVELT W J M.Binocular brightness averaging and contour information[J].British Journal of Social Psychology, 2011, 56(1):1-13.
[6] DING J, SPERLING G.A gain-control theory of binocular combination[J].Proceedings of the National Academy of Sciences, 2006, 103(4):1141-1146.
[7] SUGIE N.Neural models of brightness perception and retinal rivalry in binocular vision[J].Biological Cybernetics, 1982, 43(1):13.
[8] LEGGE G E.Binocular contrast summation-II.Quadratic summation[J].Vision Research, 1984, 24(4):385-394.
[9] HOU F, HUANG C B, LIANG J, et al.Contrast gain-control in stereo depth and cyclopean contrast perception[J].Journal of Vision, 2013, 13(8):1-19.
[10] LI X, WANG Y, ZHAO D, et al.Joint just noticeable difference model based on depth perception for stereoscopic images[C]//Proceedings of Conference on Visual Communications and Image Processing.Washington D.C., USA:IEEE Press, 2011:1-4.
[11] SHIZAWA M.Direct estimation of multiple disparities for transparent multiple surfaces in binocular stereo[C]//Proceedings of the 4th International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 1993:447-454.
[12] CHEN M J, SU C C, KWON D K, et al.Full-reference quality assessment of stereopairs accounting for rivalry[J]. Image Communication, 2013, 28(9):1143-1155.
[13] OH H, AHN S, KIM J, et al.Blind deep S3D image quality evaluation via local to global feature aggregation[J].IEEE Transactions on Image Processing, 2017, 26(10):4923-4936.
[14] ZHANG W, QU C F, MA L, et al.Learning structure of stereoscopic image for no-reference quality assessment with convolutional neural network[J].Pattern Recognition, 2016, 59:176-187.
[15] SU C C, BOVIK A C, CORMACK L K.Natural scene statistics of color and range[C]//Proceedings of the 18th IEEE International Conference on Image Processing.Washington D.C., USA:IEEE Press, 2011:257-260.
[16] ZHANG L, GU Z, LI H.SDSP:a novel saliency detection method by combining simple priors[C]//Proceedings of IEEE International Conference on Image Processing.Washington D.C., USA:IEEE Press, 2013:171-175.
[17] QIAN N, MIKAELIAN S.Relationship between phase and energy methods for disparity computation[M].Cambridge, USA:MIT Press, 2000.
[18] FIELD D J.Relations between the statistics of natural images and the response properties of cortical cells[J].Journal of the Optical Society of America, 1987, 4(12):2379-2394.
[19] KOVESI P.Image features from phase congruency[J].Journal of Computer Vision Research, 1999, 3(1):1-26.
[20] QI F, JIANG T, MA S, et al.Quality of experience assessment for stereoscopic images[C]//Proceedings of IEEE International Symposium on Circuits and Systems.Washington D.C., USA:IEEE Press, 2012:1712-1715.
[21] ZHANG L, ZHANG L, MOU X, et al.FSIM:a feature similarity index for image quality assessment[J].IEEE Transactions on Image Processing, 2011, 20(8):2378-2386.
[22] HENRIKSSON L, HYVÄRINEN A, VANNI S. Representation of cross-frequency spatial phase relationships in human visual cortex[J].Journal of Neuroence, 2009, 29(45):14342-14351.
[23] MOORTHY A K, SU C C, MITTAL A, et al.Subjective evaluation of stereoscopic image quality[J].Signal Processing Image Communication, 2013, 28(8):870-883.
[24] Video Quality Experts Group.Final report from the video quality experts group on the validation of objective models of video quality assessment, Phase II[EB/OL].[2020-11-01].https://www.its.bldrdoc.gov/vqeg/projects/frtv-phase-ii/frtv-phase-ii.aspx.
[25] SHEIKH H R, SABIR M F, BOVIK A C.A statistical evaluation of recent full reference image quality assessment algorithms[J].IEEE Transactions on Image Processing, 2006, 15(11):3440-3451.
[26] CHEN M J, CORMACK L K, BOVIK A C.No-reference quality assessment of natural stereopairs[J].IEEE Transactions on Image Processing, 2013, 22(9):3379-3391.
[27] KANG L, YE P, LI Y, et al.Convolutional neural networks for no-reference image quality assessment[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2014:1733-1740.
[28] APPINA B, KHAN S, CHANNAPPAYYA S S, et al.No-reference stereoscopic image quality assessment using natural scene statistics[J].Signal Processing:Image Communication, 2016, 43:1-14.
[29] BENOIT A, CALLET L P, CAMPISI P, et al.Quality assessment of stereoscopic images[J].EURASIP Journal on Image & Video Processing, 2009, 2008(1):1-13.
[30] YOU J Y, XING L Y, PERKIS A, et al.Perceptual quality assessment for stereoscopic images based on 2D Image quality metrics and disparity analysis[C]//Proceedings of the 5th International Workshop on Video Processing and Quality Metrics for Consumer Electronics.Arizona, USA:[s.n.], 2010:1-7.
[31] LIU Y, KONG F, ZHEN Z.Toward a quality predictor for stereoscopic images via analysis of human binocular visual perception[J].IEEE Access, 2019, 7:69283-69291.
[32] SHAO F, LI K, LIN W, et al.Full-reference quality assassessment of stereoscopic images by learning binocular receptive fied properties[J].IEEE Transactions on Image Processing, 2015, 24(10):2971-2983.

选择文件类型/文献管理软件名称

选择包含的内容