基于生成感知差异的无参考图像质量评价模型

doi:10.19678/j.issn.1000-3428.0057740

摘要/Abstract

摘要： 人眼视觉系统中的视觉感知差异是图像质量评价过程中的重要组成部分，通过感知失真图像与原始图像之间的视觉差异，可对图像的失真程度进行判断，然而在无参考图像质量评价中无法获取原始未失真的图像，且缺乏对失真图像的视觉感知差异。通过对深度学习中的生成对抗网络进行分析，提出一种基于生成视觉感知差异的无参考图像质量评价模型。利用生成对抗网络产生与失真图像相对应的视觉感知差异图像，并将其与失真图像输入质量评价网络以进一步学习图像的失真信息，从而达到评估图像质量的目的。在TID2008和TID2013数据库上的实验结果表明，与CNN、SOM、CORNIA等模型相比，该模型能够使失真图像质量预测准确度提升1个百分点以上，且对不同种类失真也表现出良好的预测性能。

关键词: 无参考图像质量评价, 视觉感知差异, 生成对抗网络, 深度学习, 损失函数

Abstract: The visual perception difference in the Human Visual System(HVS) is key to image quality evaluation.By sensing the visual difference between the distorted image and the reference image,the distortion degree of the image can be judged.However,this method is inadequate for No-Reference Image Quality Assessment(NR-IQA) where the original undistorted image cannot be obtained for the judgement of visual perception difference.By analyzing the Generative Adversarial Network(GAN) in deep learning,this paper describes a NR-IQA model based on generated perceptual differences.The model employs GAN to generate the visual perception difference image corresponding to the distorted image,and then inputs both kinds of images into the Image Quality Assessment(IQA) network to further learn the distortion information of the image,completing the evaluation of the image quality.The experimental results on TID2008 and TID2013 databases show that compared with CNN,SOM,CORNIA and other models,the proposed model can improve the prediction accuracy of distorted image quality by more than 1 percentage point,and shows excellent performance for different types of distorted images.

Key words: No-Reference Image Quality Assessment(NR-IQA), visual perceptual difference, Generative Adversarial Network(GAN), deep learning, loss function

中图分类号:

TN911.73

刘海, 杨环, 潘振宽, 黄宝香, 侯国家. 基于生成感知差异的无参考图像质量评价模型[J]. 计算机工程, 2021, 47(5): 205-212.

LIU Hai, YANG Huan, PAN Zhenkuan, HUANG Baoxiang, HOU Guojia. No-Reference Image Quality Assessment Model Based on Generated Perceptual Difference[J]. Computer Engineering, 2021, 47(5): 205-212.

https://www.ecice06.com/CN/Y2021/V47/I5/205

参考文献

[1] XU Shaoping,LIN Guanxi,ZENG Xiaoxia,et al.Research and prospect of stereoscopic image quality-aware feature extraction[J].Computer Engineering,2018,44(6):239-248.(in Chinese)徐少平,林官喜,曾小霞,等.立体图像质量感知特征提取的研究与展望[J].计算机工程,2018,44(6):239-248.
[2] WANG Z,BOVIK A C.Modern image quality assessment[J].Synthesis Lectures on Image Video and Multimedia Processing,2006,2(1):1-56.
[3] WANG Z,BOVIK A C,SHEIKH H R,et al.Image quality assessment:from error visibility to structural similarity[J].IEEE Transactions on Image Processing,2004,13(4):600-612.
[4] ZHANG Lin,SHEN Ying,LI Hongyu.VSI:a visual saliency-induced index for perceptual image quality assessment[J].IEEE Transactions on Image Processing,2014,23(10):4270-4281.
[5] YANG Chunling.Gradient-based structural similarity for image quality assessment[J].Journal of South China University of Technology (Natural Science Edition),2006,9(34):22-25.
[6] WANG Z,SIMONCELLI E P,BOVIK A C.Multiscale structural similarity for image quality assessment[C]//Proceedings of the 37th Asilomar Conference on Signals,Systems & Computers.Washington D.C.,USA:IEEE Press,2003:1122-1130.
[7] LI C F,BOVIK A C.Three-component weighted structural similarity index[EB/OL].[2020-02-10].http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=3E96A06DFCD150EFB66EF90C0CDDCA72?doi=10.1.1.153.1729&rep=rep1&type=pdf.
[8] MA Lin,LI Songnan,ZHANG Fan,et al.Reduced-reference image quality assessment using reorganized DCT-based image representation[J].IEEE Transactions on Multimedia,2011,13(4):824-829.
[9] WU Xin,ZHANG Huanlong,SHU Yunxing.Mosaic image quality evaluation method based on visual perception[J].Computer Engineering,2008,34(18):220-222.(in Chinese)武新,张焕龙,舒云星.基于视觉感知的镶嵌图像质量评价方法[J].计算机工程,2008,34(18):220-222.
[10] XIE Rui,SHAO Kun,HUO Xing,et al.An improved DIQaM_FR/NR image quality assessment model[J].Computer Engineering,2020,46(8):258-263,270.(in Chinese)谢瑞,邵堃,霍星,等.一种改进的DIQaM_FR/NR图像质量评价模型[J].计算机工程,2020,46(8):258-263,270.
[11] YAO Wang,LIU Yunpeng,ZHU Changbo.Deep learning of full-reference image quality assessment based on human visual properties[J].Infrared and Laser Engineering,2018,285(7):39-46.(in Chinese)姚旺,刘云鹏,朱昌波.基于人眼视觉特性的深度学习全参考图像质量评价方法[J].红外与激光工程,2018,285(7):39-46.
[12] XUE W F,ZHANG L,MOU X,et al.Gradient magnitude similarity deviation:a highly efficient perceptual image quality index[J].IEEE Transactions on Image Processing,2014,23(2):684-695.
[13] KUMAR V,BAWA V S.No reference image quality assessment metric based on regional mutual information among images[EB/OL].[2020-02-10].https://arxiv.org/pdf/1901.05811.pdf.
[14] LIU Yutao,GU Ke,ZHANG Yongbing,et al.Unsupervised blind image quality evaluation via statistical measurements of structure,naturalness and perception[J].IEEE Transactions on Circuits and Systems for Video Technology,2020,30(4):929-943.
[15] MOORTHY A,BOVIK A.A two-step framework for constructing blind image quality indices[J].IEEE Signal Processing Letters,2010,17(5):513-516.
[16] SAAD M A,BOVIK A C,CHARRIER C.Blind image quality assessment:a natural scene statistics approach in the DCT domain[J].IEEE Transactions on Image Processing,2012,21(8):3339-3352.
[17] KANG Le,YE Peng,LI Yi,et al.Convolutional neural networks for no-reference image quality assessment[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2014:1733-1740.
[18] BIANCO S,CELONA L,NAPOLETANO P,et al.On the use of deep learning for blind image quality assessment[EB/OL].[2020-02-10].https://xueshu.baidu.com/usercenter/paper/show?paperid=2ac3525e407c5c12640b559299838514&site=xueshu_se.
[19] ZENG H,ZHANG L,BOVIK A C.A probabilistic quality representation approach to deep blind image quality prediction[EB/OL].[2020-02-10].https://arxiv.org/pdf/1708.08190.pdf.
[20] KIM J,LEE S.Fully deep blind image quality predictor[J].IEEE Signal Processing,2017,11(1):206-220.
[21] LI Ang,SONG Xiaoying.Remote sensing image superresolution reconstruction based on GAN[J].Optics & Optoelectronic Technology,2019,17(6):39-44.(in Chinese)李昂,宋晓莹.基于生成对抗网络的遥感图像超分辨率重建[J].光学与光电技术,2019,17(6):39-44.
[22] PONOMARENKO N,IEREMEIEV O,LUKIN V,et al.Color image database TID2013:peculiarities and preliminary results[C]//Proceedings of European Workshop on Visual Information Processing. Washington D.C.,USA:IEEE Press,2013:1-12.
[23] SHEIKH H R,SABIR M F,BOVIK A C.A statistical evaluation of recent full reference image quality assessment algorithms[J].IEEE Transactions on Image Processing,2006,15(11):3440-3451.
[24] CHANDER,DAMON M.Most apparent distortion:full-reference image quality assessment and the role of strategy[J].Journal of Electronic Imaging,2010,19(1):12-36.
[25] PONOMARENKO N,LUKIN V,EGIAZARIAN K,et al.Color image database for evaluation of image quality metrics[C]//Proceedings of IEEE Workshop on Multimedia Signal Processing.Washington D.C.,USA:IEEE Press,2008:1223-1236.
[26] MITTAL A,MOORTHY A K,BOVIK A C.No-reference image quality assessment in the spatial domain[J].IEEE Transactions on Image Processing,2012,21(12):4695-4708.
[27] YE P,KUMAR J,KANG L,et al.Unsupervised feature learning framework for no-reference image quality assessment[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2012:1336-1348.
[28] ZHANG Peng,ZHOU Wengang,WU Lei,et al.SOM:semantic obviousness metric for image quality assessment[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:1-6.
[29] MOORTHY A K,BOVIK A C.Blind image quality assessment:from natural scene statistics to perceptual quality[J].IEEE Transactions on Image Processing,2012,20(12):3350-3364.

选择文件类型/文献管理软件名称

选择包含的内容