基于特征优化的无参考光场图像质量评价

doi:10.19678/j.issn.1000-3428.0064884

摘要/Abstract

摘要：

光场图像因其能够捕捉光的方向信息而备受学术界和产业界的广泛关注，然而，光场图像在压缩和重建过程中常会出现不同程度的失真，影响光场图像的后续处理和应用。因此，需要设计一个光场图像质量评估器来估计失真光场图像的质量。传统的光场图像评价方法在提取光场图像的空间特征和角度特征时，未充分考虑人眼视觉的多通道特性以及人眼对角度变化的敏感性，从而影响光场图像的质量评价结果。为此，提出一种无参考光场图像质量评价方法。设计多频带局部二值模式算法，提取光场图像的空间特征并利用优化提取的空间特征测量光场图像的空间质量。提出加权局部相位量化算法，该算法在对微透镜图像单元中提取的角度特征进行相位量化时，根据角度信息变化的强弱赋予不同的权值。在此基础上，将空间和角度纹理特征结合成一维特征向量，输入到已经训练的支持向量回归中，得到光场图像的质量分数。在Win5-LID和NBU-LF1.0数据集上的实验结果表明，该方法的斯皮尔曼等级相关系数分别为0.799 1和0.803 2，相比SSIM、FSIM、BRISQUE等方法，具有更优的质量评估准确性和稳定性。

关键词: 光场图像, 无参考质量评价, 多频带局部二值模式, 空间特征, 加权局部相位量化, 角度特征

Abstract:

Light Field Image(LFI) is receiving widespread attention from academia and industry owing to its ability to capture directional information from light.However, LFI oftens experiences varying degrees of distortion during compression and reconstruction, which affects the subsequent processing and applications. Accordingly, an LFI quality evaluator can be used to estimate the quality of distorted LFI. Traditional LFI assessment methods do not fully consider the multi-channel characteristics of human vision, or the sensitivity of the human eye to angular changes, when extracting spatial and angular features, thereby affecting the quality assessment results. This paper proposes a method for assessing the quality of no-reference LFI. The multi-band Local Binary Pattern(LBP) algorithm is designed to optimize and extract the spatial features of LFI to measure the images' spatial quality. A weighted Local Phase Quantization (LPQ) algorithm is also proposed to assign different weights according to the strength of angular changes when performing phase quantization on angular features extracted from Micro-Lens Image(MLI) units.On this basis, the spatial and angular texture features are combined into one-dimensional feature vectors, which are input into the trained Support Vector Regression(SVR) to obtain the LFI's quality score.Experimental results on the Win5-LID and NBU-LF1.0 datasets show that the Spearman Rank-order Correlation Coefficient (SRCC) of the proposed method is 0.799 1 and 0.803 2, respectively. Compared to methods such as SSIM, FSIM, and BRISQUE, the proposed approach exhibits better quality assessment accuracy and stability.

Key words: Light Field Image(LFI), no-reference quality assessment, multi-band Local Binary Pattern(LBP), spatial feature, weighted Local Phase Quantization(LPQ), angular feature

徐正梅, 刘华明, 毕学慧, 王亚. 基于特征优化的无参考光场图像质量评价[J]. 计算机工程, 2023, 49(7): 242-250.

Zhengmei XU, Huaming LIU, Xuehui BI, Ya WANG. No-Reference Light Field Image Quality Assessment Based on Feature Optimization[J]. Computer Engineering, 2023, 49(7): 242-250.

http://www.ecice06.com/CN/Y2023/V49/I7/242

图/表 12

参考文献 45

1	王安志, 任春洪, 何淋艳, 等. 基于多模态多级特征聚合网络的光场显著性目标检测. 计算机工程, 2022, 48(7): 227-233, 240 doi: 10.19678/j.issn.1000-3428.0061811
	WANG A Z, REN C H, HE L Y, et al. Light field salient object detection based on multi-modal multi-level feature aggregation network. Computer Engineering, 2022, 48(7): 227-233, 240 doi: 10.19678/j.issn.1000-3428.0061811
2	WANNER S, GOLDLUECKE B. Variational light field analysis for disparity estimation and super-resolution. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(3): 606- 619. doi: 10.1109/TPAMI.2013.147
3	CHEN J, HOU J H, NI Y, et al. Accurate light field depth estimation with superpixel regularization over partially occluded regions. IEEE Transactions on Image Processing, 2018, 27(10): 4889- 4900. doi: 10.1109/TIP.2018.2839524
4	KIM C, ZIMMER H, PRITCH Y, et al. Scene reconstruction from high spatio-angular resolution light fields. ACM Transactions on Graphics, 2013, 32(4): 1- 12.
5	FISS J, CURLESS B, SZELISKI R. Refocusing plenoptic images using depth-adaptive splatting[C]//Proceedings of International Conference on Computational Photography. Washington D. C., USA: IEEE Press, 2014: 1-9.
6	PAUDYAL P, BATTISTI F, SJÖSTRÖM M, et al. Towards the perceptual quality evaluation of compressed light field images. IEEE Transactions on Broadcasting, 2017, 63(3): 507- 522. doi: 10.1109/TBC.2017.2704430
7	刘海, 杨环, 潘振宽, 等. 基于生成感知差异的无参考图像质量评价模型. 计算机工程, 2021, 47(5): 205- 212. doi: 10.19678/j.issn.1000-3428.0057740
	LIU H, YANG H, PAN Z K, et al. No-reference image quality assessment model based on generated perceptual difference. Computer Engineering, 2021, 47(5): 205- 212. doi: 10.19678/j.issn.1000-3428.0057740
8	LIN W S, KUO C C J. Perceptual visual quality metrics: a survey. Journal of Visual Communication and Image Representation, 2011, 22(4): 297- 312. doi: 10.1016/j.jvcir.2011.01.005
9	谢瑞, 邵堃, 霍星, 等. 一种改进的DIQaMFR/NR图像质量评价模型. 计算机工程, 2020, 46(8): 258-263, 270 URL
	XIE R, SHAO K, HUO X, et al. An improved DIQaMFR/NR image quality assessment model. Computer Engineering, 2020, 46(8): 258-263, 270 URL
10	SHI L K, ZHAO S Y, ZHOU W, et al. Perceptual evaluation of light field image[C]//Proceedings of the 25th International Conference on Image Processing. Washington D. C., USA: IEEE Press, 2018: 41-45.
11	WU G C, MASIA B, JARABO A, et al. Light field image processing: an overview. IEEE Journal of Selected Topics in Signal Processing, 2017, 11(7): 926- 954. doi: 10.1109/JSTSP.2017.2747126
12	XIANG J J, JIANG G Y, YU M, et al. No-reference light field image quality assessment based on depth, structural and angular information. Signal Processing, 2021, 184, 108063. doi: 10.1016/j.sigpro.2021.108063
13	ALVES G, PEREIRA M P, DE CARVALHO M B, et al. A study on the 4D sparsity of JPEG pleno light fields using the discrete cosine transform[C]//Proceedings of the 25th International Conference on Image Processing. Washington D. C., USA: IEEE Press, 2018: 1148-1152.
14	SHI L K, ZHAO S Y, CHEN Z B. Belif: blind quality evaluator of light field image with tensor structure variation index[C]//Proceedings of International Conference on Image Processing. Washington D. C., USA: IEEE Press, 2019: 3781-3785.
15	ZHOU W, SHI L K, CHEN Z B, et al. Tensor oriented no-reference light field image quality assessment. IEEE Transactions on Image Processing, 2020, 29, 4070- 4084. doi: 10.1109/TIP.2020.2969777
16	XIANG J J, YU M, CHEN H, et al. VBLFI: visualization-based blind light field image quality assessment[C]//Proceedings of International Conference on Multimedia and Expo. Washington D. C., USA: IEEE Press, 2020: 1-6.
17	XIANG J J, YU M, JIANG G Y, et al. Pseudo video and refocused images-based blind light field image quality assessment. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31(7): 2575- 2590. doi: 10.1109/TCSVT.2020.3030049
18	IHRKE I, RESTREPO J, MIGNARD-DEBISE L. Principles of light field imaging: briefly revisiting 25 years of research. IEEE Signal Processing Magazine, 2016, 33(5): 59- 69. doi: 10.1109/MSP.2016.2582220
19	史丽坤. 光场图像质量评价研究[D]. 合肥: 中国科学技术大学, 2019.
	SHI L K. Evaluation of light field image quality[D]. Hefei: University of Science and Technology of China, 2019. (in Chinese)
20	TIAN Y, ZENG H Q, XING L, et al. A multi-order derivative feature-based quality assessment model for light field image. Journal of Visual Communication and Image Representation, 2018, 57, 212- 217. doi: 10.1016/j.jvcir.2018.11.005
21	MIN X K, ZHOU J T, ZHAI G T, et al. A metric for light field reconstruction, compression, and display quality evaluation. IEEE Transactions on Image Processing, 2020, 29, 3790- 3804. doi: 10.1109/TIP.2020.2966081
22	PAUDYAL P, BATTISTI F, CARLI M. Reduced reference quality assessment of light field images. IEEE Transactions on Broadcasting, 2019, 65(1): 152- 165. doi: 10.1109/TBC.2019.2892092
23	SHI L K, ZHOU W, CHEN Z B, et al. No-reference light field image quality assessment based on spatial-angular measurement. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(11): 4114- 4128. doi: 10.1109/TCSVT.2019.2955011
24	史婷婷, 刘卫华, 伍春晖. 一种新的图像空间特征提取方法. 计算机工程, 2012, 38(3): 218- 220. doi: 10.3969/j.issn.1000-3428.2012.03.073
	SHI T T, LIU W H, WU C H. Novel approach for image spatial feature extraction. Computer Engineering, 2012, 38(3): 218- 220. doi: 10.3969/j.issn.1000-3428.2012.03.073
25	DONG W, BIE H X, LU L K, et al. Image quality assessment by considering multiscale and multidirectional visibility differences in shearlet domain. IEEE Access, 2019, 7, 78715- 78728. doi: 10.1109/ACCESS.2019.2922011
26	LIN Y H, WU J L. Quality assessment of stereoscopic 3D image compression by binocular integration behaviors. IEEE Transactions on Image Processing, 2014, 23(4): 1527- 1542. doi: 10.1109/TIP.2014.2302686
27	PEI S C, CHEN L H. Image quality assessment using human visual DoG model fused with random forest. IEEE Transactions on Image Processing, 2015, 24(11): 3282- 3292. doi: 10.1109/TIP.2015.2440172
28	OJALA T, PIETIKAINEN M, MAENPAA T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7): 971- 987. doi: 10.1109/TPAMI.2002.1017623
29	LI Q H, LIN W S, FANG Y M. No-reference quality assessment for multiply-distorted images in gradient domain. IEEE Signal Processing Letters, 2016, 23(4): 541- 545. doi: 10.1109/LSP.2016.2537321
30	YUE G H, HOU C P, GU K, et al. Biologically inspired blind quality assessment of tone-mapped images. IEEE Transactions on Industrial Electronics, 2017, 65(3): 2525- 2536.
31	LUO Z Y, ZHOU W, SHI L K, et al. No-reference light field image quality assessment based on micro-lens image[C]//Proceedings of Conference on Picture Coding Symposium. Washington D. C., USA: IEEE Press, 2019: 1-5.
32	OJANSIVU V, HEIKKILÄ J. Blur insensitive texture classification using local phase quantization[C]//Proceedings of International Conference on Image and Signal Processing. Berlin, Germany: Springer, 2008: 236-243.
33	马允, 王晓东, 富显祖, 等. 基于GA-SVR模型的无参考立体图像质量评价. 计算机工程, 2017, 43(5): 234-239, 247 URL
	MA Y, WANG X D, FU X Z, et al. No-reference stereoscopic image quality assessment based on GA-SVR model. Computer Engineering, 2017, 43(5): 234-239, 247 URL
34	李翔. 基于SVM和PSO的图像质量评价. 计算机工程, 2012, 38(23): 215- 218. URL
	LI X. Image quality assessment based on support vector machine and particle swarm optimization. Computer Engineering, 2012, 38(23): 215- 218. URL
35	CHANG C C, LIN C J. LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2011, 2(3): 1- 27.
36	HUANG Z J, YU M, JIANG G Y, et al. Reconstruction distortion oriented light field image dataset for visual communication[C]//Proceedings of International Symposium on Networks, Computers and Communications. Washington D. C., USA: IEEE Press, 2019: 1-5.
37	RERABEK M, EBRAHIMI T. New light field image dataset[C]//Proceedings of the 8th International Conference on Quality of Multimedia Experience. Lisbon, Portugal: [s. n. ], 2016: 1-10.
38	HONAUER K, JOHANNSEN O, KONDERMANN D, et al. A dataset and evaluation methodology for depth estimation on 4D light fields[C]//Proceedings of Asian Conference on Computer Vision. Berlin, Germany: Springer, 2017: 19-34.
39	ZHANG S, SHENG H, YANG D, et al. Micro-lens based matching for scene recovery in lenslet cameras. IEEE Transactions on Image Processing, 2018, 27(3): 1060- 1075. doi: 10.1109/TIP.2017.2763823
40	WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13(4): 600- 612. URL
41	ZHANG L, ZHANG L, MOU X Q, et al. FSIM: a feature similarity index for image quality assessment. IEEE Transactions on Image Processing, 2011, 20(8): 2378- 2386. URL
42	MITTAL A, MOORTHY A K, BOVIK A C. No-reference image quality assessment in the spatial domain. IEEE Transactions on Image Processing, 2012, 21(12): 4695- 4708. URL
43	MITTAL A, SOUNDARARAJAN R, BOVIK A C. Making a "completely blind" image quality analyzer. IEEE Signal Processing Letters, 2013, 20(3): 209- 212. URL
44	LI X L, GUO Q, LU X Q. Spatiotemporal statistics for video quality assessment. IEEE Transactions on Image Processing, 2016, 25(7): 3329- 3342.
45	马小雨, 姜秀华. 基于机器学习的全参考图像质量评价模型泛化能力分析. 中国传媒大学学报(自然科学版), 2019, 26(4): 42- 49. URL
	MA X Y, JIANG X H. Analysis for the generalization ability of machine learning based full reference image quality metrics. Journal of Communication University of China (Science and Technology), 2019, 26(4): 42- 49. URL

L_I	Win5-LID			NBU-LF1.0
L_I	PLCC	SRCC	RMSE	PLCC	SRCC	RMSE
2	0.825 2	0.786 1	0.525 8	0.819 8	0.802 2	0.509 0
3	0.831 3	0.791 9	0.522 3	0.821 3	0.802 5	0.505 2
4	0.841 5	0.799 1	0.516 3	0.824 9	0.803 2	0.502 2
5	0.822 9	0.778 0	0.541 0	0.806 0	0.773 6	0.522 4
6	0.804 6	0.763 7	0.569 4	0.783 0	0.749 3	0.563 9

L_I	Win5-LID			NBU-LF1.0
L_I	PLCC	SRCC	RMSE	PLCC	SRCC	RMSE
2	0.825 2	0.786 1	0.525 8	0.819 8	0.802 2	0.509 0
3	0.831 3	0.791 9	0.522 3	0.821 3	0.802 5	0.505 2
4	0.841 5	0.799 1	0.516 3	0.824 9	0.803 2	0.502 2
5	0.822 9	0.778 0	0.541 0	0.806 0	0.773 6	0.522 4
6	0.804 6	0.763 7	0.569 4	0.783 0	0.749 3	0.563 9

B_n	Win5-LID			NBU-LF1.0
B_n	PLCC	SRCC	RMSE	PLCC	SRCC	RMSE
2	0.688 9	0.587 1	0.587 1	0.533 4	0.469 4	0.772 4
4	0.784 7	0.716 6	0.593 7	0.588 7	0.555 8	0.732 7
8	0.798 1	0.750 6	0.571 8	0.689 5	0.648 4	0.652 5
16	0.795 8	0.763 1	0.565 5	0.778 1	0.751 1	0.553 6
32	0.839 8	0.794 7	0.517 0	0.791 0	0.770 0	0.542 6
64	0.841 5	0.799 1	0.516 3	0.824 9	0.803 2	0.502 2
128	0.822 2	0.784 0	0.539 3	0.822 8	0.793 9	0.503 1
256	0.817 9	0.782 9	0.538 5	0.822 2	0.795 9	0.502 8

B_n	Win5-LID			NBU-LF1.0
B_n	PLCC	SRCC	RMSE	PLCC	SRCC	RMSE
2	0.688 9	0.587 1	0.587 1	0.533 4	0.469 4	0.772 4
4	0.784 7	0.716 6	0.593 7	0.588 7	0.555 8	0.732 7
8	0.798 1	0.750 6	0.571 8	0.689 5	0.648 4	0.652 5
16	0.795 8	0.763 1	0.565 5	0.778 1	0.751 1	0.553 6
32	0.839 8	0.794 7	0.517 0	0.791 0	0.770 0	0.542 6
64	0.841 5	0.799 1	0.516 3	0.824 9	0.803 2	0.502 2
128	0.822 2	0.784 0	0.539 3	0.822 8	0.793 9	0.503 1
256	0.817 9	0.782 9	0.538 5	0.822 2	0.795 9	0.502 8

类型	方法	Win5-LID			NBU-LF1.0
类型	方法	PLCC	SRCC	RMSE	PLCC	SRCC	RMSE
FR 2DIQA	SSIM	0.780 0	0.719 8	0.611 9	0.821 9	0.792 2	0.506 9
FR 2DIQA	FSIM	0.764 2	0.706 2	0.631 8	0.823 3	0.791 1	0.509 2
NR 2DIQA	BRISQUE	0.563 0	0.454 7	0.797 0	0.486 8	0.379 2	0.783 8
NR 2DIQA	NIQE	0.528 1	0.440 3	0.815 3	0.421 0	0.302 6	0.828 5
FR LFIQA	MDFM	0.768 6	0.733 7	0.630 9	0.800 5	0.758 4	0.539 0
FR LFIQA	文献[21]方法	0.720 7	0.642 9	0.691 8	0.685 3	0.638 0	0.668 3
NR LFIQA	BELIF	0.602 0	0.519 5	0.752 6	0.708 4	0.656 9	0.622 8
	NR-LFQA	0.698 6	0.654 5	0.679 9	0.824 6	0.786 5	0.507 7
	Tensor-NLFQ	0.579 6	0.474 3	0.776 8	0.668 2	0.595 0	0.658 3
	VBLFI	0.690 1	0.640 3	0.689 3	0.779 2	0.734 9	0.561 5
	文献[17]方法	0.749 1	0.701 4	0.627 0	0.811 3	0.770 5	0.520 5
	本文方法	0.841 5	0.799 1	0.516 3	0.824 9	0.803 2	0.502 2

选择文件类型/文献管理软件名称

选择包含的内容