Research on the Implementation and Optimization of Image Filtering Algorithm Based on OpenGL ES

doi:10.19678/j.issn.1000-3428.0067337

Abstract

Abstract:

Image filtering algorithms have wide applications in such fields as machine learning, image processing, and image recognition.They play an important role in reducing "salt and pepper" noise, image binarization, edge recognition, and feature extraction. Although common image filtering algorithms are implemented in the OpenCV open source library, a significant gap in performance exists compared with other platforms on the Android platform. With the rapid development of embedded platforms, the performance requirements for filtering algorithms on embedded platforms have become increasingly high in practical applications. Therefore, starting with filtering algorithms with wide application scenarios, such as morphological filtering, box filtering, threshold filtering, compression filtering, and arithmetic filtering, a series of high-performance image filtering algorithms designed for the Android platform based on OpenGL ES are developed and implemented. OpenGL ES calculation shaders are used to accelerate the algorithm in parallel, using texture objects for memory optimization, and in-depth optimization in image boundary processing, image data types, and data communication is conducted. This approach resulted in better performance. The optimized image filtering algorithm is compared with the corresponding algorithm in the open-source OpenCV library. The experimental results show that the overall performance of the image filtering algorithm based on the Android platform using the OpenGL ES interface is significantly better than the performances of the relevant algorithms in the OpenCV library. The larger the image size, the more obvious the computational advantage. The maximum performance improvement is 110.018 times that of the corresponding algorithm in the OpenCV library.

Key words: OpenGL ES interface, image filtering, embedded platform, calculation shader, image texture

摘要：

图像滤波算法广泛应用在机器学习、图像处理、图像识别等领域，在降低椒盐噪声、图像二值化、边缘识别和特征提取等方面发挥着重要作用。现有图像滤波算法虽然在OpenCV开源库中均有实现，但是在Android平台的性能表现与其他平台相比仍存在较大差距。随着嵌入式平台的高速发展，实际应用对嵌入式平台上滤波算法的性能要求越来越高。为此，从形态学滤波、盒式滤波、阈值滤波、压缩滤波、算术滤波等具有广泛应用场景的滤波算法入手，提出并实现针对Android平台设计、基于OpenGL ES的一系列高性能图像滤波算法。使用OpenGL ES计算着色器对算法进行并行加速，使用纹理对象进行访存优化，并在图像边界处理、图像数据类型和数据通信等方面进行深入优化，取得了较优的性能。将优化的图像滤波算法与开源OpenCV库中对应算法进行对比，实验结果表明，基于Android平台使用OpenGL ES接口的图像滤波算法整体性能明显优于OpenCV库中的相关算法，且图像规模越大计算优势越明显，性能最大提升为OpenCV库中对应算法的110.018倍。

关键词: OpenGL ES接口, 图像滤波, 嵌入式平台, 计算着色器, 图像纹理

Wenbin CHANG, Mingren MU, Haipeng JIA, Yunquan ZHANG, Sijia ZHANG. Research on the Implementation and Optimization of Image Filtering Algorithm Based on OpenGL ES[J]. Computer Engineering, 2023, 49(11): 257-266.

常文斌, 牟明任, 贾海鹏, 张云泉, 张思佳. 基于OpenGL ES的图像滤波算法实现及优化研究[J]. 计算机工程, 2023, 49(11): 257-266.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0067337

https://www.ecice06.com/EN/Y2023/V49/I11/257

Figures/Tables 15

Fig.1 Schematic diagram of principle of the box filtering algorithm

Fig.2 Schematic diagram of the principle of morphological erode filtering algorithm

Fig.3 Schematic diagram of naive implementation of adaptive threshold filtering

Fig.4 Schematic diagram of three-channel image compression filtering algorithm

Fig.5 Schematic diagram of boundary treatment method

Fig.6 Schematic diagram of two-dimensional workgroup division

Fig.7 Schematic diagram of three-channel image parallel algorithm

Fig.8 Schematic diagram of Uniform binding data

Fig.9 Comparison of morphological filtering performance

Fig.10 Comparison of 8UC1 box filtering performance

Fig.11 Comparison of 8UC3 box filtering performance

Fig.12 Comparison of adaptive threshold filtering performance

Fig.13 Comparison of arithmetic filtering performance

Fig.14 Comparison of compression filtering performance

References 28

1	KHRNOS Group. OpenCL[EB/OL]. [2023-03-01]. https://www.khronos.org/opengl/.
2	KHRONOS Group. OpenGL ES 3.2 specific[EB/OL]. [2023-03-01]. https://www.khronos.org/registry/OpenGL/specs/es/ation.3.2/es_spec_3.2.pdf.
3	WIJAYANTI S, MUKTAMAR B A, WIBIRAMA S, et al. A new native video filtering based on OpenGL ES for mobile platform. International Journal on Advanced Science, Engineering and Information Technology, 2019, 9(3): 759- 765. doi: 10.18517/ijaseit.9.3.7329
4	WU C, YANG B W, ZHU W W, et al. Toward high mobile GPU performance through collaborative workload offloading. IEEE Transactions on Parallel and Distributed Systems, 2018, 29(2): 435- 449. doi: 10.1109/TPDS.2017.2754482
5	AAEN L, DAM H H. Performance analysis of OpenGL ES 3.2 compute shaders on mobile GPUs[C]//Proceedings of the 20th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. New York, USA: ACM Press, 2017: 38-49.
6	PARK S. A GPU algorithm for matrix-matrix multiplication using OpenGL compute shader. Korean Journal of Computational Design and Engineering, 2018, 23(1): 1- 10. doi: 10.7315/CDE.2018.001
7	KANAGASABAPATHI C, SIDDAMMA Y, YELLAMPALLI S S. Design of programmable pixel shader computing unit. International Journal of Innovative Technology and Exploring Engineering, 2019, 8(9): 269- 272.
8	BOSSÉR L, BOSSÉR J D. Target echo calculations using the OpenGL graphics pipeline. Applied Acoustics, 2021, 181, 108133. doi: 10.1016/j.apacoust.2021.108133
9	MADERA-RAMIREZ F A, LOPEZ-MARTINEZ J L, MOO-MENA F, et al. CPU-GPU buffer communication using compute shader to fill volumes with spheres. The Journal of Supercomputing, 2022, 78(5): 6448- 6460. doi: 10.1007/s11227-021-04136-1
10	XU J, CHEN X. Study and analysis texture mapping technology based on OpenGL. Applied Mechanics and Materials, 2012, 229/230/231, 1990- 1993.
11	刘子璋. 基于Android的图片剧制作工具系统的设计与实现[D]. 北京: 北京交通大学, 2020.
	LIU Z Z. Design and implementation of the tool system for making picture drama based on Android[D]. Beijing: Beijing Jiaotong University, 2020. (in Chinese)
12	闫隆鑫. 基于定制Android平台的超声相控阵软件设计与实现[D]. 南京: 东南大学, 2019.
	YAN L X. Design and implementation of ultrasonic phased array software based on customized Android platform[D]. Nanjing: Southeast University, 2019. (in Chinese)
13	NVIDIA. CUDA[EB/OL]. [2023-03-01]. https://docs.nvidia.com/cuda/.
14	丁毅乐. 基于CUDA架构的遥感图像滤波算法并行处理[D]. 郑州: 解放军信息工程大学, 2017.
	DING Y L. Parallel processing of remote sensing image filtering algorithm based on CUDA architecture[D]. Zhengzhou: PLA Information Engineering University, 2017. (in Chinese)
15	韦存阳, 贾海鹏, 张云泉, 等. 基于ARMv8处理器的高性能图像处理算法实现与优化研究. 计算机工程与科学, 2022, 44(10): 1711- 1720. URL
	WEI C Y, JIA H P, ZHANG Y Q, et al. Research on implementation and optimization of high performance image processing algorithm based on ARMv8 processor. Computer Engineering & Science, 2022, 44(10): 1711- 1720. URL
16	AYSE Y M. Graph-Waving architecture: efficient execution of graph applications on GPUs. Journal of Parallel and Distributed Computing, 2021, 148, 69- 82. doi: 10.1016/j.jpdc.2020.10.005
17	ODEN L, KELLER J. Improving cryptanalytic applications with stochastic runtimes on GPUs and multicores. Parallel Computing, 2022, 112, 102944. doi: 10.1016/j.parco.2022.102944
18	LEE S, KYE H. Efficient MIP volume rendering via fast SIMD interpolation and memory access reordering. Multimedia Tools and Applications, 2023, 82(7): 10515- 10534. doi: 10.1007/s11042-022-13732-z
19	KAGEYAMA K, OGURA T, FUJITA T, et al. Implementation of max-plus algebra-based morphological wavelet transform algorithm with CAM-based massive-parallel SIMD matrix core. IEICE Proceeding Series, 2022, 71, 305- 308.
20	CHEN S Z, FANG J B, XU C F, et al. Adaptive hybrid storage format for sparse matrix-vector multiplication on multi-core SIMD CPUs. Applied Sciences, 2022, 12(19): 9812. doi: 10.3390/app12199812
21	CHOUDHARY J, CHOUDHARY A. Enhancement in morphological mean filter for image denoising using GLCM algorithm. International Journal of Computer Theory and Engineering, 2021, 13(4): 134- 137. doi: 10.7763/IJCTE.2021.V13.1302
22	张志禹, 李向月, 孟令辉. 随机投影深度排序的矢量中值滤波算法. 计算机工程, 2016, 42(10): 242- 248. URL
	ZHANG Z Y, LI X Y, MENG L H. Vector median filtering algorithm of random projection depth sorting. Computer Engineering, 2016, 42(10): 242- 248. URL
23	雷涛, 樊养余, 罗维薇, 等. 矢量自对偶形态学滤波算子. 自动化学报, 2015, 41(5): 1013- 1023. URL
	LEI T, FAN Y Y, LUO W W, et al. Vector self-dual morphological filtering operators. Acta Automatica Sinica, 2015, 41(5): 1013- 1023. URL
24	DÖLLNER J. Interactive oil paint filtering on mobile devices[EB/OL]. [2023-03-01]. https://hpi.de/doellner/people/trapp/year/2016/102999/STPD2016.html.
25	郑永恒, 程建, 曹宗杰. 改进非局部均值滤波的SAR图像降噪. 中国图象图形学报, 2012, 17(7): 886- 891. URL
	ZHENG Y H, CHENG J, CAO Z J. SAR image denoising via improved non-local means filter. Journal of Image and Graphics, 2012, 17(7): 886- 891. URL
26	PEDRINO E C, DE LIMA D P, TEMPESTI G. A multiobjective metaheuristic approach for morphological filters on many-core architectures. Integrated Computer-Aided Engineering, 2019, 26(4): 383- 397. doi: 10.3233/ICA-190607
27	PAPADOPOULOS K, VLACHOS K. Efficient projective transformation and Lanczos interpolation on ARM platform using SIMD instructions[EB/OL]. [2023-03-01]. https://www.researchgate.net/profile/Konstantinos-Papadopoulos-14/publication/322881948_Efficient_Projective_Transformation_and_Lanczos_Interpolation_on_ARM_Platform_using_SIMD_Instructions/links/5a7878920f7e9b41dbd42897/Efficient-Projective-Transformation-and-Lanczos-Interpolation-on-ARM-Platform-using-SIMD-Instructions.pdf.
28	Arm Community. Arm Mali GPU best practices developer guide Version2.2[EB/OL]. [2023-03-01]. https://developer.arm.com/documentation/101897/v2-2/Compute/Shared-memory.

[1]	MAO Ruirui, JIANG Bo. Highly Real-Time Method of Small Target Tracking and Detection for Embedded Device [J]. Computer Engineering, 2021, 47(12): 291-298.
[2]	YIN Yanqing, GONG Huajun, WANG Xinhua. Embedded Real-time Video Object Detection Algorithm Based on YOLOv3 [J]. Computer Engineering, 2020, 46(2): 230-234.
[3]	WU Yunda,ZHANG Tao,HOU Xiaodan,XU Chen. Universal Unsupervised Steganalysis Forensics Algorithm Based on Outlier Detection [J]. Computer Engineering, 2017, 43(11): 239-244,251.
[4]	ZHANG Zhiyu,LI Xiangyue,MENG Linghui. Vector Median Filtering Algorithm of Random Projection Depth Sorting [J]. Computer Engineering, 2016, 42(10): 242-248.
[5]	HU Lian-da, ZHANG Ji. Research on Embedded Graphics System Performance Optimization Based on GTK+ [J]. Computer Engineering, 2014, 40(1): 287-290,294.
[6]	GONG Li-Xia, TUN Fan, CENG Qi-Meng, ZHANG Jing-Fa. Multi-temporal Combination Filtering Method of SAR Images [J]. Computer Engineering, 2011, 37(21): 4-5,10.
[7]	SHI Bai-Ying, YANG Xiao-Guang, SHU Tong. Design and Realization of Local Intelligence System of Signalized Intersection [J]. Computer Engineering, 2010, 36(19): 5-7.
[8]	ZHANG Xiao-bo; HU Wei-wei; DING Wen-rui; ZHANG Qi-shan. Control System for Real-time Video Encoding Application [J]. Computer Engineering, 2008, 34(19): 225-227.
[9]	ZHANG Hengbo; OU Zongying. Image Retrieval Method Based on Color Primitive Co-occurrence Matrix [J]. Computer Engineering, 2007, 33(14): 171-173.

Please choose a citation manager

Content to export