加权光滑投影孪生支持向量回归算法

doi:10.19678/j.issn.1000-3428.0063542

摘要/Abstract

摘要： 现有双边移位投影孪生支持向量回归（PPTSVR）算法在训练阶段没有考虑不同位置样本对超平面构造的影响，当样本中存在异常点时会降低算法拟合性能。针对该问题，提出一种加权光滑投影孪生支持向量回归算法。采用孤立森林法赋予每个样本不同的权值，并且赋予样本中异常点很小的权值，通过将权值引入算法目标函数，削弱异常点对超平面构造的影响。为直接在原空间中寻求最优超平面，引入正号函数，将有约束优化问题转化为无约束优化问题，并采用Sigmoid光滑函数对目标函数进行光滑处理，证明其任意阶可微且严格凸的特性，进而在原空间中采用牛顿迭代法进行求解。在基准数据集和人工测试函数上的实验结果表明，该算法相比于现有代表性回归算法具备更好的拟合性能和更快的训练速度，尤其当训练样本中存在异常点时，相比于PPTSVR算法拟合性能提升更明显。

关键词: 投影, 孪生支持向量回归, 孤立森林, Sigmoid光滑函数, 牛顿迭代法

Abstract: The existing Pair-shifted Projection Twin Support Vector Regression(PPTSVR) algorithm ignores the effects of samples at different locations on the hyperplane construction during the training process.If there are outliers in the samples, the fitting capacity of the algorithm will be weakened.Therefore, this study proposes a Weighted Smooth Projection Twin Support Vector Regression(WSPTSVR) algorithm.First, an isolation forest approach is utilized to assign different weights to each sample, and the effects of outliers on the hyperplane construction are weakened by assigning tiny weights to them.Second, to find the optimal hyperplane directly in the original space, the algorithm adopts a plus function to convert the constrained optimization problems into unconstrained ones, and utilizes a Sigmoid smooth function to smooth the objective function.It is proved that the objective function is differentiable and strictly convex at any order;then, a Newton iteration method is employed to solve the unconstrained optimization problems in the primal space.Finally, the effectiveness of the proposed algorithm is validated on benchmark datasets and an artificial test function.The experimental results show that the WSPTSVR algorithm outperforms several state-of-the-art algorithms in terms of fitting capacity and training speed.Especially, if there are outliers in the training samples, the fitting capacity of the proposed algorithm is greatly improved compared to that of the PPTSVR algorithm.

Key words: projection, Twin Support Vector Regression(TSVR), isolated forest, Sigmoid smooth function, Newton iteration method

中图分类号:

TP181

徐奔业, 顾斌杰, 潘丰, 熊伟丽. 加权光滑投影孪生支持向量回归算法[J]. 计算机工程, 2022, 48(12): 104-111,118.

XU Benye, GU Binjie, PAN Feng, XIONG Weili. Weighted Smooth Projection Twin Support Vector Regression Algorithm[J]. Computer Engineering, 2022, 48(12): 104-111,118.

http://www.ecice06.com/CN/Y2022/V48/I12/104

图/表 4

参考文献

[1] VAPNIK V N.The nature of statistical learning theory[M].New York, USA:Springer, 1995.
[2] VAPNIK V N.Statistical learning theory[M].New York, USA:Wiley, 1998.
[3] 王海, 翁晨傲, 李克, 等.一种面向基站扇区方向角估计的改进SVM算法[J].计算机工程, 2021, 47(4):120-126. WANG H, WENG C N, LI K, et al.An improved SVM algorithm for azimuth estimation of base station sector[J].Computer Engineering, 2021, 47(4):120-126.(in Chinese)
[4] 鲁淑霞, 蔡莲香, 张罗幻.基于动量加速零阶减小方差的鲁棒支持向量机[J].计算机工程, 2020, 46(12):88-95, 104. LU S X, CAI L X, ZHANG L H.Robust support vector machine based on momentum acceleration zero-order variance reduction[J].Computer Engineering, 2020, 46(12):88-95, 104.(in Chinese)
[5] KIM S T, HAN I G, LEE C Y, et al.A development of unknown intrusion detection system with SVM[J].Convergence Security Journal, 2007, 7(4):23-28.
[6] CHEN Y, DING S H, HU G L, et al.Facial beautification method based on age evolution[J].Computer Aided Drafting, Design and Manufacturing, 2013, 23(4):7-12.
[7] BEN ABID F, ZGARNI S, BRAHAM A.Distinct bearing faults detection in induction motor by a hybrid optimized SWPT and aiNet-DAG SVM[J].IEEE Transactions on Energy Conversion, 2018, 33(4):1692-1699.
[8] 罗彬珅, 刘利民, 董健, 等.基于SAE-GA-SVM模型的雷达新型干扰识别[J].计算机工程, 2020, 46(6):281-287. LUO B S, LIU L M, DONG J, et al.Radar new jamming identification based on SAE-GA-SVM model[J].Computer Engineering, 2020, 46(6):281-287.(in Chinese)
[9] GU B J, FANG J W, PAN F, et al.Fast clustering-based weighted twin support vector regression[J].Soft Computing, 2020, 24(8):6101-6117.
[10] KHEMCHANDANI J R, CHANDRA S.Twin support vector machines for pattern classification[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(5):905-910.
[11] CHEN X, YANG J, YE Q.Recursive projection twin support vector machine via within-class variance minimization[J].Pattern Recognition, 2011, 44(10/11):2643-2655.
[12] PENG X J.TSVR:an efficient twin support vector machine for regression[J].Neural Networks, 2010, 23(3):365-372.
[13] CHEN X B, YANG J, LIANG J, et al.Smooth twin support vector regression[J].Neural Computing and Applications, 2012, 21(3):505-513.
[14] PENG X J, CHEN D.PTSVR:regression models via projection twin support vector machine[J].Information Sciences, 2018, 435:1-14.
[15] LIU F T, TING K M, ZHOU Z H.Isolation forest[C]//Proceedings of the 8th IEEE International Conference on Data Mining.Washington D.C., USA:IEEE Press, 2009:413-422.
[16] LIU F T, TING K M, ZHOU Z H.Isolation-based anomaly detection[J].ACM Transactions on Knowledge Discovery from Data, 2012, 6(1):1-39.
[17] FENG W, SHEN G L, XU B Y, et al.Isolation forest-based least squares twin margin distribution support vector regression[J].International Journal of Innovative Computing, Information and Control, 2021, 17(2):565-579.
[18] DING S F, HUANG H J, XU X Z, et al.Polynomial smooth twin support vector machines[J].Applied Mathematics & Information Sciences, 2014, 8(4):2063-2071.
[19] BARLOW J L.Matrix analysis[J].Computing Reviews, 2013, 54(8):462-463.
[20] WANG L D, GAO C, ZHAO N N, et al.A projection wavelet weighted twin support vector regression and its primal solution[J].Applied Intelligence, 2019, 49(8):3061-3081.
[21] BI J B, BENNETT K P.A geometric approach to support vector regression[J].Neurocomputing, 2003, 55(1/2):79-108.
[22] TANVEER M, SHUBHAM K.A regularization on Lagrangian twin support vector regression[J].International Journal of Machine Learning and Cybernetics, 2017, 8(3):807-821.
[23] LÓPEZ J, MALDONADO S.Robust twin support vector regression via second-order cone programming[J].Knowledge-Based Systems, 2018, 152:83-93.
[24] GU B J, SHEN G L, PAN F, et al.Least squares twin projection support vector regression[J].International Journal of Innovative Computing Information and Control, 2019, 15(6):2275-2288.

选择文件类型/文献管理软件名称

选择包含的内容