作者投稿和查稿 主编审稿 专家审稿 编委审稿 远程编辑

计算机工程

• •    

增量式稀疏密度加权孪生支持向量回归机

  • 发布日期:2023-12-05

Incremental Sparse Density-weighted Twin Support Vector Regression

  • Published:2023-12-05

摘要: 现有密度加权孪生支持向量回归机(DWTSVR)是一种能够反映数据内在分布的回归算法,具有预测精度高和鲁棒性等优点,然而其并不适用于训练样本是以增量形式提供的场景。针对该问题,提出一种增量式稀疏密度加权孪生支持向量回归机(ISDWTSVR)。首先,辨别新增数据是否为异常样本,并赋予有效样本适当的权重,降低异常样本对模型泛化性能的影响。接着,结合矩阵降维与主成分分析的思想,筛选出原始核矩阵中的一组特征列向量基代替原特征,实现核矩阵列稀疏化以获得稀疏解。其次,借助牛顿迭代法和增量学习策略,对上一时刻的模型信息进行调整,实现模型的增量更新。此外,引入矩阵求逆引理避免增量更新过程中直接求解逆矩阵,进一步加快训练速度。最后,在UCI基准数据集上进行仿真实验,并与现有代表性算法进行比较。结果表明,ISDWTSVR继承了DWTSVR的泛化性能,在大规模数据集Bike-Sharing上,新增一个样本模型更新所需的平均时间为5.13秒,较DWTSVR缩短了97.94%,有效地解决了模型从头开始重新进行训练的问题,适用于大规模数据集的在线学习。

Abstract: The existing density-weighted twin support vector regression machine (DWTSVR) is a regression algorithm that can reflect the internal distribution of data, and has the advantages of high prediction accuracy and robustness. However, DWTSVR is unsuitable for the scenarios where the training samples are provided in incremental form. To solve this problem, this paper proposes an incremental sparse density-weighted twin support vector regression machine (ISDWTSVR). First, in order to reduce the impact of abnormal samples on the generalization performance of the model, whether the new data is an abnormal sample are identified, and appropriate weights are assigned to valid samples. Next, combining the ideas of matrix dimensionality reduction and principal component analysis, a set of feature column vector bases in the original kernel matrix is screened to replace the original features, achieving sparsity of the kernel matrix and obtaining sparse solutions. Secondly, with the help of Newton iteration method and incremental learning strategy, the model information of the previous moment is adjusted to achieve incremental update of the model. In addition, the matrix inverse lemma is introduced to avoid solving the inverse matrix directly during the process of incremental updating, which further speeds up the training. Finally, simulation experiments are performed on UCI benchmark datasets and compared with existing representative algorithms. The results show that ISDWTSVR inherits the generalization performance of DWTSVR. When adding a new sample on the large-scale dataset Bike-Sharing, the average time elapsed for model update is 5.13 seconds, which is 97.94% shorter than DWTSVR. This effectively solves the problem of model retraining from scratch and is suitable for online learning of large-scale datasets.