基于概率矩阵分解的不完整数据集特征选择方法

doi:10.19678/j.issn.1000-3428.0061524

摘要/Abstract

摘要： 在机器学习理论与应用中，特征选择是降低高维数据特征维度的常用方法之一。传统的特征选择方法多数基于完整数据集，对实际应用中普遍存在缺失数据的情形研究较少。针对不完整数据中含有未被观察信息和存在异常值的特点，提出一种基于概率矩阵分解技术的鲁棒特征选择方法。使用基于分簇的概率矩阵分解模型对数据集中的缺失值进行近似估计，以有效测量相邻簇之间数据的相似性，缩小问题规模，同时降低填充误差。依据缺失数据值存在少量异常值的情形，利用基于l_2，1损失函数的方法进行特征选择，在此基础上给出不完整数据集的特征选择方法流程，并对其收敛性进行理论分析。该方法利用不完整数据集中的所有信息，有效应对不完整数据集中异常值带来的影响。实验结果表明，相比传统特征选择方法，该方法在合成数据集上选择更少的无关特征，可降低异常值带来的影响，在真实数据集上获得了较高的分类准确率，能够选择出更为准确的特征。

关键词: 矩阵分解, 缺失值填补, 鲁棒特征选择, 不完整数据, 1_2,1范数

Abstract: In machine learning theory and application, feature selection is one of the common methods of reducing the feature dimension of high-dimensional data.Traditional feature selection methods are mostly based on complete data sets, and a few studies have been conducted on missing data in practical applications.In this study, a robust feature selection method is proposed based on Probability Matrix Decomposition(PMF) for incomplete data containing unobserved information and outliers.First, a probabilistic matrix decomposition model, based on clustering, is used to approximate the missing values in the data set.The model can effectively measure data similarity between adjacent clusters, reduce the scale of the problem, and reduce the imputation error.Secondly, the feature selection method, based on loss function, is used in the case involving missing data values with a few outliers.Finally, the flow of feature selection method for incomplete data sets is constructed, and its convergence is theoretically analyzed.The method proposed in this studyutilizes all the information in incomplete data sets and effectively deals with the influence of outliers in incomplete data sets.Experimental results show that when compared with traditional feature selection methods, the proposed method can select fewer irrelevant features in the synthetic data set and reduce the influence of outliers.Conversely, on real data sets, the proposed method realizes higher classification accuracy and selects more accurate features.

Key words: matrix decomposition, missing value filling, Robust Feature Selection(RFS), incomplete data, l_2,1 norm

中图分类号:

TP18

范林歌, 武欣嵘, 童玮, 曾维军. 基于概率矩阵分解的不完整数据集特征选择方法[J]. 计算机工程, 2022, 48(6): 57-64.

FAN Linge, WU Xinrong, TONG Wei, ZENG Weijun. Feature Selection Method for Incomplete Data Sets Based on Probability Matrix Decomposition[J]. Computer Engineering, 2022, 48(6): 57-64.

https://www.ecice06.com/CN/Y2022/V48/I6/57

图/表 11

20220621184335

20220621184340

20220621184344

20220621184348

20220621184353

20220621184357

20220621184401

20220621184406

20220621184410

20220621184414

20220621184418

参考文献

[1] ZHU X F, LI X L, ZHANG S C, et al.Robust joint graph sparse coding for unsupervised spectral feature selection[J].IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(6):1263-1275.
[2] ZHANG Z, LIU L, SHEN F M, et al.Binary multi-view clustering[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(7):1774-1782.
[3] ZHU X F, ZHANG S C, ZHU Y H, et al.Self-weighted multi-view fuzzy clustering[J].ACM Transactions on Knowledge Discovery from Data, 2020, 14(4):1-17.
[4] HU R Y, ZHU X F, ZHU Y H, et al.Robust SVM with adaptive graph learning[J].World Wide Web, 2020, 23(3):1945-1968.
[5] TSAGRIS M, PAPADOVASILAKIS Z, LAKIOTAKI K, et al.The γ-OMP algorithm for feature selection with application to gene expression data[J].IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2020, 99(1):1-10.
[6] CISMONDI F, FIALHO A S, VIEIRA S M, et al.Missing data in medical databases:impute, delete or classify?[J].Artificial Intelligence in Medicine, 2013, 58(1):63-72.
[7] GARCIA C, LEITE D, ŠKRJANC I.Incremental missing-data imputation for evolving fuzzy granular prediction[J].IEEE Transactions on Fuzzy Systems, 2020, 28(10):2348-2362.
[8] SIMONE R.An accelerated EM algorithm for mixture models with uncertainty for rating data[J].Computational Statistics, 2021, 36(1):691-714.
[9] ZHANG S C, LI X L, ZONG M, et al.Efficient kNN classification with different numbers of nearest neighbors[J].IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(5):1774-1785.
[10] PAN R L, YANG T S, CAO J H, et al.Missing data imputation by K nearest neighbours based on grey relational structure and mutual information[J].Applied Intelligence, 2015, 43(3):614-632.
[11] SEFIDIAN A M, DANESHPOUR N.Missing value imputation using a novel grey based fuzzy c-means, mutual information based feature selection, and regression model[J].Expert Systems with Applications, 2019, 115:68-94.
[12] HUANG C C, LEE H M.A grey-based nearest neighbor approach for missing attribute value prediction[J].Applied Intelligence, 2004, 20(3):239-252.
[13] TIAN J, YU B, YU D, et al.Missing data analyses:a hybrid multiple imputation algorithm using gray system theory and entropy based on clustering[J].Applied Intelligence, 2014, 40(2):376-388.
[14] LOUS, OBRADOVIC Z.Margin-based feature selection in incomplete data[C]//Proceedings of AAAI Conference on Artificial Intelligence.[S.1.]:AAAI Press, 2012:125-136.
[15] NIE F, HUANG H, XIAO C, et al.Efficient and robust feature selection via joint ℓ_2,1-norms minimization[C]//Proceedings of International Conference on Neural Information Processing Systems.[S.1.]:Curran Associates Inc., 2010:389-397.
[16] YU H Y, GAO L R, LIAO W Z, et al.Global spatial and local spectral similarity-based manifold learning group sparse representation for hyperspectral imagery classification[J].IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(5):3043-3056.
[17] ZHANG C K, WANG C.Probabilistic matrix factorization recommendation of self-attention mechanism convolutional neural networks with item auxiliary information[J].IEEE Access, 2020, 8:208311-208321.
[18] MA C, LI Y X, CHI Y J.Beyond procrustes:balancing-free gradient descent for asymmetric low-rank matrix sensing[J].IEEE Transactions on Signal Processing, 2021, 69(1):867-877.
[19] GILAD-BACHRACH R, NAVOT A, TISHBY N.Margin based feature selection-theory and algorithms[C]//Proceedings of the 21st International Conference on Machine Learning.New York, USA:ACM Press, 2004:452-466.
[20] KIRA K, RENDELL L A.Feature selection problem:traditional methods and a new algorithm[C]//Proceedings of the 20th IEEE National Conference on Artificial Intelligence.Washington D.C., USA:IEEE Press, 1992:129-134.
[21] AMINI M R, USUNIER N, GOUTTE C.Uci-dataset-url[EB/OL].[2021-03-20]. https://dblp.org/rec/journals/prl/Brito.
[22] LIU Y, JIANG Z S, XIANG J W.An adaptive cross-validation thresholding de-noising algorithm for fault diagnosis of rolling element bearings under variable and transients conditions[J].IEEE Access, 2020, 8:67501-67518.

选择文件类型/文献管理软件名称

选择包含的内容