基于Lasso稀疏学习的径向基函数神经网络模型

doi:10.19678/j.issn.1000-3428.0048910

摘要/Abstract

摘要：

传统径向基函数(RBF)神经网络模型使用完整的隐含层节点进行模型构建时,会因缺乏隐含层节点抽取机制而使得受训模型的泛化性能下降,导致模型更加复杂。为此,提出一种改进的RBF神经网络模型。通过Lasso稀疏约束对隐含层节点和输出层连接权值进行稀疏表示,去除冗余和不相关隐含层节点的同时保留重要的隐含层节点,并使用交叉验证和网格搜索确定收缩参数以优化模型分类性能。实验结果表明,与现有RBF神经网络模型相比,该模型具有更低的计算复杂度和更高的分类精度。

关键词: 数据挖掘, Lasso稀疏学习, 径向基函数, 神经网络, 收缩参数

Abstract:

The traditional Radial Basis Function(RBF) neural network model uses all hidden layer nodes to construct the model.In this case,the generalization performance of traditional RBF neural network model is degraded because of the lackness of the effective hidden layer nodes extraction mechanism,which will easily leads to more complicated of model.In order to solve this problem,this paper proposes an improved RBF neural network model.It realizes sparse representation of hidden layer nodes and output layer connection weights by Lasso sparse constraint to remove redundant and uncorrelated hidden layer nodes,and retain important hidden layer nodes.The shrinkage parameter are determined by Cross Validation(CV) and grid search,so as to optimize model classification performance.Experimental results show that RBF neural network model based on Lasso sparse learning can not only reduce the calculation complexity of the model,but also improve the classification accuracy compared with existing RBF neural network model.

Key words: data mining, Lasso sparse learning, Radial Basis Function(RBF), neural network, shrinkage parameter

中图分类号:

TP391

崔晨,邓赵红,王士同. 基于Lasso稀疏学习的径向基函数神经网络模型[J]. 计算机工程, 2019, 45(2): 173-177.

CUI Chen,DENG Zhaohong,WANG Shitong. Radial Basis Function Neural Network Model Based on Lasso Sparse Learning[J]. Computer Engineering, 2019, 45(2): 173-177.

https://www.ecice06.com/CN/Y2019/V45/I2/173

参考文献

［1］LIU G P.Nonlinear identification and control［J］.Industrial Robot,2002,29(5):469-470.
［2］DUFOUR P,BHARTIYA S,DHURJATI P S,et al.Neural network-based software sensor:training set design and application to a continuous pulp digester［J］.Control Engineering Practice,2005,13(2):135-143.
［3］FORTUNA L,GRAZIANI S,XIBILIA M G.Soft sensors for product quality monitoring in debutanizer distillation columns［J］.Control Engineering Practice,2005,13(4):499-508.
［4］GONZAGA J C B,MELEIRO L A C,KIANG C,et al.ANN-based soft-sensor for real-time process monitoring and control of an industrial polymerization process［J］.Computers and Chemical Engineering,2009,33(1):43-49.
［5］RANI A,SINGH V,GUPTA J R P.Development of soft sensor for neural network based control of distillation column［J］.ISA Transactions,2013,52(3):438-449.
［6］RODGER J A.A fuzzy nearest neighbor neural network statistical model for predicting demand for natural gas and energy cost savings in public buildings［J］.Expert Systems with Applications,2014,41(4):1813-1829.
［7］YAN W.Toward automatic time-series forecasting using neural networks［J］.IEEE Transactions on Neural Networks and Learning Systems,2012,23(7):1028-1039.
［8］ZHANG J,MARTIN E B,MORRIS A J,et al.Inferential estimation of polymer quality using stacked neural networks［J］.Computers and Chemical Engineering,1997,21(Suppl):1037-1039.
［9］王士同.神经模糊系统及其应用［M］.北京:北京航空航天大学出版社,1998.
［10］DING Shifei,MA Gang,SHI Zhongzhi.A rough RBF neural network based on weighted regularized extreme learning machine［J］.Neural Processing Letters,2014,40(3):245-260.
［11］WU Jiansheng,LONG Jin,LIU Mingzhe.Evolving RBF neural networks for rainfall prediction using hybrid particle swarm optimization and genetic algorithm［J］.Neurocomputing,2015,148(2):136-142.
［12］NIROS A D,TSEKOURAS G E,TSOLAKIS D,et al.Hierarchical fuzzy clustering in conjunction with particle swarm optimization to efficiently design RBF neural networks［J］.Journal of Intelligent and Robotic Systems,2015,78(1):105-125.
［13］NIROSA D,TSEKOURAS G E.A novel training algorithm for RBF neural network using a hybrid fuzzy clustering approach［J］.Fuzzy Sets and Systems,2012,193(9):62-84.
［14］张江滨,邓赵红,王士同.模糊子空间聚类的径向基函数神经网络建模［J］.计算机科学与探索,2015,9(12):1513-1522.
［15］TIBSHIRANI R.Regression shrinkage and selection via the Lasso［J］.Journal of the Royal Statistical Society,1996,58(1):267-288.
［16］ZOU Hui.The adaptive Lasso and its oracle properties［J］.Journal of the American Statistical Association,2012,101(476):1418-1429.
［17］RADCHENKOP,JAMES G M.Improved variable selection with forward-Lasso adaptive shrinkage［J］.Annals of Applied Statistics,2011,5(1):427-448.
［18］HUANG Guangbin,ZHU Qinyu,SIEW C K.Extreme learning machine:theory and applications［J］.Neurocomputing,2006,70(1-3):489-501.
［19］HUANG Guangbin.Learning capability and storage capacity of two-hidden-layer feedforward networks［J］.IEEE Transactions on Neural Networks,2003,14(2):274-281.
［20］ZHANG Rong,DENG Zhaohong,WANG Shitong,et al.Robust single hidden layer feed-forward neural networks modeling for small datasets［J］.Control and Decision,2012,27(9):1308-1312.
［21］SANG Qingbing,DENG Zhaohong,WANG Shitong,et al.ε-insensitive criterion and structure risk based radius-basis-function neural-network modeling［J］.Journal of Electronics and Information Technology,2012,34(6):1414-1419.
［22］NIE F,HUANG H,CAI X,et al.Efficient and robust feature selection via joint 2,1-norms minimiza-tion［C］//Proceedings of Conference on Neural Information Processing Systems.Vancouver,Canada:Curran Associates,Inc.,2010:1813-1821.
［23］BEZDEK J C,EHRLICH R,FULL W.FCM:the fuzzy C-means clustering algorithm［J］.Computers and Geosciences,1984,10(2):191-203.
［24］LOCKLEY S W,EVANS E E,SCHEER F,et al.Short-wavelength sensitivity for the direct effects of light on alertness,vigilance,and the waking electroencephalogram in humans［J］.Sleep,2006,29(2):161-168.
［25］DEBENER S,ULLSPERGER M,SIEGEL M,et al.Trial-by-trial coupling of concurrent electroencephalogram and functional magnetic resonance imaging identifies the dynamics of performance monitoring［J］.The Journal of Neuroscience,2005,25(50):11730-11737.
［26］YUN Z,QUAN Z,CAIXIN S,et al.RBF neural network and ANFIS-based short-term load forecasting approach in real-time price environment［J］.IEEE Transactions on Power Systems,2008,23(3):853-858.
［27］WU T T,LANGE K.Coordinate descent algorithms for Lasso penalized regression［J］.Annals of Applied Statistics,2008,2(1):224-244.
［28］DENG Z,CHOI K S,JIANG Y,et al.Generalized hidden-mapping ridge regression,knowledge-leveraged inductive transfer learning for neural networks,fuzzy systems and kernel methods［J］.IEEE Transactions on Cybernetics,2014,44(12):2585.

[1]	何杏宇, 周易歆, 罗东旭, 杨桂松. 基于图神经网络和多主体评价的教学资源推荐[J]. 计算机工程, 2024, 50(7): 13-22.
[2]	耿丽丽, 牛保宁. 基于通道相似度熵的卷积神经网络裁剪[J]. 计算机工程, 2024, 50(7): 133-143.
[3]	张洋, 刘畅, 李少青. 基于可控制性度量的图神经网络门级硬件木马检测方法[J]. 计算机工程, 2024, 50(7): 164-173.
[4]	牛瑞婷, 严天峰, 高锐, 王映植. 低信噪比下基于深度学习TCNN-MobileNet的调制识别[J]. 计算机工程, 2024, 50(7): 204-215.
[5]	张溢文, 蔡满春, 陈咏豪, 朱懿, 姚利峰. 融合空间特征的多尺度深度伪造检测方法[J]. 计算机工程, 2024, 50(7): 240-250.
[6]	逯焕宇, 张永宏, 马光义, 谢东林, 田伟. 基于半监督对抗学习的遥感图像水体提取[J]. 计算机工程, 2024, 50(7): 251-263.
[7]	李云航, 潘晴, 田妮莉. 结构相似度优化的混合多尺度医学图像融合[J]. 计算机工程, 2024, 50(7): 264-270.
[8]	张正康, 杨丹, 聂铁铮, 寇月. 基于图结构聚类的自监督学习疾病诊断方法[J]. 计算机工程, 2024, 50(7): 360-371.
[9]	李亚康, 陈刚. 小角中子散射物理模型自动化筛选[J]. 计算机工程, 2024, 50(6): 56-64.
[10]	更藏措毛, 黄鹤鸣, 杨毅杰. 融合多尺度特征与上下文信息的语音增强方法[J]. 计算机工程, 2024, 50(6): 138-147.
[11]	于洋, 孙芳芳, 吕华, 李扬, 王晓民. 基于多尺度时空注意力网络的微表情检测方法[J]. 计算机工程, 2024, 50(6): 228-235.
[12]	宋庆增, 刘向东, 许康为, 刘佳辉, 任二祥, 骆丽, 魏琦, 乔飞. 基于卷积神经网络的逐级唤醒存内计算控制器设计[J]. 计算机工程, 2024, 50(6): 328-335.
[13]	孙文洁, 李宗民, 孙浩淼. 基于图神经网络的多智能体强化学习值函数分解方法[J]. 计算机工程, 2024, 50(5): 62-70.
[14]	游奔, 李晓红, 姚锦, 冯绍杰. 基于多粒度图与注意力机制的半监督短文本分类[J]. 计算机工程, 2024, 50(5): 83-90.
[15]	张宝鑫, 杨丹, 聂铁铮, 寇月. 基于自监督的多视角图协同过滤推荐方法[J]. 计算机工程, 2024, 50(5): 100-110.

选择文件类型/文献管理软件名称

选择包含的内容