作者投稿和查稿 主编审稿 专家审稿 编委审稿 远程编辑

计算机工程

• 人工智能及识别技术 • 上一篇    下一篇

基于惩罚函数泛化的神经网络剪枝算法研究

熊 俊1,王士同1,潘永惠2,包 芳2   

  1. (1. 江南大学数字媒体学院,江苏无锡214122; 2. 江阴职业技术学院计算机科学系,江苏江阴214405)
  • 收稿日期:2013-11-26 出版日期:2014-11-15 发布日期:2014-11-13
  • 作者简介:熊 俊(1991 - ),男,硕士研究生,主研方向:人工智能,模式识别;王士同,教授、博士生导师;潘永惠,副教授、博士;包 芳, 教授,博士。

Study of Neural Network Pruning Algorithm Based on Generalization of Penalty Function

XIONG Jun 1,WANG Shitong 1,PAN Yonghui 2,BAO Fang 2   

  1. (1. College of Digital Media,Jiangnan University,Wuxi 214122,China;2. Department of Computer Science,Jiangyin Polytechnic College,Jiangyin 214405,China)
  • Received:2013-11-26 Online:2014-11-15 Published:2014-11-13

摘要: 神经网络的隐层数和隐层节点数决定了网络规模,并对网络性能造成较大影响。在满足网络所需最少隐层节点数的前提下,利用剪枝算法删除某些冗余节点,减少隐层节点数,得到更加精简的网络结构。基于惩罚函数的剪枝算法是在目标函数后加入一个惩罚函数项,该惩罚函数项是一个变量为网络权值的函数。由于惩罚函数中的网络权值变量可以附加一个可调参数,将单一惩罚函数项泛化为一类随参数规律变化的新的惩罚函数,初始惩罚函数可看作泛化后惩罚函数的参数取定值的特殊情况。实验利用基于标准BP 神经网络的XOR 数据进行测试,得到隐层节点剪枝效果和网络权值随惩罚函数的泛化而发生变化,并从数据分析中得出具有更好剪枝效果及更优网络结构的惩罚函数泛化参数。

关键词: 隐层节点, 神经网络, 剪枝算法, 惩罚函数, 泛化, XOR 数据

Abstract: The number of hidden layer and hidden layer node in neural network determines the size of the network and has a great influence on the performance of the network. Therefore,when the network contains the least hidden layer node number,pruning algorithm can be used to delete some redundant node,then the network is more simple. The pruning algorithm adds a penalty function to the target function,and the penalty function regards the weights of network as variable. It adds a variable parameter to the weights of network,so the simple penalty function can be generalized to a kind of new penalty function that changes as the parameter. The initial function can be treated as a special condition after the generalization of penalty function. Experiment tests the XOR data based on the BP neural network and sums up the effect of the generalization of penalty function on the pruning of the hidden layer node with neural network and the structure of the neural network. Then the parameters which can lead to better pruning effect and more optimal network structure are obtained from data in experiment.

Key words: hidden layer node, neural network, pruning algorithm, penalty function, generalization, XOR data

中图分类号: