Gradient Descent Learning Algorithm for Spiking Neuron with Delay Adjustment

doi:10.19678/j.issn.1000-3428.0052234

Computer Engineering ›› 2019, Vol. 45 ›› Issue (9): 169-175. doi: 10.19678/j.issn.1000-3428.0052234

Previous Articles Next Articles

Gradient Descent Learning Algorithm for Spiking Neuron with Delay Adjustment

YANG Jing¹, XU Yan², ZHAO Xin¹

1. School of Management, Zhuhai Campus, Beijing Normal University, Zhuhai, Guangdong 519087, China;
2. College of Information Science and Technology, Nanjing Agricultural University, Nanjing 210095, China

Received:2018-07-27 Revised:2018-09-06 Online:2019-09-15 Published:2019-09-03

带延迟调整的脉冲神经元梯度下降学习算法

杨静¹, 徐彦², 赵欣¹

1. 北京师范大学珠海分校管理学院, 广东珠海 519087;
2. 南京农业大学信息科技学院, 南京 210095

作者简介:杨静(1980-),女,副教授、博士,主研方向为神经网络、模式识别;徐彦,讲师、博士;赵欣,副教授、博士。
基金资助:
国家自然科学基金（61503031）；广东省科技计划基金（2016A040403029）；广东省哲学社会科学"十二五"规划学科共建项目（GD15XGL02）。

Abstract

Abstract: The spiking neuron supervised learning algorithm adjusts the synaptic weight of the neuron by gradient descent method,but the accuracy gets low and the learning period gets long as the length of the target learning sequence increases.Therefore,a gradient descent learning algorithm with delay adjustment is proposed.The delay of each synapse is used as a learning parameter to adjust the weight during the learning process,and at the same time,the gradient delay adjustment of the synaptic delay time is performed to make neurons trigger a target spiking sequence.Experimental results show that the proposed algorithm can improve the ability of neurons to learn complex spiking sequences without increasing the complexity of the algorithm,and the convergence speed is faster.

Key words: spiking neuron, spiking sequence, gradient descent, neuron synapse, delay learning

摘要： 脉冲神经元有监督学习算法通过梯度下降法调整神经元的突触权值，但目标学习序列长度的增加会降低其精度并延长学习周期。为此，提出一种带延迟调整的梯度下降学习算法。将每个突触的延迟作为学习参数，在学习过程中调整权值，同时对突触的延迟时间进行梯度下降调整，从而使神经元激发出目标脉冲序列。实验结果表明，该算法在不增加算法复杂度的情况下，能够提高神经元学习复杂脉冲序列的能力，且收敛速度较快。

关键词: 脉冲神经元, 脉冲序列, 梯度下降, 神经元突触, 延迟学习

CLC Number:

TP183

YANG Jing, XU Yan, ZHAO Xin. Gradient Descent Learning Algorithm for Spiking Neuron with Delay Adjustment[J]. Computer Engineering, 2019, 45(9): 169-175.

杨静, 徐彦, 赵欣. 带延迟调整的脉冲神经元梯度下降学习算法[J]. 计算机工程, 2019, 45(9): 169-175.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0052234

http://www.ecice06.com/EN/Y2019/V45/I9/169

Figures/Tables 14

References

[1] PONULAK F,KASINSKI A.Introduction to spiking neural networks:information processing,learning and applications[J].Acta Neurobiologiae Experimentalis,2011,71(4):409-433.
[2] BOHTE S M,KOK J N,LA POUTR'E J A.Error-backpropagation in temporally encoded networks of spiking neurons[J].Neurocomputing,2002,48(1/4):17-37.
[3] MCKENNOCH S,LIU D,BUSHNELL L G.Fast modifications of the spikeprop algorithm[C]//Proceedings of International Joint Conference on Neural Networks.Washington D.C.,USA:IEEE Press,2006:3970-3977.
[4] SILVA S M,RUANO A E.Application of levenberg-marquardt method to the training of spiking neural networks[C]//Proceedings of the International Conference on Neural Networks and Brain.Washington D.C.,USA:IEEE Press,2005:1354-1358.
[5] SCHRAUWEN B,CAMPENHOUT J V.Extending spike-prop[C]//Proceedings of International Joint Conference on Neural Networks.Washington D.C.,USA:IEEE Press,2004:471-476.
[6] BOOIJ O,NGUYEN H T.A gradient descent rule for multiple spiking neurons emitting multiple spikes[J].Information Processing Letters,2005,95(6):552-558.
[7] XU Yan,ZENG Xiaoqin,HAN Lixin,et al.A supervised multi-spike learning algorithm based on gradient descent for spiking neural networks[J].Neural Networks,2013,43(4):99-113.
[8] 徐彦.基于梯度下降的脉冲神经元在线学习方法[J].计算机工程,2015,41(12):150-155,160.
[9] XU Yan,YANG Jing,ZHONG Shuiming.An online supervised learning method based on gradient descent for spiking neurons[J].Neural Networks,2017,93:7-20.
[10] FLORIAN R V.The chronotron:a neuron that learns to fire temporally precise spike patterns[EB/OL].[2018-05-21].https://www.ncbi.nlm.nih.gov/pubmed/22879876.
[11] VICTOR J D,PURPURA K P.Nature and precision of temporal coding in visual cortex:a metric-space analysis[J].Journal of Neurophysiology,1996,76(2):1310-1326.
[12] MARKRAM H,TSODYKS M.Redistribution of synaptic efficacy between neocortical pyramidal neurons[J].Nature,1996,382(6594):807-810.
[13] PONULAK F,KASINSKI A.Supervised learning in spiking neural networks with ReSuMe:sequence learning,classification and spike shifting[J].Neural Computation,2010,22(2):467-510.
[14] 徐彦,杨静.脉冲神经元序列学习方法的影响因素研究[J].计算机工程,2015,41(11):194-201.
[15] XU Yan,ZENG Xiaoqin,ZHONG Shuiming.A new supervised learning algorithm for spiking neurons[J].Neural Computation,2013,25(6):1472-1511.
[16] MOHEMMED A,SCHLIEBS S,MATSUDA S,et al.Training spiking neural networks to associate spatio-temporal input-output spike patterns[J].Neurocomputing,2013,107(4):3-10.
[17] YU Qiang,TANG Huajin,TAN K C,et al.Precise-spike-driven synaptic plasticity:learning hetero-association of spatiotemporal spike patterns[EB/OL].[2018-05-21].https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0078318.
[18] LIN Xianghong,ZHANG Ning,WANG Xiangwen.An online supervised learning algorithm based on nonlinear spike train kernels[C]//Proceedings of International Conference on Intelligent Computing.Berlin,Germany:Springer,2015:106-115.
[19] GARDNER B,GRÜNING A.Supervised learning in spiking neural networks for precise temporal encoding[EB/OL].[2018-05-21].https://www.ncbi.nlm.nih.gov/pubmed/27532262.
[20] 徐彦,熊迎军,杨静.脉冲神经元脉冲序列学习方法综述[J].计算机应用,2018,38(6):1527-1534..
[21] XU Bo,GONG Yubing,WANG Baoying.Delay-induced firing behavior and transitions in adaptive neuronal networks with two types of synapses[J].Science China Chemistry,2013,56(2):222-229.
[22] SCHRAUWEN B,CAMPENHOUT J V.Extending SpikeProp[C]//Proceedings of IEEE International Joint Conference on Neural Network.Washington D.C.,USA:IEEE Press,2004:471-475.
[23] TAHERKHANI A,BELATRECHE A,LI Yuhua,et al.DL-ReSuMe:a delay learning-based remote supervised method for spiking neurons[C]//Proceedings of International Conference on Neural Information Processing.Berlin,Germany:Springer,2015:190-197.

Please choose a citation manager

Content to export