基于FPGA的Izhikevich神经元定制计算方法

doi:10.19678/j.issn.1000-3428.0066260

摘要/Abstract

摘要：

脉冲神经网络作为第三代神经网络，其工作机理与生物大脑更接近，层内连接与反向连接的复杂拓扑结构具有解决复杂问题的潜力。神经元和突触是脉冲神经网络中最基本的计算单元，相比于带泄露积分触发神经元模型，Izhikevich神经元模型能通过模拟出更多的生物脉冲现象来支持更广泛的类脑仿真计算，但Izhikevich神经元模型的计算复杂度更高，基于其搭建的脉冲神经网络存在低性能、高功耗的问题。提出一种基于FPGA的Izhikevich神经元定制计算方法。首先，通过研究脉冲神经网络中Izhikevich神经元各参数的取值范围以及平衡膜电位的相对误差与资源消耗，设计一套混合精度的定点化方案；其次，针对单个神经元，通过平衡神经元更新计算方程的数据路径实现最小化流水；再次，针对整体脉冲神经网络，设计并行度可扩展的计算架构以适应不同规模的FPGA平台；最后，把该定制计算方法用于经典的NEST仿真器加速。实验结果表明，相比于i7-10700 CPU，经典的丘脑外侧膝状核网络模型和液体状态机模型在ZCU102上的性能平均提升2.26和3.02倍，能效比平均提升8.06和10.8倍。

关键词: Izhikevich神经元, 混合精度, 脉冲神经网络, 定制计算, FPGA

Abstract:

As a third-generation neural network, the Spiking Neural Network(SNN) uses neurons and synapses as the basic computing units, and its working mechanism is similar to that of the biological brain. Its complex topology of intra-layer connections and reverse connections has the potential to solve complex problems. Compared with the Leaky-Integrate-and-Fire(LIF) model, the Izhikevich neuron model can support a wider range of neuromorphic computing by simulating more biological impulse phenomena; however, the Izhikevich neuron model has higher computational complexity, leading to potential issues of suboptimal performance and increased power consumption within the network. To address these problem, a customized calculation method of Izhikevich neurons based on FPGA is proposed. First, by studying the value range of the parameters of Izhikevich neurons in the SNN and balancing the relative errors of the membrane potential and resource consumption, a fixed-point solution with mixed-precision is designed. Second, for a single neuron, the data path of the calculation equation is updated by balancing the neuron to achieve the minimum pipeline length. Furthermore, at the network level, a scalable computing architecture is devised to accommodate varying FPGA scales, ensuring adaptability across different configurations. Finally, the customized computing method is used to accelerate the classical NEST simulator. The experimental results reveal that, compared with that of the i7-10700 CPU, the performance of the classic lateral geniculate nucleus network model and the liquid state machine model on the ZCU102 is 2.26 and 3.02 times better in average, and the energy efficiency ratio is improved by 8.06 and 10.8 times in average.

Key words: Izhikevich neuron, mixed-precision, Spiking Neural Network(SNN), customized computing, FPGA

叶钧超, 徐聪, 黄尧, 柴志雷. 基于FPGA的Izhikevich神经元定制计算方法[J]. 计算机工程, 2023, 49(12): 35-45.

Junchao YE, Cong XU, Yao HUANG, Zhilei CHAI. FPGA-based Customized Computing Method for Izhikevich Neuron[J]. Computer Engineering, 2023, 49(12): 35-45.

https://www.ecice06.com/CN/Y2023/V49/I12/35

图/表 20

图1 不同Izhikevich模型各参数的值

Fig.1 Parameters' values of different Izhikevich models

图2 Izhikevich神经元权重值分布

Fig.2 Izhikevich neuron weight value distribution

图3 各位宽下膜电位的相对误差

Fig.3 Membrane potential's relative error with each bit width

图4 可定制数据流架构

Fig.4 Customizable data flow architecture

图5 FPGA上的数据重分配

Fig.5 Data redistribution on FPGA

图6 单Izhikevich神经元的硬件设计

Fig.6 Hardware design of a Izhikevich neuron

图7 多神经元更新优化前后对比

Fig.7 Comparison before and after multi-neuron update optimization

图8 基于NEST的系统结构

Fig.8 NEST-based system structure

图9 ARM与FPGA数据传输接口示意图

Fig.9 Schematic diagram of data transmission interface between ARM and FPGA

图10 丘脑外侧膝状核网络模型拓扑

Fig.10 Topology of the network model of the lateral geniculate nucleus of the thalamus

图11 LSM网络拓扑

Fig.11 Topology of LSM network

图12 不同数据类型下恢复变量波形图

Fig.12 Waveform diagrams of recovery variable under different data types

图13 不同数据类型下膜电位波形图

Fig.13 Waveform diagrams of membrane potential under different data types

参考文献 26

1	GERSTNER W, KISTLER W M, NAUD R, et al. Neuronal dynamics: from single neurons to networks and models of cognition. Cambridge, UK: Cambridge University Press, 2014.
2	JOHNSON A P, LIU J X, MILLARD A G, et al. Homeostatic fault tolerance in spiking neural networks: a dynamic hardware perspective. IEEE Transactions on Circuits and Systems Ⅰ: Regular Papers, 2017, 65(2): 687- 699.
3	MARCHISIO A, PIRA G, MARTINA M, et al. R-SNN: an analysis and design methodology for robustifying spiking neural networks against adversarial attacks through noise filters for dynamic vision sensors[C]//Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. Washington D. C., USA: IEEE Press, 2021: 6315-6321.
4	OSSWALD M, IENG S H, BENOSMAN R, et al. A spiking neural network model of 3D perception for event-based neuromorphic stereo vision systems. Scientific Reports, 2017, 7, 40703. doi: 10.1038/srep40703
5	张铁林, 徐波. 脉冲神经网络研究现状及展望. 计算机学报, 2021, 44(9): 1767- 1785. URL
	ZHANG T L, XU B. Research advances and perspectives on spiking neural networks. Chinese Journal of Computers, 2021, 44(9): 1767- 1785. URL
6	HAN B, SRINIVASAN G, ROY K. RMP-SNN: residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 13555-13564.
7	高嘉潞, 于强, 唐华锦, 等. 联合突触权重和延迟可塑性的高效多脉冲学习算法研究. 计算机学报, 2022, 45(10): 2065- 2079. URL
	GAO J L, YU Q, TANG H J, et al. Research on efficient multi-pulse learning algorithm combining synaptic weight and delay plasticity. Chinese Journal of Computers, 2022, 45(10): 2065- 2079. URL
8	KIM S, PARK S, NA B, et al. Spiking-YOLO: spiking neural network for energy-efficient object detection. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 11270- 11277. doi: 10.1609/aaai.v34i07.6787
9	KOUL S, HORIUCHI T K. Waypoint path planning with synaptic-dependent spike latency. IEEE Transactions on Circuits and Systems Ⅰ: Regular Papers, 2019, 66(4): 1544- 1557. doi: 10.1109/TCSI.2018.2882818
10	NAVEROS F, LUQUE N R, ROS E, et al. VOR adaptation on a humanoid iCub robot using a spiking cerebellar model. IEEE Transactions on Cybernetics, 2020, 50(11): 4744- 4757. doi: 10.1109/TCYB.2019.2899246
11	IZHIKEVICH E M. Simple model of spiking neurons. IEEE Transactions on Neural Networks, 2003, 14(6): 1569- 1572. doi: 10.1109/TNN.2003.820440
12	GERSTNER W, KISTLER W M. Spiking neuron models: single neurons, populations, plasticity. Cambridge, UK: Cambridge University Press, 2002.
13	SKOCIK M J, LONG L N. On the capabilities and computational costs of neuron models. IEEE Transactions on Neural Networks and Learning Systems, 2013, 25(8): 1474- 1483.
14	KHUN J, NOVOTNÝ M, SKRBEK M. High-performance spiking neural network simulator[C]//Proceedings of the 8th Mediterranean Conference on Embedded Computing. Washington D. C., USA: IEEE Press, 2019: 1-4.
15	栗学磊, 朱效民, 魏彦杰, 等. 神威太湖之光加速计算在脑神经网络模拟中的应用. 计算机学报, 2020, 43(6): 1025- 1037. URL
	LI X L, ZHU X M, WEI Y J, et al. Application of Sunway TaihuLight accelerating in brain neural network simulation. Chinese Journal of Computers, 2020, 43(6): 1025- 1037. URL
16	NURVITADHI E, SHEFFIELD D, SIM J, et al. Accelerating binarized neural networks: comparison of FPGA, CPU, GPU, and ASIC[C]//Proceedings of International Conference on Field-Programmable Technology. Washington D. C., USA: IEEE Press, 2017: 77-84.
17	ELNABAWY A, ABDELMOHSEN H, MOUSTAFA M, et al. A low power CORDIC-based hardware implementation of Izhikevich neuron model[C]//Proceedings of the 16th IEEE International New Circuits and Systems Conference. Washington D. C., USA: IEEE Press, 2018: 130-133.
18	HEIDARPUR M, AHMADI A, AHMADI M, et al. CORDIC-SNN: on-FPGA STDP learning with Izhikevich neurons. IEEE Transactions on Circuits and Systems Ⅰ: Regular Papers, 2019, 66(7): 2651- 2661. doi: 10.1109/TCSI.2019.2899356
19	WANG J P, PENG Z X, ZHAN Y, et al. A high-accuracy and energy-efficient CORDIC based Izhikevich neuron with error suppression and compensation[C]//Proceedings of IEEE Transactions on Biomedical Circuits and Systems. Washington D. C., USA: IEEE Press, 2022: 807-821.
20	SAPOUNAKI M, KAKAROUNTAS A. A high-performance neuron for artificial neural network based on Izhikevich model[C]//Proceedings of the 29th International Symposium on Power and Timing Modeling, Optimization and Simulation. Washington D. C., USA: IEEE Press, 2019: 29-34.
21	SAPOUNAKI M, KAKAROUNTAS A. A novel low-power neuromorphic circuit based on Izhikevich model[C]//Proceedings of the 10th International Conference on Modern Circuits and Systems Technologies. Washington D. C., USA: IEEE Press, 2021: 1-4.
22	HEIDARPUR M, AHMADI A, AHMADI M. Time step impact on performance and accuracy of Izhikevich neuron: software simulation and hardware implementation[C]//Proceedings of IEEE International Symposium on Circuits and Systems. Washington D. C., USA: IEEE Press, 2020: 1-5.
23	刘家航, 郁龚健, 李佩琦, 等. 基于SNN神经元重分布的NEST仿真器性能优化. 计算机工程, 2022, 48(3): 189- 196. URL
	LIU J H, YU G J, LI P Q, et al. Performance optimization of NEST simulator based on SNN neuron relocation. Computer Engineering, 2022, 48(3): 189- 196. URL
24	KASTNER S, SCHNEIDER K A, WUNDERLICH K. Beyond a relay nucleus: neuroimaging views on the human LGN. Progress in Brain Research, 2006, 155, 125- 143.
25	AJINA S, BRIDGE H. Blindsight relies on a functional connection between hMT+ and the lateral geniculate nucleus, not the pulvinar. PLoS Biology, 2018, 16(7): e2005769.
26	KAISER J, STAL R, SUBRAMONEY A, et al. Scaling up liquid state machines to predict over address events from dynamic vision sensors. Bioinspiration & Biomimetics, 2017, 12(5): 055001.

[1]	陈逸, 刘博生, 徐永祺, 武继刚. 混合精度频域卷积神经网络FPGA加速器设计[J]. 计算机工程, 2023, 49(12): 1-9.
[2]	刘家航, 郁龚健, 李佩琦, 华夏, 柴志雷, 陈闻杰. 基于SNN神经元重分布的NEST仿真器性能优化[J]. 计算机工程, 2022, 48(3): 189-196.
[3]	刘鹏飞, 杜欣军. 基于加权反馈三维混沌系统的调制跳变通信[J]. 计算机工程, 2021, 47(3): 183-189,195.
[4]	庄祖江, 房玉, 雷建超, 刘栋博, 王海滨. 基于STDP规则的脉冲神经网络研究[J]. 计算机工程, 2020, 46(9): 83-88,94.
[5]	尚瑛杰, 董丽亚, 何虎. 基于脉冲神经网络的迁移学习算法与软件框架[J]. 计算机工程, 2020, 46(3): 53-59.
[6]	李康, 张鲁飞, 张新伟, 郁龚健, 刘家航, 吴东, 柴志雷. 基于FPGA集群的脉冲神经网络仿真器设计[J]. 计算机工程, 2020, 46(10): 201-209.
[7]	陈俞娴, 桂良启, 任利明, 周自成, 占美娟, 杨杰波, 郎量, 陈柯, 田加胜, 郭伟. 基于ARM与FPGA的气象微波辐射计测控系统设计[J]. 计算机工程, 2019, 45(8): 86-91.
[8]	蔺想红,张宁,崔文博,冯丽霞. 基于TTFS编码的脉冲神经网络图像分割方法[J]. 计算机工程, 2015, 41(8): 227-232.
[9]	徐彦,杨静. 脉冲神经元序列学习方法的影响因素研究[J]. 计算机工程, 2015, 41(11): 194-201.
[10]	崔文博, 蔺想红, 徐满意. 脉冲神经网络图像分割的编码方法[J]. 计算机工程, 2012, 38(24): 196-199.
[11]	梁春锋, 唐艳发, 孙红亮, 张利. 一种无链表零树熵编码算法及其FPGA实现[J]. 计算机工程, 2012, 38(04): 202-204.
[12]	张威, 李跃辉, 景为平. 电子收费系统车载单元LLC子层研究[J]. 计算机工程, 2011, 37(8): 246-248.
[13]	李天顺, 肖铁军. 基于WSRF的FPGA远程实验服务[J]. 计算机工程, 2011, 37(2): 240-241.
[14]	杨敏, 吴艳霞, 顾国昌, 孙延腾. 面向可重构编译技术的RAM访问优化算法[J]. 计算机工程, 2011, 37(2): 284-285.
[15]	段欣, 陈利光, 王健, 来金梅, 鲍丽春. 可进化芯片的FPGA接口设计与实现[J]. 计算机工程, 2011, 37(13): 13-16.

选择文件类型/文献管理软件名称

选择包含的内容