FPGA-based Customized Computing Method for Izhikevich Neuron

doi:10.19678/j.issn.1000-3428.0066260

Abstract

Abstract:

As a third-generation neural network, the Spiking Neural Network(SNN) uses neurons and synapses as the basic computing units, and its working mechanism is similar to that of the biological brain. Its complex topology of intra-layer connections and reverse connections has the potential to solve complex problems. Compared with the Leaky-Integrate-and-Fire(LIF) model, the Izhikevich neuron model can support a wider range of neuromorphic computing by simulating more biological impulse phenomena; however, the Izhikevich neuron model has higher computational complexity, leading to potential issues of suboptimal performance and increased power consumption within the network. To address these problem, a customized calculation method of Izhikevich neurons based on FPGA is proposed. First, by studying the value range of the parameters of Izhikevich neurons in the SNN and balancing the relative errors of the membrane potential and resource consumption, a fixed-point solution with mixed-precision is designed. Second, for a single neuron, the data path of the calculation equation is updated by balancing the neuron to achieve the minimum pipeline length. Furthermore, at the network level, a scalable computing architecture is devised to accommodate varying FPGA scales, ensuring adaptability across different configurations. Finally, the customized computing method is used to accelerate the classical NEST simulator. The experimental results reveal that, compared with that of the i7-10700 CPU, the performance of the classic lateral geniculate nucleus network model and the liquid state machine model on the ZCU102 is 2.26 and 3.02 times better in average, and the energy efficiency ratio is improved by 8.06 and 10.8 times in average.

Key words: Izhikevich neuron, mixed-precision, Spiking Neural Network(SNN), customized computing, FPGA

摘要：

脉冲神经网络作为第三代神经网络，其工作机理与生物大脑更接近，层内连接与反向连接的复杂拓扑结构具有解决复杂问题的潜力。神经元和突触是脉冲神经网络中最基本的计算单元，相比于带泄露积分触发神经元模型，Izhikevich神经元模型能通过模拟出更多的生物脉冲现象来支持更广泛的类脑仿真计算，但Izhikevich神经元模型的计算复杂度更高，基于其搭建的脉冲神经网络存在低性能、高功耗的问题。提出一种基于FPGA的Izhikevich神经元定制计算方法。首先，通过研究脉冲神经网络中Izhikevich神经元各参数的取值范围以及平衡膜电位的相对误差与资源消耗，设计一套混合精度的定点化方案；其次，针对单个神经元，通过平衡神经元更新计算方程的数据路径实现最小化流水；再次，针对整体脉冲神经网络，设计并行度可扩展的计算架构以适应不同规模的FPGA平台；最后，把该定制计算方法用于经典的NEST仿真器加速。实验结果表明，相比于i7-10700 CPU，经典的丘脑外侧膝状核网络模型和液体状态机模型在ZCU102上的性能平均提升2.26和3.02倍，能效比平均提升8.06和10.8倍。

关键词: Izhikevich神经元, 混合精度, 脉冲神经网络, 定制计算, FPGA

Junchao YE, Cong XU, Yao HUANG, Zhilei CHAI. FPGA-based Customized Computing Method for Izhikevich Neuron[J]. Computer Engineering, 2023, 49(12): 35-45.

叶钧超, 徐聪, 黄尧, 柴志雷. 基于FPGA的Izhikevich神经元定制计算方法[J]. 计算机工程, 2023, 49(12): 35-45.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0066260

http://www.ecice06.com/EN/Y2023/V49/I12/35

Figures/Tables 20

Fig.1 Parameters' values of different Izhikevich models

Fig.2 Izhikevich neuron weight value distribution

Fig.3 Membrane potential's relative error with each bit width

Fig.4 Customizable data flow architecture

Fig.5 Data redistribution on FPGA

Fig.6 Hardware design of a Izhikevich neuron

Fig.7 Comparison before and after multi-neuron update optimization

Fig.8 NEST-based system structure

Fig.9 Schematic diagram of data transmission interface between ARM and FPGA

Fig.10 Topology of the network model of the lateral geniculate nucleus of the thalamus

Fig.11 Topology of LSM network

Fig.12 Waveform diagrams of recovery variable under different data types

Fig.13 Waveform diagrams of membrane potential under different data types

References 26

1	GERSTNER W, KISTLER W M, NAUD R, et al. Neuronal dynamics: from single neurons to networks and models of cognition. Cambridge, UK: Cambridge University Press, 2014.
2	JOHNSON A P, LIU J X, MILLARD A G, et al. Homeostatic fault tolerance in spiking neural networks: a dynamic hardware perspective. IEEE Transactions on Circuits and Systems Ⅰ: Regular Papers, 2017, 65(2): 687- 699.
3	MARCHISIO A, PIRA G, MARTINA M, et al. R-SNN: an analysis and design methodology for robustifying spiking neural networks against adversarial attacks through noise filters for dynamic vision sensors[C]//Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. Washington D. C., USA: IEEE Press, 2021: 6315-6321.
4	OSSWALD M, IENG S H, BENOSMAN R, et al. A spiking neural network model of 3D perception for event-based neuromorphic stereo vision systems. Scientific Reports, 2017, 7, 40703. doi: 10.1038/srep40703
5	张铁林, 徐波. 脉冲神经网络研究现状及展望. 计算机学报, 2021, 44(9): 1767- 1785. URL
	ZHANG T L, XU B. Research advances and perspectives on spiking neural networks. Chinese Journal of Computers, 2021, 44(9): 1767- 1785. URL
6	HAN B, SRINIVASAN G, ROY K. RMP-SNN: residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 13555-13564.
7	高嘉潞, 于强, 唐华锦, 等. 联合突触权重和延迟可塑性的高效多脉冲学习算法研究. 计算机学报, 2022, 45(10): 2065- 2079. URL
	GAO J L, YU Q, TANG H J, et al. Research on efficient multi-pulse learning algorithm combining synaptic weight and delay plasticity. Chinese Journal of Computers, 2022, 45(10): 2065- 2079. URL
8	KIM S, PARK S, NA B, et al. Spiking-YOLO: spiking neural network for energy-efficient object detection. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 11270- 11277. doi: 10.1609/aaai.v34i07.6787
9	KOUL S, HORIUCHI T K. Waypoint path planning with synaptic-dependent spike latency. IEEE Transactions on Circuits and Systems Ⅰ: Regular Papers, 2019, 66(4): 1544- 1557. doi: 10.1109/TCSI.2018.2882818
10	NAVEROS F, LUQUE N R, ROS E, et al. VOR adaptation on a humanoid iCub robot using a spiking cerebellar model. IEEE Transactions on Cybernetics, 2020, 50(11): 4744- 4757. doi: 10.1109/TCYB.2019.2899246
11	IZHIKEVICH E M. Simple model of spiking neurons. IEEE Transactions on Neural Networks, 2003, 14(6): 1569- 1572. doi: 10.1109/TNN.2003.820440
12	GERSTNER W, KISTLER W M. Spiking neuron models: single neurons, populations, plasticity. Cambridge, UK: Cambridge University Press, 2002.
13	SKOCIK M J, LONG L N. On the capabilities and computational costs of neuron models. IEEE Transactions on Neural Networks and Learning Systems, 2013, 25(8): 1474- 1483.
14	KHUN J, NOVOTNÝ M, SKRBEK M. High-performance spiking neural network simulator[C]//Proceedings of the 8th Mediterranean Conference on Embedded Computing. Washington D. C., USA: IEEE Press, 2019: 1-4.
15	栗学磊, 朱效民, 魏彦杰, 等. 神威太湖之光加速计算在脑神经网络模拟中的应用. 计算机学报, 2020, 43(6): 1025- 1037. URL
	LI X L, ZHU X M, WEI Y J, et al. Application of Sunway TaihuLight accelerating in brain neural network simulation. Chinese Journal of Computers, 2020, 43(6): 1025- 1037. URL
16	NURVITADHI E, SHEFFIELD D, SIM J, et al. Accelerating binarized neural networks: comparison of FPGA, CPU, GPU, and ASIC[C]//Proceedings of International Conference on Field-Programmable Technology. Washington D. C., USA: IEEE Press, 2017: 77-84.
17	ELNABAWY A, ABDELMOHSEN H, MOUSTAFA M, et al. A low power CORDIC-based hardware implementation of Izhikevich neuron model[C]//Proceedings of the 16th IEEE International New Circuits and Systems Conference. Washington D. C., USA: IEEE Press, 2018: 130-133.
18	HEIDARPUR M, AHMADI A, AHMADI M, et al. CORDIC-SNN: on-FPGA STDP learning with Izhikevich neurons. IEEE Transactions on Circuits and Systems Ⅰ: Regular Papers, 2019, 66(7): 2651- 2661. doi: 10.1109/TCSI.2019.2899356
19	WANG J P, PENG Z X, ZHAN Y, et al. A high-accuracy and energy-efficient CORDIC based Izhikevich neuron with error suppression and compensation[C]//Proceedings of IEEE Transactions on Biomedical Circuits and Systems. Washington D. C., USA: IEEE Press, 2022: 807-821.
20	SAPOUNAKI M, KAKAROUNTAS A. A high-performance neuron for artificial neural network based on Izhikevich model[C]//Proceedings of the 29th International Symposium on Power and Timing Modeling, Optimization and Simulation. Washington D. C., USA: IEEE Press, 2019: 29-34.
21	SAPOUNAKI M, KAKAROUNTAS A. A novel low-power neuromorphic circuit based on Izhikevich model[C]//Proceedings of the 10th International Conference on Modern Circuits and Systems Technologies. Washington D. C., USA: IEEE Press, 2021: 1-4.
22	HEIDARPUR M, AHMADI A, AHMADI M. Time step impact on performance and accuracy of Izhikevich neuron: software simulation and hardware implementation[C]//Proceedings of IEEE International Symposium on Circuits and Systems. Washington D. C., USA: IEEE Press, 2020: 1-5.
23	刘家航, 郁龚健, 李佩琦, 等. 基于SNN神经元重分布的NEST仿真器性能优化. 计算机工程, 2022, 48(3): 189- 196. URL
	LIU J H, YU G J, LI P Q, et al. Performance optimization of NEST simulator based on SNN neuron relocation. Computer Engineering, 2022, 48(3): 189- 196. URL
24	KASTNER S, SCHNEIDER K A, WUNDERLICH K. Beyond a relay nucleus: neuroimaging views on the human LGN. Progress in Brain Research, 2006, 155, 125- 143.
25	AJINA S, BRIDGE H. Blindsight relies on a functional connection between hMT+ and the lateral geniculate nucleus, not the pulvinar. PLoS Biology, 2018, 16(7): e2005769.
26	KAISER J, STAL R, SUBRAMONEY A, et al. Scaling up liquid state machines to predict over address events from dynamic vision sensors. Bioinspiration & Biomimetics, 2017, 12(5): 055001.

[1]	Qirun HONG, Qin WANG. Design of Sparse CNN Accelerator Based on Inter-Frame Data Reuse [J]. Computer Engineering, 2023, 49(12): 55-62.
[2]	Yi CHEN, Bosheng LIU, Yongqi XU, Jigang WU. FPGA Accelerator Design for Hybrid Precision Frequency Domain Convolutional Neural Network [J]. Computer Engineering, 2023, 49(12): 1-9.
[3]	HUANG Zhengwei, LIU Hongwei, XU Yuan. Design of Ultra-Low-Power RISC-V Dedicated Processor for IToF Sensor [J]. Computer Engineering, 2022, 48(9): 146-154.
[4]	FENG Yifei, DING Nan, YE Junchao, CHAI Zhilei. Design and Implementation of Domain-Specific Low-Latency and High-Bandwidth TCP/IP Offload Engine [J]. Computer Engineering, 2022, 48(9): 162-170.
[5]	XI Zhiwen, CAI Jingjing, YANG Wenmin, CHAI Zhilei. Concurrent Request Scheduling Mechanism for FPGA Cloud Platform Based on Microservice Architecture [J]. Computer Engineering, 2022, 48(7): 206-213.
[6]	LIU Jiahang, YU Gongjian, LI Peiqi, HUA Xia, CHAI Zhilei, CHEN Wenjie. Performance Optimization of NEST Simulator Based on SNN Neuron Relocation [J]. Computer Engineering, 2022, 48(3): 189-196.
[7]	GONG Jie, ZHAO Shuo, HE Hu, DENG Ning. Design of Quantized CNN Acceleration System Based on FPGA [J]. Computer Engineering, 2022, 48(3): 170-174,196.
[8]	ZHAO Chenyuan, LI Wenxin, ZHANG Qingxi. An Improved Real-Time Semi-Global Stereo Matching Algorithm and Its Hardware Implementation [J]. Computer Engineering, 2021, 47(9): 162-170.
[9]	HUANG Rui, JIN Guanghao, LI Lei, JIANG Wenchao, SONG Qingzeng. Design and Implementation of Accelerator for Lightweight Neural Network [J]. Computer Engineering, 2021, 47(9): 185-190,196.
[10]	DI Xinkai, YANG Haigang. FPGA-based Accelerator for Sparse Convolutional Neutral Network [J]. Computer Engineering, 2021, 47(7): 189-195,204.
[11]	LIU Pengfei, DU Xinjun. Modulation Hopping Communication Based on Three-Dimensional Chaotic System with Weighted Feedback [J]. Computer Engineering, 2021, 47(3): 183-189,195.
[12]	YUAN Jiawei, SONG Qingzeng, WANG Xuechun, JIANG Wenchao, JIN Guanghao. Performance and Power Consumption Measurement and Analysis of Edge Computing Devices [J]. Computer Engineering, 2021, 47(2): 233-238,245.
[13]	WU Jianfeng, ZHENG Bowen, NIE Yi, CHAI Zhilei. FPGA Accelerator for 3DES Algorithm Based on OpenCL [J]. Computer Engineering, 2021, 47(12): 147-155,162.
[14]	SHI Yongquan, JING Naifeng. Evaluation Method Based on FPGA Emulation for Resistive Neural Network Accelerators [J]. Computer Engineering, 2021, 47(12): 209-214.
[15]	ZHUANG Zujiang, FANG Yu, LEI Jianchao, LIU Dongbo, WANG Haibin. Research on Spiking Neural Network Based on STDP Rule [J]. Computer Engineering, 2020, 46(9): 83-88,94.

Please choose a citation manager

Content to export