Abstract:

Spiking Neural Network (SNN) has become increasingly important for studying and simulating the functions of various brain regions and their interconnections. Parallel-distributed computing has become an inevitable choice for SNN simulations of larger-scale brain regions. However, as the scale of computation increases, SNN simulation performance is affected primarily by load imbalances among computing nodes and communication issues. For distributed computing platforms, existing partitioning algorithms cannot find a globally optimal partition or effectively map workloads to computing cores. Therefore, this study proposes a communication and topology-aware partitioning and mapping algorithm that includes two core steps: partitioning and topology-aware mapping. Introducing a partitioning method that is aware of SNN connections improves the computational efficiency and reduces communication latency. In the topology-aware mapping method, the communication topology graph and underlying network information are utilized to efficiently allocate workloads to computing nodes and minimize the communication costs across different computing cores. Experimental results show that, when simulating SNN benchmark datasets with 96 processes on the parallel computing platform of the National Supercomputing Center in Jinan, the proposed method achieves better load balancing and communication performance than existing state-of-the-art partitioning frameworks. The synchronization and communication times are reduced by 40% and 7.1%, respectively, and the total simulation time is shortened by 30%.

Key words: Spiking Neural Network (SNN), distributed computing, load balancing, hypergraph partitioning, topology-aware mapping

摘要：

脉冲神经网络(SNN)正日益成为研究和模拟大脑各区功能及其相互关联性的重要方法。为了模拟更大规模的脑区域, 并行分布式计算已成为模拟SNN的必然选择。然而, 随着计算规模的增长, 计算节点间的负载不均衡及通信问题成为影响SNN模拟性能的主要因素。针对分布式计算平台, 现有分区算法还无法找到全局最佳分区并有效地将工作负载映射到计算核心上。因此, 提出一种基于通信和拓扑感知的分区与映射算法, 该算法包括分区和拓扑感知映射2个核心步骤。通过引入能够感知SNN连接的分区方法, 提高计算效率并降低通信延迟; 在拓扑感知映射方法中, 利用通信拓扑图和底层网络信息将工作负载高效地分配到各计算节点上, 最小化跨不同计算核心的通信成本。实验结果表明, 在国家超算济南计算中心的并行计算平台上, 采用96进程规模并行模拟SNN基准测试集时, 相比现有先进的分区框架, 所提方法具有更好的负载均衡和通信性能, 同步时间和通信时间分别减少了40%和7.1%, 最终的模拟总时间缩短了30%。

关键词: 脉冲神经网络, 分布式计算, 负载均衡, 超图分区, 拓扑感知映射