摘要：

为满足计算密集型大数据应用的实时处理需求,在Apache Storm基础上,研究开发H-Storm异构计算平台。通过多进程服务特性设计图形处理器(GPU)资源的量化和分布式调用机制,进而提出H-Storm异构集群的任务调度策略,实现GPU性能及负载的任务调度算法与协同计算下自适应的流分发决策机制。实验结果表明,在512×512矩阵乘法用例下,与原生Storm平台相比,H-Storm异构计算平台吞吐量提升54.9倍,响应延时下降77倍。

关键词: Storm平台, 异构资源, 调度算法, 协同计算, JCuda库, 多进程服务特征

Abstract:

To meet the real-time processing needs of compute-intensive big data applications,H-Storm heterogeneous computing platform TS developed based on Apache Storm.Through the Multi-process Service(MPS) feature,Graphic Process Unit(GPU) resource quantization and distributed calling mechanism are designed the task scheduling strategy of H-Storm heterogeneous clusters is proposed,and the task scheduling algorithm of GPU performance and load and adaptive flow distribution decision mechanism under cooperative computing are realized.Experimental results show that in the case of 512×512 matrix multiplication,the throughput of H-Storm heterogeneous computing platform increases by 54.9 times and the response delay decreases by 77 times compared with that of native Storm.

Key words: Storm platform, heterogeneous resource, scheduling algorithm, Co-computing, JCuda library, Multi-process Service(MPS) feature

中图分类号: 

• TP391