计算机工程

所属专题: 大数据专题

• 大数据专题 • 上一篇    下一篇

基于CPU/GPU异构资源协同调度的改进H-Storm平台

严健康,陈更生   

  1. (复旦大学 专用集成电路与系统国家重点实验室,上海 200433)
  • 收稿日期:2017-02-10 出版日期:2018-04-15 发布日期:2018-04-15
  • 作者简介:严健康(1992—),男,硕士研究生,主研方向为大数据、异构集群;陈更生,高级工程师。
  • 基金项目:

    上海市科委科技创新行动计划项目(14511108002)。

Improved H-Storm Platform Based on Co-scheduling of CPU/GPU

YAN Jiankang,CHEN Gengsheng   

  1. (State Key Laboratory of ASIC and System,Fudan University,Shanghai 200433,China)
  • Received:2017-02-10 Online:2018-04-15 Published:2018-04-15

摘要:

为满足计算密集型大数据应用的实时处理需求,在Apache Storm基础上,研究开发H-Storm异构计算平台。通过多进程服务特性设计图形处理器(GPU)资源的量化和分布式调用机制,进而提出H-Storm异构集群的任务调度策略,实现GPU性能及负载的任务调度算法与协同计算下自适应的流分发决策机制。实验结果表明,在512×512矩阵乘法用例下,与原生Storm平台相比,H-Storm异构计算平台吞吐量提升54.9倍,响应延时下降77倍。

关键词: Storm平台, 异构资源, 调度算法, 协同计算, JCuda库, 多进程服务特征

Abstract:

To meet the real-time processing needs of compute-intensive big data applications,H-Storm heterogeneous computing platform TS developed based on Apache Storm.Through the Multi-process Service(MPS) feature,Graphic Process Unit(GPU) resource quantization and distributed calling mechanism are designed the task scheduling strategy of H-Storm heterogeneous clusters is proposed,and the task scheduling algorithm of GPU performance and load and adaptive flow distribution decision mechanism under cooperative computing are realized.Experimental results show that in the case of 512×512 matrix multiplication,the throughput of H-Storm heterogeneous computing platform increases by 54.9 times and the response delay decreases by 77 times compared with that of native Storm.

Key words: Storm platform, heterogeneous resource, scheduling algorithm, Co-computing, JCuda library, Multi-process Service(MPS) feature

中图分类号: