作者投稿和查稿 主编审稿 专家审稿 编委审稿 远程编辑

计算机工程

• 体系结构与软件技术 • 上一篇    下一篇

影像数据分布并行计算处理平台体系架构研究

朱嘉舟,邵培南,陈景   

  1. (中国电子科技集团公司第三十二研究所,上海 201800)
  • 收稿日期:2016-10-11 出版日期:2017-05-15 发布日期:2017-05-15
  • 作者简介:朱嘉舟(1992—),男,硕士研究生,主研方向为大数据、云计算;邵培南,研究员;陈景,硕士。

Research on Distributed Parallel Computing Processing Platform Architecture for Image Data

ZHU Jiazhou,SHAO Peinan,CHEN Jing   

  1. (The 32nd Research Institute of China Electronics Technology Group Corporation,Shanghai 201800,China)
  • Received:2016-10-11 Online:2017-05-15 Published:2017-05-15

摘要: 遥感影像数据并行处理系统大多依赖于国外商用产品,而国内自主化并行计算处理系统的任务流程化支撑能力以及并行计算性能难以适应规模化生产。为此,基于Hadoop的HDFS,MapReduce集群并行架构、CPU和GPU协同并行处理、内存映像、BMP等技术,提出流程驱动执行的高性能分布式并行计算处理平台体系架构。实验结果表明,工作站集群和工作站内多粒度混合的并行计算架构提高了平台并行处理性能,为海量遥感影像数据产品的批量生产提供一种自主化解决方案。

关键词: 大数据, Hadoop架构, Hadoop分布式文件系统, MapReduce框架, GPU并行计算

Abstract: Remote image data parallel processing system basically relies on foreign commercial products,while domestic independent parallel processing system the task process support capability and parallel processing cannot meet the need of scale production.Therefore,based on HDFS,MapReduce cluster parallel architecture,CPU/GPU cooperative parallel processing,memory mapping,BMP and so on,this paper proposes the architecture and realization of a high-performance distributed parallel processing platform with process driven execution.Experimental results show that the multi-granularity mixing parallel processing architecture of cluster and workstation magnificently increases the parallel performance of the platform,which proposes an independent resolution to the scale production of massive remote image data products.

Key words: big data, Hadoop architecture, Hadoop Distributed File System(HDFS), MapReduce framework, GPU parallel computing

中图分类号: