计算机工程

• 体系结构与软件技术 • 上一篇    下一篇

一种面向高性能计算的分布式对象存储系统

陈曦,朱建涛,何晓斌   

  1. (江南计算技术研究所,江苏 无锡 214083)
  • 收稿日期:2016-07-12 出版日期:2017-08-15 发布日期:2017-08-15
  • 作者简介:陈曦(1992—),女,硕士研究生,主研方向为海量存储、分布式文件系统;朱建涛,高级工程师;何晓斌,工程师、硕士。
  • 基金项目:
    国家高技术研究发展计划项目(2013AA013203,2013AA01A210)。

An HPC-oriented Distributed Object Storage System

CHEN Xi,ZHU Jiantao,HE Xiaobin   

  1. (Jiangnan Institute of Computing Technology,Wuxi,Jiangsu 214083,China)
  • Received:2016-07-12 Online:2017-08-15 Published:2017-08-15

摘要:

现有分布式文件存储系统存在数据组织低效和访问语义冗余等问题,严重限制了系统性能。为此,借鉴对象存储思想,设计面向高性能计算的分布式对象存储系统。分离数据访问和数据管理,实现更精简高效的访问语义,同时采用分布式全局对象数据组织方式,运用基于内存的元数据管理方法提升系统性能。实验结果表明,在大规模并发访问时,该系统的读/写聚合带宽相比Lustre系统分别提升22.5%和50.4%,文件创建、删除性能分别达到Lustre系统的2.15倍和5.13倍。此外,该系统还具有拟线性的数据读/写和元数据管理功能,可扩展性较好。

关键词: 高性能计算, 对象存储, 存储语义, 元数据管理, 可扩展性

Abstract: The existing distributed file storage system has problems like inefficient data organization and redundant access semantics,etc.,which severely limit the performance of High Performance Computing(HPC) storage system.Therefore,this paper presents HPC-oriented distributed Object Storage System(COSS) based on the idea of object storage.It separates the data access and data management to achieve more streamlined and efficient access semantics,uses distributed global object data organization method and designs memory-based metadata management method to improve system performance.Experimental results show that,under large scale concurrent access,COSS can improve read and write aggregated bandwidth by 22.5% and 50.4% respectively compared with Lustre system.The performance of file creation and file deletion of COSS is respectively 2.15 times and 5.13 times of Lustre system.Meanwhile,COSS provides quasi-linear data read/write performance and metadata management performance.It also has excellent scalability.

Key words: High Performance Computing(HPC), object storage, storage semantics, metadata management, scalability

中图分类号: