作者投稿和查稿 主编审稿 专家审稿 编委审稿 远程编辑

计算机工程

• 体系结构与软件技术 • 上一篇    下一篇

分布式同步系统Zookeeper的优化

唐海东1,2,武延军1   

  1. (1. 中国科学院软件研究所,北京 100190;2. 中国科学院大学,北京 100049)
  • 收稿日期:2013-01-30 出版日期:2014-04-15 发布日期:2014-04-14
  • 作者简介:唐海东(1988-),男,硕士研究生,主研方向:分布式系统;武延军,研究员、博士生导师。
  • 基金资助:
    中国科学院战略先导科技专项基金资助项目(XDA06000000);国家科技重大专项基金资助项目(2012ZX01039-004)。

Optimization of Distributed Synchronous System Zookeeper

TANG Hai-dong  1,2, WU Yan-jun  1   

  1. (1. Institute of Software, Chinese Academy of Sciences, Beijing 100190, China; 2. University of Chinese Academy of Sciences, Beijing 100049, China)
  • Received:2013-01-30 Online:2014-04-15 Published:2014-04-14

摘要: 针对分布式同步系统Zookeeper在大规模计算机集群上运行效率低下的问题,在研究手动设定响应节点方法的基础上,提出一种基于成员节点选举的自动设定响应节点算法。在一个大规模Zookeeper系统中,采用选举因子可配置的选举算法(包括计算能力、磁盘读写速度、请求率、失效率和网络延迟的测试),挑选出适合完成Zookeeper响应工作的一个或数个节点,进行数据更新请求的应答,降低系统的响应时间,提高系统的性能。实验结果证明,与目前手动设定响应节点相比,自动选举算法能够选举出最合适的节点,并且效率高,性能稳定。在对系统访问延迟的测试中,自动选举算法比手动设定响应节点的平均值减少了11%,且比手动设定响应节点的最大值减少了17%。

关键词: Zookeeper系统, 分布式系统, 成员选举, Hadoop系统, 数据同步

Abstract: Aiming at distributed synchronous system Zookeeper’s low-efficiency problem in a large-scale computer cluster, this paper puts forward an automatic response-node set algorithm based on the method of member node election. In a large-scale Zookeeper system, using a factor configurable election algorithm(includes computing capacity, disk reads and writes rate, request rate, failure rate and network latency test), it picks out one or several of the most suitable nodes for completing the Zookeeper’s response work, responding to the data updating request, it reduces the system’s response time, and improves the performance of the system. Experimental results show that, compared with the manual setting response node algorithm, the automatic election algorithm can always elect the most suitable nodes, and it has high efficiency, stable performance. In the tests of system’s access latency, automatic election algorithm has a response latency decrease of 11% than manual setting node in average, and a decrease of 17% than manually set’s maximum response latency.

Key words: Zookeeper system, distributed system, election of members, Hadoop system, data synchronization

中图分类号: