Abstract:
As applications programs based on MapReduce model growing up, Hadoop’s performance depends on applications. From applications, this paper analyzes the limitations and shortcomings of Hadoop architecture and storage technology, and proposes a feasible and cost-effective solution to address the problem. It builds a series tests to convince the idea. The solution is multi-level parallel, both on job and I/O levels, which makes full use of disk and network bandwidth, reduces I/O bottlenecks and improves performance.
Key words:
distributed computing,
storage,
concurrent I/O,
performance optimization
摘要:
随着基于MapReduce模型的应用程序越来越多,Hadoop性能取决于应用程序。针对上述特性,从应用着手剖析Hadoop存在的局限和不足,提出解决方案,利用作业和任务的多重并发平衡磁盘和网络带宽,减小瓶颈出现的可能性,提高系统性能。
关键词:
分布式计算,
存储,
并发I/O,
性能优化
CLC Number:
LUAN E-Jian, HUANG Chong-Min, GONG Gao-Cheng, DIAO Tie-Zhu. Research on Performance Optimization of Hadoop Platform[J]. Computer Engineering, 2010, 36(14): 262-263.
栾亚建, 黄翀民, 龚高晟, 赵铁柱. Hadoop平台的性能优化研究[J]. 计算机工程, 2010, 36(14): 262-263.