摘要： 提出了一种基于机群的分布式数据采集系统，即采用任务调度中心、机群管理中心、采集代理多层次结构实现分布式数据采集；同时提出了机群的3种分配策略、管理中心选择机制及采集代理的设计方案。实验结果表明：该文所提出的方法适用于大规模网络的数据采集，能更加有效地提高数据采集的速度，解决传统的采集方法在大规模网络中所存在的数据传输负载过重和数据处理的瓶颈问题，在数据采集领域具有很好的应用前景，具有一定的应用价值。

关键词: 分布式, 机群, 数据采集, 采集代理

Abstract: This paper proposes a distributed data collecting system based on cluster, which implements data collection on the components, namely: task assignment center, cluster manage center, collection agents based on multilayer structure. In addition, three assign strategies of cluster, choice mechanisms of the manage center and the design project of collecting agents are also proposed. Experimental results show that the methods proposed in this paper are applicable and can effectively improve the speed of the data collecting and resolve the disadvantages of the traditional methods in the large scale networks, such as overload of the data transmission and bottleneck of the data transaction etc. They have great potential and application value in the field of the data collecting.

Key words: Distributed, Cluster, Data collection, Collection agent

中图分类号: 

• TP393