摘要： 在高速网络中，网络设备的分组转发性能在打开流采集功能后会受其影响。为此，提出一种在网络设备外基于云计算平台的并行流生成方法。在需要监测的网络设备附近部署小型云，把分组流量复制到云中，采用云的Map-Reduce并行处理框架快速地从海量分组数据中生成流记录。设计基于Map-Reduce框架的并行流生成算法，通过配置合适数量的云节点，可分析任意大小的网络流量。用实际网络分组数据对并行流生成方法的性能进行验证，实验结果表明，在由3台、5台或7台节点构成的小型云平台上，从超过40 GB的文本分组数据中共提取了15 160 052条流，与顺序处理相比，耗费时间至少可减小85%、90%和94%。

关键词: 网络流量分析, 并行处理, Map-Reduce框架, Hadoop平台

Abstract: In high speed networks, the forwarding performance of a network device which captures flows is inevitably degraded. A parallel flow generating approach based on small size cloud computing platform outside of network device is proposed. A small size cloud can be deployed near the selected network device being monitored, and packet traffic through the network device is copied into the cloud. Then flow records can be extracted fast from the large volume packet traffic by the parallel Map-Reduce framework of cloud. A parallel flow extraction algorithm based on Map-Reduce framework is proposed. In addition, cloud scale is flexible to expand, and by configuring proper cloud workers, this approach can be adopted to analyze network traffic in arbitrary size. Real network packet traffic is used to verify the performance of the parallel flow generating approach. Analysis results show that in clouds consisting of 3, 5 or 7 worker nodes, 15 160 052 flows are recognized from more than 40 GB text packet data, and comparing against sequentially processing method, the consumed time are reduced by more than 85%, 90% and 94%, respectively.

Key words: network traffic analysis, parallel processing, Map-Reduce framework, Hadoop platform

中图分类号: 

• TP393.02