摘要： 现有的海量日志统计分析方法速度慢，且对硬件配置的要求高。为此，提出一种基于文本策略和SMCS的海量日志分析方法。根据文件的软件设计策略，采用日志文件索引方法，将日志文件与日志时间关联，以加快日志提取。SMCS算法采用哈希表、文件归并、堆操作方法对海量日志进行统计分析和内存损耗控制。通过对真实软件进行对比实验，结果表明，该方法的分析速度比传统方法提高4倍。
关键词:
Syslog日志,
日志分析,
SMCS算法,
海量日志,
文本策略,
控制内存
Abstract: Now the statistical analysis for the massive logs not only is time-consuming, but also requires higher hard configuration. For these questions, a method based on the file software strategy and SMCS algorithm is proposed in this paper. By the software strategy, the log file indexing and the association between the log file and log time are adopted in this method, so as to quicken the log extraction. In order to analyze the massive logs statistically and control the memory loss, hash tables, file merging and heap operation are used in the SMCS algorithm. By experiment of the real software, results indicate that the proposed method is faster than that of traditional statistical analysis by four times.
Key words:
Syslog log,
log analysis,
SMCS algorithm,
mass log,
file strategy,
control memory
中图分类号:
张俊峰, 冯巧娟, 张晓丽. 基于文本策略和SMCS的海量日志分析方法[J]. 计算机工程, 2012, 38(3): 43-45,48.
ZHANG Dun-Feng, FENG Qiao-Juan, ZHANG Xiao-Li. Mass Log Analysis Method Based on File Strategy and SMCS[J]. Computer Engineering, 2012, 38(3): 43-45,48.