作者投稿和查稿 主编审稿 专家审稿 编委审稿 远程编辑

计算机工程 ›› 2012, Vol. 38 ›› Issue (16): 203-206. doi: 10.3969/j.issn.1000-3428.2012.16.053

• 人工智能及识别技术 • 上一篇    下一篇

基于Map Reduce的Bagging贝叶斯文本分类

冀素琴,石洪波,卫 洁   

  1. (山西财经大学信息管理学院,太原 030031)
  • 收稿日期:2011-09-15 修回日期:2011-12-08 出版日期:2012-08-20 发布日期:2012-08-17
  • 作者简介:冀素琴(1972-),女,讲师、硕士,主研方向:数据挖掘,分布式系统;石洪波,教授、博士;卫 洁,硕士研究生
  • 基金资助:

    国家自然科学基金资助项目(60873100);山西省自然科学基金资助项目(2009011017-4)

Bagging Bayes Text Classification Based on Map Reduce

JI Su-qin, SHI Hong-bo, WEI Jie   

  1. (School of Information Management, Shanxi University of Finance & Economics, Taiyuan 030031, China)
  • Received:2011-09-15 Revised:2011-12-08 Online:2012-08-20 Published:2012-08-17

摘要: 集中式系统框架难以进行海量文本数据分类。为此,提出一种基于Map Reduce的Bagging贝叶斯文本分类算法。介绍朴素贝叶斯文本分类算法,将其与Bagging算法结合,运用Map Reduce并行编程模型,在Hadoop平台上实现算法。实验结果表明,该算法分类准确率较高,运行时间较短,适用于大规模文本数据集的分类学习。

关键词: 分布式, Map Reduce模型, 文本分类, 集成学习, 朴素贝叶斯, Bagging算法

Abstract: In order to solve the problem that the classification is difficult on massive text data under the framework of a centralized system, this paper proposes a Bagging Bayes text classification algorithm based on Map Reduce. It introduces the Naive Bayes text classification algorithm. Combined with the Bagging algorithm, it uses Map Reduce parallel programming model to realize the algorithm on Hadoop platform. Experimental results show that this algorithm can be used in the classification of large-scale text data sets, have good accuracy and short running time.

Key words: distribution, Map Reduce model, text classification, ensemble learning, Naive Bayes, Bagging algorithm

中图分类号: