计算机工程

• 人工智能及识别技术 • 上一篇    下一篇

一种基于簇相合性的文本增量聚类算法

陶舒怡1,王明文1,万剑怡1,罗远胜2,左家莉3   

  1. (1. 江西师范大学计算机信息工程学院,南昌 330022;2. 江西财经大学网络信息管理中心,南昌 330013;3. 江西师范大学初等教育学院,南昌 330027)
  • 收稿日期:2013-02-28 出版日期:2014-06-15 发布日期:2014-06-13
  • 作者简介:陶舒怡(1988-),女,硕士研究生,主研方向:信息检索,数据挖掘;王明文,教授、博士生导师;万剑怡,教授;罗远胜,讲师、硕士;左家莉,讲师、博士。
  • 基金项目:
    国家自然科学基金资助项目(61272212)。

An Incremental Text Clustering Algorithm Based on Cluster Congruence

TAO Shu-yi  1, WANG Ming-wen   1, WAN Jian-yi   1, LUO Yuan-sheng   2, ZUO Jia-li    3   

  1. (1. School of Computer Information Engineering, Jiangxi Normal University, Nanchang 330022, China; 2. Network Information Management Center, Jiangxi University of Finance and Economics, Nanchang 330013, China; 3. School of Elementary Education, Jiangxi Normal University, Nanchang 330027, China)
  • Received:2013-02-28 Online:2014-06-15 Published:2014-06-13

摘要: 传统文本聚类方法只适合处理静态样本,且时间复杂度较高。针对该问题,提出一种基于簇相合性的文本增量聚类算法。采用基于词项语义相似度的文本表示模型,利用词项之间的语义信息,通过计算新增文本与已有簇之间的相合性实现对文本的增量聚类。增量处理完部分文本后,对其中错分可能性较大的文本重新指派类别,以进一步提高聚类性能。该算法可在对象数据不断增长或更新的情况下,避免大量重复计算,提高聚类性能。在20 Newsgroups数据集上进行实验,结果表明,与k-means算法和SHC算法相比,该算法可减少聚类时间,提高聚类性能。

关键词: 文本聚类, 增量聚类, 语义相似度, 簇相合性, 文本再分配

Abstract: Traditional text clustering methods are only suitable for static sample, and their time complexity is too high. Aiming at these problems, this paper proposes a new Incremental Text Clustering Algorithm Based on Congruence(ITCAC) between text and cluster. The new algorithm can avoid a lot of double counting to improve the performance of clustering. It uses text representation model based on semantic similarity of lexical items, fully takes the semantic information between terms into account and computes the congruence between new documents and existing clusters. After processing part of the documents, the algorithm reassigns the categorization of documents that has large possibility of misclassification to further improve the clustering performance. Experimental results on 20 Newsgroups datasets show that, compared with the k-means algorithm and SHC algorithm, the new algorithm not only has less clustering time, but also has better performance of clustering.

Key words: text clustering, incremental clustering, semantic similarity, cluster congruence, text redistribution

中图分类号: