作者投稿和查稿 主编审稿 专家审稿 编委审稿 远程编辑

计算机工程

• 人工智能及识别技术 • 上一篇    下一篇

改进的频繁词集短文本特征扩展方法

马慧芳,曾宪桃,李晓红,朱志强   

  1. (西北师范大学 计算机科学与工程学院,兰州 730070)
  • 收稿日期:2015-10-19 出版日期:2016-10-15 发布日期:2016-10-15
  • 作者简介:马慧芳(1981—),女,副教授、博士,主研方向为人工智能、数据挖掘、机器学习;曾宪桃,本科生;李晓红,讲师、硕士;朱志强,本科生。
  • 基金资助:
    国家自然科学基金资助项目(61363058);甘肃省青年科技基金资助项目(145RJYA259);甘肃省自然科学研究基金资助项目(145RJZA232,1606RJYA269);甘肃省互联网计算应用创新创业众创空间基金资助项目(1505JTCA007);西北师范大学2013年度青年教师科研能力提升计划基金资助项目(NWNU-LKQN-12-23)。

Short Text Feature Extension Method of Improved Frequent Term Set

(College of Computer Science and Engineering,Northwest Normal University,Lanzhou 730070,China)   

  1. (College of Computer Science and Engineering,Northwest Normal University,Lanzhou 730070,China)
  • Received:2015-10-19 Online:2016-10-15 Published:2016-10-15

摘要: 针对短文本结构短小、语义不足、难以建模的特点,提出一种利用改进频繁词集进行短文本特征扩展的方法。通过计算单词集的支持度和置信度,挖掘出具有共现关系和类别同向关系的频繁二元词集,并在挖掘出的频繁词集基础上定义关联关系对所选词集进一步扩充。同时,在TF-IDF的基础上引入词语信息增益表示词语在文本集合中的类别分布信息,以加强词语权重。由频繁词集通过改进后的词语权重构造出词语相似性矩阵,利用非负矩阵分解技术将其扩展至短文本特征空间,从而得到短文本模型。实验结果表明,该方法构造的短文本模型能显著提升短文本的聚类性能。

关键词: 词语权重, 信息增益, 频繁词集, 关联关系, 非负矩阵

Abstract: Short text is generally of short structure and semantic inadequacy which makes it difficult to model.Therefore a method for short text feature extension via improved frequent term set is proposed.By calculating support and confidence,with co-occurrence relations and same lategory tendency are extracted.Based on the extracted frequent term set correlations are defined to further extend the term set.Meanwhile,information gain is introduced based on the traditional TF-IDF,to express the category distribution information and enhance the weight of the words for each category.Finally,the word similarity matrix is constructed via the frequent term set,and the symmetric non-negative matrix factorization technique is used to extend the feature space.Experimental results show that the constructed short text model built by this method can significantly improve the performance of short text clustering.

Key words: term weight, information gain, frequent term set, incidence relation, non-negative matrix

中图分类号: