Biterm Topic Model Based on Semantic Extension of Double Words

doi:10.19678/j.issn.1000-3428.0049745

Abstract

Abstract:

Aiming at the lack of semantic connection between double words in Biterm Topic Model(BTM) short text documents,a BTM based on semantic extension of double words is proposed.Considering the semantic relationship between each word in double words,the word vector model is introduced.By training the word vector model,the semantic distance between word in double words is judged,and the BTM is extended according to the semantic distance.Experimental results show that,compared with the existing BTM,this model not only has better short text topic classification effect,but also improves the performance of semantic association and topic meaning clustering between double words.

Key words: Biterm Topic Model(BTM), double words, word vector, double words semantic, Gibbs sampling

摘要：

针对Biterm主题模型短文本文档的双词产生过程中词对之间缺乏语义联系的情况,提出一种融入词对语义扩展的Biterm主题模型。考虑双词的语义关系,引入词向量模型。通过训练词向量模型,判断词与词之间的语义距离,并根据语义距离对Biterm主题模型进行双词语义扩展。实验结果表明,与现有Biterm主题模型相比,该模型不仅具有较好的短文本主题分类效果,而且双词间的语义关联性能及主题词义聚类性能也得到明显提升。

关键词: Biterm主题模型, 双词, 词向量, 双词语义, 吉布斯采样

CLC Number:

TP391

LI Siyu,XIE Jun,ZOU Xuejun,XU Xinying,JI Xiaoping. Biterm Topic Model Based on Semantic Extension of Double Words[J]. Computer Engineering, 2019, 45(1): 210-216.

李思宇,谢珺,邹雪君,续欣莹,冀小平. 基于双词语义扩展的Biterm主题模型[J]. 计算机工程, 2019, 45(1): 210-216.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0049745

http://www.ecice06.com/EN/Y2019/V45/I1/210

References

［1］徐戈,王厚峰.自然语言处理中主题模型的发展［J］.计算机学报,2011,34(8):1423-1436.
［2］DEERWESTER S,DUMAIS S T,FURNAS G W,et al.Indexing by latent semantic analysis［J］.Journal of the American Society for Information Science,1990,41(6):391-407.
［3］HOFMANN T.Probabilistic latent semantic indexing［C］//Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and development in Information Retrieval.New York,USA:ACM Press,1999:50-57.
［4］BLEI D M,NG A Y,JORDAN M I.Latent dirichlet allocation［J］.Journal of Machine Learning Research,2003,3:993-1022.
［5］HONG L,DAVISON B.Empirical study of topic modelingin Twitter［C］//Proceedings of the 1st Workshop on Social Media Analytics.New York,USA:ACM Press,2010:80-88.
［6］SRIDHAR V K R.Unsupervised topic modeling for short texts using distributed representations of words［C］//Proceedings of the Workshop on Vector Space Modeling for Natural Language Processing.New York,USA:ACM Press,2015:192-200.
［7］杨萌萌,黄浩,程露红,等.基于LDA主题模型的短文本分类［J］.计算机工程与设计,2016,37(12):3371-3377.
［8］CHENG X,YAN X,LAN Y,et al.BTM:topic modeling over short texts［J］.IEEE Transactions on Knowledge and Data Engineering,2014,26(12):2928-2941.
［9］PANG Jianhui,LI Xiangsheng,XIE Haoran,et al.SBTM:topic modeling over short texts［C］//Proceedings of Database Systems for Advanced Applications.Berlin,Germany:Springer,2016:43-56.
［10］李振兴,王松.基于卡方特征和BTM融合的短文本分类方法［J］.兰州交通大学学报,2016,35(1):36-41.
［11］郑诚,吴文岫,代宁.融合BTM主题特征的短文本分类方法［J］.计算机工程与应用,2016,52(13):95-100.
［12］孙锐,郭晟,姬东鸿.融入事件知识的主题表示方法［J］.计算机学报,2017,40(4):791-804.
［13］奚雪峰,周国栋.面向自然语言处理的深度学习研究［J］.自动化学报,2016,42(10):1445-1465.
［14］LU Tingting,HOU Shifeng,CHEN Zhenxiang,et al.An intention-topic model based on verbs clustering and short texts topic mining［C］//Proceedings of 2015 IEEE International Conference on Computer and Information Technology;Ubiquitous Computing and Communications; Dependable,Autonomic and Secure Computing; Pervasive Intelligence and Computing.Washington D.C.,USA:IEEE Press,2015:837-842.
［15］MIKOLOV T,CHEN K,CORRADO G,et al.Efficient estimation of word representations in vector space［EB/OL］.［2017-12-07］.http://cn.arxiv.org/abs/1301.3781.
［16］MIKOLOV T,SUTSKEVER I,CHEN K,et al.Distributed representations of words and phrases and their compositionality［J］.Advances in Neural Information Processing Systems,2013,26:3111-3119.
［17］MIMNO D,WALLACH H M,TALLEY E,et al.Optimizing semantic coherence in topic models［C］//Proceedings of Conference on Empirical Methods in Natural Language Processing.Stroudsburg,USA:Association for Computational Linguistics,2011:262-272.

[1]	LI Junhuai, CHEN Miaomiao, WANG Huaijun, CUI Ying'an, ZHANG Aihua. Chinese Named Entity Recognition Method Based on ALBERT-BGRU-CRF [J]. Computer Engineering, 2022, 48(6): 89-94,106.
[2]	LI Ranran, LIU Daming, LIU Zheng, CHANG Gaoxiang. Text Classification Using Capsule Network Integrating Stroke Features [J]. Computer Engineering, 2022, 48(3): 69-73,80.
[3]	LEI Henglin, Gulanbaier Tuerhong, Mairidan Wushouer, ZENG Qi. Topic Model of Lifelong Machine Learning Based on Hellinger Distance and Word Vector [J]. Computer Engineering, 2022, 48(11): 89-95.
[4]	XU Weijia, QIN Yongbin, HUANG Ruizhang, CHEN Yanping. Multi-Source Text Topic Model Based on DMA and Feature Division [J]. Computer Engineering, 2021, 47(7): 59-66.
[5]	PENG Junli, GU Yu, ZHANG Zhen, GENG Xiaohang. Document Representation Fused with Term Contribution and Word2Vec Word Vector [J]. Computer Engineering, 2021, 47(4): 62-67.
[6]	LI Jun, LÜ Xueqiang. Keyword Extraction Method Based on BERT Semantic Weighting and Network Graph [J]. Computer Engineering, 2020, 46(9): 89-94.
[7]	WANG Yi, SHEN Yang, DAI Yueming. Sentiment Analysis of Texts Based on Fine-Grained Multi-Channel Convolutional Neural Network [J]. Computer Engineering, 2020, 46(5): 102-108.
[8]	LU Chenyang,KANG Yan,YANG Chengrong,PU Bin. Text Feature Alignment Algorithm for Transfer Learning Based on Semantic Structure [J]. Computer Engineering, 2019, 45(5): 116-121.
[9]	YAO Li,ZHANG Xihuang. An Improved Topics over Time Model Based on Label [J]. Computer Engineering, 2019, 45(4): 205-210,216.
[10]	ZHOU Jinfeng,YE Shiren,WANG Hui. Text Sentiment Classification Based on Deep Convolutional Neural Network Model [J]. Computer Engineering, 2019, 45(3): 300-308.
[11]	YU Jingmin,XIANG Lingyun,ZENG Daojian. Natural Language Steganalysis Method Based on Word2vec [J]. Computer Engineering, 2019, 45(3): 309-314.
[12]	KANG Yan, LI Jinyuan, YANG Qiyue, CUI Guorong, WANG Peiyao. Text Classification Using Convolutional Capsule Network Based on Dual-Channel Word Vectors [J]. Computer Engineering, 2019, 45(11): 177-182.
[13]	LIANG Yanhong, KAN Qixuan, SU Yi. Research on Fuzzy Text Classification Based on Topic Distribution Optimization [J]. Computer Engineering, 2019, 45(10): 221-226.
[14]	BAO Kaifang,GU Junzhong,YANG Jing. Knowledge Graph Completion Method Based on Jointly Representation of Structure and Text [J]. Computer Engineering, 2018, 44(7): 205-211.
[15]	HAO Zhifeng,HUANG Hao,CAI Ruichu,WEN Wen. Fine-grained Opinion Analysis Based on Multi-feature Fusion and Bidirectional RNN [J]. Computer Engineering, 2018, 44(7): 199-204,211.

Please choose a citation manager

Content to export