基于Hellinger距离与词向量的终身机器学习主题模型

doi:10.19678/j.issn.1000-3428.0062694

摘要/Abstract

摘要： 与传统的机器学习方法相比，终身机器学习能够有效利用知识库中积累的知识来提高当前学习任务的学习效果。然而经典的终身主题模型（LTM）在领域选择时缺乏偏向性，且在计算目标词的相似性时不能充分利用目标词的上下文信息。从词语和主题选择的角度提出改进模型HW-LTM，利用Word2vec词向量的余弦相似度和主题之间的Hellinger距离寻找相似度较大的词语和领域，实现在迭代学习中对词语和领域的更优选择和更有效的知识获取，同时通过预加载词向量相似度矩阵的方式解决词向量余弦距离的重复计算问题，利用Hellinger距离计算主题相似度，加快模型收敛速度。在京东商品评论数据集上的实验结果表明，HW-LTM模型表现优于基线主题挖掘模型，相比LTM模型，其topic coherence指标提升48，耗时缩短43.75%。

关键词: 终身机器学习, 主题模型, Hellinger距离, 词向量, 领域选择

Abstract: Lifelong machine learning, as opposed to conventional machine learning methods, can effectively use accumulated knowledge in the knowledge base to improve the current learning task performance.The classic Lifelong Topic Model(LTM), however, is unbiased in domain selection and fails to fully utilize the contextual information of the target words when calculating similarity.Therefore, from the standpoint of word and topic selection, this study proposes an improved model, named HW-LTM, that finds words and domains with higher similarity using the cosine similarity of Word2vec word vectors and the Hellinger distance between topics.The improved model achieves better word and domain selection during interactive learning and more effective knowledge acquisition.The problem of repeated calculation of the cosine distance of the word vector is solved by preloading the word vector similarity matrix, and the Hellinger distance is used to calculate the topic similarity, which accelerated the model convergence speed.A comparison study using the JD commodity review dataset reveals that HW-LTM outperforms baseline topic mining models.It not only improves the topic coherence index by 48, but it also reduces time consumption by 43.75% when compared to the LTM model.

Key words: lifelong machine learning, topic model, Hellinger distance, word vector, domain selection

中图分类号:

TP18

雷恒林, 古兰拜尔·吐尔洪, 买日旦·吾守尔, 曾琪. 基于Hellinger距离与词向量的终身机器学习主题模型[J]. 计算机工程, 2022, 48(11): 89-95.

LEI Henglin, Gulanbaier Tuerhong, Mairidan Wushouer, ZENG Qi. Topic Model of Lifelong Machine Learning Based on Hellinger Distance and Word Vector[J]. Computer Engineering, 2022, 48(11): 89-95.

http://www.ecice06.com/CN/Y2022/V48/I11/89

图/表 7

20230203202758

20230203202802

20230203202806

20230203202809

20230203202813

20230203202817

20230203202820

参考文献

[1] HOFMANN T.Probabilistic latent semantic analysis[C]//Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence.[S.l.]:Morgan Kaufmann Publishers Inc., 1999:289-296.
[2] BLEI D M, NG A Y, JORDAN M I.Latent Dirichlet allocation[J].Journal of Machine Learning Research, 2003, 3(Jan):993-1022.
[3] LIN C J.Projected gradient methods for nonnegative matrix factorization[J].Neural Computation, 2007, 19(10):2756-2779.
[4] THRUN S, MITCHELL T M.Lifelong robot learning[J].Robotics and Autonomous Systems, 1995, 15(1/2):25-46.
[5] CHEN Z Y, LIU B.Lifelong machine learning[J].Synthesis Lectures on Artificial Intelligence and Machine Learning, 2016, 10(3):1-145.
[6] CHEN Z Y, LIU B, BRACHMAN R, et al.Lifelong machine learning second edition[M].Vermont, USA:Morgan & Claypool, 2018.
[7] CHEN Z Y, LIU B.Topic modeling using topics from many domains, lifelong learning and big data[C]//Proceedings of the 31st International Conference on Machine Learning.[S.l.]:JMLR, 2014:1-10.
[8] 刘一宁, 申彦明.基于终身机器学习的主题挖掘与评分预测联合模型[J].计算机工程, 2019, 45(6):237-241, 248. LIU Y N, SHEN Y M.Topic mining and ratings prediction joint model based on lifelong machine learning[J].Computer Engineering, 2019, 45(6):237-241, 248.(in Chinese)
[9] CHEN Z Y, LIU B.Mining topics in documents:standing on the shoulders of big data[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.New York, USA:ACM Press, 2014:1116-1125.
[10] HA Q T, PHAM T N, NGUYEN V Q, et al.A new lifelong topic modeling method and its application to vietnamese text multi-label classification[C]//Proceedings of Asian Conference on Intelligent Information and Database Systems.Berlin, Germany:Springer, 2018:200-210.
[11] XU M Y, YANG R X, HARENBERG S, et al.A lifelong learning topic model structured using latent embeddings[C]//Proceedings of the 11th International Conference on Semantic Computing.Washington D.C., USA:IEEE Press, 2017:260-261.
[12] XU Y S, YIN Y Y, YIN J W.Tackling topic general words in topic modeling[J].Engineering Applications of Artificial Intelligence, 2017, 62:124-133.
[13] CHEN Y, WU J J, LIN J Y, et al.Affinity regularized non-negative matrix factorization for lifelong topic modeling[J].IEEE Transactions on Knowledge and Data Engineering, 2020, 32(7):1249-1262.
[14] QIN X R, LU Y Y, CHEN Y F, et al.Lifelong learning of topics and domain-specific word embeddings[C]//Proceedings of ACL-IJCNLP 2021.Stroudsburg, USA:Association for Computational Linguistics, 2021:2294-2309.
[15] QIAN J, WANG H, ELSHERIEF M, et al.Lifelong learning of hate speech classification on social media[C]//Proceedings of 2021 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Stroudsburg, USA:Association for Computational Linguistics, 2021:2304-2314.
[16] GUPTA P, CHAUDHARY Y, RUNKLER T, et al.Neural topic modeling with continual lifelong learning[EB/OL].[2021-08-10].https://arxiv.org/abs/2006.10909.
[17] RAHUL A.Detection of intruders and flooding in voip using IDS, jacobson fast and hellinger distance algorithms[J].IOSR Journal of Computer Engineering, 2012, 2(2):30-36.
[18] ALER R, VALLS J M, BOSTRÖM H.Study of Hellinger distance as a splitting metric for random forests in balanced and imbalanced classification datasets[J].Expert Systems with Applications, 2020, 149:113264.
[19] 董明刚, 姜振龙, 敬超.基于海林格距离和SMOTE的多类不平衡学习算法[J].计算机科学, 2020, 47(1):102-109. DONG M G, JIANG Z L, JING C.Multi-class imbalanced learning algorithm based on Hellinger distance and SMOTE algorithm[J].Computer Science, 2020, 47(1):102-109.(in Chinese)
[20] DELAMAIRE A, JUGANARU-MATHIEU M, BEIGBEDER M.Correlation between textual similarity and quality of LDA topic model results[C]//Proceedings of the 13th International Conference on Research Challenges in Information Science.Washington D.C., USA:IEEE Press, 2019:1-6.
[21] WANG Y, HOUGEN C, OSELIO B, et al.A geometry-driven longitudal topic model[J].Harvard Data Science Review, 2021, 3(2):1-10.
[22] MIKOLOV T, CHEN K, CORRADO G, et al.Efficient estimation of word representations in vector space[EB/OL].[2021-08-10].https://arxiv.org/pdf/1301.3781.pdf.
[23] 李晚莲, 田俊钦.国际数字人文领域研究前沿探测与发展趋势分析:基于词嵌入和主题建模技术[J].高校图书馆工作, 2021, 41(3):22-28. LI W L, TIAN J Q.Analysis on research frontier detection and development trend of international digital humanities:based on word embedding and topic modelling technology[J].Library Work in Colleges and Universities, 2021, 41(3):22-28.(in Chinese)
[24] 朱玉虎.基于词嵌入模型的短文本主题发现研究[D].西安:西安电子科技大学, 2020. ZHU Y H.Topic discovery for short texts based on word embeddings[D].Xi'an:Xidian University, 2020.(in Chinese)
[25] 韩亚楠, 刘建伟, 罗雄麟.概率主题模型综述[J].计算机学报, 2021, 44(6):1095-1139. HAN Y N, LIU J W, LUO X L.A survey on probabilistic topic model[J].Chinese Journal of Computers, 2021, 44(6):1095-1139.(in Chinese)
[26] 彭俊利, 谷雨, 张震, 等.融合单词贡献度与Word2Vec词向量的文档表示[J].计算机工程, 2021, 47(4):62-67. PENG J L, GU Y, ZHANG Z, et al.Document representation fused with term contribution and Word2Vec word vector[J].Computer Engineering, 2021, 47(4):62-67.(in Chinese)

选择文件类型/文献管理软件名称

选择包含的内容