融合BERT语义加权与网络图的关键词抽取方法

doi:10.19678/j.issn.1000-3428.0055368

摘要/Abstract

摘要： 结合文档本身的结构信息与外部词语的语义信息，提出一种融合BERT词向量与TextRank的关键词抽取方法。在基于网络图的TextRank方法基础上，引入语义差异性并利用BERT词向量加权方式优化TextRank转移概率矩阵计算过程，同时通过迭代运算对文档中的词语进行综合影响力得分排序，最终提取得分最高的TopN个词语作为关键词。实验结果表明，当选取Top3、Top5、Top7和Top10个关键词时，与基于词向量聚类质心与TextRank加权的关键词抽取方法相比，该方法的平均F值提升了2.5%，关键词抽取效率更高。

关键词: 关键词抽取, 语义关系, 词向量, TextRank方法, 基于Transformer的双向编码器表示

Abstract: Based on the structural information of the document and the semantic information of external words,this paper proposes a keyword extraction method based on Bidirectional Encoder Representation from Transformer(BERT) word vectors and TextRank.Using network graph-based TextRank,this method introduces the semantic difference and uses BERT word vector weighting to optimize the calculation process of the transfer possibility matrix of TextRank.At the same time,the overall influence scores of words in the document are sorted by iteration,and the words with the TopN scores are selected as keywords.Experimental results show that when keywords are selected Top3,Top5,Top7 and Top10 words,the average F value of the proposed method is 2.5% higher than that of the keyword extraction method based on word vector clustering centroid and TextRank weighting.The proposed method can improve the efficiency of keyword extraction.

Key words: keyword extraction, semantic relation, word vector, TextRank method, Bidirectional Encoder Representation from Transformer(BERT)

中图分类号:

TP391

李俊, 吕学强. 融合BERT语义加权与网络图的关键词抽取方法[J]. 计算机工程, 2020, 46(9): 89-94.

LI Jun, LÜ Xueqiang. Keyword Extraction Method Based on BERT Semantic Weighting and Network Graph[J]. Computer Engineering, 2020, 46(9): 89-94.

http://www.ecice06.com/CN/Y2020/V46/I9/89

图/表 3

参考文献

[1] ZHAO Jingsheng,ZHU Qiaoming,ZHOU Guodong,et al.Review of research in automatic keyword extraction[J].Journal of Software,2017,28(9):2431-2449.(in Chinese)赵京胜,朱巧明,周国栋,等.自动关键词抽取研究综述[J].软件学报,2017,28(9):2431-2449.
[2] MIHALCEA R,TARAU P.TextRank:bringing order into texts[C]//Proceedings of 2004 Conference on Empirical Methods in Natural Language Processing.Philadelphia,USA:Association for Computational Linguistics,2004:404-411.
[3] HABIBI M,POPESCU-BELIS A.Diverse keyword extraction from conversations[C]//Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics.Philadelphia,USA:Association for Computational Linguistics,2013:651-657.
[4] XIA Tian.Study on keyword extraction using word position weighted TextRank[J].New Technology of Library and Information Service,2013(9):30-34.(in Chinese)夏天.词语位置加权TextRank的关键词抽取研究[J].现代图书情报技术,2013(9):30-34.
[5] ZHANG Lijing,LI Yeli,ZENG Qingtao,et al.Keyword extraction algorithm based on improved TextRank[J].Journal of Beijing Institute of Graphic Communication,2016,24(4):51-55.(in Chinese)张莉婧,李业丽,曾庆涛,等.基于改进TextRank的关键词抽取算法[J].北京印刷学院学报,2016,24(4):51-55.
[6] GU Yijun,XIA Tian.Study on keyword extraction with LDA and TextRank combination[J].New Technology of Library and Information Service,2014(7):41-47.(in Chinese)顾益军,夏天.融合LDA与TextRank的关键词抽取研究[J].现代图书情报技术,2014(7):41-47.
[7] BLEI D M,NG A Y,JORDAN M I,et al.Latent Dirichlet allocation[J].Journal of Machine Learning Research,2003,3:993-1022.
[8] SIU M H,GISH H,CHAN A,et al.Unsupervised training of an HMM-based self-organizing unit recognizer with applications to topic classification and keyword discovery[J].Computer Speech and Language,2014,28(1):210-223.
[9] LI Junfeng,LÜ Xueqiang,ZHOU Shaojun.Patent keyword indexing based on weighted complex graph model[J].New Technology of Library and Information Service,2015(3):26-32.(in Chinese)李军锋,吕学强,周绍钧.带权复杂图模型的专利关键词标引研究[J].现代图书情报技术,2015(3):26-32.
[10] TIXIER A,MALLIAROS F,VAZIRGIANNIS M.A graph degeneracy-based approach to keyword extraction[C]//Proceedings of 2016 Conference on Empirical Methods in Natural Language Processing.Philadelphia,USA:Association for Computational Linguistics,2016:1860-1870.
[11] ZHANG Yuxiang,CHANG Yaocheng,LIU Xiaoqing,et al.MIKE:keyphrase extraction by integrating multidimensional information[C]//Proceedings of 2017 ACM Conference on Information and Knowledge Management.New York,USA:ACM Press,2017:1349-1358.
[12] MIKOLOV T,CHEN K,CORRADO G,et al.Efficient estimation of word representations in vector space[EB/OL].[2019-05-14].https://arxiv.org/abs/1301.3781.
[13] PENNINGTON J,SOCHER R,MANNING C.GloVe:global vectors for word representation[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing.Philadelphia,USA:Association for Computational Linguistics,2014:1532-1543.
[14] WANG R,LIU W,MCDONALD C.Corpus-independent generic keyphrase extraction using word embedding vectors[EB/OL].[2019-05-14].http://www.oalib.com/paper/4057741.
[15] JIANG Fang,LI Guohe,YUE Xiang.Semantic-based keyword extraction method for document[J].Application Research of Computers,2015,32(1):142-145.(in Chinese)姜芳,李国和,岳翔.基于语义的文档关键词提取方法[J].计算机应用研究,2015,32(1):142-145.
[16] NING Jianfei,LIU Jiangzhen.Using Word2vec with TextRank to extract keywords[J].New Technology of Library and Information Service,2016(6):20-27.(in Chinese)宁建飞,刘降珍.融合Word2vec与TextRank的关键词抽取研究[J].现代图书情报技术,2016(6):20-27.
[17] LI Yuepeng,JIN Cui,JI Junchuan.A keyword extraction algorithm based on Word2vec[J].E-science Technology & Application,2015(4):54-59.(in Chinese)李跃鹏,金翠,及俊川.基于Word2vec的关键词提取算法[J].科研信息化技术与应用,2015(4):54-59.
[18] XIA Tian.Extracting keywords with modified TextRank model[J].Data Analysis and Knowledge Discovery,2017(2):28-34.(in Chinese)夏天.词向量聚类加权TextRank的关键词抽取[J].数据分析与知识发现,2017(2):28-34.
[19] DEVLIN J,CHANG M W,LEE K,et al.BERT:pre-training of deep bidirectional transformers for language understanding[EB/OL].[2019-05-14].https://arxiv.org/abs/1810.04805.
[20] WEI Hongxi,GAO Guanglai,SU Xiangdong.LDA-based word image representation for keyword spotting on historical mongolian documents[C]//Proceedings of International Conference on Neural Information Processing.Berlin,Germany:Springer,2016:432-441.
[21] VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing.New York,USA:ACM Press,2017:6000-6010.

选择文件类型/文献管理软件名称

选择包含的内容