基于图注意力网络字词融合的中文命名实体识别

doi:10.19678/j.issn.1000-3428.0063055

计算机工程 ›› 2022, Vol. 48 ›› Issue (10): 298-305. doi: 10.19678/j.issn.1000-3428.0063055

基于图注意力网络字词融合的中文命名实体识别

宋旭晖¹, 于洪涛², 李邵梅²

1. 郑州大学网络空间安全学院, 郑州 450001;
2. 战略支援部队信息工程大学信息技术研究所, 郑州 450002

收稿日期:2021-10-27 修回日期:2021-12-15 发布日期:2021-12-17
作者简介:宋旭晖(1999—),女,硕士研究生,主研方向为自然语言处理;于洪涛,研究员、博士;李邵梅,副研究员、博士。
基金资助:
国家自然科学基金青年科学基金项目（62002384）；国家重点研发计划项目（2016QY03D0502）；郑州市协同创新重大专项（162/32410218）。

Chinese Named Entity Recognition Based on Word Fusion of Graph Attention Network

SONG Xuhui¹, YU Hongtao², LI Shaomei²

1. School of Cyber Science and Engineering, Zhengzhou University, Zhengzhou 450001, China;
2. Institute of Information Technology, PLA Strategic Support Force Information Engineering University, Zhengzhou 450002, China

Received:2021-10-27 Revised:2021-12-15 Published:2021-12-17

摘要/Abstract

摘要： 命名实体识别指识别文本中具有特定意义的实体，是自然语言处理诸多下游任务的重要基石。在命名实体识别任务中，协同图网络（CGN）模型通过引入多个图注意力网络获得较强的知识整合能力及较高的处理速度，但CGN模型在嵌入层中没有充分利用词边界信息，且采用的传统静态图注意力网络影响了图注意力的表达能力。在对CGN模型的图注意力网络进行改进的基础上，提出一种中文命名实体识别新模型，在嵌入层融入词语的分词信息，以生成包含词边界信息的字向量，从而充分利用词边界信息。通过在编码层使用BiLSTM模型获取文本的上下文信息，采用改进后的图注意力网络提取文本特征，并通过优化传统图注意力网络中相关系数的计算方式，增强模型的特征提取能力。最后，利用条件随机场对文本进行解码，从而实现对实体的标注。实验结果表明，该模型相比CGN模型在MSRA、OntoNotes4.0、Weibo数据集上的F1值分别提升了0.67%、3.16%、0.16%，验证了其在中文命名实体识别任务上的有效性。

关键词: 自然语言处理, 中文命名实体识别, 图注意力网络, 字词融合, 分词信息

Abstract: Named entity recognition refers to the recognition of entities with specific meanings in texts.It is an important cornerstone of many downstream tasks in natural language processing.In the task of named entity recognition, the Collaborative Graph Network (CGN) model gains strong knowledge integration capabilities and efficient processing speed by introducing a multiple Graph Attention Network (GAT).However, this model does not make full use of word boundary information in the embedding layer, and use of the traditional static GAT model affects the expressive ability of graph attention.On the basis of improving the GAT of CGN mode, this paper proposes a new Chinese Named Entity Recognition(NER) model.The method firstly incorporates word segmentation information in an embedding layer to generate word vectors containing word boundary information, thereby improving the use of word boundary information.Secondly, it uses a BiLSTM model to initially obtain the context information of the text, and then uses the improved GAT(GATv2) to extract text features by optimizing the calculation method of the correlation coefficient in the traditional GAT.Finally, a conditional random field is used for decoding to realize the labeling of the entity.The experimental results show that compared with CGN model, the F1 value of the proposed model on MSRA, Weibo datasets is increased by 0.67%, 3.16% and 0.16% respectively, which verifies the effectiveness of the model in the task of Chinese named entity recognition.

Key words: natural language processing, Chinese Named Entity Recognition(NER), Graph Attention Network(GAT), word fusion, word segmentation information

中图分类号:

TP393

宋旭晖, 于洪涛, 李邵梅. 基于图注意力网络字词融合的中文命名实体识别[J]. 计算机工程, 2022, 48(10): 298-305.

SONG Xuhui, YU Hongtao, LI Shaomei. Chinese Named Entity Recognition Based on Word Fusion of Graph Attention Network[J]. Computer Engineering, 2022, 48(10): 298-305.

http://www.ecice06.com/CN/Y2022/V48/I10/298

图/表 9

参考文献

[1] HAN X, GAO T Y, LIN Y K, et al.More data, more relations, more context and more openness:a review and outlook for relation extraction[EB/OL].[2021-09-10].https://arxiv.org/abs/2004.03186.
[2] XIANG W, WANG B.A survey of event extraction from text[J].IEEE Access, 2019, 7:173111-173137.
[3] DIEFENBACH D, LOPEZ V, SINGH K, et al.Core techniques of question answering systems over knowledge bases:a survey[J].Knowledge and Information Systems, 2018, 55(3):529-569.
[4] LI J, SUN A X, HAN J L, et al.A survey on deep learning for named entity recognition[J].IEEE Transactions on Knowledge and Data Engineering, 2022, 34(1):50-70.
[5] KIM Y.Convolutional neural networks for sentence classification[EB/OL].[2021-09-10].https://arxiv.org/abs/1408.5882.
[6] LIU P F, QIU X P, HUANG X J.Recurrent neural network for text classification with multi-task learning[EB/OL].[2021-09-10].https://arxiv.org/abs/1605.05101.
[7] GUI T, MA R, ZHANG Q, et al.CNN-based Chinese NER with lexicon rethinking[EB/OL].[2021-09-10].https://www.researchgate.net/publication/334844205_CNN-Based_Chinese_NER_with_Lexicon_Rethinking.
[8] LIU W, XU T, XU Q, et al.An encoding strategy based word-character LSTM for Chinese NER[C]//Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, Volume 1(Long and Short Papers).Stroudsburg, USA:Association for Computational Linguistics, 2019:2379-2389.
[9] SUI D B, CHEN Y B, LIU K, et al.Leverage lexical knowledge for Chinese named entity recognition via collaborative graph network[C]//Proceedings of 2019 Conference on Empirical Methods in Natural Language and the 9th International Joint Conference on Natural Language Stroudsburg, USA:Association for Computational Linguistics, 2019:3830-3840.
[10] VELIČKOVIĆ P, CUCURULL G, CASANOVA A, et al.Graph attention networks[EB/OL].[2021-09-10].https://arxiv.org/abs/1710.10903.
[11] BRODY S, ALON U, YAHAV E.How attentive are graph attention networks?[EB/OL].[2021-09-10].https://arxiv.org/abs/2105.14491.
[12] WOODWARD R J, CHOUEIRY B Y.Weight-based variable ordering in the context of high-level consistencies[EB/OL].[2021-09-10].https://www.xueshufan.com/publication/2767443280.
[13] 杨飘, 董文永.基于BERT嵌入的中文命名实体识别方法[J].计算机工程, 2020, 46(4):40-45, 52. YANG P, DONG W Y.Chinese named entity recognition method based on BERT embedding[J].Computer Engineering, 2020, 46(4):40-45, 52.(in Chinese)
[14] HUANG Z H, XU W, YU K.Bidirectional LSTM-CRF models for sequence tagging.[EB/OL].[2021-09-10].https://arxiv.org/abs/1508.019911.
[15] LAFFERTY J, MCCALLUM A, PEREIRA F.Conditional random fields:probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the 18th International Conference on Machine Learning.San Francisco, USA:Morgan Kaufmann Publishers, 2001:282-289.
[16] LEVOW G A.The 3rd international Chinese language processing bakeoff:word segmentation and named entity recognition[EB/OL].[2021-09-10].https://www.xueshufan.com/publication/2252066972.
[17] WEISCHEDEL R, PRADHAN S, RAMSHAW L, et al.Ontonotes release 4.0[EB/OL].[2021-09-10].https://www.researchgate.net/publication/254378913_OntoNotes_Release_20.
[18] ZHANG Y, YANG J.Chinese NER using lattice LSTM[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).Stroudsburg, USA:Association for Computational Linguistics, 2018:1554-1564.
[19] PENG N Y, DREDZE M.Named entity recognition for Chinese social media with jointly trained embeddings[C]//Proceedings of 2015 Conference on Empirical Methods in Natural Language.Stroudsburg, USA:Association for Computational Linguistics, 2015:548-554.
[20] YU S, DUAN H, WU Y.Corpus of multi-level Processing for Modern Chinese[EB/OL].[2021-09-10].https://doi.org/10.18170/DVN/SEYRX5.
[21] LI S, ZHAO Z, HU R F, et al.Analogical reasoning on Chinese morphological and semantic relations[EB/OL].[2021-09-10].https://arxiv.org/abs/1805.06504.
[22] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al.Dropout:a simple way to prevent neural networks from overfitting[J].The Journal of Machine Learning Research, 2014, 15(1):1929-1958.
[23] KINGMA D P, BA J.Adam:a method for stochastic optimization[EB/OL].[2021-09-10].https://ui.adsabs.harvard.edu/abs/2014arXiv1412.6980K.
[24] GUI T, ZOU Y C, ZHANG Q, et al.A lexicon-based graph neural network for Chinese NER[C]//Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing.Stroudsburg, USA:Association for Computational Linguistics, 2019:1040-1050.
[25] LI X N, YAN H, QIU X P, et al.FLAT:Chinese NER using flat-lattice transformer[EB/OL].[2021-09-10].https://arxiv.org/abs/2004.11795.

选择文件类型/文献管理软件名称

选择包含的内容

基于图注意力网络字词融合的中文命名实体识别

Chinese Named Entity Recognition Based on Word Fusion of Graph Attention Network

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	张家熔, 苑津莎, 许珈宁, 罗志宏. 基于多元信息嵌入与协同神经网络的力学实体识别算法[J]. 计算机工程, 2023, 49(7): 125-134.
[2]	郭艳霞, 金勇, 唐宏, 彭金枝. 基于动态卷积与残差门控的多模态情感识别[J]. 计算机工程, 2023, 49(7): 94-101.
[3]	朱红, 牛浩然, 朱彤. 基于字词融合与对抗训练的行业人物实体识别[J]. 计算机工程, 2023, 49(5): 56-62.
[4]	李静雯, 赵奎. 基于改进PCFG算法的口令猜测方法[J]. 计算机工程, 2023, 49(5): 38-47.
[5]	戎珂瑶, 熊贇. 基于多维度异质图结构的代码注释自动生成[J]. 计算机工程, 2023, 49(4): 240-248.
[6]	廖列法, 谢树松. 基于注意力机制特征融合的中文命名实体识别[J]. 计算机工程, 2023, 49(4): 256-262.
[7]	刘金硕, 刘宁. 面向招标文件的半结构化文本自动生成[J]. 计算机工程, 2023, 49(3): 67-72.
[8]	杨文忠, 丁甜甜, 康鹏, 卜文秀. 基于舆情新闻的中文关键词抽取综述[J]. 计算机工程, 2023, 49(3): 1-17.
[9]	蔡瑞初, 张盛强, 许柏炎. 基于结构感知混合编码模型的代码注释生成方法[J]. 计算机工程, 2023, 49(2): 61-69.
[10]	张晓晖, 马慧芳, 王文涛, 高子皓. 基于跨会话知识图谱的图注意力网络推荐方法[J]. 计算机工程, 2023, 49(2): 136-142,149.
[11]	王春东, 孙嘉琪, 杨文军. 基于矫正理解的中文文本对抗样本生成方法[J]. 计算机工程, 2023, 49(2): 37-45.
[12]	田乔鑫, 孔韦韦, 滕金保, 王照乾. 基于并行混合网络与注意力机制的文本情感分析模型[J]. 计算机工程, 2022, 48(8): 266-273.
[13]	司逸晨, 管有庆. 基于Transformer编码器的中文命名实体识别模型[J]. 计算机工程, 2022, 48(7): 66-72.
[14]	张吉祥, 张祥森, 武长旭, 赵增顺. 知识图谱构建技术综述[J]. 计算机工程, 2022, 48(3): 23-37.
[15]	张奕, 郑婧, 蔡钢生, 王真梅. 基于GAT双聚合运算与归纳式矩阵补全的关联预测[J]. 计算机工程, 2022, 48(12): 72-78.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于图注意力网络字词融合的中文命名实体识别

Chinese Named Entity Recognition Based on Word Fusion of Graph Attention Network

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献

相关文章 15

编辑推荐

Metrics

本文评价