基于知识增强的中文命名实体识别

doi:10.19678/j.issn.1000-3428.0059810

摘要/Abstract

摘要： 基于字词联合的中文命名实体识别模型能够兼顾字符级别与词级别的信息，近些年受到了广泛关注，但这类模型受未登录词影响较大，且在小规模数据集存在学习不充分等问题。为此，在 LR-CNN 模型的基础上，提出一种融入先验知识的 AKE 模型。采用相对位置编码的多头注意力机制，提高上下文信息捕捉能力，并通过实体词典融入先验知识，降低未登录词的影响、增强模型学习能力。实验结果表明，与 SoftLexicon、FLAT 等模型相比，在维持较快解码速度和较低资源占用量的基础上，该模型在 MSRA、People Daily、Resume、Weibo 等数据集上 F1 值分别提升了 0.12%、0.12%、0.06%、2.69%，在 OntoNotes5.0、Boson 等数据集上也有较好表现。

Abstract: C hinese named entity recognition(CNER) models that use different character-word information fusion tactics are getting lots of attention recently for their remarkable capacity for information integration but, nonetheless, such models can hardly handle certain problems either, including the negative impact of out-of-vocabulary words, the instability of model performance on small datasets, etc. To address this problem, based on LR-CNN model, this paper proposed Additional Knowledge Enhanced CNER (AKE) model that effectively utilizes priori knowledge. The model uses multi-head attention mechanism with relative position embedding to improve the context modelling competence, and adds prior knowledge to reduce the impact of out-of-vocabulary(OOV) words and to enhance model’s generalization ability. Experimental results show that, compared with SoftLexicon, FLAT, and other models, the proposed one increases the F1 value by 0.12%, 0.12%, 0.06%, 2.69% on MSRA, People Daily, Resume, and Weibo respectively, and gets satisfactory results on OntoNotes5.0 and Boson, while maintain high inference speed and low resources occupancy at the same time.

胡新棒, 于溆乔, 李邵梅, 张建朋 . 基于知识增强的中文命名实体识别[J]. 计算机工程, doi: 10.19678/j.issn.1000-3428.0059810.

HU Xinbang , YU Xuqiao , LI Shaomei , ZHANG Jianpeng . Additional Knowledge Enhanced Chinese Name Entity Recognition[J]. Computer Engineering, doi: 10.19678/j.issn.1000-3428.0059810.

参考文献

[1] MIWA M, BANSAL M. End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures[C]//. 2016:1105-1116.
[2] KOO T, COLLINS M. Efficient third-order dependency parsers[C]//: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. 2010:1-11.
[3] DIEFENBACH D, LOPEZ V, SINGH K, et al. Core techniques of question answering systems over knowledge bases: a survey[J]. Knowledge and Information Systems, 2018,55(3):529-569.
[4] BORTHWICK A. A maximum entropy approach to named entity recognition[Z]. Citeseer, 1999.
[5] BIKEL D M., SCHWARTZ R, WEISCHEDEL R M. An Algorithm that Learns What's in a Name[J]. Machine Learni ng, 1999,34(1):211-231.DOI:10.1023/A:1007558221122.
[6] BIKEL D M., MILLER S, SCHWARTZ R, et al. Nymble: a High-Performance Learning Name-finder[C]//: Fifth Conference on Applied Natural Language Processing. 1997:194-201.
[7] MCCALLUM A, LI Wei. Early results for Named Entity Recognition with Conditional Random Fields, Feature Induction and Web-Enhanced Lexicons[C]//: Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003. 2003:188-191.
[8] LI Xiaoya, MENG Yuxian, SUN Xiaofei, et al. Is Word Segmentation Necessary for Deep Learning of Chinese Representations?[C]//: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019:3242-3252.
[9] 张若彬, 刘嘉勇, 何祥. 基于BLSTM-CRF模型的安全漏洞领域命名实体识别[J]. 四川大学学报(自然科学版), 2019,56(03):469-475. ZHANG Ruobin, LIU Jiayong, HE Xiang. Named entity recognition for vulnerabilities based on BLSTM-CRF model[J]. Journal of Sichuan University (Natural Science Edition), 2019,56(03):469-475.
[10] ZHANG Yue, YANG Jie. Chinese NER Using Lattice LSTM[C]//: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018:1554-1564.
[11] SUI Dianbo, CHEN Yubo, LIU Kang, et al. Leverage lexical knowledge for chinese named entity recognition via collaborative graph network[C]//: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019:3821-3831.
[12] GUI Tao, ZOU Yicheng, ZHANG Qi, et al. A lexicon-based graph neural network for chinese ner[C]//: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019:1039-1049.
[13] DING Ruixue, XIE Pengjun, ZHANG Xiaoyan, et al. A neural multi-digraph model for Chinese NER with gazetteers[C]//: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019:1462-1467.
[14] MA Ruotian, PENG Minlong, ZHANG Qi, et al. Simplify the usage of lexicon in Chinese NER[C]//: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020:5951-5960.
[15] GUI Tao, MA Ruotian, ZHANG Qi, et al. CNN-Based Chinese NER with Lexicon Rethinking[C]//: Twenty-Eighth International Joint Conference on Artificial Intelligence IJCAI-19. 2019
[16] LI Xiaonan, YAN Hang, QIU Xipeng, et al. FLAT: Chinese NER Using Flat-Lattice Transformer[C]//: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2020
[17] XUE Mengge, YU Bowen, LIU Tingwen, et al. Porous Lattice Transformer Encoder for Chinese NER[C]//: Proceedings of the 28th International Conference on Computational Linguistics. 2020:3831-3841.
[18] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[J]. arXiv preprint arXiv:1301.3781, 2013.
[19] YAN Hang, DENG Bocao, LI Xiaonan, et al. TENER: Adapting Transformer Encoder for Named Entity Recognition[J]. arXiv, 2019:1911.
[20] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//: Advances in neural information processing systems. 2017:5998-6008.
[21] LEVOW G. The third international Chinese language processing bakeoff: Word segmentation and named entity recognition[C]//: Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing. 2006:108-117.
[22] HE Hangfeng, SUN Xu. F-Score Driven Max Margin Neural Network for Named Entity Recognition in Chinese Social Media[C]//: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. 2017:713-718.
[23] PENG Nanyun, DREDZE Mark. Named entity recognition for chinese social media with jointly trained embeddings[C]//: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015:548-554.
[24] WEISCHEDEL R, PALMER M, MARCUS Mitchell, et al. Ontonotes release 5.0 ldc2013t19[Z]. 2013: 23.
[25] YU Shiwen, DUAN Huiming, WU Yunfang. Corpus of Multi-level Processing for Modern Chinese[Z]. V1 ed. Peking University Open Research Data Platform, 2018.
[26] MIKOLOV T, SUTSKEVER I, CHEN Kai, et al. Distributed representations of words and phrases and their compositionality[C]//: Advances in neural information processing systems. 2013:3111-3119.
[27] KINGMA D P., BA J. Adam: A Method for Stochastic Optimization[J]. arXiv preprint arXiv:1412.6980, 2014.
[28] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout: a simple way to prevent neural networks from overfitting[J]. The journal of machine learning research, 2014,15(1):1929-1958.