Chinese Nested Named Entity Recognition Based on Location Embedding and Multilevel Prediction

doi:10.19678/j.issn.1000-3428.0066379

Abstract

Abstract:

Traditional Chinese nested Named Entity Recognition(NER) models often face problems, such as difficulty in accurately locating entity boundaries and blurred boundaries between Chinese characters and vocabulary. A nested NER model based on position embedding and multilevel result boundary prediction is proposed to address this problem. The position information of nested entities is encoded with the text position information in the embedding layer. An absolute position sequence is then generated, which further examines the relationship between the nested entities and characters and strengthens the connection between the nested entities and the original text by focusing on the position information in the Chinese text. At the encoding layer, the nested entities are initially identified using a hidden matrix that excludes the best path with multilevel prediction. At the decoding layer, the offset of entity boundaries is calculated at the multilevel prediction layer to redefine the entity boundaries, and improve the accuracy of Chinese entity prediction. The experimental results show that the proposed model improves the precision, recall, and F1-value by 0.34, 1.06, and 0.80 percentage points, respectively, on the medical domain dataset, and by 11.90, 0.78, and 6.23 percentage points, respectively, on the daily domain dataset compared to the highest value in the baseline models. This study demonstrates that the proposed model exhibits high performance in recognizing Chinese nested named entities.

Key words: nested Named Entity Recognition(NER), location embedding, Boundary Prediction Unit(BPU), Conditional Random Field(CRF), multilevel prediction

摘要：

针对传统中文嵌套命名实体识别模型通常存在实体边界难以准确定位及中文字符与词汇之间边界模糊的问题，构建一种基于位置嵌入和多级结果边界预测的嵌套命名实体识别模型。在嵌入层，将嵌套实体位置信息与文本位置信息同时编码后生成绝对位置序列，通过关注中文文本中自带的位置信息，进一步挖掘嵌套实体与字符之间的关系，并且增强了嵌套实体与原始文本之间的联系。在编码层，利用排除最优路径的隐藏矩阵实现嵌套实体的初步识别。在解码层，计算实体边界的偏移量，重新确定实体边界，从而提高中文嵌套实体识别准确率。实验结果表明，在医疗和日常两个领域的数据集上，该模型的准确率、召回率、F1值相比于基线模型中的最优值分别提高了0.34、1.06、0.80和11.90、0.78、6.23个百分点，具有较好的识别性能。

关键词: 嵌套命名实体识别, 位置嵌入, 边界预测单元, 条件随机场, 多级预测

Jianyong DUAN, Yifei ZHU, Hao WANG, Li HE, Xin LI. Chinese Nested Named Entity Recognition Based on Location Embedding and Multilevel Prediction[J]. Computer Engineering, 2023, 49(12): 71-77.

段建勇, 朱奕霏, 王昊, 何丽, 李欣. 基于位置嵌入和多级预测的中文嵌套命名实体识别[J]. 计算机工程, 2023, 49(12): 71-77.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0066379

http://www.ecice06.com/EN/Y2023/V49/I12/71

Figures/Tables 5

References 29

1	FANG Z, CAO Y N, LI R, et al. High quality candidate generation and sequential graph attention network for entity linking[C]//Proceedings of Web Conference 2020. New York, USA: ACM Press, 2020: 640-650.
2	GEKHMAN Z, AHARONI R, BERYOZKIN G, et al. KoBE: knowledge-based machine translation evaluation[EB/OL]. [2022-10-14]. https://arxiv.org/abs/2009.11027.pdf.
3	LI B Z, MIN S, IYER S, et al. Efficient one-pass end-to-end entity linking for questions[EB/OL]. [2022-10-14]. https://arxiv.org/abs/2010.02413.pdf.
4	NADEAU D, SEKINE S. A survey of named entity recognition and classification. Lingvisticae Investigationes, 2007, 30(1): 3- 26. doi: 10.1075/li.30.1.03nad
5	GRAVES A. Supervised sequence labelling. Berlin, Germany: Springer, 2012.
6	DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional Transformers for language understanding[EB/OL]. [2022-10-14]. https://arxiv.org/abs/1810.04805.pdf.
7	张汝佳, 代璐, 王邦, 等. 基于深度学习的中文命名实体识别最新研究进展综述. 中文信息学报, 2022, 36(6): 20- 35. URL
	ZHANG R J, DAI L, WANG B, et al. Recent advances of Chinese named entity recognition based on deep learning. Journal of Chinese Information Processing, 2022, 36(6): 20- 35. URL
8	WANG B L, LU W. Neural segmental hypergraphs for overlapping mention recognition[EB/OL]. [2022-10-14]. https://arxiv.org/abs/1810.01817.pdf.
9	SOHRAB M G, MIWA M. Deep exhaustive model for nested named entity recognition[C]//Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2018: 2843-2849.
10	SHEN Y L, MA X Y, TAN Z Q, et al. Locate and label: a two-stage identifier for nested named entity recognition[EB/OL]. [2022-10-14]. https://arxiv.org/abs/2105.06804.pdf.
11	连艺谋, 张英俊, 谢斌红. 用于嵌套命名实体识别的边界强化分类模型. 计算机工程, 2022, 48(8): 313- 320. URL
	LIAN Y M, ZHANG Y J, XIE B H. Boundary enhanced classification model for nested named entity recognition. Computer Engineering, 2022, 48(8): 313- 320. URL
12	SHIBUYA T, HOVY E. Nested named entity recognition via second-best sequence learning and decoding. Transactions of the Association for Computational Linguistics, 2020, 8, 605- 620. doi: 10.1162/tacl_a_00334
13	WANG Y R, SHINDO H, MATSUMOTO Y, et al. Nested named entity recognition via explicitly excluding the influence of the best path. Journal of Natural Language Processing, 2022, 29(1): 23- 52. doi: 10.5715/jnlp.29.23
14	HUMPHREYS K, GAIZAUSKAS R, AZZAM S, et al. University of Sheffield: description of the LaSIE-II system as used for MUC-7[EB/OL]. [2022-10-14]. https://aclanthology.org/M98-1007.pdf.
15	KRUPKA G, HAUSMAN K. IsoQuest Inc. : description of the NetOwl™ extractor system as used for MUC-7[EB/OL]. [2022-10-14]. https://aclanthology.org/M98-1015.pdf.
16	BLACK W J, RINALDI F, MOWATT D. FACILE: description of the NE system used for MUC-7[EB/OL]. [2022-10-14]. https://aclanthology.org/M98-1014.pdf.
17	AONE C, HALVERSON L, HAMPTON T, et al. SRA: description of the IE2 system used for MUC-7[EB/OL]. [2022-10-14]. https://aclanthology.org/M98-1012.pdf.
18	EDDY S R. Hidden Markov models. Current Opinion in Structural Biology, 1996, 6(3): 361- 365. doi: 10.1016/S0959-440X(96)80056-X
19	QUINLAN J R. Induction of decision trees. Machine Learning, 1986, 1(1): 81- 106.
20	KAPUR J N. Maximum-entropy models in science and engineering. Biometrics, 1992, 48(1): 333.
21	SUTHAHARAN S. Support vector machine. Berlin, Germany: Springer, 2016.
22	LAFFERTY J, MCCALLUM A, PEREIRA F. Conditional random fields: probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the 18th International Conference on Machine Learning. New York, USA: ACM Press, 2001: 282-289.
23	ZHANG Y, YANG J. Chinese NER using lattice LSTM[EB/OL]. [2022-10-14]. https://arxiv.org/abs/1805.02023.pdf.
24	崔丽平, 古丽拉·阿东别克, 王智悦. 基于有向图模型的旅游领域命名实体识别. 计算机工程, 2022, 48(2): 306- 313. URL
	CUI L P, Gulila Altenbek, WANG Z Y. Named entity recognition in tourism based on directed graph model. Computer Engineering, 2022, 48(2): 306- 313. URL
25	WU S, SONG X N, FENG Z H. MECT: multi-metadata embedding based cross-transformer for Chinese named entity recognition[EB/OL]. [2022-10-14]. https://arxiv.org/abs/2107.05418.pdf.
26	SUI D B, TIAN Z K, CHEN Y B, et al. A large-scale Chinese multimodal NER dataset with speech clues[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing(Volume 1: Long Papers). Stroudsburg, USA: Association for Computational Linguistics, 2021: 2807-2818.
27	廖涛, 黄荣梅, 张顺香, 等. 基于交互式特征融合的嵌套命名实体识别. 计算机工程, 2022, 48(12): 119-126, 133. URL
	LIAO T, HUANG R M, ZHANG S X, et al. Nested named entity recognition based on interactive feature fusion. Computer Engineering, 2022, 48(12): 119-126, 133. URL
28	LI X N, YAN H, QIU X P, et al. FLAT: Chinese NER using flat-lattice Transformer[EB/OL]. [2022-10-14]. https://arxiv.org/abs/2004.11795.pdf.
29	JU M, MIWA M, ANANIADOU S. A neural layered model for nested named entity recognition[C]//Proceedings of 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1(Long Papers). Philadelphia, USA: ACL Press, 2018: 1446-1459.

[1]	LIAN Yimou, ZHANG Yingjun, XIE Binhong. Boundary Enhanced Classification Model for Nested Named Entity Recognition [J]. Computer Engineering, 2022, 48(8): 313-320.
[2]	SI Yichen, GUAN Youqing. Chinese Named Entity Recognition Model Based on Transformer Encoder [J]. Computer Engineering, 2022, 48(7): 66-72.
[3]	LI Junhuai, CHEN Miaomiao, WANG Huaijun, CUI Ying'an, ZHANG Aihua. Chinese Named Entity Recognition Method Based on ALBERT-BGRU-CRF [J]. Computer Engineering, 2022, 48(6): 89-94,106.
[4]	CUI Liping, Altenbek Gulila, WANG Zhiyue. Named Entity Recognition in Tourism Based on Directed Graph Model [J]. Computer Engineering, 2022, 48(2): 306-313.
[5]	LIAO Tao, HUANG Rongmei, ZHANG Shunxiang, DUAN Songsong. Nested Named Entity Recognition Based on Interactive Feature Fusion [J]. Computer Engineering, 2022, 48(12): 119-126,133.
[6]	Lü Jianghai, DU Junping, ZHOU Nan, XUE Zhe. Entity Name Recognition Method Based on Dilated Convolutional Iterative and Attention Mechanism [J]. Computer Engineering, 2021, 47(1): 58-65,71.
[7]	HE Yangyu, YAN Lei, YI Mianzhu, LI Hongxin. Named Entitiy Recognition Method for Laotian in Military Field Combining CRF and Rules [J]. Computer Engineering, 2020, 46(8): 297-304.
[8]	YANG Piao, DONG Wenyong. Chinese Named Entity Recognition Method Based on BERT Embedding [J]. Computer Engineering, 2020, 46(4): 40-45,52.
[9]	CAI Kai, LI Xinfu, TIAN Xuedong. 3D Content Generation Method Based on Visual Attention Analysis [J]. Computer Engineering, 2020, 46(4): 266-272.
[10]	WANG Renwu, ZHANG Wenhui. Implicit Evaluation Object Recognition Method Based on Deep Learning [J]. Computer Engineering, 2019, 45(8): 315-320.
[11]	ZHANG Jiemei, YANG Cihui. Automatic Segmentation Algorithm of CT Liver Image Based on RV-FCN [J]. Computer Engineering, 2019, 45(7): 258-263.
[12]	ZHANG Yingcheng,YANG Yang,JIANG Rui,QUAN Bing,ZHANG Lijun,REN Xiaolei. Commercial intelligence entity recognition model based on BiLSTM-CRF [J]. Computer Engineering, 2019, 45(5): 308-314.
[13]	WANG Junqiang, LI Jiansheng, ZHOU Huachun, ZHANG Xu. Typical Element Extraction Method of Remote Sensing Image Based on Deeplabv3+ and CRF [J]. Computer Engineering, 2019, 45(10): 260-265,271.
[14]	CHEN Bingfeng,HAO Zhifeng,CAI Ruichu,WEN Wen,LIANG Lixin. Method of Microblog Emotional Tendency Classification Based on AWCRF Model [J]. Computer Engineering, 2017, 43(7): 187-192.
[15]	YI Meng,SUI Lichun. Aerial Image Semantic Classification Method Based on Improved Full Convolution Neural Network [J]. Computer Engineering, 2017, 43(10): 216-221.

Please choose a citation manager

Content to export