Entity Name Recognition Method Based on Dilated Convolutional Iterative and Attention Mechanism

doi:10.19678/j.issn.1000-3428.0055986

Abstract

Abstract: The traditional entity name recognition methods fail to balance the effectiveness of feature extraction of text sequence and the training speed of neural network models.To address the problem,this paper proposes an entity name recognition method based on Iterated Dilated Convolutional Neural Network and Attention Mechanism(IDCNN-ATT).The IDCNN can fully utilize the optimization ability of GPU parallel computing,and retain the ability of Long Short-Term Memory(LSTM) neural network to remember as much information as possible based on simple structure.IDCNN can accurately extract the features of text sequences while greatly accelerating the training of neural network models.Moreover,the Attention Mechanism(ATT) is introduced to use the grammar information of the text and part-of-speech information of the words to select the features that are more critical to entity name recognition from multiple text features,which effectively improves the accuracy of text feature extraction.Experimental results show that the proposed attention-based method improves the indicators of entity name recognition by 2% compared with the traditional non-attention mechanism method.

Key words: entity name recognition, Attention Mechanism(ATT), dilated convolution, Long Short-Term Memory(LSTM) network, Conditional Random Field(CRF)

摘要： 针对传统实体名识别方法无法兼顾文本序列提取特征的有效性和神经网络模型训练速度的问题，提出一种基于迭代膨胀卷积神经网络（IDCNN）与注意力机制（ATT）的实体名识别方法。IDCNN可利用GPU并行计算的优化能力，保留长短期记忆神经网络的特性，即用简单的结构记录尽可能多的输入信息，并在准确提取文本序列特征的同时加快神经网络模型的训练速度。通过引入ATT运用文本语法信息和单词词性信息，从众多文本特征中选择对实体名识别更关键的特征，从而提高文本特征提取的准确性。在新闻数据集和微博数据集上的实验结果表明，神经网络模型的训练速度比传统的双向长短期记忆神经网络有显著提升，基于注意力的实体名识别方法的评价指标比传统的无注意力机制方法提高2%左右。

关键词: 实体名识别, 注意力机制, 膨胀卷积, 长短期记忆网络, 条件随机场

CLC Number:

TP391

Lü Jianghai, DU Junping, ZHOU Nan, XUE Zhe. Entity Name Recognition Method Based on Dilated Convolutional Iterative and Attention Mechanism[J]. Computer Engineering, 2021, 47(1): 58-65,71.

吕江海, 杜军平, 周南, 薛哲. 基于膨胀卷积迭代与注意力机制的实体名识别方法[J]. 计算机工程, 2021, 47(1): 58-65,71.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0055986

http://www.ecice06.com/EN/Y2021/V47/I1/58

Figures/Tables 7

References

[1] ZHU Q,LI X,CONESA A,et al.GRAM-CNN:a deep learning approach with local context for named entity recognition in biomedical text[J].Bioinformatics,2017,34(9):1547-1554.
[2] GRIDACH M.Character-level neural network for biomedical named entity recognition[J].Journal of Biomedical Informatics,2017,70:85-91.
[3] ZHOU Xiaolei,ZHAO Xuejiao,LIU Tangliang,et al.Named entity recognition method of judgment documents with SVM-BiLSTM-CRF[J].Computer Systems & Applications,2019,28(1):245-250.(in Chinese)周晓磊,赵薛蛟,刘堂亮,等.基于SVM-BiLSTM-CRF模型的财产纠纷命名实体识别方法[J].计算机系统应用,2019,28(1):245-250.
[4] MAI K,PPHAM T H,NGUYEN M T,et al.An empirical study on fine-grained named entity recognition[C]//Proceedings of the 27th IEEE International Conference on Computational Linguistics.Washington D.C.,USA:IEEE Press,2018:711-722.
[5] CHEN Ningyu,ZHOU Yaqian,HUANG Xuanjing,et al.Named entity recognition with un-annotated data[J].Journal of Chinese Information Processing,2005,19(2):8-12.(in Chinese)陈宁昱,周雅倩,黄萱菁,等.利用未标注语料改进实体名识别性能[J].中文信息学报,2005,19(2):8-12.
[6] WANG Haochang,ZHAO Tiejun.SVM-based biomedical name entity recognition[J].Journal of Harbin Engineering University,2006,27(z1):570-574.(in Chinese)王浩畅,赵铁军.基于SVM的生物医学命名实体的识别[J].哈尔滨工程大学学报,2006,27(z1):570-574.
[7] SHAN Yidong,WANG Hengjun,HUANG He,et al.Study on named entity recognition model based on attention mechanism-taking military text as example[J].Computer Science,2019,46(z1):111-114.(in Chinese)单义栋,王衡军,黄河,等.基于注意力机制的命名实体识别模型研究——以军事文本为例[J].计算机科学,2019,46(z1):111-114.
[8] LAMPLE G,BALLESTEROS M,SUBRAMANIAN S,et al.Neural architectures for named entity recognition[C]//Proceedings of 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Washington D.C.,USA:IEEE Press,2016:260-270.
[9] XU Jingjing,HE Hangfang,SUN Xu,et al.Cross-domain and semi-supervised named entity recognition in Chinese social media:a unified model[J].IEEE/ACM Transactions on Audio,Speech,and Language Processing,2018,26(11):2142-2152.
[10] CHIU J P C,NICHOLS E.Named entity recognition with bidirectional LSTM-CNNs[J].Transactions of the Association for Computational Linguistics,2016,4(1):357-370.
[11] MA X,HOVY E.End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF[C]//Proceedings of the 54th Annual Meeting of the Association for Computa-tional Linguistics.Washington D.C.,USA:IEEE Press,2016:1-12.
[12] ZHONG Jiang,LIU Longhai,LIANG Chuanwei. Active semi-supervised text clustering based on pairwise constraints[J].Computer Engineering,2011,37(13):183-186.(in Chinese)钟将,刘龙海,梁传伟.基于成对约束的主动半监督文本聚类[J].计算机工程,2011,37(13):183-186.
[13] SU Fang,WANG Xiaoyu,ZHANG Zhi.Review summariza-tion generation based on attention mechanism[J].Journal of Beijing University of Posts and Telecommunications,2018,41(3):7-13.(in Chinese)苏放,王晓宇,张治.基于注意力机制的评论摘要生成[J].北京邮电大学学报,2018,41(3):7-13.
[14] WANG Hong,SHI Jinchuan,ZHANG Zhiwei.Text semantic relation extraction of LSTM based on attention mechanism[J].Application Research of Computers,2018,35(5):1417-1420.(in Chinese)王红,史金钏,张志伟.基于注意力机制的LSTM的语义关系抽取[J].计算机应用研究,2018,35(5):1417-1420.
[15] QIN Pengda,XU Weiran,GUO Jun.Designing an adaptive attention mechanism for relation classification[C]//Proceedings of 2017 International Joint Conference on Neural Networks.Washington D.C.,USA:IEEE Press,2017:334-346.
[16] ZHOU Ying,LIU Yue,CAI Jun.Sentiment analysis of micro-blogs based on attention mechanism[J].Information Studies:Theory & Application,2018,41(3):89-94.(in Chinese)周瑛,刘越,蔡俊.基于注意力机制的微博情感分析[J].情报理论与实践,2018,41(3):89-94.
[17] YU F,KOLTUN V.Multi-scale context aggregation by dilated convolutions[EB/OL].[2019-08-10].https://arxiv.org/abs/1511.07122.
[18] YANG Z,HU Z,SALAKHUTDINOV R,et al.Improved variational autoencoders for text modeling using dilated convolutions[C]//Proceedings of the 34th International Conference on Machine Learning-Volume.Washington D.C.,USA:IEEE Press,2017:3881-3890.
[19] GAO Lin,SONG Weidong,TAN Hai,et al.Cloud detection based on multi-scale dilation convolutional neural network for ZY-3 satellite remote sensing imagery[J].Acta Optica Sinica,2019,39(1):299-307.(in Chinese)高琳,宋伟东,谭海,等.多尺度膨胀卷积神经网络资源三号卫星影像云识别[J].光学学报,2019,39(1):299-307.
[20] SUN Jun,HE Xiaofei,TAN Wenjun,et al.Recognition of crop seedling and weed recognition based on dilated convolution and global pooling in CNN[J].Transactions of the Chinese Society of Agricultural Engineering,2018,34(11):159-165.(in Chinese)孙俊,何小飞,谭文军,等.空洞卷积结合全局池化的卷积神经网络识别作物幼苗与杂草[J].农业工程学报,2018,34(11):159-165.
[21] YANG Wenming,CHU Weijie.Named entity recognition of online medical question answering text[J].Computer Systems & Applications,2019,28(2):10-16.(in Chinese)杨文明,褚伟杰.在线医疗问答文本的命名实体识别[J].计算机系统应用,2019,28(2):10-16.

Please choose a citation manager

Content to export