Vietnamese Event Entity Recognition Combining Dictionary and Adversarial Transfer

doi:10.19678/j.issn.1000-3428.0060466

Abstract

Abstract: The problem of the scarcity of Vietnamese event annotated corpus, comprising several unregistered words, reduces the accuracy of entity recognition.This study proposes an entity recognition model that combines a dictionary and adversarial transfer.It uses Vietnamese as the target language, and English and Chinese as the source languages.Furthermore, the entity tagging information of the source language and bilingual dictionary are used to improve the entity recognition of the target language.The semantic space is shared between the source and target languages by word-level adversarial transfer.Moreover, multi-granular features are embedded into bilingual dictionary to enrich the semantic representation of target language words, and sentence-level adversarial transfer is used to extract language-independent sequence features.Finally, the entity recognition result is marked by a Conditional Random Field(CRF) inference module.The experimental results on the Vietnamese news dataset demonstrate that the proposed model has improved entity recognition compared to the mainstream monolingual entity recognition model and transfer learning model when the source languages are English and Chinese.After adding the target semantic annotation data, the F1-score of the monolingual entity recognition transfer learning model when the source languages are English and Chinese increased by 19.61% and 18.73%, respectively.

Key words: entity recognition, adversarial transfer, bilingual dictionary, multi-granular feature, sequence feature

摘要： 针对越南语事件标注语料稀缺且标注语料中未登陆词过多导致实体识别精度降低的问题，提出一种融合词典与对抗迁移的实体识别模型。将越南语作为目标语言，英语和汉语作为源语言，通过源语言的实体标注信息和双语词典提升目标语言的实体识别效果。采用词级别对抗迁移实现源语言与目标语言的语义空间共享，融合双语词典进行多粒度特征嵌入以丰富目标语言词的语义表征，再使用句子级别对抗迁移提取与语言无关的序列特征，最终通过条件随机场推理模块标注实体识别结果。在越南语新闻数据集上的实验结果表明，在源语言为英语和汉语的情况下，该模型相比主流的单语实体识别模型和迁移学习模型的实体识别性能有明显提升，并且在加入目标语义标注数据后，相比单语实体识别模型的F1值分别增加了19.61和18.73个百分点。

关键词: 实体识别, 对抗迁移, 双语词典, 多粒度特征, 序列特征

CLC Number:

TP391

XUE Zhenyu, XIAN Yantuan, YU Zhengtao, GAO Shengxiang, PU Liuqing. Vietnamese Event Entity Recognition Combining Dictionary and Adversarial Transfer[J]. Computer Engineering, 2022, 48(3): 107-114,145.

薛振宇, 线岩团, 余正涛, 高盛祥, 普浏清. 融合词典与对抗迁移的越南语事件实体识别[J]. 计算机工程, 2022, 48(3): 107-114,145.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0060466

http://www.ecice06.com/EN/Y2022/V48/I3/107

Figures/Tables 5

References

[1] LAMPLE G, BALLESTEROS M, SUBRAMANIAN S, et al.Neural architectures for named entity recognition[C]//Proceedings of 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Stroudsburg, USA:Association for Computational Linguistics, 2016:260-270.
[2] DOAN X D, DANG T T, NGUYEN L M.Effectiveness of character language model for Vietnamese named entity recognition[C]//Proceedings of the 32nd Pacific Asia Conference on Language, Information and Computation.Stroudsburg, USA:Association for Computational Linguistics, 2018:157-163.
[3] YANG Z, SALAKHUTDINOV R, COHEN W.Multi-task cross-lingual sequence tagging from scratch[EB/OL].[2020-12-28].https://arxiv.org/pdf/1603.06270.pdf.
[4] LIN Y, YANG S Q, STOYANOV V, et al.A multi-lingual multi-task architecture for low-resource sequence labeling[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.Stroudsburg, USA:Association for Computational Linguistics, 2018:799-809.
[5] FANG M, COHN T.Learning when to trust distant supervision:an application to low-resource POS tagging using cross-lingual projection[C]//Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning.Stroudsburg, USA:Association for Computational Linguistics, 2016:178-186.
[6] WANG D, PENG N, DUH K.A multi-task learning approach to adapting bilingual word embeddings for cross-lingual named entity recognition[C]//Proceedings of the 8th International Joint Conference on Natural Language Processing.Stroudsburg, USA:Association for Computational Linguistics, 2017:383-388.
[7] SHI G, FENG C, HUANG L F, et al.Genre separation network with adversarial training for cross-genre relation extraction[C]//Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing.Stroudsburg, USA:Association for Computational Linguistics, 2018:1018-1023.
[8] CONNEAU A, LAMPLE G, RANZATO M A, et al.Word translation without parallel data[EB/OL].[2020-12-28].http://arxiv.org/abs/1710.04087.
[9] FANG M, COHN T.Model transfer for tagging low-resource languages using a bilingual dictionary[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Stroudsburg, USA:Association for Computational Linguistics, 2017:587-593.
[10] ZIRIKLY A.Cross-lingual transfer of named entity recognizers without parallel corpora[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing.Stroudsburg, USA:Association for Computational Linguistics, 2015:390-396.
[11] XIE J T, YANG Z L, NEUBIG G, et al.Neural cross-lingual named entity recognition with minimal resources[C]//Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing.Stroudsburg, USA:Association for Computational Linguistics, 2018:369-379.
[12] HUANG L F, JI H, MAY J.Cross-lingual multi-level adversarial transfer to enhance low-resource name tagging[C]//Proceedings of 2019 Conference of the North.Stroudsburg, USA:Association for Computational Linguistics, 2019:3823-3833.
[13] ZHANG M, LIU Y, LUAN H B, et al.Adversarial training for unsupervised bilingual lexicon induction[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Stroudsburg, USA:Association for Computational Linguistics, 2017:1959-1970.
[14] CAO P F, CHEN Y B, LIU K, et al.Adversarial transfer learning for Chinese named entity recognition with self-attention mechanism[C]//Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing.Stroudsburg, USA:Association for Computational Linguistics, 2018:182-192.
[15] XING C, WANG D, LIU C, et al.Normalized word embedding and orthogonal transform for bilingual word translation[C]//Proceedings of 2015 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Stroudsburg, USA:Association for Computational Linguistics, 2015:1006-1011.
[16] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al.Generative adversarial nets[C]//Proceedings of the 27th Annual Conference on Neural Information Processing Systems.New York, USA:ACM Press, 2014:2672-2680.
[17] SUKHBAATAR S, WESTON J, FERGUS R.End-to-end memory networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems.New York, USA:ACM Press, 2015:2440-2448.
[18] YIN R C, WANG Q, LI P, et al.Multi-granularity Chinese word embedding[C]//Proceedings of 2016 Conference on Empirical Methods in Natural Language Processing.Stroudsburg, USA:Association for Computational Linguistics, 2016:981-986.
[19] 张应成, 杨洋, 蒋瑞, 等.基于BiLSTM-CRF的商情实体识别模型[J].计算机工程, 2019, 45(5):308-314. ZHANG Y C, YANG Y, JIANG R, et al.Commercial intelligence entity recognition model based on BiLSTM-CRF[J].Computer Engineering, 2019, 45(5):308-314.(in Chinese)
[20] 何阳宇, 晏雷, 易绵竹, 等.融合CRF与规则的老挝语军事领域命名实体识别方法[J].计算机工程, 2020, 46(8):297-304. HE Y Y, YAN L, YI M Z, et al.Named entitiy recognition method for Laotian in military field combining CRF and rules[J].Computer Engineering, 2020, 46(8):297-304.(in Chinese)
[21] 买买提阿依甫, 吾守尔·斯拉木, 帕丽旦·木合塔尔, 等.基于BiLSTM-CNN-CRF模型的维吾尔文命名实体识别[J].计算机工程, 2018, 44(8):230-236. Maimaitiayifu, Silamu Wushouer, Muhetaer Palidan, et al.Uyghur named entity recognition based on BiLSTM-CNN-CRF model[J].Computer Engineering, 2018, 44(8):230-236.(in Chinese)
[22] JOULIN A, GRAVE E, BOJANOWSKI P, et al.Bag of tricks for efficient text classification[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics.Stroudsburg, USA:Association for Computational Linguistics, 2017:427-431.

Please choose a citation manager

Content to export