面向多源文本的越南语文本检错方法

doi:10.19678/j.issn.1000-3428.0069492

摘要/Abstract

摘要： 文本检错是自然语言处理的研究方向之一,目标是自动检测输入文本中错误单词的位置和类型,该任务不仅在文本处理的各种下游环节中应用广泛,而且关系到日常生活中方方面面。目前,针对英文、中文的文本检错模型已经能够达到较高的准确率,然而,因越南语语料资源稀缺、人工标注数据不足,面向越南语的文本检错任务深受训练样本匮乏和低质量的困扰。此外,还存在不同场景来源的文本包含错误类型不同,以及错误类型数量不均衡的情况,导致通用文本检错模型无法学习到特定错误类型的检测方法,检错能力较弱。基于上述问题,首先,提出一种面向多源文本的越南语文本检错语料库构建方法,利用越南语光学字符识别(OCR)、越南语语音识别和越南语-英语翻译数据集构建初始语料库,并根据多源越南语检错语料生成方法得到错误语料,通过检错语料自动标注算法获得带标签的训练数据。其次,提出融入多源信息特征的越南语文本检错序列标注模型,在多语言来自变换器的双向编码器表征量(BERT)编码端融入场景特征,使模型能够根据当前输入文本场景适应错误类型。实验结果表明,该方法相比基线模型,F0.5值和F1值提升了1.91和1.80百分点,并进一步验证了模型各组件的必要性以及数据集构建方法的有效性。

关键词: 自然语言处理, 机器学习, 深度学习, 文本检错, 越南语

Abstract: Text error detection is a topic of significant research interest within natural language processing, focusing on automatically identifying the location and type of erroneous words in input text. This task has broad applications across various downstream aspects of text processing and directly impacts daily life. While models for text error detection in English and Chinese have achieved high accuracy, models for Vietnamese face challenges due to a scarcity of corpus resources and manually labeled data. The low quality of training samples hampers the performance of error detection models for Vietnamese. Additionally, the task of error detection for multi-source text introduces complexities, including varied error types across sources and an uneven distribution of error types. Consequently, generalized text error detection models struggle to learn specific error type detection methods, leading to suboptimal performance. To address these challenges, this study proposes a Vietnamese text error detection corpus construction method for multi-source text. The approach leverages datasets from Vietnamese Optical Character Recognition (OCR), Vietnamese speech recognition, and Vietnamese-English translation to create an initial corpus. Using the multi-source Vietnamese error detection corpus generation method, an error corpus is constructed. An error detection corpus automatic labeling algorithm is then employed to generate labeled training data. Furthermore, a Vietnamese text error detection sequence annotation model is introduced, incorporating multi-source information features. By integrating scene features into the multilingual Bidirectional Encoder Representations from Transformers (BERT) encoding layer, the model adapts to specific error types based on the context of the input text. Experimental results demonstrate that the proposed method enhances the F0.5 and F1 values by 1.91 and 1.80 percentage points, respectively, compared to the baseline model. These results validate the necessity of each component of the model as well as the effectiveness of the dataset construction approach.

Key words: natural language processing, machine learning, deep learning, text error detection, Vietnamese

中图分类号:

TP391

庄紫薇, 朱俊国. 面向多源文本的越南语文本检错方法[J]. 计算机工程, 2025, 51(5): 93-102.

ZHUANG Ziwei, ZHU Junguo. Vietnamese Text Error Detection Method for Multi-source Text[J]. Computer Engineering, 2025, 51(5): 93-102.

https://www.ecice06.com/CN/Y2025/V51/I5/93

参考文献

[1] 梁远,祝仰修.现代越南语语法[M].广东:世界图书出版广东有限公司, 2021. LIANG Y, ZHU Y X. Modern Vietnamese grammar[M]. Guangdong:World Book Publishing Guangdong Co., Ltd., 2021.(in Chinese)
[2] FELICE M, YUAN Z. Generating artificial errors for grammatical error correction[C]//Proceedings of the Student Research Workshop at the 14th Conference of the European Chapter of the Association for Computational Linguistics. Stroudsburg, USA:USAACL, 2014:116-126.
[3] ZHAO Z W, WANG H F. MaskGEC:improving neural grammatical error correction via dynamic masking[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(1):1226-1233.
[4] REI M, FELICE M, YUAN Z, et al. Artificial error generation with machine translation and syntactic patterns[C]//Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications. Stroudsburg, USA:USAACL, 2017:287-292.
[5] WANG L H, ZHENG X Q. Improving grammatical error correction models with purpose-built adversarial examples[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, USA:USAACL, 2020:2858-2869.
[6] AL-SABAHI K, YANG K. Supervised copy mechanism for grammatical error correction[J]. IEEE Access, 2023, 11:72374-72383.
[7] 赵国红.中文语法纠错方法的研究综述[J].现代计算机, 2021, 27(28):65-69. ZHAO G H. A survey of researches on Chinese grammar error correction methods[J]. Modern Computer, 2021, 27(28):65-69.(in Chinese)
[8] HAN N R, CHODOROW M, LEACOCK C. Detecting errors in English article usage by non-native speakers[J]. Natural Language Engineering, 2006, 12(2):115-129.
[9] ULLMANN J R. A binary n-gram technique for automatic correction of substitution, deletion, insertion and reversal errors in words[J]. The Computer Journal, 1977, 20(2):141-147.
[10] AFLI H, BARRAULT L, SCHWENK H. OCR error correction using statistical machine translation[J]. International Journal of Computational Linguistics and Applications, 2016, 7(1):175-191.
[11] JUNCZYS-DOWMUNT M, GRUNDKIEWICZ R. The AMU system in the CoNLL-2014 shared task:grammatical error correction by data-intensive and feature-rich statistical machine translation[C]//Proceedings of the 18th Conference on Computational Natural Language Learning:Shared Task. Stroudsburg, USA:USAACL, 2014:25-33.
[12] WANG Y Y, SHIH C. A hybrid approach combining statistical knowledge with conditional random fields for Chinese grammatical error detection[C]//Proceedings of the 5th Workshop on Natural Language Processing Techniques for Educational Applications. Stroudsburg, USA:USAACL, 2018:194-198.
[13] YANG Y, XIE P, TAO J, et al. Alibaba at IJCNLP-2017 task 1:embedding grammatical features into LSTMs for Chinese grammatical error diagnosis task[C]//Proceedings of the International Joint Conference on Natural Language Processing (IJCNLP).[S. l.]:Asian Federation of Natural Language Processing, 2017:41-46.
[14] WANG S L, WANG B X, GONG J F, et al. Combining ResNet and transformer for Chinese grammatical error diagnosis[C]//Proceedings of the 6th Workshop on Natural Language Processing Techniques for Educational Applications.Stroudsburg, USA:USAACL, 2020:36-43.
[15] DEVLIN J, CHANG M W, LEE K, et al. BERT:pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of Association for Computational Linguistics:Human Language Technologies.[S. l.]:Association for Computational Linguistics, 2019:4171-4186.
[16] LI Y H, LIU X B, WANG S, et al. TemplateGEC:improving grammatical error correction with detection template[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA:USAACL, 2023:6878-6892.
[17] MALMI E, KRAUSE S, ROTHE S, et al. Encode, tag, realize:high-precision text editing[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Stroudsburg, USA:USAACL, 2019:5054-5065.
[18] OMELIANCHUK K, ATRASEVYCH V, CHERNODUB A,et al. GECToR-grammatical error correction:tag, not rewrite[C]//Proceedings of the 15th Workshop on Innovative Use of NLP for Building Educational Applications. Stroudsburg, USA:USAACL, 2020:163-170.
[19] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York, USA:ACM Press, 2017:6000-6010.
[20] DO D T, NGUYEN H T, BUI T N, et al. VSEC:transformer-based model for Vietnamese spelling correction[C]//Proceedings of the 18th Pacific Rim International Conference on Artificial Intelligence. Berlin, Germany:Springer, 2021:259-272.
[21] PHAM N L, NGUYEN T H, NGUYEN V V. Grammatical error correction for Vietnamese using machine translation[C]//Proceedings of the 16th International Conference of the Pacific Association for Computational Linguistics. Singapore:Springer Singapore, 2020:505-512.
[22] NGUYEN Q D, PHAN N M, KROMER P. OCR error correction for Vietnamese OCR text with different edit distances[C]//Proceedings of International Conference on Intelligent Networking and Collaborative Systems. Berlin, Germany:Springer, 2022:130-139.
[23] NGO C, TRINH T H, PHAN L, et al. MTet:multi-domain translation for English and Vietnamese[EB/OL].[2024-02-10] . https://arxiv.org/abs/2210.05610v2.
[24] ATIENZA R. Data augmentation for scene text recognition[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). Washington D. C., USA:IEEE Press, 2021:1561-1570.
[25] 计峰.自然语言处理中序列标注模型的研究[D].上海:复旦大学, 2012. JI F. Research on sequence labeling model in natural language processing[D]. Shanghai:Fudan University, 2012.(in Chinese)
[26] RAO G Q, GONG Q, ZHANG B L, et al. Overview of NLPTEA-2018 share task Chinese grammatical error diagnosis[C]//Proceedings of the 5th Workshop on Natural Language Processing Techniques for Educational Applications. Stroudsburg, USA:USAACL, 2018:42-51.
[27] PIRES T, SCHLINGER E, GARRETTE D. How multilingual is multilingual BERT?[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA:USAACL, 2019:4996-5001.
[28] LEVENSHTEIN V I. Binary codes capable of correcting deletions, insertions, and reversals[J]. Soviet Physics Doklady, 1966, 10(8):707-710.
[29] PENNINGTON J, SOCHER R, MANNING C. GloVe:global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, USA:USAACL, 2014:1532-1543.
[30] REI M, YANNAKOUDAKIS H. Compositional sequence labeling models for error detection in learner writing[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA:USAACL, 2016:1181-1191.
[31] CUI L Y, ZHANG Y. Hierarchically-refined label attention network for sequence labeling[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Stroudsburg, USA:USAACL, 2019:4115-4128.
[32] PISLAR M, REI M. Seeing both the forest and the trees:multi-head attention for joint classification on different compositional levels[C]//Proceedings of the 28th International Conference on Computational Linguistics. Barcelona, Spain:International Committee on Computational Linguistics, 2020:3761-3775.
[33] DAHLMEIER D, NG H T, DAHLMEIER D, et al. Better evaluation for grammatical error correction[C]//Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. New York, USA:ACM Press, 2012:568-572.

选择文件类型/文献管理软件名称

选择包含的内容