结合向量化方法与掩码机制的术语干预翻译模型

doi:10.19678/j.issn.1000-3428.0066031

摘要/Abstract

摘要：

术语干预神经机器翻译模型通常借助人为给定的术语翻译来改变译文，从而改善翻译质量。向量化干预方法为术语干预任务提供了新的范式，但仅考虑将术语与句子信息以向量的形式融合，没有关注术语信息对术语翻译效果的影响。为此，构建一种结合向量化方法与掩码机制的术语干预机器翻译模型，将人为给定的源端术语与目标端术语编码为特征向量，显式地融入机器翻译模型的编码器、解码器以及输出层。在训练阶段，借助掩码机制屏蔽注意力机制中源端术语对应的关键字，增强模型编码器与解码器对术语特征向量的关注。在推理阶段，利用掩码机制优化术语干预输出层的概率分布，进一步提高术语字符的翻译准确率。在WMT2014德英和WMT2021英中数据集上的实验结果表明，相较于基于原始向量化方法的Code-Switching机器翻译模型，所提模型的术语翻译准确率分别提升了9.27和2.95个百分点，并且能大幅度提升长术语的翻译准确率。

关键词: 机器翻译, 术语干预, 向量化, 注意力机制, 掩码机制

Abstract:

The terminology intervention Neural Machine Translation(NMT) model optimizes translations with the help of human-provided translations; this improves the translation quality. Recently, vectorization methods have emerged to provide a new paradigm for terminology intervention tasks; however, these methods consider only fusing terminology information with sentence information and neglect the low contribution of terminology vectors to terminology translation. To address these issues, a terminology intervention machine translation model combining the vectorization method and mask mechanism is built. This model encodes human-provided source terminology and target terminology into feature vectors and integrates them into the encoder, decoder, and output layers of the machine translation model. To enhance its attention to term feature vectors, the model uses a mask mechanism to mask the keys corresponding to the source-side terminologies in the attention mechanism during the training phase. In the inference phase, the probability distribution of the output layer is optimized to improve terminology generation. The experimental results on the WMT 2014 German-English and WMT2021 English-Chinese datasets show that, compared with the Code-Switching machine translation model based on the original vectorization method, the proposed model has improved the terminology translation accuracy by 9.27 and 2.95 percentage points, respectively, and can significantly improve the translation accuracy of long-terms.

Key words: machine translation, terminology intervention, vectorization, attention mechanism, mask mechanism

张金鹏, 段湘煜. 结合向量化方法与掩码机制的术语干预翻译模型[J]. 计算机工程, 2023, 49(11): 70-76, 84.

Jinpeng ZHANG, Xiangyu DUAN. Terminology Intervention Translation Model Combining Vectorization Method and Mask Mechanism[J]. Computer Engineering, 2023, 49(11): 70-76, 84.

https://www.ecice06.com/CN/Y2023/V49/I11/70

图/表 7

图1 基于向量化方法的术语干预机器翻译模型

Fig.1 Terminology intervention machine translation model based on vectorization method

图2 结合掩码机制与编码器的自注意力机制

Fig.2 Self-attention mechanism combining mask mechanism and encoder

图3 结合掩码机制的编码器-解码器注意力机制

Fig.3 Attention mechanism of encoder-decoder combining mask mechanism

参考文献 26

1	冯洋, 邵晨泽. 神经机器翻译前沿综述. 中文信息学报, 2020, 34 (7): 1- 18. URL
	FENG Y, SHAO C Z. Frontiers in neural machine translation: a literature review. Journal of Chinese Information Processing, 2020, 34 (7): 1- 18. URL
2	JIA H, GU S, SHI Y, et al. Bilingual terminology extraction from non-parallel e-commerce corpora[EB/OL]. [2022-09-16]. https://arxiv.org/abs/2203.05880.
3	张知行, 张佳影, 高大启, 等. 临床检验指标术语库的构建与病历挖掘应用. 中文信息学报, 2020, 34 (12): 100- 110. URL
	ZHANG Z X, ZHANG J Y, GAO D Q, et al. Construction of clinic indicator terminology base and its application in medical record mining. Journal of Chinese Information Processing, 2020, 34 (12): 100- 110. URL
4	游新冬, 杨海翔, 陈海涛, 等. 融合术语信息的新能源专利机器翻译研究. 中文信息学报, 2021, 35 (12): 76-83, 93. URL
	YOU X D, YANG H X, CHEN H T, et al. Research on new energy patent machine translation integrating terminology information. Journal of Chinese Information Processing, 2021, 35 (12): 76-83, 93. URL
5	KOEHN P, ZENS R, DYER C, et al. Moses: open source toolkit for statistical machine translation[C]//Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions. Stroudsburg, USA: Association for Computational Linguistics, 2007: 177-180.
6	张泽锋, 毛存礼, 余正涛, 等. 融入领域术语词典的司法舆情敏感信息识别. 中文信息学报, 2022, 36 (9): 76-83, 92. URL
	ZHANG Z F, MAO C L, YU Z T, et al. Identification of sensitive information of judicial public opinion incorporated into domain terminology dictionary. Journal of Chinese Information Processing, 2022, 36 (9): 76-83, 92. URL
7	WANG S, TAN Z X, LIU Y. Integrating vectorized lexical constraints for neural machine translation[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2022: 7063-7073.
8	HOKAMP C, LIU Q. Lexically constrained decoding for sequence generation using grid beam search[EB/OL]. [2022-09-16]. https://arxiv.org/abs/1704.07138.
9	POST M, VILAR D. Fast lexically constrained decoding with dynamic beam allocation for neural machine translation[EB/OL]. [2022-09-16]. https://arxiv.org/abs/1804.06609.
10	HU J E, KHAYRALLAH H, CULKIN R, et al. Improved lexically constrained decoding for translation and monolingual rewriting[C]//Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, USA: Association for Computational Linguistics, 2019: 839-850.
11	SONG K, ZHANG Y, YU H, et al. Code-switching for enhancing NMT with pre-specified translation[C]//Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, USA: Association for Computational Linguistics, 2019: 449-459.
12	DINU G, MATHUR P, FEDERICO M, et al. Training neural machine translation to apply terminology constraints[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2019: 3063-3068.
13	WANG K, GU S, CHEN B, et al. TermMind: Alibaba's WMT21 machine translation using terminologies task submission[C]//Proceedings of the 6th Conference on Machine Translation. Stroudsburg, USA: Association for Computational Linguistics, 2021: 851-856.
14	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need?[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2017: 6000-6010.
15	DU C, TU Z, JIANG J. Order-agnostic cross entropy for non-autoregressive machine translation[C]//Proceedings of International Conference on Machine Learning. New York, USA: ACM Press, 2021: 2849-2859.
16	VOITA E, SENNRICH R, TITOV I. The bottom-up evolution of representations in the Transformer: a study with machine translation and language modeling objectives[C]//Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2019: 4396-4406.
17	PASCUAL D, EGRESSY B, MEISTER C, et al. A plug-and-play method for controlled text generation[C]//Proceedings of EMNLP'21. Stroudsburg, USA: Association for Computational Linguistics, 2021: 3973-3997.
18	DYER C, CHAHUNEAU V, SMITH N A. A simple, fast, and effective reparameterization of IBM Model 2[C]//Proceedings of 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, USA: Association for Computational Linguistics, 2013: 644-648.
19	CHEN G H, CHEN Y, LI V O K. Lexically constrained neural machine translation with explicit alignment guidance[C]//Proceedings of AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2021: 12630-12638.
20	SENNRICH R, HADDOW B, BIRCH A. Neural machine translation of rare words with subword units[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2016: 1715-1725.
21	OTT M, EDUNOV S, BAEVSKI A, et al. Fairseq: a fast, extensible toolkit for sequence modeling[C]//Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2016: 1715-1725.
22	DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional Transformers for language understanding[C]//Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, USA: Association for Computational Linguistics, 2019: 4171-4186.
23	POST M. A call for clarity in reporting BLEU scores[C]//Proceedings of the 3rd Conference on Machine Translation. Stroudsburg, USA: Association for Computational Linguistics, 2018: 186-191.
24	ALAM M M I, ANASTASOPOULOS A, BESACIER L, et al. On the evaluation of machine translation for terminology consistency[EB/OL]. [2022-09-16]. https://arxiv.org/abs/2106.11891.
25	MATTHEW S, BONNIE D, RICH S, et al. A study of translation edit rate with targeted human annotation[C]//Proceedings of the 7th Conference of the Association for Machine Translation in the Americas. Cambridge, USA: Association for Machine Translation in the Americas, 2006: 223-231.
26	董兴华, 陈丽娟, 周喜, 等. 汉维统计机器翻译中的形态学处理. 计算机工程, 2011, 37 (12): 150- 152. URL
	DONG X H, CHEN L J, ZHOU X, et al. Morphology processing in Chinese-Uyghur statistical machine translation. Computer Engineering, 2011, 37 (12): 150- 152. URL

[1]	李俊俊, 董建刚, 李坤. 基于Kubernetes的集群节能策略研究[J]. 计算机工程, 2024, 50(9): 82-91.
[2]	林畅, 郭伟, 任哲聪, 金海波. 基于Transformer的目标跟踪与分割统一算法[J]. 计算机工程, 2024, 50(9): 130-141.
[3]	李泽霖, 吕兆峰, 陈富强, 李克. 基于多跳信息融合的实体对齐模型[J]. 计算机工程, 2024, 50(9): 142-152.
[4]	王汝英, 马嘉骏, 董建强, 刘万龙, 张海涛, 尹凯, 赵博超. 基于MTS-BiGRU-DMHSA的工业负荷预测方法[J]. 计算机工程, 2024, 50(9): 169-178.
[5]	朱凯, 李理, 张彤, 江晟, 别一鸣. 基于Transformer的多阶段运动模糊图像修复网络[J]. 计算机工程, 2024, 50(9): 276-285.
[6]	张天鹏, 韩晶, 吕学强. 基于多任务学习的超分辨率辅助小目标检测[J]. 计算机工程, 2024, 50(9): 304-312.
[7]	郭敏, 张熙涵, 李阳. 融合注意力的教师互一致性半监督医学图像分割[J]. 计算机工程, 2024, 50(9): 313-323.
[8]	曾钰琦, 刘博, 钟柏昌, 钟瑾. 智慧教育下基于改进YOLOv8的学生课堂行为检测算法[J]. 计算机工程, 2024, 50(9): 344-355.
[9]	饶日昕, 王怡文, 曾砺志, 童心恬, 赵海涛. 面向废旧电缆检测的轻量化网络模型[J]. 计算机工程, 2024, 50(8): 22-30.
[10]	李华昱, 张智康, 闫阳, 岳阳. 基于知识图谱增强的领域多模态实体识别[J]. 计算机工程, 2024, 50(8): 31-39.
[11]	王蕾, 党时鹏, 潘丰. 基于卷积神经网络的隐匿性旁路预测模型[J]. 计算机工程, 2024, 50(8): 40-49.
[12]	陈瀚, 赵春蕾, 蒋昊达, 王春东. 基于融合模型与语义网络的App用户意图识别研究[J]. 计算机工程, 2024, 50(8): 50-63.
[13]	王夙喆, 张雪英, 陈晓玉, 李凤莲, 吴泽林. 基于有效注意力和GAN结合的脑卒中EEG增强算法[J]. 计算机工程, 2024, 50(8): 336-344.
[14]	王宇, 祁琦, 王纯, 许才. 储能变流器信号高精度故障诊断方法[J]. 计算机工程, 2024, 50(8): 389-396.
[15]	王炼红, 林飞鹏, 李潇瑶, 谌桂枝, 周莉. 融入课程知识图谱的KMAKT预测[J]. 计算机工程, 2024, 50(7): 23-31.

选择文件类型/文献管理软件名称

选择包含的内容