AMR文本生成的数据扩充方法

doi:10.19678/j.issn.1000-3428.0061106

摘要/Abstract

摘要： 在抽象语义表示（AMR）文本生成过程中，AMR图到文本形式的转换在很大程度上受语料规模的影响。提出一种简单有效的动态数据扩充方法，在已标注数据集规模有限的情况下提高AMR文本生成性能。将AMR文本生成模型解码端视作一个语言模型，使用单词级别的扩充方法，通过动态地对目标端单词进行随机替换，得到带噪声的数据，从而增强模型的泛化能力。在加载数据时，随机选择目标句子中的部分单词做噪声化处理，利用约束编码器预测被覆盖的单词并还原出原始语句，使模型具有更深层的语言表征能力。基于AMR2.0和AMR3.0英文标准数据集进行实验，结果表明，该方法可有效提升AMR文本生成系统性能，与未引入噪声的基准Transformer模型相比，能够获得更优的BLEU、Meteor和chrF++指标，其中BLEU值在人工标注语料场景下分别提升0.68和0.64，且在大规模自动标注语料场景下也能提升0.60和0.68。

关键词: 抽象语义表示, 语料规模, AMR文本生成, 动态数据扩充, 噪声

Abstract: In the process of Abstract Meaning Representation(AMR)-to-text generation, the transformation from AMR graph to text is largely affected by the size of the corpus.A simple and effective dynamic data augmentation method is proposed to improve the performance of AMR-to-text generation for limited scale labeled datasets.The decoding end of the AMR-to-text generation model is regarded as a language model, and the word level augmentation method is used to dynamically replace the words at the target end to obtain noisy data to enhance the generalization ability of the model.When loading data, some words in the target sentence are randomly selected as noise.The covered words are predicted by the constraint encoder, and the original sentences are restored to ensure that the model acquires deeper language representation ability.Experimental results show that, the method proposed in this study has achieved significant improvement on the AMR English standard dataset.Experiment is based on AMR2.0 and AMR3.0 English standard dataset.The results show that this method can effectively improve the performance of AMR-to-text generation system.Compared with the benchmark Transformer model without noise, it can obtain better BLEU, Meteor and chrF + + indicators.Among them, BLEU value can be improved by 0.68 and 0.64 respectively in the scene of manually labeling corpus, while it can still be improved by 0.60 and 0.68 in the scene of large-scale automatic labeling corpus.

Key words: Abstract Meaning Representation(AMR), corpus size, AMR-to-text generation, dynamic data augmentation, noise

中图分类号:

TP18

付叶蔷, 李军辉. AMR文本生成的数据扩充方法[J]. 计算机工程, 2022, 48(5): 91-97.

FU Yeqiang, LI Junhui. Data Augmentation Method for AMR-to-Text Generation[J]. Computer Engineering, 2022, 48(5): 91-97.

http://www.ecice06.com/CN/Y2022/V48/I5/91

图/表 8

20220723173345

20220723173349

20220723173352

20220723173356

20220723173400

20220723173404

20220723173407

20220723173411

参考文献

[1] BANARESCU L, BONIAL C, CAI S, et al.Abstract meaning representation for sembanking[C]//Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse.Washington D.C., USA:IEEE Press, 2013:178-186.
[2] XU D, LI J, ZHU M, et al.Improving AMR parsing with squence-to-squence pre-training[C]//Proceeding of 2020 Conference on Empirical Methods in Natural Language Processing.Stroudsburg, USA:Association for Computational Linguistics, 2020:2501-2511.
[3] TAMCHYNA A, QUIRK C, GALLEY M.A discriminative model for semantics-to-string translation[C]//Proceedings of the 1st Workshop on Semantics-Driven Statistical Machine Translation.Stroudsburg, USA:Association for Computational Linguistics, 2015:30-36.
[4] MITRA A, BARAL C.Addressing a question answering challenge by combining statistical methods with inductive rule learning and reasoning[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence.Palo Alto, USA:AAAI Press, 2016:2779-2785.
[5] FLANIGAN J, DYER C, SMITH N A, et al.Generation from abstract meaning representation using tree transducers[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Stroudsburg, USA:Association for Computational Linguistics, 2016:731-739.
[6] SONG L F, PENG X C, ZHANG Y, et al.AMR-to-text generation with synchronous node replacement grammar[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2:Short Papers).Stroudsburg, USA:Association for Computational Linguistics, 2017:7-13.
[7] POURDAMGHANI N, KNIGHT K, HERMJAKOB U.Generating English from abstract meaning representations[C]//Proceedings of the 9th International Natural Language Generation Conference.Stroudsburg, USA:Association for Computational Linguistics, 2016:21.
[8] KONSTAS I, IYER S, YATSKAR M, et al.Neural AMR:sequence-to-sequence models for parsing and generation[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).Stroudsburg, USA:Association for Computational Linguistics, 2017:146-157.
[9] CAO K, CLARK S.Factorising AMR generation through syntax[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, Volume 1(Long and Short Papers).Stroudsburg, USA:Association for Computational Linguistics, 2019:2157-2163.
[10] ZHU J, LI J H, ZHU M H, et al.Modeling graph structure in transformer for better AMR-to-text generation[C]//Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing.Stroudsburg, USA:Association for Computational Linguistics, 2019:5462-5471.
[11] SONG L F, WANG A T, SU J S, et al.Structural information preserving for graph-to-text generation[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.Stroudsburg, USA:Association for Computational Linguistics, 2020:7987-7998.
[12] WANG T, WAN X, YAO S.Better AMR-To-Text generation with graph structure reconstruction[C]//Proceedings of the 29th International Joint Conference on Artificial Intelligence.Washington D.C., USA:IEEE Press, 2020:3919-3925.
[13] JIN L S, GILDEA D.Generalized shortest-paths encoders for AMR-to-text generation[C]//Proceedings of the 28th International Conference on Computational Linguistics.Stroudsburg, USA:International Committee on Computational Linguistics, 2020:2004-2013.
[14] FAN A, GARDENT C.Multilingual AMR-to-text generation[C]//Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing.Stroudsburg, USA:Association for Computational Linguistics, 2020:2889-2901.
[15] BECK D, HAFFARI G, COHN T.Graph-to-sequence learning using gated graph neural networks[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.Stroudsburg, USA:Association for Computational Linguistics, 2018:273-283.
[16] DAMONTE M, COHEN S B.Structural neural encoders for AMR-to-text generation[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, Volume 1(Long and Short Papers).Stroudsburg, USA:Association for Computational Linguistics, 2019:3649-3658.
[17] ZHAO Y B, CHEN L, CHEN Z, et al.Line graph enhanced AMR-to-text generation with mix-order graph attention networks[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.Stroudsburg, USA:Association for Computational Linguistics, 2020:732-741.
[18] VASWANI A, SHAZEER N, PARMAR N, et al.Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.Berlin, Germany:Springer, 2017:6000-6010.
[19] SENNRICH R, HADDOW B, BIRCH A.Improving neural machine translation models with monolingual data[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).Stroudsburg, USA:Association for Computational Linguistics, 2016:7-12.
[20] ARTETXE M, LABAKA G, AGIRRE E, et al.Unsupervised neural machine translation[C]//Proceedings of the 6th International Conference on Learning Representations.Vancouver, Canada:ICLR, 2018:1-5.
[21] FADAEE M, BISAZZA A, MONZ C.Data augmentation for low-resource neural machine translation[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2:Short Papers).Stroudsburg, USA:Association for Computational Linguistics, 2017:567-573.
[22] GAO F, ZHU J H, WU L J, et al.Soft contextual data augmentation for neural machine translation[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.Stroudsburg, USA:Association for Computational Linguistics, 2019:5539-5544.
[23] DEVLIN J, CHANG M W, LEE K, et al.BERT:pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, Volume 1(Long and Short Papers).Stroudsburg, USA:Association for Computational Linguistics, 2019:4171-4186.
[24] SENNRICH R, HADDOW B, BIRCH A.Neural machine translation of rare words with subword units[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).Stroudsburg, USA:Association for Computational Linguistics, 2016:1715-1725.
[25] GE D L, LI J H, ZHU M H, et al.Modeling source syntax and semantics for neural AMR parsing[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence.Stockholm, Sweden:IJCAI, 2019:4975-4981.
[26] VINCENT P, LAROCHELLE H, LAJOIE I, et al.Stacked denoising autoencoders:learning useful representations in a deep network with a local denoising criterion[J].Journal of Machine Learning Research, 2010, 11:3371-3408.
[27] 徐东钦, 李军辉, 朱慕华, 等.基于多任务预训练的AMR文本生成研究[J].软件学报, 2021, 32(10):3036-3050. XU D Q, LI J H, ZHU M H, et al.Improving AMR-to-text generation with multi-task pre-training[J].Journal of Software, 2021, 32(10):3036-3050.(in Chinese)
[28] PAPINENI K, ROUKOS S, WARD T, et al.BLEU:a method for automatic evaluation of machine translation[C]//Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Stroudsburg, USA:Association for Computational Linguistics, 2002:311-318.
[29] BANERJEE S, LAVIE A.METEOR:an automatic metric for MT evaluation with improved correlation with human judgments[C]//Proceedings of 2005 ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization.Stroudsburg, USA:Association for Computational Linguistics, 2005:65-72.
[30] POPOVIĆ M.chrF++:words helping character n-grams[C]//Proceedings of the 2nd Conference on Machine Translation.Stroudsburg, USA:Association for Computational Linguistics, 2017:612-618.
[31] KINGMA D P, BA J.Adam:a method for stochastic optimization[EB/OL].[2020-10-10].https://arxiv.org/abs/1412.6980.
[32] CHENG Y, TU Z P, MENG F D, et al.Towards robust neural machine translation[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).Stroudsburg, USA:Association for Computational Linguistics, 2018:1756-1766.

选择文件类型/文献管理软件名称

选择包含的内容