Research on Proper Noun-Enhanced Method for Paraphrase Generation

doi:10.19678/j.issn.1000-3428.0067010

Abstract

Abstract:

Existing Chinese paraphrase generation models often lose proper nouns in the original sentence when generating paraphrased sentences, which results in semantic deviation and reduces the usability of the paraphrased sentence, decreasing performance on downstream tasks. To solve these problems, this study proposes proper noun-enhanced method for paraphrase generation. Specifically, a placeholder-based paraphrase generation model is proposed for an original sentence containing a single proper noun. The model retains the placeholder by replacing the proper noun in the training sentence pair with a placeholder, to train the moeel's ability to retain placeholder. A lexically constrained paraphrase generation model is proposed for an original sentence containing multiple proper nouns. By concatenating and distinguishing the list of proper nouns from the original sentence, the model is trained to recognize and reproduce multiple proper nouns, improving the appropriate noun retention rate in the paraphrased sentences. In addition, a reference-free metric is proposed to evaluate the quality of the generated paraphrased sentences by considering both semantic consistency and expression diversity. This study considers the intent recognition cold-start task in a real dialogue business system as the downstream task. By comparing the quality of the paraphrased sentences generated by different models and the accuracy of the intent recognition task, the experimental results show that the lexical-constrained paraphrase generation model can generate a high-quality paraphrase corpus, and the related model has the highest accuracy rate. Compared to the paraphrase model which does not consider proper nouns, the accuracy is increased by 5.38%.

Key words: paraphrase generation, semantic deviation, placeholder, lexical-constraint, intention recognition

摘要：

现有的中文复述生成模型在对包含专有名词的原句生成复述句时经常丢失原句中的专有名词，造成复述句的语义偏移，降低复述句的可用性，进而影响其在下游任务中的应用效果。为了解决这类问题，提出专有名词增强的复述生成方法。针对包含单个专有名词的原句构建基于占位符的复述生成模型，通过将训练句对中的专有名词用占位符替换，训练模型对占位符的保留能力；针对包含多个专有名词的原句构建词汇约束的复述生成模型，通过将专有名词列表与原句拼接并进行区分，训练模型对多个专有名词的识别和复制能力，提高复述句对专有名词的保留率。此外，综合考虑语义一致性和表达多样性，提出参考句无关的复述句质量评价指标用来评估生成复述句的质量。以真实对话系统业务中的意图识别冷启动任务为下游任务，对比不同模型生成复述句的质量以及在意图识别任务上的准确率。实验结果表明，词汇约束的复述生成模型能够生成与原句语义一致且表达具有多样性的高质量复述语料，对应语料训练得到的意图识别模型准确率最高，相较于未考虑专有名词的复述模型，意图识别模型的准确率提高了5.38%。

关键词: 复述生成, 语义偏移, 占位符, 词汇约束, 意图识别

Xue ZHANG, Yufeng CHEN, Jin'an XU, Fengzhan TIAN. Research on Proper Noun-Enhanced Method for Paraphrase Generation[J]. Computer Engineering, 2024, 50(3): 98-105.

张雪, 陈钰枫, 徐金安, 田凤占. 专有名词增强的复述生成方法研究[J]. 计算机工程, 2024, 50(3): 98-105.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0067010

https://www.ecice06.com/EN/Y2024/V50/I3/98

Figures/Tables 10

Fig.1 Structure of BART model

Fig.2 Training data of PBPG model

Fig.3 Input of LCPG model encoder

Fig.4 Function image of

$ \text{d}\text{i}\text{v}\left({x}, {p}\right) $

Fig.5 The quality changes of paraphrases under different sampling strategies in the LCPG model

Fig.6 Accuracy of LCPG model in intention recognition under different parameters

References 29

1	MCKEOWN K R. Paraphrasing using given and new information in a question-answer system[C]//Proceedings of the 17th Annual Meeting on Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 1979: 67-72.
2	JOLLY S, FALKE T, TIRKAZ C, et al. Data-efficient paraphrase generation to bootstrap intent classification and slot labeling for new features in task-oriented dialog systems[C]//Proceedings of the 28th International Conference on Computational Linguistics: Industry Track. Stroudsburg, USA: International Committee on Computational Linguistics, 2020: 10-20.
3	IORDANSKAJA L, KITTREDGE R, POLGUÈRE A. Lexical selection and paraphrase in a meaning-text generation model[M]//PARIS C L, SWARTOUT W R, MANN W C. Natural language generation in artificial intelligence and computational linguistics. Berlin, Germany: Springer, 1991: 293-312.
4	CALLISON-BURCH C, KOEHN P, OSBORNE M. Improved statistical machine translation using paraphrases[C]//Proceedings of Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2006: 17-24.
5	鲍宇, 黄书剑, 周浩, 等. 基于句法模板采样的无监督复述生成方法. 中国科学: 信息科学, 2022, 52 (10): 1808- 1821. URL
	BAO Y, HUANG S J, ZHOU H, et al. Unsupervised retelling generation method based on syntactic template sampling. Scientia Sinica(Informationis), 2022, 52 (10): 1808- 1821. URL
6	IYYER M, WIETING J, GIMPEL K, et al. Adversarial example generation with syntactically con-trolled paraphrase networks[C]//Proceedings of 2018 Conference of the North American Chapter of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2018: 1875-1885.
7	RADFORD A, NARASIMHAN K, SALIMANS T, et al. Improving language understanding by generative pre-training[EB/OL]. [2022-09-10]. https://www.semanticscholar.org/paper/Improving-Language-Understanding-by-Generative-Radford-Narasimhan/cd18800a0fe0b668a1cc19f2ec95b5003d0a5035.
8	LEWIS M, LIU Y H, GOYAL N, et al. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2020: 7871-7880.
9	ZHOU J N, BHAT S. Paraphrase generation: a survey of the state of the art[C]//Proceedings of 2021 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2021: 5075-5086.
10	LIN Z, WAN X J. Pushing paraphrase away from original sentence: a multi-round paraphrase generation approach[C]//Proceedings of ACL-IJCNLP 2021. Stroudsburg, USA: Association for Computational Linguistics, 2021: 1548-1557.
11	侯志荣, 范晓东, 张华, 等. J-SGPGN: 基于序列与图的联合学习复述生成网络. 计算机应用, 2023, 43 (5): 1365- 1371. URL
	HOU Z R, FAN X D, ZHANG H, et al. J-SGPGN: paraphrase generation networks based on joint learning of sequence and graph. Journal of Computer Applications, 2023, 43 (5): 1365- 1371. URL
12	SU Y X, VANDYKE D, BAKER S, et al. Keep the primary, rewrite the secondary: a two-stage approach for paraphrase generation[C]//Proceedings of ACL-IJCNLP 2021. Stroudsburg, USA: Association for Computational Linguistics, 2021: 560-569.
13	YANG H R, LAM W, LI P J. Contrastive representation learning for exemplar-guided paraphrase generation[C]//Proceedings of EMNLP 2021. Stroudsburg, USA: Association for Computational Linguistics, 2021: 4754-4761.
14	SUN J A, MA X Z, PENG N Y. AESOP: paraphrase generation with adaptive syntactic control[C]//Proceedings of 2021 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2021: 5176-5189.
15	YANG E G, BAI C L, XIONG D Y, et al. Learning structural information for syntax-controlled paraphrase generation[C]//Proceedings of NAACL 2022. Stroudsburg, USA: Association for Computational Linguistics, 2022: 2079-2090.
16	YANG E G, LIU M T, XIONG D Y, et al. Syntactically-informed unsupervised paraphrasing with non-parallel data[C]//Proceedings of 2021 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2021: 2594-2604.
17	HUANG K H, CHANG K W. Generating syntactically controlled paraphrases without using annotated parallel pairs[C]//Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2021: 1022-1033.
18	张琳, 刘明童, 张玉洁, 等. 探索低资源的迭代式复述生成增强方法. 智能系统学报, 2022, 17 (4): 680- 687. URL
	ZHANG L, LIU M T, ZHANG Y J, et al. Explore the low-resource iterative paraphrase generation enhancement method. CAAI Transactions on Intelligent Systems, 2022, 17 (4): 680- 687. URL
19	BANDEL E, AHARONOV R, SHMUELI-SCHEUER M, et al. Quality controlled paraphrase generation[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2022: 596-609.
20	WESTON J, LENAIN R, MEEPEGAMA U, et al. Generative pretraining for paraphrase evaluation[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2022: 4052-4073.
21	PAPINENI K, ROUKOS S, WARD T, et al. BLEU: a method for automatic evaluation of machine translation[C]//Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2022: 311-318.
22	LIN C Y. ROUGE: a package for automatic evaluation of summaries[EB/OL]. [2022-09-10]. https://www.docin.com/p-1736180490.html.
23	LAVIE A, AGARWAL A. METEOR: an automatic metric for MT evaluation with high levels of correlation with human judgments[C]//Proceedings of StatMT'07. Stroudsburg, USA: Association for Computational Linguistics, 2007: 228-231.
24	POPOVIĆ M. chrF++: words helping character n-grams[C]//Proceedings of the 2nd Conference on Machine Translation. Stroudsburg, USA: Association for Computational Linguistics, 2017: 612-618.
25	ZHANG T Y, KISHORE V, WU F, et al. BERTScore: evaluating text generation with BERT[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1904.09675.pdf.
26	ZHAO W, PEYRARD M, LIU F, et al. MoverScore: text generation evaluating with contextualized embeddings and earth mover distance[C]//Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2019: 563-578.
27	SELLAM T, DAS D, PARIKH A. BLEURT: learning robust metrics for text generation[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2020: 7881-7892.
28	安波. 大规模多粒度中文复述语料库. 情报工程, 2022, 8 (2): 19- 33. URL
	AN B. A large scale multi-granularity Chinese paraphrase corpus. Technology Intelligence Engineering, 2022, 8 (2): 19- 33. URL
29	KIM Y. Convolutional neural networks for sentence classification[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2014: 1746-1751.

[1]	ROU Te,CAI Rangjia,SUN Maosong. Tibetan Paraphrase Sentence Generation Method Based on Word Order Transformation [J]. Computer Engineering, 2018, 44(4): 231-235.
[2]	LIU Feng, QIN Wei, ZHOU Yong-Hui, GUO Xi-Qi. Automatic Generation Method of Word Report Based on Dynamic Web Page [J]. Computer Engineering, 2012, 38(5): 279-280,284.

Please choose a citation manager

Content to export