Cross-Lingual Sentence Summarization System Based on Contrastive Attention Mechanism

doi:10.19678/j.issn.1000-3428.0054793

Abstract

Abstract: Nowadays,research in sentence summarization mainly focuses on monolingual materials,which means the source sentences and the target summarized phrases are in the same language,reducing the availability of information from texts in different languages.To solve the problem,this paper proposes a cross-lingual sentence summarization system.The system borrows the idea of back translation,using the neural machine translation system to translate the source end of parallel corpus of monolingual sentence summarization into another language.Then the translation is combined with summarized phrases in the target end of the parallel corpus of sentence summarization to construct a cross-lingual pseudo parallel corpus.On this basis,the contrastive attention mechanism is used to obtain most irrelevant information from the sequences of the source end and target end,solving the mismatching of lengths of source sentences and target sentences in the traditional attention mechanism.Experimental results show that compared with pipeline-based monolingual sentence summarization systems,the proposed cross-lingual system can generate more fluent summarized phrases that match the representation of human languages and are closer to the level of monolingual sentence summarization.

Key words: cross-lingual sentence summarization, parallel corpus, pseudo corpus, contrastive attention mechanism, pipeline method

摘要： 当今句子摘要研究主要针对单语，即源端句子和目标端摘要短语属于同种语言，然而单语句子摘要严重制约了不同语言文本信息的快速获取。为解决该问题，提出一种跨语言句子摘要系统。借鉴回译思想，将单语句子摘要平行语料中的源端通过神经机器翻译系统翻译成另一种语言，将其与句子摘要平行语料中目标端的摘要短语共同构成跨语言的伪平行语料。在此基础上，利用对比注意力机制，实现目标端与源端序列中不相关信息的获取，解决了传统注意力机制中源端和目标端句子长度不匹配的问题。实验结果表明，与基于管道方法的单语句子摘要系统相比，该跨语言系统生成的摘要短语更流畅且符合人类语言表述方式，可达到接近单语的句子摘要水平。

关键词: 跨语言句子摘要, 平行语料, 伪语料, 对比注意力机制, 管道方法

CLC Number:

TP312

YIN Mingming, SHI Xiaojing, YU Hongfei, DUAN Xiangyu. Cross-Lingual Sentence Summarization System Based on Contrastive Attention Mechanism[J]. Computer Engineering, 2020, 46(5): 86-93.

殷明明, 史小静, 俞鸿飞, 段湘煜. 基于对比注意力机制的跨语言句子摘要系统[J]. 计算机工程, 2020, 46(5): 86-93.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0054793

http://www.ecice06.com/EN/Y2020/V46/I5/86

Figures/Tables 7

References

[1] JIANG Kaizhong,LI Zicheng,GU Junzhong.Method for automatic text summarization[J].Computer Engineering,2008,34(1):221-223.(in Chinese)江开忠,李子成,顾君忠.自动文本摘要方法[J].计算机工程,2008,34(1):221-223.
[2] HU Xia,LIN Yan,WANG Can,et al.Summary of automatic text summarization techniques[J].Journal of Intelligence,2010,29(8):144-147.(in Chinese)胡侠,林晔,王灿,等.自动文本摘要技术综述[J].情报杂志,2010,29(8):144-147.
[3] LI Yachao,XIONG Deyi,ZHANG Min.A survey of neural machine translation[J].Chinese Journal of Computers,2018,41(12):100-121.(in Chinese)李亚超,熊德意,张民.神经机器翻译综述[J].计算机学报,2018,41(12):100-121.
[4] SUTSKEVER I,VINYALS O,QUOC V.Sequence to sequence learning with neural networks[EB/OL].[2019-04-14].https://arxiv.org/abs/1409.3215.
[5] BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate[C]//Proceedings of ICLR'15.Washington D.C.,USA:IEEE Press,2015:1-5.
[6] GEHRING J,AULI M,GRANGIER D,et al.Convolutional sequence to sequence learning[EB/OL].[2019-04-14].https://arxiv.org/abs/1705.03122.
[7] VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[EB/OL].[2019-04-14].https://www.jianshu.com/p/418825ae767d.
[8] SHEN Shiqi,CHEN Yun,CHENG Yang,et al.Zero-shot crosslingual neural headline generation[J].IEEE/ACM Transactions on Audio,Speech,and Language Processing,2018,26(12):2319-2327.
[9] CHEN Yun,YANG Liu,YONG Cheng,et al.A teacher-student framework for zero-resource neural machine translation[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver,Canada:Association for Computational Linguistics,2017:1-8.
[10] SENNRICH R,HADDOW B,BIRCH A.Improving neural machine translation models with monolingual data[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.Berlin,Germany:Springer,2016:86-96.
[11] SERGEY E,OTT M,AULI M,et al.Understanding back-translation at scale[C]//Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing.Brussels,Belgium:Association for Computational Linguistics,2018:1-15.
[12] RUSH A M,SUMIT CHOPRA S,WESTON J.A neural attention model for abstractive sentence summarization[C]//Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing.Lisbon,Portugal:Association for Computational Linguistics,2015:15-35.
[13] RAMESH N,ZHOU B W,GULCEHRE C,et al.Abstractive text summarization using sequence-to-sequence RNNs and beyond[C]//Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning.Berlin,Germany:Springer,2016:35-48.
[14] LI Wang,YAO Junlin,TAO Yunzhe,et al.A reinforced topic-aware convolutional sequence-to-sequence model for abstractive text summarization[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence.Stockholm,Sweden:[s.n.],2018:1-14.
[15] ZHOU Qingyu,YANG Nan,WEI Furu,et al.Selective encoding for abstractive sentence summarization[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver,Canada:Association for Computational Linguistics,2017:33-58.
[16] XU Xintao,CHAI Xiaoli,XIE Bin,et al.Extraction of Chinese text summarization based on improved TextRank algorithm[J].Computer Engineering,2019,45(3):273-277.(in Chinese)徐馨韬,柴小丽,谢彬,等.基于改进TextRank算法的中文文本摘要提取[J].计算机工程,2019,45(3):273-277.
[17] MING Tuosiyu,CHEN Hongchang,HUANG Ruiyang,et al.Semantic subgraph predictive summary algorithm based on weighted AMR graph[J].Computer Engineering,2018,44(10):292-297,302.(in Chinese)明拓思宇,陈鸿昶,黄瑞阳,等.基于加权AMR图的语义子图预测摘要算法[J].计算机工程,2018,44(10):292-297,302.
[18] WAN Xiaojun,LI Huiying,XIAO Jianguo.Cross-language document summarization based on machine translation quality prediction[C]//Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics.Uppsala,Sweden:Association for Computational Linguistics,2010:45-62.
[19] ZHANG Jiajun,ZHOU Yu,ZONG Chengqing.Abstractive cross-language summarization via translation model enhanced predicate argument structure fusing[J].IEEE/ACM Transactions on Audio,Speech,and Language Processing,2016,24(10):1842-1853.
[20] COURTNEY N,GORMLEY M,DURME B.Annotated Gigaword[C]//Proceedings of Joint Workshop on Automatic Knowledge Base Construction and Web-scale Knowledge Extraction.New York,USA:ACM Press,2012:95-100.
[21] OVER P,DANG H,HARMAN D.DUC in context[J].Information Processing & Management,2007,43(6):1506-1520.
[22] HU Baotian,CHEN Qingcai,ZHU Fangze.A large scale Chinese short text summarization dataset[C]//Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing.Lisbon,Portugal:Association for Computational Linguistics,2015:1-7.
[23] RICO S,BARRY H,ALEXANDRA B.Neural machine translation of rare words with subword units[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.Berlin,Germany:Springer,2016:25-48.
[24] LIN C Y.Rouge:a package for automatic evaluation of summaries[J].Technology and Culture,1989,30(1):155-156.
[25] CAO Ziqiang,WEI Furu,LI Wenjie,et al.Faithful to the original:fact aware neural abstractive summarization[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence.Palo Alto,USA:AAAI Press,2018:1-8.
[26] CHENG Yong,TU Zhaopeng,MENG Fandong,et al.Towards robust neural machine translation[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.Philadelphia,USA:Association for Computational Linguistics,2018:1756-1766.

Please choose a citation manager

Content to export