面向法律文书的分段式摘要模型

doi:10.19678/j.issn.1000-3428.0061119

摘要/Abstract

摘要： 文本摘要是指对文本信息内容进行概括、提取主要内容进而形成摘要的过程。现有的文本摘要模型通常将内容选择和摘要生成独立分析，虽然能够有效提高句子压缩和融合的性能，但是在抽取过程中会丢失部分文本信息，导致准确率降低。基于预训练模型和Transformer结构的文档级句子编码器，提出一种结合内容抽取与摘要生成的分段式摘要模型。采用BERT模型对大量语料进行自监督学习，获得包含丰富语义信息的词表示。基于Transformer结构，通过全连接网络分类器将每个句子分成3类标签，抽取每句摘要对应的原文句子集合。利用指针生成器网络对原文句子集合进行压缩，将多个句子集合生成单句摘要，缩短输出序列和输入序列的长度。实验结果表明，相比直接生成摘要全文，该模型在生成句子上ROUGE-1、ROUGE-2和ROUGE-L的F1平均值提高了1.69个百分点，能够有效提高生成句子的准确率。

关键词: 司法摘要, 预训练模型, Transformer编码器, 序列标注, 指针生成器网络, 分段式摘要模型

Abstract: Text summary refers to the process of summarization the content of text information, extracting the relevant content, and then formulating a summarization.Existing text summarization models usually analyze content selection and summarization generation separately.Although they can effectively improve the performance of sentence compression and fusion models, some text information will be lost in the extraction process, resulting in reduced accuracy.Based on the pre-training model and the document level sentence encoder with Transformer structure, a segmented summarization model combining content extraction and summarization generation is proposed.The BERT model is used to conduct self-supervised learning on a large corpus to arrive at a word representation containing rich semantic information.Based on the Transformer structure, each sentence is divided into three types of tags through the fully connected network classifier, and the original sentence set corresponding to each sentence summarization extracted.The Pointer-Generator(PG) network is used to compress the original sentence set, and multiple sentence sets generated into a single sentence summarization to shorten the length of the output and input sequences.The experimental results show that compared with the direct generation of summarization full text, the F1 average value of ROUGE-1, ROUGE-2, and ROUGE-L in generating sentences increased by 1.69 percentage points, which can effectively improve the accuracy of generating sentences.

Key words: judical summarization, pre-training model, Transformer encoder, sequence labeling, Pointer-Generator(PG) network, segmented summarization model

中图分类号:

TP391

王刚, 孙媛媛, 陈彦光, 林鸿飞. 面向法律文书的分段式摘要模型[J]. 计算机工程, 2022, 48(6): 288-294.

WANG Gang, SUN Yuanyuan, CHEN Yanguang, LIN Hongfei. Segmented Summarization Model for Legal Documents[J]. Computer Engineering, 2022, 48(6): 288-294.

https://www.ecice06.com/CN/Y2022/V48/I6/288

图/表 7

20220625181402

20220625181405

20220625181411

20220625181415

20220625181420

20220625181423

20220625181427

参考文献

[1] LEBANOFF L, SONG K, DERNONCOURT F, et al.Scoring sentence singletons and pairs for abstractive summarization[EB/OL].[2021-02-15].https://arxiv.org/abs/1906.00077.
[2] SUBRAMANIAN S, LI R, PILAULT J, et al.On extractive and abstractive neural document summarization with transformer language models[EB/OL].[2021-02-15]. https://arxiv.org/abs/1909.03186.
[3] VASWANI A, SHAZEER N, PARMAR N, et al.Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.NewYork, USA:ACM Press, 2017:6000-6010.
[4] SEE A, LIU P J, MANNING C D.Get to the point:summarization with pointer-generator networks[EB/OL].[2021-02-15].https://arxiv.org/pdf/1704.04368.pdf.
[5] LUHN H P.The automatic creation of literature abstracts[J].IBM Journal of Research and Development, 1958, 2(2):159-165.
[6] EDMUNDSON H P.New methods in automatic extracting[J].Journal of the ACM, 1969, 16(2):264-285.
[7] LIN C Y, HOVY E.Identifying topics by position[EB/OL].[2021-02-15].http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=18A483239494376ACA4520EFBD742575?doi=10.1.1.13.8985&rep=rep1&type=pdf.
[8] KUPIEC J, PEDERSEN J, CHEN F.A trainable document summarizer[C]//Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.New York, USA:ACM Press, 1995:68-73.
[9] LIN C Y.Training a selection function for extraction[C]//Proceedings of the 18th International Conference on Information and Knowledge Management.New York, USA:ACM Press, 1999:55-62.
[10] MIHALCEA R, TARAU P. Textrank:bringing order into text[EB/OL].[2021-02-15].https://www.researchgate.net/profile/Paul-Tarau/publication/200042361_TextRank_Bringing_Order_into_Text/links/0912f508a98af2fe24000000/TextRank-Bringing-Order-into-Text.pdf.
[11] ERKAN G, RADEV D R.LexRank:graph-based lexical centrality as salience in text summarization[EB/OL].[2021-02-15].https://arxiv.org/abs/1109.2128v1.
[12] NALLAPATI R, ZHAI F, ZHOU B.SummaRuNNer:a recurrent neural network based sequence model for extractive summarization of documents[C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence.[S.l.]:AAAI Press, 2017:3075-3081.
[13] ZHOU Q, YANG N, WEI F, et al.Neural document summarization by jointly learning to score and select sentences[EB/OL].[2021-02-15]. https://arxiv.org/pdf/1807.02305.pdf.
[14] DEVLIN J, CHANG M W, LEE K, et al.BERT:pre-training of deep bidirectional transformers for language understanding[EB/OL].[2021-02-15].https://arxiv.org/pdf/1810.04805.pdf.
[15] LIU Y, LAPATA M.Text summarization with pretrainedencoders[EB/OL].[2021-02-15].https://arxiv.org/abs/1908.08345.
[16] SUTSKEVER I, VINYALS O, LE Q V.Sequence to sequence learning with neural networks[J].Advances in Neural Information Processing Systems, 2014, 4(1):3104-3112.
[17] CHO K, VAN MERRIËNBOER B, GULCEHRE C, et al.Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]//Proceedings of Conference on Empirical Methods in Natural Language Processing.[S.l.]:AAAI Press, 2014:1724-1734.
[18] BAHDANAU D, CHO K H, BENGIO Y.Neural machine translation by jointly learning to align and translate[EB/OL].[2021-02-15].https://arxiv.org/pdf/1409.0473.pdf.
[19] VINYALS O, FORTUNATO M, JAITLY N.Pointer networks[EB/OL].[2021-02-15].https://arxiv.org/pdf/1506.03134.pdf.
[20] RUSH A M, CHOPRA S, WESTON J.A neural attention model for abstractive sentence summarization[EB/OL].[2021-02-15].https://arxiv.org/pdf/1509.00685.pdf.
[21] SANKARAN B, MI H, AL-ONAIZAN Y, et al.Temporal attention model for neural machine translation[EB/OL].[2021-02-15].https://arxiv.org/abs/1608.02927.
[22] PAULUS R, XIONG C, SOCHER R.A deep reinforced model for abstractive summarization[EB/OL].[2021-02-15].https://arxiv.org/pdf/1705.04304.pdf.
[23] KINGMA D P, BA J L.Adam:a method for stochastic optimization[EB/OL].[2021-02-15].http://de.arxiv.org/pdf/1412.6980.
[24] LIN C Y.Rouge:a package for automatic evaluation of summaries[EB/OL].[2021-02-15].http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=657245BA32160BCB52B6B8972D0FE238?doi=10.1.1.126.4764&rep=rep1&type=pdf.
[25] LAFFERTY J., MCCALLUM A, PEREIRA F.Conditional random fields:Probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the 18th International Conference on Machine Learning.New York, USA:ACM Press, 2001:282-289.

选择文件类型/文献管理软件名称

选择包含的内容