基于槽位语义增强提示学习的篇章级事件抽取方法

doi:10.19678/j.issn.1000-3428.0066170

计算机工程 ›› 2023, Vol. 49 ›› Issue (9): 23-31. doi: 10.19678/j.issn.1000-3428.0066170

基于槽位语义增强提示学习的篇章级事件抽取方法

李鸿鹏¹^,²^,³, 马博¹^,²^,³^,*, 杨雅婷¹^,²^,³, 王磊¹^,²^,³, 王震¹^,²^,³, 李晓¹^,²^,³

1. 中国科学院新疆理化技术研究所, 乌鲁木齐 830011
2. 中国科学院大学, 北京 100049
3. 新疆民族语音语言信息处理实验室, 乌鲁木齐 830011

收稿日期:2022-11-04 出版日期:2023-09-15 发布日期:2023-02-09
通讯作者: 马博
作者简介:
李鸿鹏（1996—），男，硕士研究生，主研方向为自然语言处理、事件抽取
杨雅婷，研究员、博士
王磊，研究员、博士
王震，研究实习员、硕士
李晓，研究员
基金资助:
中国科学院青年创新促进会项目(科发人函字[2019]26号); 新疆天山创新团队项目(2020D14045); 中国科学院特色科学数据库建设项目(CASWX2021SF031); 新疆维吾尔自治区自然科学基金重点基金项目(2022D01D81); 新疆维吾尔自治区自然科学基金重点基金项目(2022D01D04)

Document-level Event Extraction Method Based on Slot Semantic Enhanced Prompt Learning

Hongpeng LI¹^,²^,³, Bo MA¹^,²^,³^,*, Yating YANG¹^,²^,³, Lei WANG¹^,²^,³, Zhen WANG¹^,²^,³, Xiao LI¹^,²^,³

1. Xinjiang Technical Institute of Physics and Chemistry, Chinese Academy of Sciences, Urumqi 830011, China
2. University of Chinese Academy of Sciences, Beijing 100049, China
3. Xinjiang Laboratory of Minority Speech and Language Information Processing, Urumqi 830011, China

Received:2022-11-04 Online:2023-09-15 Published:2023-02-09
Contact: Bo MA

摘要/Abstract

摘要：

事件抽取旨在将非结构化自然语言文本中的事件信息以结构化形式进行识别提取。传统事件抽取方法抽取范围局限于单个句子，且依赖较大规模的标注数据，在篇章级抽取任务与低资源目标领域中表现不佳。现有研究利用提示学习方法，以模板槽位填空方式实现篇章级事件抽取，其缺点在于传统提示模板槽位对论元角色分类准确度不高，容易造成论元角色抽取错误。针对上述问题，提出一种基于槽位语义增强提示学习的篇章级事件抽取方法，在提示学习方法的基础上，将传统事件抽取范式中的论元角色语义信息融入提示模板槽位中，为模型的槽位预测生成环节提供论元类型约束，提高篇章级事件抽取的准确率。通过使预训练语言模型上下游任务保持一致，提高模型的泛化能力，同时以较低成本实现知识迁移，在低资源事件抽取场景下提升模型性能。实验结果表明，相较于表现次优的传统基线方法，在包含59种论元类型的英文事件抽取数据集、包含92种论元类型的中文数据集以及低资源数据规模下，该方法的F1值分别取得了2.6、2.9和4.0个百分点的提升。

关键词: 事件抽取, 提示学习, 信息抽取, 自然语言处理, 预训练语言模型

Abstract:

Event extraction aims to recognize and extract event information from unstructured natural language texts in a structured form.Traditional methods extract events at the sentence level, relying on massive labeled data for training, which are unqualified for document-level event extraction and lack performance in low-resource scenarios.Existing research utilizes prompt learning methods to achieve document-level event extraction by filling in template slots.However, traditional prompt template slots have low accuracy in classifying argument roles, which can easily lead to errors in argument role extraction.To address the above issues, this paper proposes a document-level event extraction method based on slot semantic enhancement prompt learning.Based on the prompt learning method, the argument role semantic information in the traditional event extraction paradigm is integrated into the slot of the prompt template, providing argument type constraints for the slot prediction generation process of the model and improving the accuracy of document-level event extraction.By keeping the upstream and downstream tasks of the pretrained language model consistent, the generalization ability of the model is improved, and knowledge transfer is achieved at a lower cost to improve model performance in low-resource event extraction scenarios.Experimental results show that compared to the traditional baseline method with suboptimal performance, this method achieved an F1 score improvement of 2.6, 2.9, and 4.0 percentage points on an English event extraction dataset containing 59 argument types, Chinese dataset containing 92 argument types, and low-resource data scale, respectively.

Key words: event extraction, prompt learning, information extraction, natural language processing, pretrained language model

李鸿鹏, 马博, 杨雅婷, 王磊, 王震, 李晓. 基于槽位语义增强提示学习的篇章级事件抽取方法[J]. 计算机工程, 2023, 49(9): 23-31.

Hongpeng LI, Bo MA, Yating YANG, Lei WANG, Zhen WANG, Xiao LI. Document-level Event Extraction Method Based on Slot Semantic Enhanced Prompt Learning[J]. Computer Engineering, 2023, 49(9): 23-31.

http://www.ecice06.com/CN/Y2023/V49/I9/23

图/表 12

图1 篇章级事件抽取示例

Fig.1 Example of document-level event extraction

图2 事件抽取模型结构

Fig.2 The structure of event extraction model

图3 模板槽位语义增强过程

Fig.3 Template slot semantic enhancement process

图4 提示模板知识库

Fig.4 Knowledge base of prompt template

图5 模型编码层结构

Fig.5 Model encoding layer structure

图6 不同数据规模下的知识迁移效果

Fig.6 Knowledge transfer effects under different data scales

参考文献 25

1	AHN D. The stages of event extraction[C]//Proceedings of the Workshop on Annotating and Reasoning About Time and Events. New York, USA: ACM Press, 2006: 1-8.
2	DODDINGTON G R, MITCHELL A, PRZYBOCKI M A, et al. The Automatic Content Extraction(ACE) program-tasks, data, and evaluation[EB/OL]. [2022-10-05]. http://www.lrec-conf.org/proceedings/lrec2004/pdf/5.pdf.
3	CHEN Y B, LIU S L, ZHANG X, et al. Automatically labeled data generation for large scale event extraction[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. [S. l. ]: Association for Computational Linguistics, 2017: 409-419.
4	JI H, GRISHMAN R. Refining event extraction through cross-document inference[EB/OL]. [2022-10-05]. https://aclanthology.org/P08-1030.pdf.
5	YANG H, CHEN Y B, LIU K, et al. DCFEE: a document-level Chinese financial event extraction system based on automatically labeled training data[EB/OL]. [2022-10-05]. https://aclanthology.org/P18-4009.pdf.
6	陈斌, 周勇, 刘兵. 基于卷积双向长短期记忆网络的事件触发词抽取. 计算机工程, 2019, 45 (1): 153- 158. URL
	CHEN B, ZHOU Y, LIU B. Event trigger word extraction based on convolutional bidirectional long short term memory network. Computer Engineering, 2019, 45 (1): 153- 158. URL
7	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2017: 6000-6010.
8	ZHENG S, CAO W, XU W, et al. Doc2EDAG: an end-to-end document-level framework for Chinese financial event extraction[C]//Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. [S. l. ]: Association for Computational Linguistics, 2019: 337-346.
9	仲伟峰, 杨航, 陈玉博, 等. 基于联合标注和全局推理的篇章级事件抽取. 中文信息学报, 2019, 33 (9): 88-95, 106. URL
	ZHONG W F, YANG H, CHEN Y B, et al. Document-level event extraction based on joint labeling and global reasoning. Journal of Chinese Information Processing, 2019, 33 (9): 88-95, 106. URL
10	EBNER S, XIA P, CULKIN R, et al. Multi-sentence argument linking[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. [S. l. ]: Association for Computational Linguistics, 2020: 8057-8077.
11	LI S, JI H, HAN J W. Document-level event argument extraction by conditional generation[EB/OL]. [2022-10-05]. https://arxiv.org/pdf/2104.05919.pdf.
12	LIU P, YUAN W, FU J, et al. Pre-train, prompt, and predict: a systematic survey of prompting methods in natural language processing[EB/OL]. [2022-10-05]. https://arxiv.org/pdf/2107.13586.pdf.
13	LEVY O, SEO M, CHOI E, et al. Zero-shot relation extraction via reading comprehension[C]//Proceedings of the 21st Conference on Computational Natural Language Learning. [S. l. ]: Association for Computational Linguistics, 2017: 333-342.
14	PETRONI F, ROCKTÄSCHEL T, RIEDEL S, et al. Language models as knowledge bases?[C]//Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. [S. l. ]: Association for Computational Linguistics, 2019: 2463-2473.
15	SHIN T, RAZEGHI Y, LOGAN R L, et al. AutoPrompt: eliciting knowledge from language models with automatically generated prompts[C]//Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing. [S. l. ]: Association for Computational Linguistics, 2020: 4222-4235.
16	LI X Y, FENG J R, MENG Y X, et al. A unified MRC framework for named entity recognition[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. [S. l. ]: Association for Computational Linguistics, 2020: 5849-5859.
17	DU X Y, CARDIE C. Event extraction by answering (almost) natural questions[C]//Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing. [S. l. ]: Association for Computational Linguistics, 2020: 671-683.
18	李珂, 陈彦如, 郑文蛟, 等. 基于机器阅读理解的新闻时间线挖掘与展示. 情报理论与实践, 2022, 45 (4): 184- 189. URL
	LI K, CHEN Y R, ZHENG W J, et al. News timeline mining and presentation based on machine reading comprehension. Information Studies (Theory & Application), 2022, 45 (4): 184- 189. URL
19	LIU J, CHEN Y F, XU J N. Machine reading comprehension as data augmentation: a case study on implicit event argument extraction[C]//Proceedings of 2021 Conference on Empirical Methods in Natural Language Processing. [S. l. ]: Association for Computational Linguistics, 2021: 2716-2725.
20	LEWIS M, LIU Y H, GOYAL N, et al. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. [S. l. ]: Association for Computational Linguistics, 2020: 7871-7880.
21	FAN A, LEWIS M, DAUPHIN Y. Hierarchical neural story generation[EB/OL]. [2022-10-05]. https://arxiv.org/pdf/1805.04833.pdf.
22	庄福振, 罗平, 何清, 等. 迁移学习研究进展. 软件学报, 2015, 26 (1): 26- 39. URL
	ZHUANG F Z, LUO P, HE Q, et al. Survey on transfer learning research. Journal of Software, 2015, 26 (1): 26- 39. URL
23	SHI P, LIN J. Simple BERT models for relation extraction and semantic role labeling[EB/OL]. [2022-10-05]. https://arxiv.org/abs/1904.05255.
24	XU R X, LIU T Y, LI L, et al. Document-level event extraction via heterogeneous graph-based interaction model with a tracker[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. [S. l. ]: Association for Computational Linguistics, 2021: 3533-3546.
25	ZHU T, QU X, CHEN W, et al. Efficient document-level event extraction via pseudo-trigger-aware pruned complete graph[EB/OL]. [2022-10-05]. https://arxiv.org/abs/2112.06013.

[1]	郭艳霞, 金勇, 唐宏, 彭金枝. 基于动态卷积与残差门控的多模态情感识别[J]. 计算机工程, 2023, 49(7): 94-101.
[2]	张博旭, 蒲智, 程曦. 基于提示学习的维吾尔语文本分类研究[J]. 计算机工程, 2023, 49(6): 292-299,313.
[3]	李静雯, 赵奎. 基于改进PCFG算法的口令猜测方法[J]. 计算机工程, 2023, 49(5): 38-47.
[4]	衡红军, 苗菁. 语义与句法信息加强的二元标记实体关系联合抽取[J]. 计算机工程, 2023, 49(4): 77-84.
[5]	陈柏霖, 王天极, 任丽娜, 黄瑞章. 融合ELECTRA和文本局部信息的中文语法错误检测方法[J]. 计算机工程, 2023, 49(3): 304-311.
[6]	杨文忠, 丁甜甜, 康鹏, 卜文秀. 基于舆情新闻的中文关键词抽取综述[J]. 计算机工程, 2023, 49(3): 1-17.
[7]	蔡瑞初, 张盛强, 许柏炎. 基于结构感知混合编码模型的代码注释生成方法[J]. 计算机工程, 2023, 49(2): 61-69.
[8]	杨红菊, 靳新宇. 一个实体关系与事件抽取的通用模型[J]. 计算机工程, 2023, 49(2): 143-149.
[9]	王春东, 孙嘉琪, 杨文军. 基于矫正理解的中文文本对抗样本生成方法[J]. 计算机工程, 2023, 49(2): 37-45.
[10]	禹克强, 黄芳, 吴琪, 欧阳洋. 基于双向语义的中文实体关系联合抽取方法[J]. 计算机工程, 2023, 49(1): 92-99,112.
[11]	黄君扬, 王振宇, 梁家卿, 肖仰华. 基于自裁剪异构图的NL2SQL模型[J]. 计算机工程, 2022, 48(9): 71-77,88.
[12]	田乔鑫, 孔韦韦, 滕金保, 王照乾. 基于并行混合网络与注意力机制的文本情感分析模型[J]. 计算机工程, 2022, 48(8): 266-273.
[13]	司逸晨, 管有庆. 基于Transformer编码器的中文命名实体识别模型[J]. 计算机工程, 2022, 48(7): 66-72.
[14]	王士浩, 王中卿, 李寿山, 周国栋. 基于知识蒸馏与模型集成的事件论元抽取方法[J]. 计算机工程, 2022, 48(7): 97-103.
[15]	李军怀, 陈苗苗, 王怀军, 崔颖安, 张爱华. 基于ALBERT-BGRU-CRF的中文命名实体识别方法[J]. 计算机工程, 2022, 48(6): 89-94,106.

选择文件类型/文献管理软件名称

选择包含的内容

基于槽位语义增强提示学习的篇章级事件抽取方法

Document-level Event Extraction Method Based on Slot Semantic Enhanced Prompt Learning

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 25

相关文章 15

编辑推荐

Metrics

本文评价

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于槽位语义增强提示学习的篇章级事件抽取方法

Document-level Event Extraction Method Based on Slot Semantic Enhanced Prompt Learning

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 25

相关文章 15

编辑推荐

Metrics

本文评价