Abstract:

Relation extraction is a basic and important task that aims to extract the relations between entities from unstructured text. Recent developments show that Large-Language Model (LLM) and basic models can improve the performance of several Natural Language Processing (NLP) tasks. These models utilize the language-representation ability of deep-learning and pre-training models and can automatically learn the semantic features of relations. A method to effectively use of a large model for solving the problems of entity overlap and unsatisfactory information exchange is yet to be revealed. Hence, a relational-extraction model based on large language is proposed. First, the Large-Language model Meta AI (LLaMA) is adapted to the task in this study via fine-tuning. To extract relations, the self-attention mechanism is used to enhance the correlation between entity pairs and information sharing between entities. Subsequently, average pooling is performed to generalize an entire sentence. A filtering matrix is designed for entity pairs, part-of-speech information is introduced to enhance semantics, and invalid triples are filtered out based on the relevance of entity pairs in the filtering matrix. Experimental results show that the F1 value results of the proposed model on the New York Times (NYT) and WebNLG open datasets are 93.1% and 90.4%, respectively. In the case where the LLaMA model becomes an encoder after fine-tuning, the proposed algorithm is superior to the baseline model in terms of accuracy and the F1 value index, thus verifying its effectiveness.

Key words: relation extraction, artificial intelligence, attention mechanism, Large-Language Model(LLM), part of speech

摘要：

关系抽取是一项基础且重要的任务, 旨在从非结构化文本中提取出实体之间的关系。最近研究证明, 大型语言模型(LLM)和基础模型相结合可以改进许多自然语言处理(NLP)任务的性能。这些模型利用深度学习和预训练模型的语言表示能力, 能够自动学习关系的语义特征。有效利用大模型来解决实体重叠和信息交互差等问题仍是一个挑战。针对以上问题, 提出基于大语言模型的关系抽取算法。对大型语言模型Meta AI(LLaMA)进行微调训练, 使其更加适应关系抽取的任务, 在提取关系的基础上, 使用自注意力机制增强实体对之间关联程度, 增强关系和实体之间的信息共享, 接着使用平均池化泛化到整个句子中。针对实体对设计一个过滤矩阵, 并引入词性信息进行语义增强, 根据过滤矩阵中实体对的相关性过滤掉无效的三元组。实验结果表明, 该算法在纽约时报(NYT)和WebNLG公开数据集上的F1值结果分别为93.1%、90.4%。在微调之后的LLaMA模型作为编码器的情况下, 所提算法在准确率和F1值指标上均优于基线模型, 验证了算法的有效性。

关键词: 关系抽取, 人工智能, 注意力机制, 大语言模型, 词性