摘要：

生成模型在许多问答推理任务中表现良好, 但是这往往需要人工花费大量成本为每条数据匹配对应的相关知识文本, 以保证模型输出的可靠性。如果语言模型可以经过充分训练内化知识库, 输出较为可靠的问答知识, 那么就可以降低在问答推理任务中提供相关知识的成本。此外, 输出问答相关知识文本也有利于探究模型在推理任务中依据哪些知识进行推理, 这对探究模型的可解释性有重要意义。为此, 提出一个新的自然语言生成任务, 将问答对作为输入, 使模型直接生成与问答相关的知识文本(依据这些问答知识文本能够辅助答案的推理)以帮助模型通过新任务形成知识库。为新任务提供基准模型, 结果显示了生成模型具有较高的生成质量, 表明了新任务是有可行性的, 并且当问答对的陈述形式也包含在输入中时, 模型的生成效果可以显著提高。实验比较了3种生成模型, 结果显示了参数更多的模型可能包含更全面的知识库, 具有更好的生成效果。此外, 实验设计了不同的输入融合方法和输出相关文本的数量, 确定最佳的任务形式。实验分析表明, 新任务对于未来的研究是可行和有价值的。

关键词: 生成模型, 可解释性, 知识文本, 开放领域问答, 注意力机制

Abstract:

Generative models show satisfactory performance in many question and answer reasoning tasks. However, significant manual effort is required for matching each data point with the corresponding relevant knowledge text to ensure the reliability of the model's output. If a language model can be sufficiently trained to internalize a knowledge base and reliably output question and answer knowledge, it can eliminate the cost of providing relevant knowledge explicitly in question and answer reasoning tasks. In addition, generating knowledge texts related to the answers can help explore which knowledge the model relies on for reasoning, which is crucial for investigating the interpretability of the model. For this purpose, this paper proposes a new natural language generation task. This task takes a question-answer pair as the input and requires the model to directly generate the relevant knowledge text. The generated text should support the reasoning behind the given answer, thereby helping the model consolidate its internal knowledge base during the training process. Benchmark models have been established for this new task. The results demonstrate the remarkable text generation quality of the model, confirming the feasibility of the task. When the statement forms of the question-answer pairs are also included in the input, the generation effect of the model can be significantly improved. A comparison of three generative models reveals that models with more parameters achieve superior generation performance, likely owing to their more comprehensive internal knowledge bases. Furthermore, experiments are conducted with different input fusion methods while varying the number of knowledge statements generated to identify the optimal task configuration. The results indicate that this task is feasible and valuable for future research.

Key words: generative model, interpretability, knowledge text, open domain question and answer, attention mechanism