作者投稿和查稿 主编审稿 专家审稿 编委审稿 远程编辑

计算机工程 ›› 2021, Vol. 47 ›› Issue (7): 55-58,66. doi: 10.19678/j.issn.1000-3428.0058557

• 人工智能与模式识别 • 上一篇    下一篇

基于单向Transformer和孪生网络的多轮任务型对话技术

王涛, 刘超辉, 郑青青, 黄嘉曦   

  1. 深圳市易马达科技有限公司, 广东 深圳 518055
  • 收稿日期:2020-06-05 修回日期:2020-07-12 发布日期:2020-07-17
  • 作者简介:王涛(1993-),男,工程师,主研方向为人机交互;刘超辉、郑青青,高级工程师;黄嘉曦,研究员。
  • 基金资助:
    中美绿色基金(MA009RX18)。

Multi-turn Task-oriented Dialogue Technology Based on Unidirectional Transformer and Siamese Network

WANG Tao, LIU Chaohui, ZHENG Qingqing, HUANG Jiaxi   

  1. Shenzhen Immotor Technology Co., Ltd., Shenzhen, Guangdong 518055, China
  • Received:2020-06-05 Revised:2020-07-12 Published:2020-07-17

摘要: 循环神经网络和Transformer在多轮对话系统的建模上依赖大量的样本数据且回复准确率过低。为此,提出一种针对任务型对话系统的建模方法。引入预训练模型对句子语意和对话过程进行深度编码,对Transformer模型进行精简,仅保留编码器部分的单向Transformer,将应答部分抽象成不同的指令,采用孪生网络对指令进行相似度排序,选择相似度最高的指令生成应答。在MultiWOZ数据集上的实验结果表明,与LSTM和基于Transformer模型相比,该方法预测速度更快,在小数据集上具有更好的性能,在大数据集上也能取得与当前先进模型相当的效果。

关键词: 循环神经网络, 多轮对话系统, 预训练模型, Transformer模型, 孪生网络

Abstract: The existing Recurrent Neural Network(RNN) and Transformer models rely on a large amount of sample data for the modeling of the multi-turn dialogue system,and the accuracy of answering is low.To address the problem,a new modeling method for the task-oriented dialogue system is proposed.Some pre-trained models are introduced for deep encoding of the sentence semantics and the dialog contents.At the same time,the Transformer model is simplified to a unidirectional transformer with only the encoder retained.On this basis,the answering part is abstracted to different commands,which are sorted based on similarity by using the siamese network.The command with the highest similarity is chosen to generate the answer.The experimental results on the MultiWOZ dataset show that compared to LSTM and Transformer-based models,the proposed method has a faster prediction speed,providing better performance on small datasets and equal performance on large datasets.

Key words: Recurrent Neural Network(RNN), multi-turn dialogue system, pre-training model, Transformer model, siamese network

中图分类号: