摘要： 意图识别是自然语言理解的一项重要任务，以往的意图识别研究主要关注于特定任务的单模态意图识别。然而，在现实世界的场景中，人类的意图是复杂的，需要通过整合诸如语言、语调、表情和动作等信息来判断。因此提出以注意力为主的多模态融合的意图识别方法，用于在真实世界的多模态场景中进行意图识别。为了能够捕捉和融合不同模态之间的长距离依赖关系、自适应地调整各模态信息的重要性和提供更丰富的表示，对每个模态特征分别使用自注意力机制。通过在每个模态的特征中添加明确的模态标识，使模型能够区分并有效融合不同模态的信息，提升整体理解和决策能力。考虑到在跨模态交互时文本模态信息的重要性，使用以跨注意力机制为核心、以文本为主导其他模态辅助引导交互的多模态融合，旨在促进文本与视觉、听觉模态之间的交互。最后对多模态意图识别的MIntRec和MIntRec2.0基准数据集进行了实验评估。结果显示，模型在准确性、精确度、召回率和F1分数等指标上均优于现有的多模态学习方法，比目前最好的基线有0.1%∼0.5％的提升。

Abstract: Intent recognition is an important task in natural language understanding. Previous research on intent recognition has mainly focused on single-modal intent recognition for specific tasks. However, in real-world scenarios, human intentions are complex and need to be inferred by integrating information such as language, tone, expressions, and actions. Therefore, a novel attention-based multimodal fusion method for intent recognition is proposed to address intent recognition in real-world multimodal scenarios. To capture and integrate long-range dependencies between different modalities, adaptively adjust the importance of information from each modality, and provide richer representations, a self-attention mechanism is used for each modality feature separately. By adding explicit modality identifiers to the data of each modality, the model can distinguish and effectively fuse information from different modalities, thereby enhancing overall understanding and decision-making capabilities. Given the importance of textual information in cross-modal interactions, a multimodal fusion method centered on a cross-attention mechanism is employed, with text as the primary modality and other modalities assisting and guiding the interactions. This approach aims to facilitate interaction between textual, visual, and auditory modalities. Finally, experiments were conducted on the MIntRec and MIntRec2.0 benchmark datasets for multimodal intent recognition. The results show that the model outperforms existing multimodal learning methods in terms of accuracy, precision, recall, and F1 score, with an improvement of 0.1% to 0.5% over the current best baseline.