Author Login Chief Editor Login Reviewer Login Editor Login Remote Office

Computer Engineering ›› 2026, Vol. 52 ›› Issue (10): 350-362. doi: 10.19678/j.issn.1000-3428.0252354

• Large Language Models and Generative Artificial Intelligence • Previous Articles     Next Articles

Multimodal Fake News Detection Based on Vision-Language Large Model

LIN Rongxin1, LI Shuohao1,*(), DONG Liming2, HAO Siqi1   

  1. 1. Laboratory for Big Data and Decision, National University of Defense Technology, Changsha 410073, Hunan, China
    2. The Publicity Department of the CPC Chongqing Municipal Committee, Chongqing 400000, China
  • Received:2025-04-22 Revised:2025-07-09 Online:2026-10-15 Published:2026-10-25
  • Contact: LI Shuohao

基于视觉语言大模型的多模态虚假新闻检测

林荣鑫1, 李硕豪1,*(), 董力铭2, 郝思齐1   

  1. 1. 国防科技大学大数据与决策实验, 湖南 长沙 410073
    2. 中共重庆市委宣传部, 重庆 400000
  • 通讯作者: 李硕豪
  • 作者简介:

    林荣鑫, 男, 硕士研究生, 主研方向为多模态融合、智能算法对抗

    李硕豪(通信作者), 副教授、博士

    董力铭, 硕士研究生

    郝思齐, 硕士研究生

  • 基金资助:
    国家自然科学基金(62101571); 军队高层次人才项目

Abstract:

With the exponential growth of information dissemination on social media platforms, false news detection has become a critical challenge in information authenticity verification. Existing methods focus on single-modal semantic analysis, which not only inadequately models cross-modal semantic contradictions in multimodal news but also suffers from limited interpretability due to the absence of explainable auxiliary information. To address these issues, this study proposes a multimodal fake news detection framework that utilizes a vision-language large model. The framework introduces the following innovations: 1) utilization of the vision-language large model Qwen2.5-VL to reason for news content and generate multimodal description sets that enhance interpretability; 2) design of a multi-granularity co-attention mechanism to achieve cross-modal feature alignment across textual semantics, visual features, and auxiliary descriptions. News-prompting templates are designed to guide Qwen2.5-VL in extracting key objects, scene elements, and contextual semantic enhancements from the news, thus generating explainable auxiliary decision-making evidence. Based on the co-attention layers, the multi-grained co-attention fusion mechanism employs hierarchical feature interactions to capture latent fake patterns in multimodal news within high-dimensional semantic spaces. In addition, experiments are conducted on three multimodal fake news datasets, namely, Weibo, GossipCop, and Pheme, and the results show that the accuracy rates reach 90.4%, 88.1%, and 86.6%, respectively.

Key words: fake news detection, multimodal learning, cross modal fusion, vision-language large model, interpretability

摘要:

随着社交媒体平台传播的信息量呈指数级增长, 虚假新闻检测成为信息鉴伪领域的关键任务。当前研究方法聚焦于单一模态的语义分析, 未能有效地解决多模态新闻的跨模态语义矛盾问题。现有模型决策过程可信度不足, 缺乏可解释的辅助信息支撑。针对上述问题, 提出一种面向多模态新闻的视觉语言大模型检测框架。首先, 引入视觉语言大模型推理新闻内容, 通过生成图文描述集来增强检测的可解释性; 其次, 设计多粒度协同注意力机制, 实现文本、图像, 以及辅助描述的多粒度特征对齐。采用多模态视觉语言大模型Qwen2.5-VL作为新闻解释性工具, 设计新闻提示模板, 引导模型对新闻图像提取关键对象和场景要素, 利用模型的语言生成能力增强新闻文本的上下文信息, 形成可解释的辅助决策依据。多粒度协同注意力融合机制以协同注意力层为基础, 通过多层级特征交互, 在高维语义空间中捕捉新闻图文中的潜在伪造模式。在Weibo、GossipCop和Pheme多模态虚假新闻数据集上的实验结果表明, 所提模型的准确率分别为90.4%、88.1%和86.6%。

关键词: 虚假新闻检测, 多模态学习, 跨模态融合, 视觉语言大模型, 可解释性