作者投稿和查稿 主编审稿 专家审稿 编委审稿 远程编辑

计算机工程 ›› 2021, Vol. 47 ›› Issue (9): 227-234. doi: 10.19678/j.issn.1000-3428.0058303

• 图形图像处理 • 上一篇    下一篇

基于双注意力多层特征融合的视觉情感分析

蔡国永1,2, 储阳阳1,2   

  1. 1. 桂林电子科技大学 计算机与信息安全学院, 广西 桂林 541004;
    2. 桂林电子科技大学广西可信软件重点实验室, 广西 桂林 541004
  • 收稿日期:2020-05-12 修回日期:2020-07-07 发布日期:2020-08-27
  • 作者简介:蔡国永(1971-),男,教授,主研方向为社交媒体数据挖掘、情感计算;储阳阳,硕士研究生。
  • 基金资助:
    国家自然科学基金(61763007);广西自然科学基金重点项目(2017JJD160017)。

Visual Sentiment Analysis Based on Multi-level Features Fusion of Dual Attention

CAI Guoyong1,2, CHU Yangyang1,2   

  1. 1. School of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin, Guangxi 541004, China;
    2. Guangxi Key Laboratory of Trusted Software, Guilin University of Electronic Technology, Guilin, Guangxi 541004, China
  • Received:2020-05-12 Revised:2020-07-07 Published:2020-08-27

摘要: 为获得更具判别性的视觉特征并提升情感分类效果,构建融合双注意力多层特征的视觉情感分析模型。通过卷积神经网络提取图像多通道的多层次特征,根据空间注意力机制对多通道的低层特征赋予空间注意力权重,利用通道注意力机制对多通道的高层特征赋予通道注意力权重,分别强化不同层次的特征表示,将强化后的高层特征和低层特征进行融合,形成用于训练情感分类器的判别性特征。在3个真实数据集Twitter Ⅰ、Twitter Ⅱ和EmotionROI上进行对比实验,结果表明,该模型的分类准确率分别达到79.83%、78.25%和49.34%,有效提升了社交媒体视觉情感分析的效果。

关键词: 社交媒体, 视觉情感分析, 卷积神经网络, 注意力, 特征融合

Abstract: In order to obtain more discriminative image features and improve the accuracy of sentiment classification, a visual sentiment analysis model based on multi-level feature fusion of dual attention is proposed.Firstly, the model extracts multi-channel and multi-level features of images with Convolution Neural Network(CNN).Secondly, the extracted low-level features are given spatial attention weight with spatial attention mechanism, and the extracted high-level features are given channel attention weight with channel attention mechanism to saparately enhance the low and high level features.And finally, the enhaned high-level features and low-level features are fused to form discriminant features for training sentiment classifiers.Experiments on three real datasets TwitterⅠ, TwitterⅡ and EmotionROI showed that the classification accuracy of the method reached 79.83%, 78.25% and 49.34%, which improved the effect of social media visual sentiment analysis.

Key words: social media, visual sentiment analysis, Convolutional Neural Network(CNN), attention, features fusion

中图分类号: