计算机工程 ›› 2019, Vol. 45 ›› Issue (2): 202-206.doi: 10.19678/j.issn.1000-3428.0050237

• 图形图像处理 • 上一篇    下一篇

基于MFF-GAN的图像集视觉总结

张文凯1,2,3,孙皓1,2,孙显1,2,王宏琦1,2   

  1. 1.中国科学院电子学研究所,北京 100190; 2.中国科学院空间信息处理与应用系统技术重点实验室,北京 100190; 3.中国科学院大学,北京 100190)
  • 收稿日期:2018-01-23 出版日期:2019-02-15 发布日期:2019-02-15
  • 作者简介:张文凯(1990—),男,博士研究生,主研方向为计算机视觉、图像处理;孙皓、孙显,副研究员、博士;王宏琦,研究员、博士。
  • 基金项目:

    国家自然科学基金(41501485)。

Image Set Visual Summarization Based on MFF-GAN

ZHANG Wenkai 1,2,3,SUN Hao 1,2,SUN Xian 1,2,WANG Hongqi 1,2   

  1. 1.Institute of Electronics,Chinese Academy of Sciences,Beijing 100190,China; 2.Key Laboratory of Spatial Information Processing and Application System Technology, Chinese Academy of Sciences,Beijing 100190,China; 3.University of Chinese Academy of Sciences,Beijing 100190,China
  • Received:2018-01-23 Online:2019-02-15 Published:2019-02-15

摘要:

现有图像集视觉总结方法主要使用浅层视觉特征,或者直接应用已训练的卷积神经网络模型提取图像深层特征,选取的图像不具代表性。为此,分析并研究图像集视觉总结的图像特征表示方法,提出多特征图融合生成对抗网络(MFF-GAN)模型。该模型中的判别器通过多特征图融合的方式提取图像特征,使提取的特征能表示图像细节和高层语义信息,并在多特征图融合层后添加自编码网络对特征进行降维,避免特征维度灾难问题。NUS-WIDE数据集上的实验结果验证了MFF-GAN模型的有效性,并表明其能有效提升图像集视觉总结多样性。

关键词: 生成对抗网络, 特征学习, 视觉总结, 多特征图融合, 自编码网络

Abstract:

Existing image set visual summarization methods primarily consider the low-level visual features of images,or deep features,which extracted from trained Convolutional Neural Network(CNN) model.It makes the selected image not representative.In order to solve the problem,this paper analyzes and studies the image feature representation method in the image set visual summarization,proposes a Multi-Feature Fusion Generative Adversarial Networks(MFF-GAN) model.The discriminator in the model extracts image features by means of multi-feature image fusion,so that the extracted features can represent image details and high-level semantic information.To reduce the dimensionality of feature,the encoder network is added after the fusion layer.Experimental results on NUS-WIDE dataset valify the effectiveness of the MFF-GAN model,and show it can improve the diversity of visual summarization.

Key words: Generative Adversarial Network(GAN), feature learning, visual summarization, multi-feature fusion, autoencoder network

中图分类号: