摘要： 针对语义鸿沟问题，在语义学习的基础上设计图像的多模态检索系统。该系统结合3种查询方式进行图像检索。基于视觉特征的查询通过特征提取与相似度匹配进行排位。基于标签的查询建立在图像自动标注的基础上，但在语义空间之外的泛化能力较差。基于语义图例的查询能够在很大程度上克服这个缺陷，通过在显式或隐式的语义空间上进行查询，使检索结果更符合人类感知。实验结果表明，与基于纹理特征的图像检索相比，基于语义图例的检索具有更高的精度及召回率。

关键词: 图像多模态检索, 图像自动标注, 概率主题建模, 概率潜在语义分析, 语义鸿沟, 语义学习, 语义多项式

Abstract: In order to bridge the semantic gap, a multi-modal image retrieval system is proposed based on semantic learning. The system combines three query modes to retrieval images. The paradigm of query by visual feature ranks images by feature extraction and similarity matching; The paradigm of query by label is based on automatic image annotation, but its generalization ability is not good outside the semantic space; The paradigm of Query by Semantic Example(QBSE) can overcome the problem to a great extent. It makes the retrieval more agreeable with human perception by executing the query in either explicit or implicit semantic space. Experimental results show that the paradigm of query by semantic example has higher precision and recall rate than image retrieval based on the texture feature.

Key words: multi-modal image retrieval, automatic image annotation, probabilistic topic modeling, probabilistic latent semantic analysis, semantic gap, semantic learning, semantic multinomial

中图分类号: 

• TN911.73