作者投稿和查稿 主编审稿 专家审稿 编委审稿 远程编辑

计算机工程 ›› 2020, Vol. 46 ›› Issue (6): 256-265. doi: 10.19678/j.issn.1000-3428.0055351

• 图形图像处理 • 上一篇    下一篇

融合施工场景及空间关系的图像描述生成模型

徐守坤, 吉晨晨, 倪楚涵, 李宁   

  1. 常州大学 信息科学与工程学院 数理学院, 江苏 常州 213164
  • 收稿日期:2019-07-02 修回日期:2019-08-28 发布日期:2019-09-03
  • 作者简介:徐守坤(1972-),男,教授、博士,主研方向为图像理解、人工智能、普适计算;吉晨晨、倪楚涵,硕士研究生;李宁,副教授、博士。
  • 基金资助:
    国家自然科学基金(61803050)。

Image Description Generation Model Integrating Construction Scenes and Spatial Relationship

XU Shoukun, JI Chenchen, NI Chuhan, LI Ning   

  1. School of Information Science and Engineering, School of Mathematics and Physics, Changzhou University, Changzhou, Jiangsu 213164, China
  • Received:2019-07-02 Revised:2019-08-28 Published:2019-09-03

摘要: 为解决施工场景中缺少空间关系图像描述的问题,提出一种融合施工场景及空间关系的图像描述生成模型。采用YOLOv3网络进行目标检测,以TransE算法为基础在传统对象检测模型中加入特征提取层形成关系检测模型,结合对象坐标框信息得到对象之间的关系,并采用基于规则和模板的方法生成图像描述。实验结果表明,与m-RNN、NIC、Soft-Attention等模型相比,该模型能生成更准确的空间关系图像描述。

关键词: YOLOv3网络, 施工场景, 空间关系, 图像描述, 关系检测模型

Abstract: To solve the problem of lack of image description of spatial relationship in construction scenes,this paper proposes an image description generation model integrating construction scenes and spatial relationship.YOLOv3 network is used for target detection,and a feature extraction layer is added to the traditional object detection model on the basis of the transE algorithm to form the relationship detection model.The coordinate frame information of the object is combined to obtain the relationship between the objects,and the method based on rules and templates is used to generate the image description.Experimental results show that compared with m-RNN,NIC,Soft-Attention and Hard-Attention models,the proposed model can generate accurate description of spatial relationship.

Key words: YOLOv3 network, construction scene, spatial relationship, image description, relationship detection model

中图分类号: