计算机工程 ›› 2019, Vol. 45 ›› Issue (5): 243-248.doi: 10.19678/j.issn.1000-3428.0050768

• 多媒体技术及应用 • 上一篇    下一篇

基于SRU的时域金字塔构建方法

智洪欣,于洪涛,李邵梅,高超   

  1. 国家数字交换系统工程技术研究中心,郑州 450002
  • 收稿日期:2018-03-14 出版日期:2019-05-15 发布日期:2019-05-15
  • 作者简介:智洪欣(1987—),男,硕士研究生,主研方向为计算机视觉、视频处理;于洪涛,研究员、博士;李邵梅、高超,副研究员、博士。
  • 基金项目:

    国家自然科学基金(61601513)。

Temporal pyramid construction method based on SRU

ZHI Hongxin,YU Hongtao,LI Shaomei,GAO Chao   

  1. China National Digital Switching System Engineering and Technological R & D Center,Zhengzhou 450002,China
  • Received:2018-03-14 Online:2019-05-15 Published:2019-05-15

摘要:

现有基于时域金字塔的特征提取方法不能学习视频帧和视频段各自之间的时间依赖性信息以及未充分利用视频时域的分层结构信息,造成视频分类特征提取不充分。为此,提出一种基于SRU的多层次多粒度时空域深度特征提取方法。利用卷积神经网络提取视频的低、中、高3个层次的帧特征,构建时域金字塔,同时采用级联SRU学习视频时间依赖性和时域的分层结构特征,通过聚合3个层次的时域金字塔得到视频的多层次多粒度全局特征。在数据集UCF101和HMDB51上的实验结果表明,与DTPP方法、TLE方法相比,该方法提取的特征具有较好的表征能力和鲁棒性。

关键词: 时域金字塔, 时间依赖性, 分层结构, 多层次多粒度, 级联SRU

Abstract:

Existing feature extraction methods based on temporal pyramid cannot learn the time dependence between video frames and video segments and the hierarchical structure information of video temporal is not fully utilized,resulting in insufficient video classification feature extraction.Therefore,a multi-level multi-granularity spatial-temporal depth feature extraction method based on SRU is proposed.The Convolutional Neural Network(CNN) is used to extract the low,medium and high frame features of the video,and the temporal pyramid is constructed.At the same time,the cascade SRU is used to learn the temporal characteristics of video time and the hierarchical structure of the temporal,and the three-level temporal pyramid is aggregated to obtain the multi-level and multi-granular global features of the video.Experimental results in the datasets UCF101 and HMDB51 show that compared with DTPP method and TLE method,this method has better characterization ability and robustness.

Key words: temporal pyramid, time dependence, hierarchical structure, multi-level multi-granularity, cascad Simple Recurrent Units(SRU)

中图分类号: