基于改进型时间分段网络的视频异常检测

doi:10.19678/j.issn.1000-3428.0062691

计算机工程 ›› 2022, Vol. 48 ›› Issue (11): 137-144. doi: 10.19678/j.issn.1000-3428.0062691

基于改进型时间分段网络的视频异常检测

黄涛¹, 邬开俊¹, 王迪聪^1,2, 白晨帅¹, 陶小苗¹

1. 兰州交通大学电子与信息工程学院, 兰州 730070;
2. 天津大学智能与计算学部, 天津 300350

收稿日期:2021-09-14 修回日期:2022-03-17 发布日期:2022-07-18
作者简介:黄涛(1996—),男,硕士研究生,主研方向为视频异常检测;邬开俊(通信作者),教授、博士、博士生导师;王迪聪,博士研究生;白晨帅,硕士研究生;陶小苗,讲师、博士研究生。
基金资助:
国家自然科学基金（61966022）；甘肃省教育厅优秀研究生“创新之星”项目（2021CXZX-555）。

Video Anomaly Detection Based on Improved Time Segmentation Network

HUANG Tao¹, WU Kaijun¹, WANG Dicong^1,2, BAI Chenshuai¹, TAO Xiaomiao¹

1. School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China;
2. College of Intelligence and Computing, Tianjin University, Tianjin 300350, China

Received:2021-09-14 Revised:2022-03-17 Published:2022-07-18

摘要/Abstract

摘要： 视频异常检测是计算机视觉领域的一个重要研究课题，广泛应用于道路监控、异常事件监测等方面。考虑到异常行为的外观、运动特征与正常行为存在明显差异，提出一种改进型时间分段网络，利用该网络学习视频中的外观和运动信息，从而对视频异常行为进行预测。为了提取更多的视频信息，将RGB图和RGB帧差图相融合作为输入，以提取RGB图中的外观信息并通过RGB帧差图获得更有效的运动特征。将卷积注意力机制模块加入到时间分段网络模型中，从空间和通道2个不同的维度学习注意力图，利用学习到的注意力权重区分异常和正常的视频片段，同时运用焦点损失函数降低大量简单负样本在训练过程中所占的权重，使得模型更专注于难分类的样本，从而解决视频异常检测中正负样本比例不平衡的问题。实验结果表明，改进型时间分段网络在UCF-Crime和CUHK Avenue数据集上的AUC值分别达到77.6%和83.3%，检测性能优于基准方法TSN（RGB流）以及ISTL、3D-ConvAE等方法。

关键词: 视频异常检测, 卷积注意力机制, RGB帧差图, 焦点损失函数, 时间分段网络

Abstract: Video anomaly detection is an important research topic in the field of computer vision, that is widely used in road monitoring and abnormal event monitoring.Considering the obvious differences between the appearance and motion characteristics of abnormal and normal behavior, an improved time segmentation network is proposed to learn the appearance and motion information in video so as to predict abnormal video behavior.An RGB image and RGB frame difference image are fused as inputs to extract the appearance information from the RGB image and obtain more effective motion features from the RGB frame difference image.The convolution attention mechanism module is added to the time segmentation network model to learn the attention graph from two different dimensions of space and channel, and the learned attention weight is used to better distinguish between abnormal and normal video clips.Focal Loss function reduces the weight of a large number of simple negative samples during the training process, allowing the model to focus on samples that are difficult to classify, thus resolving the problem of the imbalance between the proportion of positive and negative samples during video anomaly detection.The experimental results show that the Area Under Curve(AUC) values of the improved time segmentation network on UCF-Crime and CUHK Avenue datasets reach 77.6% and 83.3%, respectively, and the detection performance is better than the benchmark methods TSN(RGB stream), ISTL, 3D-ConvAE, and other methods.

Key words: video anomaly detection, convolutional attention mechanism, RGB frame difference image, Focal Loss(FL) function, time segmentation network

中图分类号:

TP391

黄涛, 邬开俊, 王迪聪, 白晨帅, 陶小苗. 基于改进型时间分段网络的视频异常检测[J]. 计算机工程, 2022, 48(11): 137-144.

HUANG Tao, WU Kaijun, WANG Dicong, BAI Chenshuai, TAO Xiaomiao. Video Anomaly Detection Based on Improved Time Segmentation Network[J]. Computer Engineering, 2022, 48(11): 137-144.

http://www.ecice06.com/CN/Y2022/V48/I11/137

图/表 7

20230211175831

20230211175834

20230211175838

20230211175842

20230211175845

20230211175848

20230211175852

参考文献

[1] NGUYEN T N, MEUNIER J.Anomaly detection in video sequence with appearance-motion correspondence[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2019:1273-1283.
[2] TU Z G, XIE W, ZHANG D J, et al.A survey of variational and CNN-based optical flow techniques[J].Signal Processing:Image Communication, 2019, 72:9-24.
[3] WANG L M, XIONG Y J, WANG Z, et al.Temporal segment networks for action recognition in videos[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(11):2740-2755.
[4] WANG H, SCHMID C.Action recognition with improved trajectories[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2013:3551-3558.
[5] SALIGRAMA V, KONRAD J, JODOIN P M.Video anomaly identification[J].IEEE Signal Processing Magazine, 2010, 27(5):18-33.
[6] RANZATO M, POULTNEY C, CHOPRA S, et al.Efficient learning of sparse representations with an energy-based model[EB/OL].[2021-08-05].https://proceedings.neurips.cc/paper/2006/file/87f4d79e36d68c3031ccf6c55e9bbd39-Paper.pdf.
[7] 傅博, 王瑞子, 王丽妍, 等.基于深度卷积神经网络的水下偏色图像增强方法[J].吉林大学学报(理学版), 2021, 59(4):891-899. FU B, WANG R Z, WANG L Y, et al.Enhancement method of underwater color cast image based on deep convolutional neural network[J].Journal of Jilin University(Science Edition), 2021, 59(4):891-899.(in Chinese)
[8] RIFAI S, VINCENT P, MULLER X, et al.Contractive auto-encoders:explicit invariance during feature extraction[EB/OL].[2021-08-05].http://www.icml-2011.org/papers/455_icmlpaper.pdf.
[9] YU T, REN Z, LI Y C, et al.Temporal structure mining for weakly supervised action detection[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2019:5521-5530.
[10] VAROL G, LAPTEV I, SCHMID C.Long-term temporal convolutions for action recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(6):1510-1517.
[11] CARREIRA J, ZISSERMAN A.Quo vadis, action recognition?A new model and the kinetics dataset[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:4724-4733.
[12] TRAN D, BOURDEV L, FERGUS R, et al.Learning spatiotemporal features with 3D convolutional networks[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2015:4489-4497.
[13] WANG L M, XIONG Y J, WANG Z, et al.Temporal segment networks:towards good practices for deep action recognition[EB/OL].[2021-08-05].https://arxiv.org/pdf/1608.00859.pdf.
[14] SIMONYAN K, ZISSERMAN A.Two-stream convolutional networks for action recognition in videos[EB/OL].[2021-08-05].https://arxiv.org/abs/1406.2199.
[15] XU D, YAN Y, RICCI E, et al.Detecting anomalous events in videos by learning deep representations of appearance and motion[J].Computer Vision and Image Understanding, 2017, 156:117-127.
[16] RAVANBAKHSH M, NABI M, SANGINETO E, et al.Abnormal event detection in videos using generative adversarial nets[C]//Proceedings of IEEE International Conference on Image Processing.Washington D.C., USA:IEEE Press, 2017:1577-1581.
[17] LIN T Y, GOYAL P, GIRSHICK R, et al.Focal loss for dense object detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2):318-327.
[18] RABIEE H, HADDADNIA J, MOUSAVI H, et al.Novel dataset for fine-grained abnormal behavior understanding in crowd[C]//Proceedings of the 13th IEEE International Conference on Advanced Video and Signal Based Surveillance.Washington D.C., USA:IEEE Press, 2016:95-101.
[19] SULTANI W, CHEN C, SHAH M.Real-world anomaly detection in surveillance videos[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:6479-6488.
[20] SIMONYAN K, ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[EB/OL].[2021-08-05].https://arxiv.org/abs/1409.1556.
[21] SZEGEDY C, LIU W, JIA Y Q, et al.Going deeper with convolutions[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2015:1-9.
[22] LI W X, MAHADEVAN V, VASCONCELOS N.Anomaly detection and localization in crowded scenes[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(1):18-32.
[23] LU C W, SHI J P, JIA J Y.Abnormal event detection at 150 FPS in MATLAB[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2013:2720-2727.
[24] ZIMEK A, SCHUBERT E, KRIEGEL H P.A survey on unsupervised outlier detection in high-dimensional numerical data[J].Statistical Analysis and Data Mining, 2012, 5(5):363-387.
[25] HASAN M, CHOI J, NEUMANN J, et al.Learning temporal regularity in video sequences[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:733-742.
[26] NAWARATNE R, ALAHAKOON D, DE SILVA D, et al.Spatiotemporal anomaly detection using deep learning for real-time video surveillance[J].IEEE Transactions on Industrial Informatics, 2020, 16(1):393-402.
[27] CHONG Y S, TAY Y H.Abnormal event detection in videos using spatiotemporal autoencoder[EB/OL].[2021-08-05].https://arxiv.org/pdf/1701.01546.pdf.
[28] ZHAO Y R, DENG B, SHEN C, et al.Spatio-temporal autoencoder for video anomaly detection[C]//Proceedings of the 25th ACM International Conference on Multimedia.New York, USA:ACM Press, 2017:1933-1941.
[29] LIU W, LUO W X, LIAN D Z, et al.Future Frame prediction for anomaly detection-A new baseline[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:6536-6545.
[30] CHANG Y P, TU Z G, XIE W, et al.Video anomaly detection with spatio-temporal dissociation[J].Pattern Recognition, 2022, 122:108213.

选择文件类型/文献管理软件名称

选择包含的内容

基于改进型时间分段网络的视频异常检测

Video Anomaly Detection Based on Improved Time Segmentation Network

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献

相关文章 2

编辑推荐

Metrics

本文评价

[1]	古平, 邱嘉涛, 罗长江, 张志鹏. 基于目标时空上下文融合的视频异常检测算法[J]. 计算机工程, 2022, 48(10): 169-175.
[2]	周红志，程向阳. 一种基于局部时空特征的视频异常检测方案[J]. 计算机工程, 2014, 40(4): 203-208,213.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于改进型时间分段网络的视频异常检测

Video Anomaly Detection Based on Improved Time Segmentation Network

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献

相关文章 2

编辑推荐

Metrics

本文评价