基于知识图谱的异常个体提前识别模型研究

doi:10.19678/j.issn.1000-3428.0069349

摘要/Abstract

摘要：

识别视频中的异常个体是计算机视觉领域重要的研究课题，已有算法主要研究如何检测出异常行为的爆发期，而忽略了异常行为的发展阶段，同时存在异常定义不明确、可解释性差、应用场景泛化能力不强等问题。针对上述问题，提出一种基于知识图谱的异常个体提前识别模型。对视频进行行人检测与跟踪、行人视觉关注目标检测和行人行为识别，以捕获与异常行为相关的行人属性特征；建立针对异常个体的知识图谱网络，提出年龄属性、社交距离等4种节点建模算法，根据行人属性对节点进行建模，更好地分析异常个体在异常行为发展期的特征；提出拐卖儿童、偷窃/抢劫和打架3种基于节点状态转移的异常个体推理算法，对知识图谱节点进行状态推理，得出个体在未来发生异常行为的概率值，实现对异常个体的提前识别，采用的推理算法增强了模型的可解释性；制作并标注异常个体提前发现数据集，定义偷窃、打架、抢劫和拐卖儿童4种异常行为，其中的样本源自不同的拍摄场景。在该数据集上评估模型的有效性，实验结果表明，该模型的均值平均精度(mAP)为22.83%，优于其他主流行为识别模型，其中与SlowFast模型相比提升了18.96百分点，表明所提模型能在异常行为爆发之前有效地识别出异常个体，且模型对应用场景具有良好的泛化能力。

关键词: 异常行为, 提前识别, 行为识别, 异常检测, 行人属性, 知识图谱

Abstract:

The identification of abnormal individuals in videos is an important research topic in the field of computer vision. Existing algorithms primarily focus on detecting the outbreak phase of abnormal behaviors but overlook their developmental stage. Moreover, they suffer from issues such as unclear definitions of abnormalities, poor interpretability, and weak generalizability across application scenarios. To address these problems, this study proposes a knowledge graph-based model for the early identification of abnormal individuals. The model performs pedestrian detection and tracking, pedestrian visual attention target detection, and pedestrian behavior recognition from videos to capture pedestrian attribute features related to abnormal behaviors. Moreover, the study establishes a knowledge graph network targeting abnormal individuals and proposes four node modeling algorithms, including those for age attributes and social distance. Nodes are modeled based on pedestrian attributes to better analyze the characteristics of abnormal individuals during the developmental stage of abnormal behaviors. Additionally, the study proposes three abnormal individual reasoning algorithms based on node state transitions for child abduction, theft, robbery, and fighting. These algorithms perform state reasoning on knowledge graph nodes to derive the probability of an individual engaging in abnormal behavior in the future, thereby enabling the early identification of abnormal individuals. The reasoning algorithms adopted enhance the interpretability of the model. An early abnormal individual detection dataset is created and annotated, defining four types of abnormal behaviors: theft, fighting, robbery, and child abduction. The samples in the dataset are sourced from various shooting scenarios. The effectiveness of the model is evaluated on this dataset, and the experimental results show that the model achieves a mean Average Precision (mAP) of 22.83%, outperforming other mainstream behavior recognition models. Specifically, it demonstrates an 18.96 percentage point improvement over the SlowFast model, indicating that the proposed model can effectively identify abnormal individuals before the outbreak of abnormal behaviors and is generalizable across application scenarios.

Key words: abnormal behavior, advance recognition, behavior recognition, anomaly detection, pedestrian attribute, knowledge graph

徐式芃, 王雷, 盛捷. 基于知识图谱的异常个体提前识别模型研究[J]. 计算机工程, 2025, 51(9): 59-70.

XU Shipeng, WANG Lei, SHENG Jie. Knowledge Graph-based Advance Recognition Model for Abnormal Individuals[J]. Computer Engineering, 2025, 51(9): 59-70.

https://www.ecice06.com/CN/Y2025/V51/I9/59

图/表 13

图1 模型总体框架

Fig.1 Overall framework of model

图2 行人属性提取模块

Fig.2 Pedestrian attribute extraction module

图3 行人检测与跟踪效果

Fig.3 The effect of pedestrian detection and tracking

图4 行人视觉关注目标检测效果

Fig.4 The detection effect of pedestrian visual attention targets

图5 行人行为识别效果

Fig.5 Pedestrian behavior recognition effect

图6 基于知识图谱的异常个体推理模块

Fig.6 Abnormal individual inference module based on knowledge graph

图7 年龄属性节点建模效果

Fig.7 Modeling effect of age attribute nodes

图8 部分数据集样本

Fig.8 Partial dataset samples

图9 实验效果

Fig.9 Experimental effect

参考文献 34

1	胡兴航. 结合多模态信息的行为识别研究[D]. 成都: 电子科技大学, 2022.
	HU X H. Research on behavior recognition based on multimodal information[D]. Chengdu: University of Electronic Science and Technology of China, 2022. (in Chinese)
2	罗会兰. 视频动作识别研究. 长沙: 中南大学出版社, 2019.
	LUO H L . Study on video action recognition. Changsha: Central South University Press, 2019.
3	FEICHTENHOFER C, FAN H Q, MALIK J, et al. SlowFast networks for video recognition[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Washington D.C., USA: IEEE Press, 2019: 6202-6211.
4	LEE J, KANG S J. Skeleton action recognition using two-stream adaptive graph convolutional networks[C]//Proceedings of the 36th International Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC). Washington D.C., USA: IEEE Press, 2021: 1-3.
5	RYALI C, HU Y T, BOLYA D, et al. Hiera: a hierarchical vision transformer without the bells-and-whistles[C]//Proceedings of International Conference on Machine Learning. New York, USA: ACM Press, 2023: 29441-29454.
6	GU C H, SUN C, ROSS D A, et al. AVA: a video dataset of spatio-temporally localized atomic visual actions[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2018: 6047-6056.
7	KAY W, CARREIRA J, SIMONYAN K, et al. The Kinetics human action video dataset[EB/OL]. [2023-10-05]. https://arxiv.org/abs/1705.06950v1.
8	张蔚澜. 商场监控系统中的人体异常行为检测方法研究[D]. 西安: 西安工业大学, 2022.
	ZHANG W L. Research on human abnormal behavior detection method in shopping mall monitoring system[D]. Xi'an: Xi'an Technological University, 2022. (in Chinese)
9	LIU W, LUO W X, LIAN D Z, et al. Future frame prediction for anomaly detection-a new baseline[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2018: 6536-6545.
10	NGUYEN K T, DINH D T, DO M N, et al. Anomaly detection in traffic surveillance videos with GAN-based future frame prediction[C]//Proceedings of the 2020 International Conference on Multimedia Retrieval. Washington D.C., USA: IEEE Press, 2020: 457-463.
11	NGUYEN T N, MEUNIER J. Anomaly detection in video sequence with appearance-motion correspondence[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2019: 1273-1283.
12	LEE S , KIM H G , RO Y M . BMAN: bidirectional multi-scale aggregation networks for abnormal event detection. IEEE Transactions on Image Processing, 2019, 29, 2395- 2408.
13	ZHU X H, SHENG Q, LI D A, et al. Local attention base on time masks for weakly supervised video anomaly detection[C]//Proceedings of the 8th International Conference on Information Systems Engineering (ICISE). Washington D.C., USA: IEEE Press, 2023: 121-125.
14	GUO B, LIU M R, HE Q, et al. Video anomaly detection with video vision transformer[C]//Proceedings of the 8th International Conference on Signal and Image Processing (ICSIP). Washington D.C., USA: IEEE Press, 2023: 131-135.
15	MEHRAN R, OYAMA A, SHAH M. Abnormal crowd behavior detection using social force model[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2009: 935-942.
16	KHAN S S , NOGAS J , MIHAILIDIS A . Spatio-temporal adversarial learning for detecting unseen falls. Pattern Analysis and Applications, 2021, 24 (1): 381- 391. doi: 10.1007/s10044-020-00901-9
17	ZHANG Y F, SUN P Z, JIANG Y, et al. ByteTrack: multi-object tracking by associating every detection box[EB/OL]. [2023-10-05]. https://arxiv.org/abs/2110.06864.
18	CHONG E, WANG Y X, RUIZ N, et al. Detecting attended visual targets in video[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2020: 5396-5406.
19	BALCI B, ALKAN B, ELIHOS A, et al. Front seat child occupancy detection using road surveillance camera images[C]//Proceedings of the 25th IEEE International Conference on Image Processing (ICIP). Washington D.C., USA: IEEE Press, 2018: 1927-1931.
20	BRANDAO M. Age and gender bias in pedestrian detection algorithms[EB/OL]. [2023-10-05]. https://arxiv.org/abs/1906.10490.
21	KOGURE S, WATABE K, YAMADA R, et al. Age should not matter: towards more accurate pedestrian detection via self-training[EB/OL]. [2023-10-05]. https://www.mdpi.com/2813-0324/3/1/11.
22	HALL E T . The hidden dimension. New York, USA: Doubleday, 1966.
23	NAYAK R, BEHERA M M, GIRISH V, et al. Deep learning based loitering detection system using multi-camera video surveillance network[C]//Proceedings of the IEEE International Symposium on Smart Electronic Systems (iSES)(Formerly iNiS). Washington D.C., USA: IEEE Press, 2019: 215-220.
24	ZHANG Y Y, WANG J C, WANG X B, et al. Research and application of loitering detection based on deep learning[C]//Proceedings of the 10th International Conference on Dependable Systems and Their Applications (DSA). Washington D.C., USA: IEEE Press, 2023: 339-346.
25	NAM Y . Loitering detection using an associating pedestrian tracker in crowded scenes. Multimedia Tools and Applications, 2015, 74 (9): 2939- 2961. doi: 10.1007/s11042-013-1763-7
26	ELHAMOD M , LEVINE M D . Automated real-time detection of potentially suspicious behavior in public transport areas. IEEE Transactions on Intelligent Transportation Systems, 2012, 14 (2): 688- 699.
27	HUANG T M , HAN Q , MIN W D , et al. Loitering detection based on pedestrian activity area classification. Applied Sciences, 2019, 9 (9): 1866. doi: 10.3390/app9091866
28	HE S, LI P. A campus virtual fence system based on loitering detection[C]//Proceedings of the 6th IEEE International Conference on Pattern Recognition and Artificial Intelligence (PRAI). Washington D.C., USA: IEEE Press, 2023: 107-110.
29	SHANMUGHAPRIYA M, GUNASUNDARI S, BHARATHY S. Loitering detection in home surveillance system[C]//Proceedings of the 10th International Conference on Emerging Trends in Engineering and Technology-Signal and Information Processing (ICETET-SIP-22). Washington D.C., USA: IEEE Press, 2022: 1-6.
30	LI W T, ZHANG D P, SUN M, et al. Loitering detection based on trajectory analysis[C]//Proceedings of the 8th International Conference on Intelligent Computation Technology and Automation (ICICTA). Washington D.C., USA: IEEE Press, 2015: 530-533.
31	MAHADEVAN V, LI W X, BHALODIA V, et al. Anomaly detection in crowded scenes[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2010: 11-23.
32	LU C W, SHI J P, JIA J Y. Abnormal event detection at 150 FPS in MATLAB[C]//Proceedings of the IEEE International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2013: 2720-2727.
33	CARREIRA J, ZISSERMAN A. Quo vadis, action recognition?A new model and the Kinetics dataset[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2017: 6299-6308.
34	LI Y H, WU C Y, FAN H Q, et al. MViTv2: improved multiscale vision transformers for classification and detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2022: 4804-4814.

[1]	符家成, 田瑾, 张玉金, 方志军. 结合前置三元组集的知识图谱推荐[J]. 计算机工程, 2025, 51(9): 101-109.
[2]	吴杰辉, 柳毅. 面向多元时间序列的联合优化异常检测模型[J]. 计算机工程, 2025, 51(9): 166-176.
[3]	王光明, 李冬青, 蒋从锋. 不平衡数据集下的数据中心网络流量异常检测[J]. 计算机工程, 2025, 51(8): 227-237.
[4]	刘文杰, 陈亮, 任智杰. 基于图神经网络与元学习的小样本关系推理模型[J]. 计算机工程, 2025, 51(5): 124-132.
[5]	蒋杰平, 王明文. 基于时空置换注意力机制的残差行为识别模型[J]. 计算机工程, 2025, 51(4): 119-128.
[6]	徐永刚, 孙琦烜, 李凡甲, 程健维, 戴佳俊. 基于扩展时间和时空特征融合图卷积网络的骨架行为识别[J]. 计算机工程, 2025, 51(4): 281-292.
[7]	张安勤, 丁志锋. 融合动态图嵌入和Transformer自编码器的网络异常检测[J]. 计算机工程, 2025, 51(4): 47-56.
[8]	刘春雨, 陈庆锋, 莫少聪, 谢泽. 基于逻辑规则和图神经网络的知识图谱补全[J]. 计算机工程, 2025, 51(3): 131-143.
[9]	朱红, 王阔然, 朱彤. 基于多侧面信息表征联合的实体相似性度量及对齐方法[J]. 计算机工程, 2025, 51(3): 64-75.
[10]	马恒志, 钱育蓉, 冷洪勇, 吴海鹏, 陶文彬, 张依杨. 知识图谱嵌入研究进展综述[J]. 计算机工程, 2025, 51(2): 18-34.
[11]	李猛坤, 袁晨, 王琪, 赵冲, 陈景轩, 刘立峰. 基于改进YOLOv8算法的在线听课行为识别模型研究[J]. 计算机工程, 2025, 51(1): 287-294.
[12]	钟忺, 陈亮, 刘文璇, 叶舒, 江奎, 王正, 林嘉文. 时空语义驱动的渐进多视角行为去偏置研究[J]. 计算机工程, 2025, 51(1): 1-10.
[13]	李泽霖, 吕兆峰, 陈富强, 李克. 基于多跳信息融合的实体对齐模型[J]. 计算机工程, 2024, 50(9): 142-152.
[14]	汤志康, 武毓琦, 李春英, 汤庸. 基于知识图谱卷积网络的学习资源推荐[J]. 计算机工程, 2024, 50(9): 153-160.
[15]	刘锁兰, 王炎, 王洪元, 朱生升. 基于多流语义图卷积网络的人体行为识别[J]. 计算机工程, 2024, 50(8): 64-74.

选择文件类型/文献管理软件名称

选择包含的内容