Multi-Time-Series Variable Indirect Truck Lifting Detection Based on Motor Data Visualization

doi:10.19678/j.issn.1000-3428.0068533

Abstract

Abstract:

In automated container terminal operations, simultaneous lifting of containers and trucks has led to significant safety issues, including damage to goods and vehicles and, in severe cases, human fatalities. To mitigate these issues, a novel approach, Motor-data Engineering and image processing-based Indirect truck lifting detection Network (MEIN), is introduced. MEIN collects three-phase current and voltage data from cranes and employs feature engineering based on physical formulas to calculate numerous related time-series physical quantities. It combines a Sliding window with Tomek-Synthetic Minority Oversampling Technique(SMOTE) for sampling, which enhances sample size and balances category quantities. This process converts multi-time-series variables into an image format for classification via EfficientNet. In tests, MEIN demonstrated robust detection capabilities, even in challenging conditions such as adverse weather or obscured tires, achieving an Area Under the receiver operator characteristic Curve (AUC) above 0.997 across various test regions. Compared to traditional detection methodologies such as lidar and computer vision, MEIN offers notable advantages including reduced cost, heightened accuracy, reduced computational demands, and enhanced resilience to environmental disturbances. Its successful implementation in Wuhan, Qingdao, Qinzhou, and Meishan underscores its its effectiveness in improving safety at automated container terminals.

Key words: time-series classification, Convolutional Neural Network(CNN), Synthetic Minority Oversampling Technique(SMOTE), Tomek Links downsampling technique, truck lifting detection

摘要：

自动化集装箱码头的装卸作业中经常发生集装箱与卡车同时被吊起的安全事故, 导致人员伤亡及货品、车辆的损坏。为解决该问题, 提出一种基于电机数据图像化处理的多时序变量间接卡车误吊起检测方法(MEIN)。该方法通过神经网络分析异步电机在吊起集装箱和卡车的过程中产生的电流和电压异常, 从而判断是否发生了误吊起事故。采集吊机的三相电流和电压数据, 并基于物理公式进行特征工程计算出多个相关时序物理量, 采用滑动窗口、SMOTE-Tomek综合采样的方式扩大样本总数并平衡类别数量, 最后将多时序变量转换为图像的形式以EfficientNet进行分类。实验结果表明, 该方法能在复杂的环境下(例如雨雾天气或轮胎被遮挡)保持稳定的检测性能, 各测试地区的AUC均在0.997以上。相较于传统的基于激光雷达和计算机视觉的检测方法, MEIN方法具有成本低、精度高、计算量小并且抗环境干扰能力强等优点。该方法已在武汉、青岛、钦州、梅山等多地部署, 为提高自动化集装箱码头的作业安全提供一种有效的解决方案。

关键词: 时间序列分类, 卷积神经网络, 合成少数类样本的过采样技术, Tomek Links欠采样技术, 卡车误吊起检测

LIU Jiajie, LIU Guoping, HU Wenshan. Multi-Time-Series Variable Indirect Truck Lifting Detection Based on Motor Data Visualization[J]. Computer Engineering, 2024, 50(10): 370-380.

刘嘉杰, 刘国平, 胡文山. 基于电机数据图像化的多时序变量间接卡车误吊起检测[J]. 计算机工程, 2024, 50(10): 370-380.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0068533

https://www.ecice06.com/EN/Y2024/V50/I10/370

Figures/Tables 8

Fig.1 Scene of a truck being lifted during the experiment

Fig.2 Overall architecture of the MEIN algorithm

Fig.3 Kernel density estimation of physical quantities

Fig.4 SMOTE-Tomek combined sampling results

Fig.5 Graphicalization of motor multi-time series signals

References 29

1	童巍, 吴翔. 轮胎起重机集卡防吊起系统的研究与应用. 起重运输机械, 2017,(12): 95- 97.
	TONG W, WU X. Research and application of anti-lifting system for tire crane. Hoisting and Conveying Machinery, 2017,(12): 95- 97.
2	BRINKMANN B. Operations systems of container terminals: a compendious overview. Berlin, Germany: Springer, 2011.
3	黄伟, 赵德安, 刘晓洋. 基于机器视觉的港口集装箱卡车防吊方法研究. 软件导刊, 2019, 18(5): 37- 40.
	HUANG W, ZHAO D A, LIU X Y. Research on port container truck anti-lifting method based on machine version. Software Guide, 2019, 18(5): 37- 40.
4	王润生. 基于目标检测与跟踪的集装箱卡车防吊起研究[D]. 东营: 中国石油大学(华东), 2020.
	WANG R S. Research on anti-lifting of container truck based on target detection and tracking[D]. Dongying: China University of Petroleum (Huadong), 2020. (in Chinese)
5	刘燕欣, 田彤坤, 唐波, 等. 基于机器视觉的港口集卡防吊起系统. 中国质量, 2020,(12): 45- 49.
	LIU Y X, TIAN T K, TANG B, et al. Anti-hoisting system for port container trucks based on machine vision. China Quality, 2020,(12): 45- 49.
6	孙嘉, 张建辉, 卜佑军, 等. 基于CNN-BiLSTM模型的日志异常检测方法. 计算机工程, 2022, 48(7): 151-158, 167. URL
	SUN J, ZHANG J H, BU Y J, et al. Log anomaly detection method based on CNN-BiLSTM model. Computer Engineering, 2022, 48(7): 151-158, 167. URL
7	KWON D H, KIM J B, HEO J S, et al. Time series classification of cryptocurrency price trend based on a recurrent LSTM neural network. Journal of Information Processing Systems, 2019, 15(3): 694- 706.
8	VASWANI A, SHAZEER N, PAMMAR N, et al. Attention is all you need[C]//Proceedings of NIPS'17. Cambridge, USA: MIT Press, 2017: 30-39.
9	ZERVEAS G, JAYARAMAN S, PATEL D, et al. A transformer-based framework for multivariate time series representation learning[C]//Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. New York, USA: ACM Press, 2021: 2114-2124.
10	LIU M, REN S, MA S, et al. Gated transformer networks for multivariate time series classification[EB/OL]. [2023-09-01]. http://arxiv.org/abs/2103.14438v1.
11	孙彦玺, 赵婉婉, 武东辉, 等. 基于卷积长短时记忆网络的人体行为识别研究. 计算机工程, 2021, 47(10): 260- 268. URL
	SUN Y X, ZHAO W W, WU D H, et al. Research of human activity recognition based on convolutional long short-term memory network. Computer Engineering, 2021, 47(10): 260- 268. URL
12	ZHAO B D, LU H Z, et al. Convolutional neural networks for time series classification. Journal of Systems Engineering and Electronics, 2017, 28(1): 162- 169. doi: 10.21629/JSEE.2017.01.18
13	WANG Z G, YAN W Z, OATES T. Time series classification from scratch with deep neural networks: a strong baseline[C]//Proceedings of International Joint Conference on Neural Networks. Washington D. C., USA: IEEE Press, 2017: 1578-1585.
14	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 770-778.
15	RAHIMILARKI R, GAO Z W, JIN N L, et al. Convolutional neural network fault classification based on time-series analysis for benchmark wind turbine machine. Renewable Energy, 2022, 185, 916- 931. doi: 10.1016/j.renene.2021.12.056
16	TAN M, LE Q. Efficientnet: rethinking model scaling for convolutional neural networks[C]//Proceedings of International Conference on Machine Learning. Washington D. C., USA: IEEE Press, 2019: 6105-6114.
17	ALEX S A, JHANJHI N Z, HUMAYUN M, et al. Deep LSTM model for diabetes prediction with class balancing by SMOTE. Electronics, 2022, 11(17): 2737. doi: 10.3390/electronics11172737
18	SWANA E F, DOORSAMY W, BOKORO P. Tomek link and SMOTE approaches for machine fault classification with an imbalanced dataset. Sensors, 2022, 22(9): 3246. doi: 10.3390/s22093246
19	PETMEZAS G, CHEIMARIOTIS G A, STEFANOPOULOS L, et al. Automated lung sound classification using a hybrid CNN-LSTM network and focal loss function. Sensors, 2022, 22(3): 1232. doi: 10.3390/s22031232
20	CHEN Y C. A tutorial on kernel density estimation and recent advances. Biostatistics & Epidemiology, 2017, 1(1): 161- 187.
21	LUI W, LI Y, URTASUN R, et al. Understanding the effective receptive field in deep convolutional neural networks[C]//Proceedings of NIPS'17. Cambridge, USA: MIT Press, 2016, 29.
22	HOANG V T, JO K H. Practical analysis on architecture of EfficientNet[C]//Proceedings of the 14th International Conference on Human System Interaction. Washington D. C., USA: IEEE Press, 2021: 1-4.
23	DENG J, DONG W, SOCHER R, et al. ImageNet: a large-scale hierarchical image database[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2009: 248-255.
24	YANG Z, XU B B, LUO W, et al. Autoencoder-based representation learning and its application in intelligent fault diagnosis: a review. Measurement, 2022, 189, 110460. doi: 10.1016/j.measurement.2021.110460
25	LING C X, HUANG J, ZHANG H. AUC: a better measure than accuracy in comparing learning algorithms. Berlin, Germany: Springer, 2003.
26	CHEN T Q, GUESTRIN C. XGBoost: a scalable tree boosting system[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM Press, 2016: 785-794.
27	常婷婷, 翟江涛, 戴跃伟. 一种基于Xgboost的Skype时间式隐信道检测方法. 计算机工程, 2021, 47(7): 88- 94. URL
	CHANG T T, ZHAI J T, DAI Y W. An Xgboost-based method for detecting covert timing channel of Skype. Computer Engineering, 2021, 47(7): 88- 94. URL
28	IBRAHEM AHMED OSMAN A, NAJAH AHMED A, CHOW M F, et al. Extreme gradient boosting (Xgboost) model to predict the groundwater levels in Selangor Malaysia. Ain Shams Engineering Journal, 2021, 12(2): 1545- 1556. doi: 10.1016/j.asej.2020.11.011
29	AKIBA T, SANO S, YANASE T, et al. Optuna: a next-generation hyperparameter optimization framework[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, USA: ACM Press, 2019: 2623-2631.

[1]	WANG Zhihao, QIAN Yuntao. Super-Resolution Reconstruction of Spatiotemporal Fusion for Dual-Stream Remote Sensing Images Based on Swin Transformer [J]. Computer Engineering, 2024, 50(9): 33-45.
[2]	GAO Yubao, WEN Zhicheng. Dual Decoder Image Denoising Method Based on Attention Mechanism [J]. Computer Engineering, 2024, 50(9): 324-332.
[3]	ZHANG Lu, TIAN Chunwei, SONG Huansheng, LIU Shigang. Multi-Level Dual-Tree Complex Wavelet Network for Low-Dose CT Image Denoising [J]. Computer Engineering, 2024, 50(9): 266-275.
[4]	Lei WANG, Shipeng DANG, Feng PAN. Model for Predicting Concealed Accessory Pathway Based on Convolutional Neural Network [J]. Computer Engineering, 2024, 50(8): 40-49.
[5]	Lili GENG, Baoning NIU. Convolutional Neural Network Pruning Based on Channel Similarity Entropy [J]. Computer Engineering, 2024, 50(7): 133-143.
[6]	Ruiting NUI, Tianfeng YAN, Rui GAO, Yingzhi WANG. Deep Learning TCNN-MobileNet-Based Modulation Recognition Under Low Signal-to-Noise Radio [J]. Computer Engineering, 2024, 50(7): 204-215.
[7]	Lei ZHANG, Guochen SHEN, Dongxiu OU. Filtering Method for Feature Maps from Convolutional Neural Network for Thermal Imaging Data [J]. Computer Engineering, 2024, 50(4): 31-40.
[8]	ZHANG Lei, SHEN Guochen, OU Dongxiu. Filtering Method for Feature Maps from Convolutional Neural Network for Thermal Imaging Data [J]. Computer Engineering, 2024, 50(4): 31-40.
[9]	Baihao JIANG, Jing LIU, Dawei QIU, Liang JIANG. Review of Deep Learning Applications in Spinal Image Segmentation [J]. Computer Engineering, 2024, 50(3): 1-15.
[10]	Bohan WANG, Xiaoyan JIANG, Liuyi FAN. Semantic Segmentation Improvement Method Based on Deep Supervision for the Construction of Latent Space [J]. Computer Engineering, 2024, 50(3): 191-199.
[11]	Xinlin XIE, Dongxu YIN, Taoyuan ZHANG, Gang XIE. Multiscale Fusion Crowd Counting Algorithm Based on Attention Mechanism [J]. Computer Engineering, 2024, 50(3): 290-297.
[12]	Haochen XU, Manhua LIU. Facial Landmark Detection Based on Hierarchical Self-Attention Network [J]. Computer Engineering, 2024, 50(2): 239-246.
[13]	Qian LI, Weibo WEI, Guangyu YANG, Jintao SONG, Lu SUN, Zhenkuan PAN. Poisson Denoising Variational Model Based on Prior-Driven Deep Neural Network [J]. Computer Engineering, 2024, 50(2): 273-280.
[14]	WANG Yaru, TANG Lu, CHEN Aibin, PENG Weixiong, SHEN Ping. Birdsong Recognition Based on LPDMR-NET [J]. Computer Engineering, 2024, 50(10): 174-184.
[15]	LAN Hong, WANG Huizhao. Combining Lightweight and Multiscale Fusion for Traffic Sign Detection Algorithm [J]. Computer Engineering, 2024, 50(10): 381-392.

Please choose a citation manager

Content to export