不平衡数据集下的数据中心网络流量异常检测

doi:10.19678/j.issn.1000-3428.0069281

摘要/Abstract

摘要：

数据中心作为信息化时代的重要基础设施, 承载着各类关键信息服务。目前, 数据中心是网络攻击的主要攻击目标。为了提高网络安全, 提出数据中心网络流量异常检测方法。研究内容包括特征选择、不平衡数据集分类和异常流量检测。首先, 提出了一种处理不平衡数据集的分类方法, 通过基于集成的特征选择和混合采样算法提高分类性能; 其次, 引入基于随机森林(RF)和LightGBM的流量异常检测方法, 充分利用它们在处理不平衡数据和噪声抵抗方面的优势。在CSE-CIC-IDS2018公开数据集上进行验证, 实验结果表明, 所提方法具有较高的精确率和召回率, 在15种流量类型中有9种类型的分类精确率都高于90%, 并且有13种类型的分类精确率都在74%以上, 对提高数据中心安全、保障服务质量和改善网络流量异常检测具有重要意义。

关键词: 数据中心, 网络流量, 异常检测, 不平衡数据集, 集成学习

Abstract:

As an important infrastructure in the information age, data centers provide all types of key information services. Currently, data centers face high levels of network attacks and are the main targets of network attacks. To improve network security, this study focuses on an anomaly detection method for data center network traffic. This study includes feature selection, dataset distribution balance, and abnormal traffic detection. First, a classification method for imbalanced datasets is proposed, and the classification performance is improved using feature engineering and a mixed sampling algorithm. Second, traffic anomaly detection methods based on Random Forest (RF) and Light Gradient Boosting Machine (LightGBM) are introduced to fully utilize their advantages in processing imbalanced data and noise resistance. The experiment uses the CSE-CIC-IDS2018 public dataset for verification. The results show that the proposed algorithm has a high precision and recall; among the 15 traffic types, the classification precision of 9 types is higher than 90%, and the classification precision of 13 types is higher than 74%. This study is significant for improving data center security, service quality, and network traffic anomaly detection. It not only provides an effective means to address escalating network threats but also makes a positive contribution to the stable operation of data centers and the reliability of information services.

Key words: data center, network traffic, anomaly detection, imbalanced dataset, ensemble learning

王光明, 李冬青, 蒋从锋. 不平衡数据集下的数据中心网络流量异常检测[J]. 计算机工程, 2025, 51(8): 227-237.

WANG Guangming, LI Dongqing, JIANG Congfeng. Network Traffic Anomaly Detection for Data Centers in Imbalanced Datasets[J]. Computer Engineering, 2025, 51(8): 227-237.

https://www.ecice06.com/CN/Y2025/V51/I8/227

图/表 10

图1 不平衡数据集分类方法总体结构

Fig.1 Overall structure of classification methods for imbalanced datasets

图2 基于聚类的自适应混合采样算法流程

Fig.2 Procedure of adaptive hybridsampling algorithm based on clustering

图3 直方图算法示意图

Fig.3 Schematic diagram of histogram algorithm

图4 LightGBM建模流程

Fig.4 Procedure of LightGBM modeling

图5 4种算法在CSE-CIC-IDS2018数据集上的分类性能

Fig.5 The classification performance of the four algorithms on the CSE-CIC-IDS2018 dataset

参考文献 37

1	HOLT T J , BOSSLER A M . The palgrave handbook of international cybercrime and cyberdeviance. Berlin, Germany: Palgrave Macmillan, 2020: 135- 154.
2	ABDELRAHMAN A M , RODRIGUES J J , MAHMOUD M M , et al. Software‐defined networking security for private data center networks and clouds: vulnerabilities, attacks, countermeasures, and solutions. International Journal of Communication Systems, 2021, 34 (4): e4706. doi: 10.1002/dac.4706
3	JIAO J H, YE B J, ZHAO Y, et al. Detecting TCP-based DDoS attacks in Baidu cloud computing data centers[C]//Proceedings of the 36th Symposium on Reliable Distributed Systems (SRDS). Washington D.C., USA: IEEE Press, 2017: 256-258.
4	国家计算机网络应急技术处理协调中心. 2020年中国互联网网络安全报告[EB/OL]. [2023-12-19]. https://www.cert.org.cn/publish/main/46/2021/20210721130944504525772/20210721130944504525772_.html.
	National Computer Network Emergency Response Technical Coordination Center of China. 2020 China Internet Network Security Report[EB/OL][2023-12-19]. https://www.cert.org.cn/publish/main/46/2021/20210721130944504525772/20210721130944504525772_.html. (in Chinese)
5	GAO Y L , LI X Y , PENG H , et al. Hincti: a cyber threat intelligence modeling and identification system based on heterogeneous information network. IEEE Transactions on Knowledge and Data Engineering, 2022, 34 (2): 708- 722. doi: 10.1109/TKDE.2020.2987019
6	SUN N , ZHANG J , RIMBA P , et al. Data-driven cybersecurity incident prediction: a survey. IEEE Communications Surveys & Tutorials, 2019, 21 (2): 1744- 1772.
7	李艳霞, 柴毅, 胡友强, 等. 不平衡数据分类方法综述. 控制与决策, 2019, 34 (4): 673- 688.
	LI Y X , CHAI Y , HU Y Q , et al. Review of imbalanced data classification methods. Control and Decision, 2019, 34 (4): 673- 688.
8	HOU X D , ZHANG T , JI L , et al. Combating highly imbalanced steganalysis with small training samples using feature selection. Journal of Visual Communication and Image Representation, 2017, 49, 243- 256. doi: 10.1016/j.jvcir.2017.09.016
9	VIEGAS F , ROCHA L , GONCALVES M , et al. A genetic programming approach for feature selection in highly dimensional skewed data. Neurocomputing, 2018, 273, 554- 569. doi: 10.1016/j.neucom.2017.08.050
10	ZHOU F , YANG S , FUJITA H , et al. Deep learning fault diagnosis method based on global optimization GAN for unbalanced data. Knowledge-Based Systems, 2020, 187, 104837- 104855. doi: 10.1016/j.knosys.2019.07.008
11	LIN W C , TSAI C F , HU Y H , et al. Clustering-based undersampling in class-imbalanced data. Information Sciences, 2017, 409, 17- 26.
12	CHAWLA N V , BOWYER K W , HALL L O , et al. SMOTE: synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 2002, 16, 321- 357. doi: 10.1613/jair.953
13	BATISTA G E , PRATI R C , MONARD M C . A study of the behavior of several methods for balancing machine learning training data. ACM SIGKDD Explorations Newsletter, 2004, 6 (1): 20- 29. doi: 10.1145/1007730.1007735
14	FUQUA D , RAZZAGHI T . A cost-sensitive convolution neural network learning for control chart pattern recognition. Expert Systems with Applications, 2020, 150, 113275. doi: 10.1016/j.eswa.2020.113275
15	YIN S , ZHU X P , JING C . Fault detection based on a robust one class support vector machine. Neurocomputing, 2014, 145, 263- 268. doi: 10.1016/j.neucom.2014.05.035
16	FENG W , HUANG W J , REN J C . Class imbalance ensemble learning based on the margin theory. Applied Sciences, 2018, 8 (5): 815. doi: 10.3390/app8050815
17	徐玉华, 孙知信. 软件定义网络中的异常流量检测研究进展. 软件学报, 2020, 31 (1): 183- 207.
	XU Y H , SUN Z X . Research development of abnormal traffic detection in software defined networks. Journal of Software, 2020, 31 (1): 183- 207.
18	YANG C . Anomaly network traffic detection algorithm based on information entropy measurement under the cloud computing environment. Cluster Computing, 2019, 22 (1): 8309- 8317.
19	HOANG D H, NGUYEN H D. A PCA-based method for IoT network traffic anomaly detection[C]//Proceedings of the 20th International Conference on Advanced Communication Technology. Washington D. C., USA: IEEE Press, 2018: 381-386.
20	程云观, 台宪青, 马治杰. 一种云环境下的高效异常检测策略研究. 计算机应用与软件, 2020, 37 (1): 326- 333.
	CHENG Y G , TAI X Q , MA Z J . An efficient anomaly detection strategy in cloud environments. Computer Applications and Software, 2020, 37 (1): 326- 333.
21	PEREZ-BUENO F , GARCIA L , MACIA-FERNANDEZ G , et al. Leveraging a probabilistic PCA model to understand the multivariate statistical network monitoring framework for network security anomaly detection. IEEE/ACM Transactions on Networking, 2022, 30 (3): 1217- 1229. doi: 10.1109/TNET.2021.3138536
22	杜臻, 马立鹏, 孙国梓, 等. 一种基于小波分析的网络流量异常检测方法. 计算机科学, 2019, 46 (8): 178- 182.
	DU Z , MA L P , SUN G Z , et al. Network traffic anomaly detection method based on wavelet analysis. Computer Science, 2019, 46 (8): 178- 182.
23	LAVROVA D S , ALEKSEEV I V , SHTYRKINA A A . Security analysis based on controlling dependences of network traffic parameters by wavelet transformation. Automatic Control and Computer Sciences, 2018, 52 (8): 931- 935. doi: 10.3103/S0146411618080187
24	GOZDE K , ONDER D , KORAY S O . Increasing the performance of machine learning-based IDSs on an imbalanced and up-to-date dataset. IEEE Access, 2020, 8, 32150- 32162. doi: 10.1109/ACCESS.2020.2973219
25	GAO X W , SHAN C , HU C Z , et al. An adaptive ensemble machine learning model for intrusion detection. IEEE Access, 2019, 7, 82512- 82521. doi: 10.1109/ACCESS.2019.2923640
26	ZHANG H , LI Y D , LÜ Z H , et al. A real-time and ubiquitous network attack detection based on deep belief network and support vector machine. IEEE/CAA Journal of Automatica Sinica, 2020, 7 (3): 790- 799. doi: 10.1109/JAS.2020.1003099
27	WANG Z H , FOK K W , THING V L L . Machine learning for encrypted malicious traffic detection: approaches, datasets and comparative study. Computers & Security, 2022, 113, 102542.
28	ZHANG P , HE F , ZHANG H , et al. Real-time malicious traffic detection with online isolation forest over SD-WAN. IEEE Transactions on Information Forensics and Security, 2023, 18, 2076- 2090. doi: 10.1109/TIFS.2023.3262121
29	FERRAG M A , MAGLARAS L . DeliveryCoin: an IDS and blockchain-based delivery framework for drone-delivered services. Computers, 2019, 8 (3): 58- 73. doi: 10.3390/computers8030058
30	LU X L, LIU P J, LIN J Y. Network traffic anomaly detection based on information gain and deep learning[C]//Proceedings of the 3rd International Conference on Information System and Data Mining. New York, USA: ACM Press, 2019: 11-15.
31	ZHANG Y , CHEN X , JIN L , et al. Network intrusion detection: based on deep hierarchical network and original flow data. IEEE Access, 2019, 7, 37004- 37016. doi: 10.1109/ACCESS.2019.2905041
32	LIN P, YE K J, XU C Z. Dynamic network anomaly detection system by using deep learning techniques[C]//Proceedings of International Conference on Cloud Computing. Berlin, Germany: Springer, 2019: 161-176.
33	AHMAD R , ALSMADI I , ALHAMDANI W , et al. A deep learning ensemble approach to detecting unknown network attacks. Journal of Information Security and Applications, 2022, 67, 103196.
34	LIU F R, LI X F, XIONG W, et al. An accuracy network anomaly detection method based on ensemble model[C]//Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Washington D. C., USA: IEEE Press, 2021: 125-131.
35	HE H B, BAI Y, GARCIA E A, et al. ADASYN: adaptive synthetic sampling approach for imbalanced learning[C]//Proceedings of IEEE International Joint Conference on Neural Networks. Washington D. C., USA: IEEE Press, 2008: 1322-1328.
36	ISHWARAN H , LU M . Standard errors and confidence intervals for variable importance in random forest regression, classification, and survival. Statistics in Medicine, 2019, 38 (4): 558- 582.
37	University of New Brunswick. CSE-CIC-IDS2018 on AWS[EB/OL]. [2023-12-19]. https://www.unb.ca/cic/datasets/ids-2018.html.

[1]	莫定涛, 俱莹, 李文进, 张亚生, 何辞, 董飞虎. 面向卫星网络的流量分类方法研究[J]. 计算机工程, 2025, 51(5): 9-19.
[2]	张安勤, 丁志锋. 融合动态图嵌入和Transformer自编码器的网络异常检测[J]. 计算机工程, 2025, 51(4): 47-56.
[3]	孟凡丰, 王子聪, 张金涛, 王彦景, 欧洋, 吴利舟, 肖侬. 基于gem5的CXL内存池系统设计与实现[J]. 计算机工程, 2025, 51(3): 180-188.
[4]	周炫余, 吴莲华, 郑勤华, 肖天星, 王紫璇, 张思敏. 联合语义提示和记忆增强的弱监督跳绳视频异常检测方法[J]. 计算机工程, 2024, 50(7): 87-95.
[5]	李永飞, 李铭洋, 常鑫, 曹可欣. 基于可解释性深度学习的物联网水质监测数据异常检测[J]. 计算机工程, 2024, 50(6): 179-187.
[6]	魏德宾, 杨力, 潘成胜, 沈婷. 基于流量自相似性的网络队列管理算法[J]. 计算机工程, 2024, 50(5): 306-312.
[7]	王俊, 赖会霞, 万玥, 张仕. 基于角度的图神经网络高维数据异常检测方法[J]. 计算机工程, 2024, 50(3): 156-165.
[8]	赵博超, 马嘉骏, 崔磊, 栾文鹏, 朱静. 基于改进VMD-XGBoost-BiLSTM组合模型的光伏发电异常检测[J]. 计算机工程, 2024, 50(3): 306-316.
[9]	单永航, 张希, 胡川, 丁涛军, 姚远. 基于集成学习的交通事故严重程度预测研究与应用[J]. 计算机工程, 2024, 50(2): 33-42.
[10]	宋航, 周凤, 熊伟. 基于自相关-变分对抗学习的物理系统异常检测[J]. 计算机工程, 2024, 50(12): 358-366.
[11]	曲坤, 王震龙, 刘志锋. 基于多表征学习的交叉熵集成图像分类方法[J]. 计算机工程, 2024, 50(10): 322-333.
[12]	包善书, 车波, 邓林红. 基于双源域迁移学习的肺音信号识别[J]. 计算机工程, 2023, 49(9): 295-302, 312.
[13]	陈仲磊, 伊鹏, 陈祥, 胡涛. 基于集成学习的系统调用实时异常检测框架[J]. 计算机工程, 2023, 49(6): 162-169,179.
[14]	王爱玲, 马文臻, 邹自明, 钟佳. 基于领域自适应的卫星工程参数异常检测[J]. 计算机工程, 2023, 49(5): 29-37,47.
[15]	张子宣, 宗学军, 何戡, 连莲. 基于CVAE-CatBoost的工业控制网络异常流量检测研究[J]. 计算机工程, 2023, 49(5): 173-180.

选择文件类型/文献管理软件名称

选择包含的内容