网络流量异常检测中的维数约简研究

doi:10.19678/j.issn.1000-3428.0056532

计算机工程 ›› 2020, Vol. 46 ›› Issue (2): 11-20. doi: 10.19678/j.issn.1000-3428.0056532

网络流量异常检测中的维数约简研究

陈良臣^1,2,3, 高曙¹, 刘宝旭², 陶明峰⁴

1. 武汉理工大学计算机科学与技术学院, 武汉 430063;
2. 中国科学院信息工程研究所, 北京 100049;
3. 中国劳动关系学院应用技术学院, 北京 100048;
4. 国网山东省电力公司淄博供电公司, 山东淄博 255000

收稿日期:2019-11-07 修回日期:2019-12-12 发布日期:2019-12-20
作者简介:陈良臣(1982-),男,讲师、博士研究生,主研方向为大数据、网络攻防技术、安全态势感知;高曙,教授、博士、博士生导师;刘宝旭(通信作者),研究员、博士、博士生导师;陶明峰,教授级高级工程师。
基金资助:
国家自然科学基金（61802404，61602470）；国家信息安全专项（发改办高技[2015]289号）；中国科学院战略性先导C类课题（XDC020400100）；中国劳动关系学院科研项目（20XYJS003，20ZYJS017）；北京市科委重点研究项目（D181100000618003）；中国科学院网络测评技术重点实验室基金；网络安全防护技术北京市重点实验室基金。

Research on Dimensionality Reduction in Network Traffic Anomaly Detection

CHEN Liangchen^1,2,3, GAO Shu¹, LIU Baoxu², TAO Mingfeng⁴

1. School of Computer Science and Technology, Wuhan University of Technology, Wuhan 430063, China;
2. Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100049, China;
3. School of Application Technology, China University of Labor Relations, Beijing 100048, China;
4. Zibo Power Supply Company, State Grid Shandong Electric Power Company, Zibo, Shandong 255000, China

Received:2019-11-07 Revised:2019-12-12 Published:2019-12-20

摘要/Abstract

摘要： 对包含大流量数据的高维度网络进行异常检测，必须加入维数约简处理以减轻系统在传输和存储方面的压力。介绍高速网络环境下网络流量异常检测过程以及维数约简方式，阐述流量数据常用特征和维数约简技术研究的最新进展。针对网络流量特征选择和流量特征提取2种特征降维方式，对现有算法进行归纳分类，分别描述算法原理及优缺点。此外，给出维数约简常用的数据集和评价指标，分析网络流量异常检测中维数约简技术研究面临的挑战，并对未来发展方向进行展望。

关键词: 网络异常检测, 流量维数约简, 流量特征提取, 流量特征选择, 网络空间安全

Abstract: To implement anomaly detection for a high dimensional network with mass flow data,data dimensionality should be reduced to relieve transmission and storage burdens from the system.This paper introduces network traffic anomaly detection process and dimensionality reduction ways in hig-speed network environment.Then it summarizes common features of feature in network traffic anomaly detection and latest research developments of dimensionality reduction for traffic data.Aiming at two kinds of feature dimensionality reduction ways,network traffic feature selection and network traffic feature extraction,this paper lists and classifies frequently used algorithms and describes the principles,advantages and disadvantages respectively.On this basis,this paper analyzes existing datasets and evaluation indexes used in research of dimensionality reduction.Finally,this paper discusses development directions and challenges of dimensionality reduction technologies in network traffic anomaly detection.

Key words: network anomaly detection, traffic dimensionality reduction, traffic feature extraction, traffic feature selection, cyberspace security

中图分类号:

TP301

陈良臣, 高曙, 刘宝旭, 陶明峰. 网络流量异常检测中的维数约简研究[J]. 计算机工程, 2020, 46(2): 11-20.

CHEN Liangchen, GAO Shu, LIU Baoxu, TAO Mingfeng. Research on Dimensionality Reduction in Network Traffic Anomaly Detection[J]. Computer Engineering, 2020, 46(2): 11-20.

http://www.ecice06.com/CN/Y2020/V46/I2/11

图/表 8

20200218140531

20200218140535

20200218140537

20200218140541

20200218140544

20200218140547

20200218140551

20200218140554

参考文献

[1] JIN Hai.Report on the development of China’s cyberspace security cutting edge science and technology in 2018[M].Beijing:Posts and Telecom Press,2019.(in Chinese) 金海.2018年中国网络空间安全前沿科技发展报告[M].北京:人民邮电出版社,2019.
[2] CHEN Liangchen,GAO Shu,LIU Baoxu,et al.Research status and development trends on network encrypted traffic identification[J].Netinfo Security,2019,19(3):19-25.(in Chinese) 陈良臣,高曙,刘宝旭,等.网络加密流量识别研究进展及发展趋势[J].信息网络安全,2019,19(3):19-25.
[3] CAO Jie.Research of feature reduction and traffic classification method based on SVM[D].Changchun:Jilin University,2017.(in Chinese) 曹杰.基于SVM的网络流量特征降维与分类方法研究[D].长春:吉林大学,2017.
[4] NI X,HE D,CHAN S,et al.Network anomaly detection using unsupervised feature selection and density peak clustering[C]//Proceedings of International Conference on Applied Cryptography and Network Security.Berlin,Germany:Springer,2016:212-227.
[5] NIU Weina.Research on exfiltration complex network attack modeling and identification method[D].Chengdu:University of Electronic Science and Technology in China,2018.(in Chinese) 牛伟纳.窃密型复杂网络攻击建模与识别方法研究[D].成都:电子科技大学,2018.
[6] CHEN Liangchen,LIU Baoxu,GAO Shu.Research on traffic data sampling technology in network attack detection[J].Netinfo Security,2019,19(8):22-28.(in Chinese) 陈良臣,刘宝旭,高曙.网络攻击检测中流量数据抽样技术研究[J].信息网络安全,2019,19(8):22-28.
[7] LUO Ling.Anomaly detection of backbone network based on dimensionality reduction[D].Hefei:University of Science and Technology of China,2015.(in Chinese) 罗玲.基于降维的骨干网流量异常检测研究[D].合肥:中国科学技术大学,2015.
[8] WEI Zekun,XIA Jingbo,ZHANG Xiaoyan,et al.Research on traffic multi-feature extraction and classification based on random forest[J].Transducer and Microsystem Technologies,2016,35(12):55-59.(in Chinese) 韦泽鲲, 夏靖波,张晓燕,等.基于随机森林的流量多特征提取与分类研究[J].传感器与微系统,2016,35(12):55-59.
[9] ZHOU Ya.Research on the network traffic identification technology based on semi-supervised learning[D].Nanjing:Southeast University,2017.(in Chinese) 周雅.基于半监督学习的网络业务流量识别方法研究[D].南京:东南大学,2017.
[10] HUANG Yinxiang.Research on feature engineering in Internet traffic classification[D].Nanjing:Nanjing University of Posts and Telecommunications,2017.(in Chinese) 黄引翔.网络流量分类中特征工程的研究[D].南京:南京邮电大学,2017.
[11] ANUSHA K,SATHIYAMOORTHY E.Comparative study for feature selection algorithms in intrusion detection system[J].Automatic Control and Computer Sciences,2016,50(1):1-9.
[12] KWON D,KIM H,KIM J,et al.A survey of deep learning-based network anomaly detection[J].Cluster Computing,2019,22:949-961.
[13] YIN Xiu.Network intrusion detection based on improved dictionary learning[D].Nanjing:Nanjing University of Posts and Telecommunications,2018.(in Chinese) 尹秀.基于改进的字典学习的网络入侵检测方法研究[D].南京:南京邮电大学,2018.
[14] YAO Yepeng,SU Liya,LU Zhigang.DeepGFL:deep feature learning via graph for attack detection on flow-based network traffic[C]//Proceedings of IEEE Military Communications Conference.Washington D.C.,USA:IEEE Press,2018:29-31.
[15] TANG Jian,SUN Chunlai,MAO Kefeng,et al.Network intrusion anomaly detection model based on dimension reduction strategy using principal component analysis and mutual information[J].Netinfo Security,2015,15(9):78-83.(in Chinese) 汤健,孙春来,毛克峰,等.基于主元分析和互信息维数约简策略的网络入侵异常检测[J].信息网络安全,2015,15(9):78-83.
[16] SHRIVAS A K,DEWANGAN A K.An ensemble model for classification of attacks with feature selection based on KDD99 and NSL-KDD dataset[J].International Journal of Computer Applications,2014,99:8-13.
[17] YE Kai.Key feature recognition algorithm of network intrusion signal based on neural network and support vector machine[J].Symmetry,2019,11(3):380.
[18] PALECHOR F M,HOZMANOTAS A,HOZFRANCO E,et al.Feature selection,learning metrics and dimension reduction in training and classification processes in intrusion detection systems[J].Journal of Theoretical and Applied Information Technology,2015,82(2):291-298.
[19] GUPTA J,SINGH J.Detecting anomaly based network intrusion using feature extraction and classification techniques[J].International Journal of Advanced Research in Computer Science,2017,8(5):1-5.
[20] ABUROMMAN A A,REAZ M B I.Ensemble of binary SVM classifiers based on PCA and LDA feature extraction for intrusion detection[C]//Proceedings of IMCEC’16.Washington D.C.,USA:IEEE Press,2016:636-640.
[21] LIU Zhan,JIE Ling,PENG Lin.An industrial control intrusion detection method combining semi-supervised LDA and PSO-SVM[C]//Proceedings of CNCI’19.[S.l.]:Atlantis Press,2019:1-6.
[22] NATESAN P,RAJALAXMI R R,GOWRISON G,et al.Hadoop based parallel binary bat algorithm for network intrusion detection[J].International Journal of Parallel Programming,2017,45(5):1194-1213.
[23] TANG Jian,ZHUO Liu,JIA Meiying,et al.Supervised nonlinear latent feature extraction and regularized random weights neural network modeling for intrusion detection system[C]//Proceedings of International Conference on Cloud Computing and Security.Berlin,Germany:Springer,2016:343-354.
[24] SUGANYA S,MUTHUMARI G,BALASUBRAMANIAN C.Propitiating behavioral variability for mouse dynamics using dimensionality reduction based approach[C]//Proceedings of ICCTIDE’16.Washington D.C.,USA:IEEE Press,2016:1-6.
[25] KEERTHIVASAN K,SURENDIRAN B.Dimensionality reduction using principal component analysis for network intrusion detection[J].Perspectives in Science,2016(8):510-512.
[26] ABDULHAMMED R,MUSAFER H.Features dimensionality reduction approaches for machine learning based network intrusion detection[J].Electronics,2019,8(3):322-232.
[27] TSANG C H,KWONG S.Multi-agent intrusion detection system in industrial network using ant colony clustering approach and unsupervised feature extraction[C]//Proceedings of IEEE International Conference on Industrial Technology.Washington D.C.,USA:IEEE Press,2005:51-56.
[28] PALMIERI F,FIORE U,CASTIGLIONE A.A distributed approach to network anomaly detection based on independent component analysis[J].Concurrency and Computation:Practice and Experience,2015,26(5):1113-1129.
[29] YANG Qing,JIA Cangzhi,LI Taoying.Prediction of aptamer-protein interacting pairs based on sparse autoencoder feature extraction and an ensemble classifier[J].Mathematical Biosciences,2019(311):103-108.
[30] YU Lei,LIU Huan.Feature selection for high-dimensional data:a fast correlation-based filter solution[C]//Proceedings of ICML’03.Washington D.C.,USA:[s.n.],2003:856-863.
[31] SUN Xingbin,RUI Yun.Feature selection method based on statistic frequency in network traffic classification[J].Journal of Chinese Computer Systems,2016,37(11):2483-2487.(in Chinese) 孙兴斌,芮赟.一种基于统计频率的网络流量特征选择方法[J].小型微型计算机系统,2016,37(11):2483-2487.
[32] LI Zhanshan,LIU Zhaogeng.Feature selection algorithm based on XGBoost[J].Journal on Communications,2019,40(10):101-108.(in Chinese) 李占山,刘兆赓.基于XGBoost的特征选择算法[J].通信学报,2019,40(10):101-108.
[33] WANG Wei,HE Yongzhong,LIU Jiqiang,et al.Constructing important features from massive network traffic for lightweight intrusion detection[J].Constructing IET Information Security,2015,9(6):374-379.
[34] SELVAKUMAR B,MUNEESWARAN K.Firefly algorithm based feature selection for network intrusion detection[J].Computers and Security,2019,81(3):148-155.
[35] AMBUSAIDI M A,HE X J,NANDA P,et al.Building an intrusion detection system using a filter-based feature selection algorithm[J].IEEE Transactions on Computers,2016,65(10):2986-2998.
[36] WANG Jie,XU Jing,ZHAO Cheng’an,et al.An ensemble feature selection method for high-dimensional data based on sort aggregation[J].Systems Science and Control Engineering,2019,7(2):32-39.
[37] BALKANLI E,ZINCIR-HEYWOOD A N,HEYWOOD M I.Feature selection for robust backscatter DDoS detection[C]//Proceedings of the 40th Local Computer Networks Conference.Washington D.C.,USA:IEEE Press,2015:611-618.
[38] SHADI A,MONTHER A.Anomaly-based intrusion detection system through feature selection analysis and building hybrid efficient model[J].Journal of Computational Science,2018,25,152-160.
[39] OSANAIYE O,CHOO K K R,DLODLO M.Analysing feature selection and classification techniques for DDoS detection in cloud[C]//Proceedings of Southern Africa Telecommunication Networks and Applications Conference.Fancourt,South Africa:[s.n.],2016:198-203.
[40] HUANG T,CHEN W,ZHANG R.A combined feature selection method based on clustering in intrusion detection[C]//Proceedings of the 2nd International Conference on Automation,Mechanical Control and Computational Engineering.[S.l.]:Atlantis Press,2017:2352-5401.
[41] KHOR K C,TING C Y,AMNUAISUK S P.A feature selection approach for network intrusion detection[C]//Proceedings of the 11th International Conference on Information Management and Engineering.London,UK:[s.n.],2019:133-140.
[42] BENAICHA S E,SAOUDI L,GUERMECHE S E B,et al.Intrusion detection system using genetic algorithm[C]//Proceedings of Science and Information Conference.London,UK:[s.n.],2014:564-568.
[43] VIJAYAN R,DEVARAJ D,KANNAPIRAN B.Intrusion detection system for wireless mesh network using multiple support vector machine classifiers with genetic-algorithm-based feature selection[J].Computers and Security,2018,77,304-314.
[44] EL-ALFY E S M,AL-OBEIDAT F N.A multicriterion fuzzy classification method with greedy attribute selection for anomaly-based intrusion detection[J].Procedia Computer Science,2014,34:55-62.
[45] BAHL S,SHARMA S K.A minimal subset of features using correlation feature selection model for intrusion detection system[C]//Proceedings of the 2nd International Conference on Computer and Communication Technologies.Berlin,Germany:Springer,2016:337-346.
[46] EESA A S,ORMAN Z,BRIFCANI A M A.A novel feature selection approach based on the cuttlefish optimization algorithm for intrusion detection systems[J].Expert System with Applications,2015,42(5):2670-2679.
[47] KAMBATTAN K R,MANIMEGALAI R,GANAPATHY S.An increment feature selection approach for intrusion detection system in MANET[J].International Journal for Research in Applied Science and Engineering Technology,2017,5(1):325-329.
[48] MALIK A J,SHAHZAD W,KHAN F A.Network intrusion detection using hybrid binary PSO and random forests algorithm[J].Security and Communication Networks,2015,8(16):2646-2660.
[49] USTEBAY S,TURGUT Z,AYDIN M A.Intrusion detection system with recursive feature elimination by using random forest and deep learning classifier[C]//Proceedings of International Congress on Big Data,Deep Learning and Fighting Cyber Terrorism.Washington D.C.,USA:IEEE Press,2018:71-76.
[50] SHAH R,QIAN Y,KUMAR D,et al.Network intrusion detection through discriminative feature selection by using sparse logistic regression[J].Future Internet,2017,9(4):81.
[51] GHOSH P,MITRA R.Proposed GA-BFSS and logistic regression based intrusion detection system[C]//Proceedings of the 3rd International Conference on Computer,Communication,Control and Information Technology.Washington D.C.,USA:IEEE Press,2015:1-6.
[52] ZHENG Huiting,YUAN Jiabin,CHEN Long.Short-term load forecasting using EMD-LSTM neural networks with an Xgboost algorithm for feature importance evaluation[J].Energies,2017,10(8):1168-1176.
[53] HASAN M,NASSER M.Feature selection for intrusion detection using random forest[J].Journal of Information Security,2016(7):129-140.
[54] SHARAFALDIN I,LASHKARI A H,GHORBAN A A.Toward generating a new intrusion detection dataset and intrusion traffic characterization[C]//Proceedings of the 4th International Conference on Information Systems Security and Privacy.Funchal,Portugal:[s.n.],2018:22-24.
[55] ALDWAIRI M,KHAMAYSEH Y,AL-MASRI M.Application of artificial beecolony for intrusion detection systems[J].Security and Communication Networks,2015,8(16):2730-2740.
[56] POPOOLA E,ADEWUMI A.Efficient feature selection technique for network intrusion detection system using discrete differential evolution and decision tree[J].International Journal of Network Security,2017,19(5):660-669.
[57] RAO H D,SHI X Z,RODRIGU A K.Feature selection based on artificial bee colony and gradient boosting decision tree[J].Applied Soft Computing,2019(74):634-642.
[58] CHEBROLU S,ABRAHAM A,THOMAS J P.Feature deduction and ensemble design of intrusion detection systems[J].Computers and Security,2005,24(4): 295-307.

选择文件类型/文献管理软件名称

选择包含的内容

网络流量异常检测中的维数约简研究

Research on Dimensionality Reduction in Network Traffic Anomaly Detection

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献

相关文章 4

编辑推荐

Metrics

本文评价

[1]	张海军, 陈映辉. 基于类图像处理与向量化的大数据脚本攻击智能检测[J]. 计算机工程, 2020, 46(3): 129-137,143.
[2]	李向军,张华薇,郑思维,霍艳丽,张新萍. 基于相对邻域熵的直推式网络异常检测算法[J]. 计算机工程, 2015, 41(8): 132-139.
[3]	黄强盛;程久军;康钦马. 基于高阶AR模型的网络异常检测[J]. 计算机工程, 2010, 36(3): 174-176.
[4]	宿娇娜;李程;李巍;唐发根;李云春. 基于改进NB分类方法的网络异常检测模型[J]. 计算机工程, 2008, 34(5): 148-149,.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

网络流量异常检测中的维数约简研究

Research on Dimensionality Reduction in Network Traffic Anomaly Detection

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献

相关文章 4

编辑推荐

Metrics

本文评价