Encrypted Malicious Traffic Detection Based on Stacking and Multi-Feature Fusion

doi:10.19678/j.issn.1000-3428.0064805

Abstract

Abstract: Although encryption technology protects network communications，plenty malware uses encryption protocols to hide malicious behavior.For the existing Transport Layer Security（TLS） encrypted malicious traffic detection techniques based on machine learning，a single model detection algorithm is available for multi-granularity features，poor applicability，and a high false alarm rate of mixed traffic detection problems.A non-decryption TLS-encrypted malicious traffic detection method based on Stacking strategy and multi-feature fusion is proposed.The multigranularity of encrypted malicious traffic features is analyzed to extract the flow features，connection features，and TLS handshake features of the traffic.The extracted features are statutorily processed using feature engineering to reduce computational overhead.The Random Forest（RF），XGBoost，and Gaussian Naive Bayesian（GNB） classifier models are built for the three classes of features after statute processing to learn the hidden patterns inside them.Using the multidimensional features processed via stream fingerprint fusion，three classifier models are combined using a Stacking strategy to form DMMFC detection model to identify TLS-encrypted malicious traffic in the network.The performance of the constructed model is evaluated on the CTU-13 public dataset.The experimental results show that the identification recall of the proposed method is dimensionality of 99.93% in binary classification experiments and a False Alarm Rate（FAR） is less than 0.10% in malicious traffic detection.In can effectively detect non-decrypted TLS encrypted malicious traffic.

Key words: encrypted malicious traffic, Transport Layer Security（TLS）protocol, Stacking strategy, feature dimensionality reduction, muti-feature fusion

摘要： 加密技术保护网络通信安全的同时，大量恶意软件也采用加密协议来隐藏其恶意行为。在现有基于机器学习的TLS加密恶意流量检测模型中，存在单模型检测算法对多粒度特征适用性差和混合流量检测误报率高的问题。提出基于Stacking策略和多特征融合的非解密TLS加密恶意流量检测方法。分析加密恶意流量特征多粒度的特点，提取流量的流特征、连接特征和TLS握手特征。对所提取的特征通过特征工程进行规约处理，从而减少计算开销。对规约处理后的3类特征分别建立随机森林、XGBoost和高斯朴素贝叶斯分类器模型学习隐藏在流量内部的规律。在此基础上，使用流指纹融合处理后的多维特征，利用Stacking策略组合3个分类器，构成DMMFC检测模型来识别网络中的TLS加密恶意流量。基于CTU-13公开数据集对构建的模型进行性能评估，实验结果表明，该方法在二分类实验上识别召回率高达99.93%，恶意流量检测的误报率低于0.10%，能够有效检测非解密的TLS加密恶意流量。

关键词: 加密恶意流量, TLS协议, Stacking策略, 特征降维, 多特征融合

CLC Number:

TP393

HUO Yuehua, ZHAO Faqi. Encrypted Malicious Traffic Detection Based on Stacking and Multi-Feature Fusion[J]. Computer Engineering, 2023, 49(5): 165-172,180.

霍跃华, 赵法起. 基于Stacking与多特征融合的加密恶意流量检测[J]. 计算机工程, 2023, 49(5): 165-172,180.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0064805

http://www.ecice06.com/EN/Y2023/V49/I5/165

Figures/Tables 12

References

[1] DAI R,GAO C,LANG B,et al.SSL malicious traffic detection based on multi-view features[C]//Proceedings of the 9th International Conference on Communication and Network Security.New York,USA:ACM Press,2019:40-46.
[2] GALLAGHER S.Nearly half of malware now use TLS to conceal communications[EB/OL].(2021-04-21)[2022-04-20].https://news.sophos.com/en-us/2021/04/21/nearly-half-of-malware-now-use-tls-to-conceal-communications/.
[3] 潘吴斌,程光,郭晓军,等.网络加密流量识别研究综述及展望[J].通信学报,2016,37(9):154-167. PAN W B,CHENG G,GUO X J,et al.Review and perspective on encrypted traffic identification research[J].Journal on Communications,2016,37(9):154-167.(in Chinese)
[4] 骆子铭,许书彬,刘晓东.基于机器学习的TLS恶意加密流量检测方案[J].网络与信息安全学报,2020,6(1):77-83. LUO Z M,XU S B,LIU X D.Scheme for identifying malware traffic with TLS data based on machine learning[J].Chinese Journal of Network and Information Security,2020,6(1):77-83.(in Chinese)
[5] FERRAG M A,MAGLARAS L,MOSCHOYIANNIS S,et al.Deep learning for cyber security intrusion detection:approaches,datasets,and comparative study[J].Journal of Information Security and Applications,2020,50:102419.
[6] REZAEI S,LIU X.Deep learning for encrypted traffic classification:an overview[J].IEEE Communications Magazine,2019,57(5):76-81.
[7] SHEN M,WEI M W,ZHU L H,et al.Classification of encrypted traffic with second-order Markov chains and application attribute bigrams[J].IEEE Transactions on Information Forensics and Security,2017,12(8):1830-1843.
[8] DE LUCIA M J,COTTON C.Detection of encrypted malicious network traffic using machine learning[C]//Proceedings of IEEE Military Communications Conference.Washington D.C.,USA:IEEE Press,2019:1-6.
[9] YU K F,HARANG R E.Machine learning in malware traffic classifications[C]//Proceedings of IEEE Military Communications Conference.Washington D.C.,USA:IEEE Press,2017:6-10.
[10] 蒋彤彤,尹魏昕,蔡冰,等.基于层次时空特征与多头注意力的恶意加密流量识别[J].计算机工程,2021,47(7):101-108. JIANG T T,YIN W X,CAI B,et al.Encrypted malicious traffic identification based on hierarchical spatiotemporal feature and multi-head attention[J].Computer Engineering,2021,47(7):101-108.(in Chinese)
[11] LIU C,HE L,XIONG G,et al.FS-net:a flow sequence network for encrypted traffic classification[C]//Proceedings of IEEE Conference on Computer Communications.Washington D.C.,USA:IEEE Press,2019:1171-1179.
[12] YANG H,HE Q,LIU Z,et al.Malicious encryption traffic detection based on NLP[J].Security and Communication Networks,2021,2021:13-22.
[13] ANDERSON B,PAUL S,MCGREW D.Deciphering malware's use of TLS(without decryption)[J].Journal of Computer Virology and Hacking Techniques,2018,14(3):195-211.
[14] CHEN Y C,LI Y J,TSENG A,et al.Deep learning for malicious flow detection[C]//Proceedings of the 28th Annual International Symposium on Personal,Indoor,and Mobile Radio Communications.Washington D.C.,USA:IEEE Press,2017:1-7.
[15] BARUT O,ZHU R,LUO Y,et al.TLS encrypted application classification using machine learning with flow feature engineering[C]//Proceedings of the 10th International Conference on Communication and Network Security.New York,USA:ACM Press,2020:32-41.
[16] 胡斌,周志洪,姚立红,等.结合报文负载与流指纹特征的恶意流量检测[J].计算机工程,2020,46(11):157-163. HU B,ZHOU Z H,YAO L H,et al.Malicious traffic detection combining features of packet payload and stream fingerprint[J].Computer Engineering,2020,46(11):157-163.(in Chinese)
[17] GARCIA S,GRILL M,STIBOREK J,et al.An empirical comparison of botnet detection methods[J].Computers & Security,2014,45:100-123.
[18] TORROLEDO I,CAMACHO L D,BAHNSEN A C.Hunting malicious TLS certificates with deep neural networks[C]//Proceedings of the 11th ACM Workshop on Artificial Intelligence and Security.New York,USA:ACM Press,2018:64-73.
[19] PAI K C,MITRA S,CHARI S M.Novel TLS signature extraction for malware detection[C]//Proceedings of IEEE International Conference on Electronics,Computing and Communication Technologies.Washington D.C.,USA:IEEE Press,2020:1-3.
[20] PAPADOGIANNAKI E,IOANNIDIS S.A survey on encrypted network traffic analysis applications,techniques,and countermeasures[J].ACM Computing Surveys,2022,54(6):123.
[21] ANDERSON B,MCGREW D.Machine learning for encrypted malware traffic classification:accounting for noisy labels and non-stationarity[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.New York,USA:ACM Press,2017:1723-1732.
[22] GUO J,SANG Y,CHANG P,et al.MGEL:a robust malware encrypted traffic detection method based on ensemble learning with multi-grained features[C]//Proceeding of the 21st International Conference on Computational Science.Berlin,Germany:Springer,2021:195-208.
[23] 李小剑,谢晓尧,徐洋,等.基于CNN-SIndRNN的恶意TLS流量快速识别方法[J].计算机工程,2022,48(4):148-157,164. LI X J,XIE X Y,XU Y,et al.Fast identification method of malicious TLS traffic based on CNN-SIndRNN[J].Computer Engineering,2022,48(4):148-157,164.(in Chinese)
[24] YU B,FANG Y,YANG Q,et al.A survey of malware behavior description and analysis[J].Frontiers of Information Technology & Electronic Engineering,2018,19(5):583-603.
[25] 李慧慧,张士庚,宋虹,等.结合多特征识别的恶意加密流量检测方法[J].信息安全学报,2021,6(2):129-142. LI H H,ZHANG S G,SONG H,et al.Robust malicious encrypted traffic detection based with multiple features[J].Journal of Cyber Security,2021,6(2):129-142.(in Chinese)
[26] ANDERSON B,MCGREW D.Identifying encrypted malware traffic with contextual flow data[C]//Proceedings of ACM Workshop on Artificial Intelligence and Security.New York,USA:ACM Press,2016:35-46.
[27] 霍跃华,赵法起,李晓宇,等.一种加密恶意流量检测方法:CN 202210124869.1[P].2022-04-15. HUO Y H,ZHAO F Q,LI X Y,et al.An encrypted malicious traffic detection method:CN 202210124869.1[P].2022-04-15.(in Chinese)
[28] PEDREGOSA F,VAROQUAUX G,GRAMFORT A,et al.Scikit-learn:machine learning in python[J].The Journal of Machine Learning Research,2011,12:2825-2830.

Please choose a citation manager

Content to export