面向大数据系统的检测器快速筛选算法

doi:10.3969/j.issn.1000-3428.2015.09.008

摘要/Abstract

摘要： 筛选成熟检测器是决定人工免疫系统性能和效率的关键因素,在大数据环境下由于初始检测器的数量极其庞大,会造成现有检测器筛选算法时间开销过大。针对该问题,提出一种新的海量初始检测器快速筛选算法。设计海量初始检测器的分布存储模式,利用Map/Reduce模型,给出混合式初始检测器快速筛选架构、海量初始检测器分区检查策略和成熟检测器集优化策略,以提高筛选初始检测器的效率,优化成熟检测器。在Hadoop集群中实现面向大数据系统检测器快速筛选算法原型系统,使用CERT synthethic sendmail data数据集进行测试与分析,结果表明,与传统算法相比,该算法能减少58.87%的时间开销,并在初始检测器数量不断增加时保持时间开销的稳定。

关键词: 检测器生成算法, 大数据系统, 人工免疫系统, Map/Reduce模型

Abstract: The detector screening algorithm is important for the performance and efficiency of Artificial Immune System(AIS).But the time overhead of current detector generation algorithm is too big due to the large number of initial detector in the big data system.In this paper,the new distributed initial detector storage strategy is designed to provide the basis for improving the massive initial detector selection efficiency.The new fast detector screening algorithm for the big data system is given by analyzing the different character of map and reduce stage in Map/Reduce model,and it is used to improve the initial detector selection efficiency and optimize the mature detector set.It includes hybrid initial detector selection architecture,massive initial detection partition inspection algorithm and mature detector set optimization algorithm.The fast detector generation algorithm prototype for big data system is realized in Hadoop system.CERT synthethic sendmail data set is used to evaluate and compared with the current detector screening algorithm.The result shows that the 58.87% time overhead is reduced,and time overhead can be maintained stability with the number of initial detector increasing.

Key words: detector generation algorithm, big data system, Artificial Immune System(AIS), Map/Reduce model

中图分类号:

TP391

蔡涛,倪晓蓉,王伟生,牛德姣. 面向大数据系统的检测器快速筛选算法[J]. 计算机工程, doi: 10.3969/j.issn.1000-3428.2015.09.008.

CAI Tao,NI Xiaorong,WANG Weisheng,NIU Dejiao. Fast Detector Screening Algorithm for Big Data System[J]. Computer Engineering, doi: 10.3969/j.issn.1000-3428.2015.09.008.

https://www.ecice06.com/CN/Y2015/V41/I9/45

参考文献

参考文献［1］Forrest S,Helman P.An Immunological Approach to Change Detection:Algorithms,Analysis,and Implica-tions［C］//Proceedings of IEEE Symposium on Computer Security and Privacy.Washington D.C.,USA:IEEE Press,1996:192-211. ［2］ Forrest S,Perelson A S,Allen S,et al.Self-nonself Discrimination in a Computer［C］//Proceedings of IEEE Symposium on Research in Security and Privacy.Washington D.C.,USA:IEEE Press,1994:202-212. ［3］杨东勇,陈晋因.基于多种群遗传算法的检测器生成算法研究［J］.自动化学报,2009,35(4):425-432. ［4］ de Castro L N,von Zuben F J.The Clonal Selection Algorithm with Engineering Applications［C］//Proceedings of GECCO’00.San Francisco,USA:Morgan Kaufmann Publishers,2000:36-37. ［5］ de Castro L N,von Zuben F J.Learning and Optimization Using the Clonal Selection Principle［J］.IEEE Transactions on Evolutionary Computation,2002,6(3):239-251. ［6］ Kim J,Bentley P J.Towards an Artificial Immune System for Network Intrusion Detection:An Investigation of Clonal Selection with a Negative Selection Operator［C］//Proceedings of Congress on Evolutionary Computation.Piscataway,USA:IEEE Press,2001:1244-1252. ［7］焦李成,杜海峰,刘芳,等.免疫优化计算、学习与识别［M］.北京:科学出版社,2006. ［8］张立宁,公茂果,焦李成,等.抗独特型克隆选择算法［J］.软件学报,2009,20(5):1269-1281. ［9］ Agrawal R,Srikant R.Fast Algorithms for Mining Associ-ation Rules［C］//Proceedings of the 20th VLDB Confe-rence.Santiago,USA:Morgan Kaufmann,1994:487-499. ［10］ Luo Wenjian,Zhang Zeming,Wang Xufa.A Heuristic Detector Generation Algorithm for Negative Selection Algorithm with Hamming Distance Partial Matching Rule［C］//Proceedings of the 5th International Conference on Artificial Immune Systems.Oeiras,Portugal:［s.n.］,2006:229-243. ［11］ Jiang Hua,Wang Xin,Fan Xiaofeng.Research on Linear Time Detector Generating Algorithm Based on Nega-tive Selection Model［C］//Proceedings of CCDC’09.Washington D.C.,USA:IEEE Press,2009:3058-3061. ［12］ Dean J,Ghemawat S.MapReduce:Simplified Data Processing on Large Clusters［J］.Communications of the ACM,2008,51(1):107-113. ［13］孙大为,张广艳,郑纬民.大数据流式计算:关键技术及系统实例［J］.软件学报,2014,25(4):839-862. ［14］ Scalosub G,Marbach P,Liebeherr J.Buffer Management for Aggregated Streaming Data with Packet Depen-dencies［J］.IEEE Transactions on Parallel and Distributed Systems,2013,24(3):439-449. ［15］ Malensek M,Pallickara S L,Pallickara S.Exploiting Geospatial and Chronological Characteristics in Data Streams to Enable Efficient Storage and Retrievals［J］.Future Generation Computer Systems,2013,29(4):1049-1061. ［16］ Cugola G,Margara A.Processing Flows of Information:From Data Stream to Complex Event Processing［J］.ACM Computing Surveys,2012,44(3):151-162. ［17］覃雄派,王会举,杜小勇,等.大数据分析——RDBMS与MapReduce的竞争与共生［J］.软件学报,2012,23(1):32-45. 编辑金胡考 (上接第44页) ［15］Youssef M,Agrawala A.The Horus WLAN Location Determination System［EB/OL］.(2011-12-12).http://www.docin.com/p-305713112.html. ［16］Azizyan M,Constandache I,Choudhury R R.SurroundSense:Mobile Phone Localization via Ambience Fingerprinting ［C］//Proceedings of the 15th Annual International Conference on Mobile Computing and Networking.Washington D.C.,USA:IEEE Press,2009:261-272. ［17］Jiang Y,Pan X,Li K,et al.ARIEL:Automatic Wi-Fi Based Room Fingerprinting for Indoor Localization［C］//Proceedings of 2012 ACM Conference on Ubiquitous Computing.New York,USA:ACM Press,2012:441-450. ［18］Wang H,Sen S,Elgohary A,et al.No Need to War-drive:Unsupervised Indoor Localization［C］//Proceedings of the 10th International Conference on Mobile Systems.Washington D.C.,USA:IEEE Press,2012:157-165. ［19］Zhang C,Luo J,Wu J.A Dual-sensor Enabled Indoor Localization System with Crowdsensing Spot Survey［C］//Proceedings of DCSS’14.Washington D.C.,USA:IEEE Press,2014:148-156. ［20］Sen S,Lee J,Kim K H,et al.Avoiding Multipath to Revive Inbuilding WiFi Localization［C］//Proceedings of the 11th Annual International Conference on Mobile Systems.Washington D.C.,USA:IEEE Press,2013:269-278. ［21］Roy N,Wang He,Choudhury R R.I Am a Smartphone and I Can Tell My User’s Walking Direction［C］//Proceedings of the 12th Annual International Conference on Mobile Systems.Washington D.C.,USA:IEEE Press,2014:156-162. ［22］Chen Y,Lymberopoulos D,Liu J,et al.Fm-based Indoor Localization［C］//Proceedings of the 10th International Conference on Mobile Systems.Washington D.C.,USA:IEEE Press,2012:359-368. ［23］Robertson P,Angermann M,Krach B.Simultaneous Localization and Mapping for Pedestrians Using Only Foot-mounted Inertial Sensors［C］//Proceedings of the 11th International Conference on Ubiquitous Computing.Washington D.C.,USA:IEEE Press,2009:93-96. ［24］Ravi N,Shankar P,Frankel A,et al.Indoor Localization Using Camera Phones［C］//Proceedings of MCSA’05.Washington D.C.,USA:IEEE Press,2005:299-307. 编辑索书志

[1]	张凯林, 董红斌. 基于mRMR与基尼重要性的树突状细胞模型[J]. 计算机工程, 2023, 49(5): 129-138.
[2]	张艺, 周雯, 梁意文, 谭成予. 基于数字微分的函数化树突状细胞算法模型[J]. 计算机工程, 2020, 46(9): 54-60.
[3]	孙靖杰, 赵建军, 杨利斌, 桑德一, 姚刚. 一种人工免疫分类方法在故障诊断中的应用[J]. 计算机工程, 2013, 39(8): 208-214.
[4]	方贤进，蔡妙琪. 基于人工免疫系统的入侵检测研究[J]. 计算机工程, 2013, 39(11): 136-138,142.
[5]	张洁, 裴芳. 利用云模型表示危险信号的入侵风险评估方法[J]. 计算机工程, 2012, 38(19): 89-91.
[6]	魏建刚, 张曦煌. 树突状细胞算法在检测任务超时异常中的应用[J]. 计算机工程, 2012, 38(15): 240-243.
[7]	刘锦伟, 唐俊. 一种改进的实值负选择算法[J]. 计算机工程, 2011, 37(14): 195-196.
[8]	常志英, 韩莉, 姜大伟. 改进的克隆选择算法及其应用[J]. 计算机工程, 2011, 37(01): 173-174,177.
[9]	徐佳;张卫. 人工免疫系统中的抗体生成与匹配算法[J]. 计算机工程, 2010, 36(9): 181-183.
[10]	陈岳兵;冯超;张权;唐朝京. 树突状细胞算法原理及其应用[J]. 计算机工程, 2010, 36(8): 173-176.
[11]	彭敏, 唐俊. 检测器实时生成算法及其应用[J]. 计算机工程, 2010, 36(19): 180-181.
[12]	王飞, 钱玉文, 王执铨. 基于混合AIS/SOM的入侵检测模型[J]. 计算机工程, 2010, 36(12): 164-166.
[13]	苏洁;乔佩利;刘亚辉. 基于免疫进化计算的分布式入侵检测方法[J]. 计算机工程, 2010, 36(06): 163-165.
[14]	洪征;吴礼发;王元元. 基于阴性选择的网络蠕虫抑制模型[J]. 计算机工程, 2008, 34(7): 126-128.
[15]	王磊;刘戈峰;李园园. 基于AIS的服务匹配模型[J]. 计算机工程, 2008, 34(24): 186-189.

选择文件类型/文献管理软件名称

选择包含的内容