摘要： 分析了网络机器人的访问特征,给出了从Web日志中检测Web Robot的一般方法,并提出了一个新的基于页面成员表的检测算法。经实验验证,该算法可以有效地检测未知的和不遵守网络机器人排斥标准的Robot。

关键词: 搜索引擎, 网络机器人, 页面成员表, 检测, Web日志

Abstract: After analyzing the navigational patterns of Web robots,the general techniques of detecting Web robots are discussed,and a new algorithm based on the webpage member list is proposed.The experimental result shows that the new algorithm can detect the unknown robots and unfriendly robots who do not obey the standard for robot exclusion.

中图分类号: 

• TP393.092