摘要： 为了提高聚焦爬虫的搜索效率，提出一种结合内容评价和链接结构搜索策略的优点并利用小生境遗传算法进行全局寻优的搜索策略。改进遗传算子和小生境遗传算法，将待搜索的网页URL作为遗传个体，采用概率变迁规则和小生境淘汰运算引导搜索方向。实验结果证明，与聚焦爬虫的其他实现技术相比，该策略在抓取主题相关网页时具有更高的查准率和查全率。

关键词: 聚焦爬虫, 遗传算法, 小生境, 主题相关度

Abstract: In order to improve the search efficiency of focused crawler, based on Niche Genetic Algorithm(NGA), this paper proposes a global optimization of search strategy which combines the advantages of content evaluation and link structure. URL search direction is guided by improving the genetic operators and NGA. Compared with other algorithms, experimental results indicate that this strategy has higher precision and recall in searching the topic pages.

Key words: focused crawler, genetic algorithm, niche, topic relevancy

中图分类号: 

• TP311.13