摘要： 定题爬虫的研究是定题搜索引擎的关键技术。该文提出一种基于多Agent系统的爬虫算法，采用本题语义主题关键词过滤的方法来抓取与主题相关的网页，利用本体库语义网络实现本体领域中同近义词的过滤。凭借HTML网页标记对关键字识别的不同权重和超链接锚文本对主题相关网页进行预测，通过黑板的通信机制实现多Agent交互。实验结果表明算法在抓取网页的查准率、查全率方面有一定的改善。

关键词: 定题爬虫, 主题关键字过滤, 语义

Abstract: Focused crawling research is key to search engine technology. In this paper, a focused crawling algorithm based on multi-Agent system is presented, which presents a core issue of a theme key words filtering method based on ontology to collect the URL related to the themes. Semantic network based on the ontology is to achieve filtering of similar meaning. It also introduces keyword identification of different weights by HTML page tags and anchor text, which are important for the website forecast use. And system model based on the blackboard communication mechanism is explained. The experimental results show that the system has an increasing promotion in both precision and extension for crawling website.

Key words: focused crawling, theme key words filtering, semantics

中图分类号: 

• TP301.6