摘要：

传统TF-IDF算法仅从词频与逆向文档频率的角度计算关键词与文档之间的相关性权重,忽略了用户兴趣对权重计算的影响。为此,以满足用户信息检索目的为研究背景,提出一种基于日志关联的相关性权重算法。从面向用户相关性的角度出发,通过分析用户的搜索日志建立用户兴趣模型,并结合分布式计算的思想,运用MapReduce编程框架实现计算任务的并行化处理。实验结果表明,该算法在处理海量数据时,不仅能够提高算法效率,而且可以根据用户的历史检索记录动态地改变检索词的权重,提升用户与系统的交互能力。

关键词: 分布式计算, TF-IDF算法, 日志, 兴趣模型, 信息检索

Abstract:

The traditional TF-IDF algorithm calculates the correlation weights between keywords and documents only by using the perspective of word frequency and reverse document frequency,which ignoes the influence of user interest on weight calculation.In order to meet the purpose of user information retrieval,a correlation weight algorithm based on journal association is proposed.From the perspective of user-oriented comelation,the user interest model is built by analyzing the user's search journal,and combined with the idea of distributed computing,the MapReduce programming framework is used to realize the parallel processing of computing tasks.Experimental results show that it can not only improve the efficiency of the algorithm when dealing with massive data,but also dynamically change the weight of retrieval word according to the user's historical retrieval records,so as to enhance the interaction ability between users and the system.

Key words: distributed computing, TF-IDF algorithm, journal, interest model, information retrieval

中图分类号: 

• TP391