摘要： 为实现数据仓库中数据的高效集成,针对数据偏斜分布现象,提出一种改进的数据流更新算法EH-JOIN。该算法对传统散列连接方法进行改进,利用索引将部分频繁使用的主数据存储在内存中,解决了高速数据流下的磁盘频繁访问问题。实验结果表明,与MESHJOIN 算法和R-MESHJOIN 算法相比,EH-JOIN 算法的服务速率在磁盘存储关系集保持适当大小时分别提高了96% 和81% ,在内存大小不同时提高了57% 和48% 。

关键词: 实时数据仓库, 数据转换, 数据流更新, 基于流的连接, 哈希索引, 偏斜分布

Abstract: distribution,this paper proposes an improved data stream update algorithm———Extended Hybrid Join(EH-JOIN). The algorithm improves the traditional Hash join method,and it can adapt to common skewed data and greatly reduce the disk I / O cost through using index structure and storing some parts of the master data in memory. Experimental results show that the service rate of proposed algorithm is improved by 96% and 80% compared with MESHJOIN algorithm and R-MESHJOUIN algorithm as the relation set keeps an appropriate size,and the service rate of proposed algorithm is improved by 57% and 48% compared with MESHJOIN algorithm and R-MESHJOUIN algorithm as the memory size differs.

Key words: real-time data warehouse, data transformation, data stream update, stream-based join, Hash index, skewed distribution

中图分类号: 

• TP311． 13