摘要： 传统的文件系统管理工具通过实时遍历文件目录树获取文件元数据信息,以实现管理监视功能。但对于
大型文件系统,元数据信息的获取需要耗费大量时间,传统管理工具无法适应当前大数据背景下的管理需求。为
此,基于数据库技术设计一种新的文件系统管理工具。该工具集成Robinhood 策略引擎与TORQUE 作业管理系
统,通过分布式并行计算获取文件系统元数据信息,并将信息保存到MySQL 数据库,同时基于数据库中的元数据
信息,实现文件系统的监视、管理和备份功能。实验结果表明,采用分布式并行计算能够充分利用集群的计算资
源,提高文件系统的遍历速率,保证文件系统监视、管理和备份的顺利进行。
关键词:
元数据,
大数据,
Robinhood 策略引擎,
TORQUE 作业管理系统,
分布式处理,
MySQL 数据库
Abstract: Traditional file system management tools achieve management and monitoring functions by traversing file
directory tree to get metadata information. For large file systems,it is a time consumed task to get metadata which can not meet the demand of the current management of large data background. This paper integrates policy engine Robinhood and TORQUE job management system. A distributed parallel computing is used to get the file system metadata information which is saved into MySQL database. Based on metadata information saved in database,the tool achieves monitoring,file management and system backup. The tests indicate that distributed computing is able to fully use the computing utilities of the cluster,not able to enhance the speed of traversing file system,and makes sure of the progressing on monitoring, management and back up of the file system.
Key words:
metadata,
big data,
Robinhood policy engine,
TORQUE job management system,
distributed processing,
MySQL database
中图分类号: