File Data Synchronization Method Based on Ordered Hash Chain

doi:10.19678/j.issn.1000-3428.0063814

Abstract

Abstract: This study aims to tackle the problems of low efficiency and local update difficulties in the existing file data synchronization transmission methods.To this end, it proposes a Hash chain construction and file data synchronization method.It takes the changes of server files or directories in the C/S architecture as a series of Hash nodes, iterates the Hash values of files or directories through the Hash function according to the time sequence, and forms an ordered Hash chain that can record all the operation states of the file library.The client only needs to perform the same file operation according to the Hash chain node for synchronous updates, without synchronous authentication for each file data, to ensure the integrity, non-repudiation, traceability, and tamper resistance of the file library.The sequential Hash chain synchronization method is used to monitor the difference and consistency of file data for different terminals to quickly obtain file changes and perform logical synchronization.The experimental results show that the average synchronization acceleration ratio of the proposed method is 94.85% under the mode of unchanged file library, in the mode of changed file library, the average synchronization acceleration ratio is 6.5% and 69.99% respectively compared with the Rsync algorithm with ‘quick check’ strategy and conventional strategy, effectively reduces the time and resource consumption during synchronization.

Key words: Hash chain, file synchronization, data backup, Rsync algorithm, file monitoring

摘要： 针对现有文件数据同步传输方法效率低、局部更新困难的问题，提出一种哈希链构建及文件数据同步方法。将C/S架构中服务器端文件或目录的变化作为一系列哈希节点，根据时间先后顺序，通过哈希函数迭代文件或目录的哈希值，形成能够记录文件库所有操作状态的有序哈希链。客户端只需根据哈希链节点执行相同文件操作并进行同步更新，而不需要对每个文件数据进行同步认证，确保文件库的完整性、不可抵赖性、可溯源性和防篡改性。采用有序哈希链的同步方法对不同终端进行文件数据差异监视和一致性检测，以快速获取文件变化并进行逻辑同步。实验结果表明，该方法在文件库未变动模式下的平均同步加速比为94.85%，在文件库变动的模式下，相较于“quick check”策略和常规策略的Rsync算法，平均同步加速比分别为6.5%和69.99%。有效地减少了同步过程中时间和资源的消耗。

关键词: 哈希链, 文件同步, 数据备份, Rsync算法, 文件监控

CLC Number:

TP393

ZENG Chang, JIANG Wenbao, GUO Yangnan. File Data Synchronization Method Based on Ordered Hash Chain[J]. Computer Engineering, 2023, 49(1): 181-190,200.

曾畅, 蒋文保, 郭阳楠. 基于有序哈希链的文件数据同步方法[J]. 计算机工程, 2023, 49(1): 181-190,200.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0063814

http://www.ecice06.com/EN/Y2023/V49/I1/181

Figures/Tables 13

References

[1] 余春雷.分布式存储系统中数据存储研究[D].西安:长安大学, 2020. YU C L.Study on data storage in distributed storage systems[D].Xi'an:Changan University, 2020.(in Chinese)
[2] SALA F, SCHOENY C, BITOUZÉ N, et al.Synchronizing files from a large number of insertions and deletions[J].IEEE Transactions on Communications, 2016, 64(6):2258-2273.
[3] 丁建立, 周冬雪, 王静, 等.基于连接驱动SQL捕获的远程数据库同步机制[J].计算机工程, 2017, 43(9):39-42, 50. DING J L, ZHOU D X, WANG J, et al.Remote database synchronization mechanism based on SQL capture from connection driver[J].Computer Engineering, 2017, 43(9):39-42, 50.(in Chinese)
[4] WANG T, ZHOU J Y, LIU A F, et al.Fog-based computing and storage offloading for data synchronization in IoT[J].IEEE Internet of Things Journal, 2019, 6(3):4272-4282.
[5] 何倩, 赵帮, 王勇, 等.基于本地快照和Rsync算法的Web文件保护[J].计算机工程, 2013, 39(6):190-193, 199. HE Q, ZHAO B, WANG Y, et al.Web file protection based on local snapshot and Rsync algorithm[J].Computer Engineering, 2013, 39(6):190-193, 199.(in Chinese)
[6] FAIZ M, SHANKER U.Data synchronization in distributed client-server applications[C]//Proceedings of International Conference on Engineering and Technology.Washington D.C., USA:IEEE Press, 2016:611-616.
[7] GHOBADI A, ESWARAN C, HO C K, et al.Awrapper algorithm improve data transfer and synchornization performance of RSYNC[J].International Journal Artificial Intelligent and Informatics, 2021, 2(1):40-52.
[8] ZHANG C J, QI D Y, LI W L, et al.SimpleSync:a parallel delta synchronization method based on Flink[J].Concurrency and Computation:Practice and Experience, 2021, 33(20):1-10.
[9] IRMAK U, MIHAYLOV S, SUEL T.Improved single-round protocols for remote file synchronization[C]//Proceedings of the 24th Annual Joint Conference of the Computer and Communications Societies.Washington D.C., USA:IEEE Press, 2005:1665-1676.
[10] 王宾, 刘钊远.基于Rsync的远程文件同步优化模型[J].计算机与现代化, 2015(4):10-13. WANG B, LIU Z Y.Optimization model of remote file synchronization based on Rsync algorithm[J].Computer and Modernization, 2015(4):10-13.(in Chinese)
[11] 蒋文保, 曾畅.一种哈希链构建及文件数据同步方法、装置及系统:CN113114749A[P].2021-07-13. JIANG W B, ZENG C.Hash chain construction and file data synchronization method, device and system:CN113114749A[P].2021-07-13.(in Chinese)
[12] KHAN M I, FAISAL F, AZAM S, et al.Using blockchain technology for file synchronization[J].IOP Conference Series:Materials Science and Engineering, 2019, 561(1):1-10.
[13] 刘扬, 颜光伟, 夏凡, 等.基于Inotify的文件备份系统研究[J].电声技术, 2021, 45(3):46-51. LIU Y, YAN G W, XIA F, et al.Research of file backup system based on Inotify[J].Audio Engineering, 2021, 45(3):46-51.(in Chinese)
[14] LAMPORT L.Password authentication with insecure communication[J].Communications of the ACM, 1981, 24(11):770-772.
[15] HUANG N, ZHU J X, GUO C N, et al.A novel hash chain-based data availability monitoring method for off-site disaster recovery architecture[J].Journal of Circuits, Systems and Computers, 2021, 30(16):1-10.
[16] MOTOHASHI T, HIRANO T, OKUMURA K, et al.Secure and scalable mHealth data management using blockchain combined with client hashchain:system design and validation[J].Journal of Medical Internet Research, 2019, 21(5):1-8.
[17] LU Y, FENG T.Research on trusted DNP₃-BAE protocol based on hash chain[J].EURASIP Journal on Wireless Communications and Networking, 2018, 10:108.
[18] ZHAO C D, SHI M X, HUANG M M, et al.Authentication scheme based on Hashchain for space-air-ground integrated network[C]//Proceedings of International Conference on Communications.Washington D.C., USA:IEEE Press, 2019:1-6.
[19] 程万旭, 金鑫, 肖勇.基于改进Rsync算法计量自动化系统与营销系统档案同步技术[J].计算机应用研究, 2020, 37(S1):199-200, 203. CHENG W X, JIN X, XIAO Y.Advanced metering system and power sales management system file synchronization technology based on improved Rsync algorithm[J].Application Research of Computers, 2020, 37(S1):199-200, 203.(in Chinese)
[20] GHOBADI A, ESWARAN C, MUTHUVELU N, et al.An adaptive wrapper algorithm for file transfer applications to support optimal large file transfers[C]//Proceedings of the 11th International Conference on Advanced Communication Technology.Washington D.C., USA:IEEE Press, 2009:315-320.
[21] 李帅, 刘晓洁, 徐兵.一种基于目录哈希树的磁盘数据同步方法研究[J].信息网络安全, 2019(2):53-59. LI S, LIU X J, XU B.Research on a disk data synchronization method based on directory Hash tree[J].Netinfo Security, 2019(2):53-59.(in Chinese)
[22] 许圣明, 马迪, 毛伟, 等.基于有序哈希树的RPKI资料库数据同步方法[J].计算机系统应用, 2016, 25(6):141-146. XU S M, MA D, MAO W, et al.RPKI repository synchronization method based on ordered Hash tree[J].Computer Systems & Applications, 2016, 25(6):141-146.(in Chinese)
[23] 胡晓勤, 卢正添, 刘晓洁, 等.远程文件快速同步方法[J].电子科技大学学报, 2008, 37(4):594-597. HU X Q, LU Z T, LIU X J, et al.A remote file fast synchronization method[J].Journal of University of Electronic Science and Technology of China, 2008, 37(4):594-597.(in Chinese)
[24] 徐旦, 生拥宏, 鞠大鹏, 等.高效的两轮远程文件快速同步算法[J].计算机科学与探索, 2011, 5(1):38-49. XU D, SHENG Y H, JU D P, et al.High effective two-round remote file fast synchronization algorithm[J].Journal of Frontiers of Computer Science & Technology, 2011, 5(1):38-49.(in Chinese)
[25] GRAMOLI V.More than you ever wanted to know about synchronization:synchrobench, measuring the impact of the synchronization on concurrent algorithms[C]//Proceedings of the 20th Symposium on Principles and Practice of Parallel Programming.New York, USA:ACM Press, 2015:1-10.

Please choose a citation manager

Content to export