LoCI

Bert Dempsey, Debra Weiss

Technical Report TR-1999-01, University of North Carolina at Chapel Hill School of Information and Library Science (UNC SILS), April, 1999

Available as: .PDF

Abstract:
This paper presents the development of new functionality for the open-source rsync utility aimed at producing an efficient, scalable solution for multiple-site file synchronization. The context of our work is the Internet2 Distributed Storage Infrastructure (I2-DSI) project, which is developing a reliable, scalable, high performance storage service infrastructure for advanced applications in research and education. Specifically, the I2-DSI project is working on middleware software to enable the replication of applications across a set of geographically distributed hosts. This paper presents a new mechanism for replicating filesystems, rsync+, which is a modification of an open-source rsync file synchronization utility. Using rsync+ for file updates, a flexible, powerful replication mechanism can be developed for publishing source objects into the I2-DSI replication service, and the approach enables scalable network distribution through multicast-based solutions. The paper presents the technical details behind the rsync+ tool, its use as a replication solution within I2-DSI, and performance results from a large-scale (multi-gigabyte) WWW mirroring experiment using rsync+ that demonstrate correct operation and efficiency gains with actual data from an active WWW document archive.

Citation Info:

Authors:	Bert Dempsey, Debra Weiss
Title:	Towards an Efficient, Scalable Replication Mechanism for the I2-DSI Project
Institution:	University of North Carolina at Chapel Hill School of Information and Library Science (UNC SILS)
Year:	1999
Month:	April
Number:	TR-1999-01
Where:	http://loci.cs.utk.edu/publications/1999_Towards_Efficient_Scalable_Replication.php

Links