LoCI logo
Home
About Us
Projects
Publications
Software
Documents
Screencasts
Related Projects
Related Pubs

People
FAQs
News Archive
Contact Us


Links

IBPvo
Record your favorite show on TV!
Linux ISOs
Download the latest version of Linux!
 
 



 
Towards an Efficient, Scalable Replication Mechanism for the I2-DSI Project
Bert Dempsey, Debra Weiss

Technical Report TR-1999-01, University of North Carolina at Chapel Hill School of Information and Library Science (UNC SILS), April, 1999

Available as: .PDF

Abstract:
This paper presents the development of new functionality for the open-source rsync utility aimed at producing an efficient, scalable solution for multiple-site file synchronization. The context of our work is the Internet2 Distributed Storage Infrastructure (I2-DSI) project, which is developing a reliable, scalable, high performance storage service infrastructure for advanced applications in research and education. Specifically, the I2-DSI project is working on middleware software to enable the replication of applications across a set of geographically distributed hosts. This paper presents a new mechanism for replicating filesystems, rsync+, which is a modification of an open-source rsync file synchronization utility. Using rsync+ for file updates, a flexible, powerful replication mechanism can be developed for publishing source objects into the I2-DSI replication service, and the approach enables scalable network distribution through multicast-based solutions. The paper presents the technical details behind the rsync+ tool, its use as a replication solution within I2-DSI, and performance results from a large-scale (multi-gigabyte) WWW mirroring experiment using rsync+ that demonstrate correct operation and efficiency gains with actual data from an active WWW document archive.

Citation Info:

Authors: Bert Dempsey, Debra Weiss
Title: Towards an Efficient, Scalable Replication Mechanism for the I2-DSI Project
Institution: University of North Carolina at Chapel Hill School of Information and Library Science (UNC SILS)
Year: 1999
Month: April
Number: TR-1999-01
Where: http://loci.cs.utk.edu/publications/1999_Towards_Efficient_Scalable_Replication.php
UTK home        DOE NSF
 
corner
Home   About Us   Projects   Publications   Software   Docs   Screencasts   Related Projects   Related Pubs   People   FAQs   News  Contact Us
corner