[lug] Replication of files

Nate Duehr nate at natetech.com
Sat Feb 17 17:59:14 MST 2001


On Fri, Feb 16, 2001 at 06:17:26PM -0600, Carlos Hern?ndez L?pez wrote:
> Yes, I was too vague. I hope this works.
> 
> For an application that we have at office, I need to replicate  a number of  files, some of them could be really huge
> (80-100 Mbytes).
> 
> The replication it would be  from Linux to Solaris
> 
> The  files grow substantially during the day because they are some kind of log files and, as part of the application,
> can not be deleted. They only grow and grow.
> 
> rsync seems appropiate but I'd like to see if  there are other options.

rsync should work fine, but I'd worry about whether or not the files can
be written to during the rsync run, stuff like that.  You may get a
completely broken copy of the files if the files are "in use" at the
time of the rsync run.  Think carefully about that before you implement
rsync (or anything else, really...).

You can also specify larger than normal block-sizes to rsync if that
helps performance issues -- try to make the block sizes some percentage
of the average sizes of the files you're copying. 

Remember rsync looks at files as chunks of smaller blocks and only
bothers with copying the parts that have changed... so if the files are
simply growing and growing, rsync should be pretty efficient at bringing
over only the additional data at the end of the file every day, not the
whole file again.  Very nice stuff.

-- 
Nate Duehr <nate at natetech.com>

GPG Key fingerprint = DCAF 2B9D CC9B 96FA 7A6D AAF4 2D61 77C5 7ECE C1D2
Public Key available upon request, or at wwwkeys.pgp.net and others.



More information about the LUG mailing list