[lug] Commercial rsync service?

[Rob Nagler]

David L. Anselmi writes:
> Ok, thanks.  So your experience rsync'ing 30k files isn't representative 
> of using rsync between local directories where one is davfs.  No big 
> deal, just wanted to clarify.

Yes, that's correct so I just did a little test, and was very
surprised at the results:

$ date; rsync -e ssh -azlCSR Somedir example.com:; date
17:01:44
17:01:59
$ date; rsync -e ssh -azlCSR Somedir example.com:; date
17:02:02
17:02:05
$ date; rsync -alCSR Somedir /Volumes/dav.example.com; date
17:02:44
17:04:50
$ date; rsync -alCSR Somedir /Volumes/dav.example.com; date
17:05:01
17:06:58

Note that the first rsync over ssh took about 15 secs.  The second
finished in 3 secs.  That's what I'm used to.

However, the first rsync over Mac DAV (WebDAVFS) took 2+ min, and the
second one took about 2min.  Huh, why?  So I looked at the logs.  Here
is the stripped entries for just one file:

17:05:08 GET 	    /Somedir/Somefile  	    	200 1137 
17:05:08 PROPFIND   /Somedir/.Somefile.99iChc  	404 83 
17:05:08 PUT 	    /Somedir/.Somefile.99iChc  	200 60 
17:05:09 PROPFIND   /Somedir/._.Somefile.99iChc 404 85 
17:05:09 LOCK 	    /Somedir/.Somefile.99iChc  	200 449 
17:05:09 GET 	    /Somedir/.Somefile.99iChc  	200 0 
17:05:09 PROPFIND   /Somedir/._.Somefile.99iChc 404 85 
17:05:09 PUT 	    /Somedir/.Somefile.99iChc  	200 60 
17:05:10 PROPFIND   /Somedir/.Somefile.99iChc  	207 528 
17:05:10 PROPFIND   /Somedir/.Somefile.99iChc  	207 528 
17:05:10 PROPFIND   /Somedir/._.Somefile.99iChc 404 85 
17:05:10 MOVE 	    /Somedir/.Somefile.99iChc  	204 20 
17:05:10 PROPFIND   /Somedir/._.Somefile.99iChc 404 85 
17:05:11 PROPFIND   /Somedir/._Somefile     	404 77

You'll notice that WebDAVFS appears to be extremely repetitive,
because rsync thinks it is talking to a local file system.  I got
better performance with:

$ date; rsync -alCSR --inplace Somefile /Volumes/dav.example.com; date 
17:34:24
17:35:19

About a minute with --inplace.  I got almost reasonable performance
with:

$ date; rsync -alCSR --size-only Somefile /Volumes/dav.example.com; date
17:38:32
17:38:51

About 20 seconds with --size-only.  Turns out that WebDAVFS doesn't
support updating of file times (while the WebDAV protocol technically
does) so the files are always modified when they are copied.  This
makes rsync useless.  It's faster to copy everything every time.

Well, that's an interesting lesson for me.  I don't have time to try
this experiment over Netdrive, but I suspect it might be better.

Rob