[lug] help with the infamous rsync include/exclude patterns

[Simos]

Hi Davide,

I'm possibly not understanding your situation, but you don't need to run the
find commands on the server. I just used them in my example to show you that
the rsync command correctly copies only what you need (just data under *mon
directories). You can just ignore the finds. You just need this to do the sync:

rsync -arvm --include='*/' --include='*mon/***' --exclude='*' /path/to/source /path/to/dest

HTH, but if not, just ignore my ramblings!

Simos

On Fri, 13 May 2016 16:26:40 -0600
Davide Del Vento <davide.del.vento at gmail.com> wrote:

> Hi Simo,
> I don't have shell access on the server where I'm downloading files from,
> so I can't run find there. So far the best option seems to be to use rsync
> --list-only --recursive to get a file with the list of files, then prune
> everything that I don't need from such a list_of_files.txt and do something
> like
> 
> rsync -av --files-from=list_of_files.txt / /my/local/path/
> 
> However I was frankly hoping for something easier and which would not
> require an intermediate file to massage.
> 
> Thanks a lot for your suggestion!
> 
> On Fri, May 13, 2016 at 4:02 PM, Simos <blug at chinesetearoom.com> wrote:
> 
> > On Fri, 13 May 2016 11:21:57 -0600
> > Davide Del Vento <davide.del.vento at gmail.com> wrote:
> >
> > > I have this question at
> > >
> > http://stackoverflow.com/questions/37198595/how-to-properly-match-directories-with-rsync
> > > if you prefer to answer there. Otherwise keep reading
> > >
> > > I need to rsync some directories from one machine to another, but only
> > > part of the tree. I think the easiest way to explain it would be to
> > > run, on the server, tree -f (which prints the full path of each file)
> > > and then grep'ing mon in the full path and rsynch'ing only the matches
> > > of grep. Actually I would be happy to do it that way, but I don't have
> > > shell access to the server, only rsync. Two step solution based on
> > > greping output of rsync --list-only is totally acceptable.
> > >
> > > It's actually simpler, because I also know that this mon appears only
> > > as a suffix to a directory (the hierarchy is pretty rigid), so I tried
> > > doing
> > >
> > > rsync --dry-run -vrlpt --include='*/' --include='**/*mon/'
> > > --exclude='*' --prune-empty-dirs
> > >
> > > but that did not match anything. Ditto for --include='*mon/' instead
> > > of the one with the double asterisk. Avoiding the --prune-empty-dirs
> > > matched all the directories but no files, which I think is my problem.
> > > So I tried to add an include pattern for some file extensions right
> > > before the exclude pattern (which I would rather not do because I
> > > don't want to track them all and they might change), but that simply
> > > matched all those files, including the ones not in the "mon" trees.
> > >
> > > And yes, I did see countless examples of people doing similar stuff,
> > > which helped me making some progress (i.e. what you can see above),
> > > but not solve the problem...
> > > _______________________________________________
> > > Web Page:  http://lug.boulder.co.us
> > > Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
> > > Join us on IRC: irc.hackingsociety.org port=6667 channel=#hackingsociety
> >
> > Hi Davide,
> >
> > How about this?
> >
> > $ find toplevel | sort
> > toplevel
> > toplevel/amon
> > toplevel/amon/file3
> > toplevel/dir1
> > toplevel/dir1/bmon
> > toplevel/dir1/bmon/dir3
> > toplevel/dir1/bmon/dir3/file5
> > toplevel/dir1/bmon/file2
> > toplevel/dir1/file1
> > toplevel/dir2
> > toplevel/dir2/file6
> > toplevel/emon
> > toplevel/file4
> >
> > $ rsync -arvm --include='*/' --include='*mon/***' --exclude='*' toplevel /tmp
> >
> > $ find /tmp/toplevel | sort
> > /tmp/toplevel
> > /tmp/toplevel/amon
> > /tmp/toplevel/amon/file3
> > /tmp/toplevel/dir1
> > /tmp/toplevel/dir1/bmon
> > /tmp/toplevel/dir1/bmon/dir3
> > /tmp/toplevel/dir1/bmon/dir3/file5
> > /tmp/toplevel/dir1/bmon/file2
> >
> > Only files under *mon directories are copied.
> >
> > Regards,
> >
> > Simos
> > _______________________________________________
> > Web Page:  http://lug.boulder.co.us
> > Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
> > Join us on IRC: irc.hackingsociety.org port=6667 channel=#hackingsociety
> >