[lug] help with the infamous rsync include/exclude patterns

[Davide Del Vento]

Hi Simo,
I don't have shell access on the server where I'm downloading files from,
so I can't run find there. So far the best option seems to be to use rsync
--list-only --recursive to get a file with the list of files, then prune
everything that I don't need from such a list_of_files.txt and do something
like

rsync -av --files-from=list_of_files.txt / /my/local/path/

However I was frankly hoping for something easier and which would not
require an intermediate file to massage.

Thanks a lot for your suggestion!

On Fri, May 13, 2016 at 4:02 PM, Simos <blug at chinesetearoom.com> wrote:

> On Fri, 13 May 2016 11:21:57 -0600
> Davide Del Vento <davide.del.vento at gmail.com> wrote:
>
> > I have this question at
> >
> http://stackoverflow.com/questions/37198595/how-to-properly-match-directories-with-rsync
> > if you prefer to answer there. Otherwise keep reading
> >
> > I need to rsync some directories from one machine to another, but only
> > part of the tree. I think the easiest way to explain it would be to
> > run, on the server, tree -f (which prints the full path of each file)
> > and then grep'ing mon in the full path and rsynch'ing only the matches
> > of grep. Actually I would be happy to do it that way, but I don't have
> > shell access to the server, only rsync. Two step solution based on
> > greping output of rsync --list-only is totally acceptable.
> >
> > It's actually simpler, because I also know that this mon appears only
> > as a suffix to a directory (the hierarchy is pretty rigid), so I tried
> > doing
> >
> > rsync --dry-run -vrlpt --include='*/' --include='**/*mon/'
> > --exclude='*' --prune-empty-dirs
> >
> > but that did not match anything. Ditto for --include='*mon/' instead
> > of the one with the double asterisk. Avoiding the --prune-empty-dirs
> > matched all the directories but no files, which I think is my problem.
> > So I tried to add an include pattern for some file extensions right
> > before the exclude pattern (which I would rather not do because I
> > don't want to track them all and they might change), but that simply
> > matched all those files, including the ones not in the "mon" trees.
> >
> > And yes, I did see countless examples of people doing similar stuff,
> > which helped me making some progress (i.e. what you can see above),
> > but not solve the problem...
> > _______________________________________________
> > Web Page:  http://lug.boulder.co.us
> > Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
> > Join us on IRC: irc.hackingsociety.org port=6667 channel=#hackingsociety
>
> Hi Davide,
>
> How about this?
>
> $ find toplevel | sort
> toplevel
> toplevel/amon
> toplevel/amon/file3
> toplevel/dir1
> toplevel/dir1/bmon
> toplevel/dir1/bmon/dir3
> toplevel/dir1/bmon/dir3/file5
> toplevel/dir1/bmon/file2
> toplevel/dir1/file1
> toplevel/dir2
> toplevel/dir2/file6
> toplevel/emon
> toplevel/file4
>
> $ rsync -arvm --include='*/' --include='*mon/***' --exclude='*' toplevel
> /tmp
>
> $ find /tmp/toplevel | sort
> /tmp/toplevel
> /tmp/toplevel/amon
> /tmp/toplevel/amon/file3
> /tmp/toplevel/dir1
> /tmp/toplevel/dir1/bmon
> /tmp/toplevel/dir1/bmon/dir3
> /tmp/toplevel/dir1/bmon/dir3/file5
> /tmp/toplevel/dir1/bmon/file2
>
> Only files under *mon directories are copied.
>
> Regards,
>
> Simos
> _______________________________________________
> Web Page:  http://lug.boulder.co.us
> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
> Join us on IRC: irc.hackingsociety.org port=6667 channel=#hackingsociety
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lug.boulder.co.us/pipermail/lug/attachments/20160513/2a41e96c/attachment.html>