[lug] help with the infamous rsync include/exclude patterns

[Davide Del Vento]

Hi Simos,
Thanks for clarifying. This indeed works great.
Very appreciated!
Davide

On Fri, May 13, 2016 at 4:44 PM, Simos <blug at chinesetearoom.com> wrote:
> Hi Davide,
>
> I'm possibly not understanding your situation, but you don't need to run the
> find commands on the server. I just used them in my example to show you that
> the rsync command correctly copies only what you need (just data under *mon
> directories). You can just ignore the finds. You just need this to do the sync:
>
> rsync -arvm --include='*/' --include='*mon/***' --exclude='*' /path/to/source /path/to/dest
>
> HTH, but if not, just ignore my ramblings!
>
> Simos
>
> On Fri, 13 May 2016 16:26:40 -0600
> Davide Del Vento <davide.del.vento at gmail.com> wrote:
>
>> Hi Simo,
>> I don't have shell access on the server where I'm downloading files from,
>> so I can't run find there. So far the best option seems to be to use rsync
>> --list-only --recursive to get a file with the list of files, then prune
>> everything that I don't need from such a list_of_files.txt and do something
>> like
>>
>> rsync -av --files-from=list_of_files.txt / /my/local/path/
>>
>> However I was frankly hoping for something easier and which would not
>> require an intermediate file to massage.
>>
>> Thanks a lot for your suggestion!
>>
>> On Fri, May 13, 2016 at 4:02 PM, Simos <blug at chinesetearoom.com> wrote:
>>
>> > On Fri, 13 May 2016 11:21:57 -0600
>> > Davide Del Vento <davide.del.vento at gmail.com> wrote:
>> >
>> > > I have this question at
>> > >
>> > http://stackoverflow.com/questions/37198595/how-to-properly-match-directories-with-rsync
>> > > if you prefer to answer there. Otherwise keep reading
>> > >
>> > > I need to rsync some directories from one machine to another, but only
>> > > part of the tree. I think the easiest way to explain it would be to
>> > > run, on the server, tree -f (which prints the full path of each file)
>> > > and then grep'ing mon in the full path and rsynch'ing only the matches
>> > > of grep. Actually I would be happy to do it that way, but I don't have
>> > > shell access to the server, only rsync. Two step solution based on
>> > > greping output of rsync --list-only is totally acceptable.
>> > >
>> > > It's actually simpler, because I also know that this mon appears only
>> > > as a suffix to a directory (the hierarchy is pretty rigid), so I tried
>> > > doing
>> > >
>> > > rsync --dry-run -vrlpt --include='*/' --include='**/*mon/'
>> > > --exclude='*' --prune-empty-dirs
>> > >
>> > > but that did not match anything. Ditto for --include='*mon/' instead
>> > > of the one with the double asterisk. Avoiding the --prune-empty-dirs
>> > > matched all the directories but no files, which I think is my problem.
>> > > So I tried to add an include pattern for some file extensions right
>> > > before the exclude pattern (which I would rather not do because I
>> > > don't want to track them all and they might change), but that simply
>> > > matched all those files, including the ones not in the "mon" trees.
>> > >
>> > > And yes, I did see countless examples of people doing similar stuff,
>> > > which helped me making some progress (i.e. what you can see above),
>> > > but not solve the problem...
>> > > _______________________________________________
>> > > Web Page:  http://lug.boulder.co.us
>> > > Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
>> > > Join us on IRC: irc.hackingsociety.org port=6667 channel=#hackingsociety
>> >
>> > Hi Davide,
>> >
>> > How about this?
>> >
>> > $ find toplevel | sort
>> > toplevel
>> > toplevel/amon
>> > toplevel/amon/file3
>> > toplevel/dir1
>> > toplevel/dir1/bmon
>> > toplevel/dir1/bmon/dir3
>> > toplevel/dir1/bmon/dir3/file5
>> > toplevel/dir1/bmon/file2
>> > toplevel/dir1/file1
>> > toplevel/dir2
>> > toplevel/dir2/file6
>> > toplevel/emon
>> > toplevel/file4
>> >
>> > $ rsync -arvm --include='*/' --include='*mon/***' --exclude='*' toplevel /tmp
>> >
>> > $ find /tmp/toplevel | sort
>> > /tmp/toplevel
>> > /tmp/toplevel/amon
>> > /tmp/toplevel/amon/file3
>> > /tmp/toplevel/dir1
>> > /tmp/toplevel/dir1/bmon
>> > /tmp/toplevel/dir1/bmon/dir3
>> > /tmp/toplevel/dir1/bmon/dir3/file5
>> > /tmp/toplevel/dir1/bmon/file2
>> >
>> > Only files under *mon directories are copied.
>> >
>> > Regards,
>> >
>> > Simos
>> > _______________________________________________
>> > Web Page:  http://lug.boulder.co.us
>> > Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
>> > Join us on IRC: irc.hackingsociety.org port=6667 channel=#hackingsociety
>> >
>
> _______________________________________________
> Web Page:  http://lug.boulder.co.us
> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
> Join us on IRC: irc.hackingsociety.org port=6667 channel=#hackingsociety