[lug] Reliable SATA server?

[Lee Woodworth]

Hi Rob,

pbzip2 is an almost linear improvement in this case
where all of the source is cached. Parallel bzip2 is
0:59, regular is 6:52. Similar compression too:
774MB vs 772MB.

Average write speed is now 13.1MB/s at about 3.1 : 1 compression.
So the source read needs to average better than 50MB/s to keep up.
With non-cached source and small files, it might be harder to
do that.

time tar --one-file-system -cpf - / | (time pbzip2 > /mnt/backup/root_fs.tar.bz2)

    real    0m58.801s
    user    7m39.549s
    sys     0m2.712s

    real    0m58.802s
    user    7m40.037s
    sys     0m4.828s

8 cores running at 100%. Some write bursts of 130MB/s in the middle,
and then 2-3 seconds of 130MB/s writes at the end.


On 05/09/12 08:13, Rob Nagler wrote:
> Hi Lee,
> 
>> There are two processes: tar (1.0%) and bzip2 (99.0%) effectively running
> 
> This would improve things then:
> 
> http://compression.ca/pbzip2/
> 
> I tend to use gzip, because it is faster to compress, and many of our
> files are already compressed (docx, mov, etc.):
> 
> http://www.zlib.net/pigz/
> 
> Rob
> _______________________________________________
> Web Page:  http://lug.boulder.co.us
> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
> Join us on IRC: irc.hackingsociety.org port=6667 channel=#hackingsociety