[lug] Reliable SATA server?

[Lee Woodworth]

On 05/08/12 08:36, Rob Nagler wrote:

A data point for you.

....

> 
> The refurb box with rails is about $900 delivered.  It has 2 x
> quadcore 3ghz Xeons, 8GB,
> redundant power supply, dual enet, and remote access card.  It's
> really quite a beast,
> and way over kill for my problem, and quite a deal imiho.

On an 8 core Dell R515
    dual Opteron 4130 2.6GHZ, 512KB L1, 2MB L2, 5MB L3
    16GB RAM,
    Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]
    SATA 6Gbps backplane
    2 500GB Seagate SATA 3Gbps
    2 TB WD Sata 3Gbps

Doing 4 of these in parallel:

    dd if=/dev/zero of=/dev/sdx oflag=direct,nonblock bs=32768

When finished, dd reports a rate of about 108MB/s for each copy,
with starting rates of 147MB/s for the WD disks (from kill -SIGUSR1).

A guaranteed-not-to-exceed number for this box. I suspect I need to
also use dsync to completely bypass the buffer cache.

....

> that in plenty of time. The biggest job, in fact, is not the cp
> --link, but the weekly
> tars.  They take well over 24 hours on both machines.  The tar runs between
> different disks, and the writes are very sequential (typical file size
> is 100MB).
> The compression is probably a big cost on the slow boxes.  I'm curious how
> the Dell will fair.

On the same R515 system:

The / file system is xfs, on an lvm volume, on a software radi1:

    121908 files/7646 dirs using 2,391,508 KB

The target file system for the following is an xfs file system
on an lvm volume on a software raid1.

tar bzip2:

    time tar --bzip2 --one-file-system -cpf /mnt/backup/root_fs.tar.bz2 /

    real    6m51.587s
    user    6m49.650s
    sys     0m8.073s

    occasional burst writes of 30MB/s (every 10s or so) with a final
    100MB/s burst at the end

    only one core is ever active and is pegged at 100%

rsync copy:

    time rsync --archive --one-file-system / /mnt/backup/root_fs/

    real    1m24.066s
    user    0m15.161s
    sys     0m18.561s

    bursty i/o, periodically 100MB/s over 1 second intervals
    ~20% of 1 cpu for the rsync processes

tree copy:

    time cp -rp --one-file-system / /mnt/backup/root_fs/

    real    0m19.888s
    user    0m0.408s
    sys     0m6.852s

    bursty i/o, periodically 100MB/s over 1 second intervals

tree hard link:

    time cp -rp -l /mnt/backup/root_fs/* /mnt/backup/root_fs_links/

    real    0m6.136s
    user    0m0.140s
    sys     0m1.648s

tree delete:

    time rm -rf /mnt/backup/root_fs_links/*

    real    0m4.527s
    user    0m0.096s
    sys     0m1.052s

    time rm -rf /mnt/backup/root_fs/*

    real    0m6.395s
    user    0m0.084s
    sys     0m3.624s