[lug] "Poor" RAID performance?

[D. Stimits]

Sean Reifschneider wrote:
> 
> I was recently running some performance tests (bonnie++) on a RAID array
> and was fairly suprised at the performance, or lack thereof...  Since Rob
> in the past has mentioned that current SCSI discs can saturate 80MB/sec
> controllers, I'd assume that he's seeing something in that neighborhood.
> 
> The setup is a Mylex AcceleRAID 352 dual channel U160 controller, with 6
> 10KRPM drives set up in a RAID-5 array.  According to the controller, all
> drives are talking to the controller at 160MB/sec.

You will get an enormous difference in bandwidth depending on cluster
density. Larger drives tend to have larger density. I have
timed/measured an average transfer of a hair over 30 MB/sec on a single
10k rpm 36 GB drive that was 1/3 full (u160, 64 bit/66MHz PCI, dual
cpu), whereas the same rpm drive at 20 GB will only put out about 20 to
25 MB/sec. The really big drives can actually sustain over 40 MB/sec at
10k rpm. The burst rate is over 140 MB/sec. You did not mention what
kind of raid it is, but 3 high density 10k rpm drives could challenge a
regular PCI bus, then there is the controller itself. You said it is
u160, so the 80 MB/sec limit doesn't apply, it goes up to 160 MB/sec,
and your limit is then the PCI bus (about 133 MB/sec on a normal pci
bus, multiply by 4 for a 64 bit/66MHz bus). If you are using raid 0
(striping), you could easily exceed and saturate a poor little ordinary
pci bus. Even raid 1 has to worry about that during a read, since it can
read from pairs of drives. If one of the setups is not hardware raid
(yours obviously is, but you mentioned comparing it to something else),
then it is highly cpu dependent, and a dual cpu system will benefit
enormously from software raid, whereas single cpu will be helped but the
load will be high enough that it won't achieve its best. In all cases,
you also have to figure out the chunk size, the size of data segments
that are written to one disk before another part gets split to the next
disk. If your chunk size is wrong, you'll be writing so much of your
operation to a single disk it really isn't splitting things evenly. If
it gets too small, it might become more responsive but the total
throughput could drop quite a bit. For people running raid 5, it is
always bad performance, for raid 1, it helps only on read, it hurts on
write. For random access your cache ram in the controller matters a lot
for hardware raid, but hardware raid does not do so great at linear
access, which is where software raid wins the race. The only raid I have
experience with though is scsi, running raid on IDE of course requires
separate channels for every drive.

D. Stimits, stimits at idcomm.com

> 
> So, what is this "poor" performance?
> 
>             ------Sequential Output------ --Sequential Input- --Random-
>             -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
>        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
>       8000M  7089  56  7196  11  5969   5 12242  97 32905  21 381.1   3
> 
> Am I just expecting too much from a mid-end RAID setup?  These numbers are
> not all that much better (and are in some cases lower) than my laptop's
> single 4500RPM IDE drive.
> 
>             -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
>        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
>        700M  5508  82 15684  17  6775   5  4918  77 12615   5  90.4   0
> 
> For comparison sake, I've also run bonnie against 3 7200RPM IDE drives in a
> RAID-0 array (though these drives were kind of mixed and matched, some at
> least a year or two old -- not running the high bit densities seen
> currently):
> 
>             -Per Chr- --Block--  -Rewrite-   -Per Chr- --Block--  --Seeks--
>        Size  K/sec %CP K/sec %CP  K/sec %CP  K/sec %CP  K/sec %CP  /sec %CP
>        2000  8616 78.3 49499 56.7 29567 35.2 10695 94.9 66362 36.1 237.5 2.1
> 
> Am I just wrong to be expecting a $3k RAID array to be faster than, say, a
> laptop IDE drive?
> 
> Sean
> --
>  "The big bad wolf, he learned the rule.  You gotta get hot to play real cool."
> Sean Reifschneider, Inimitably Superfluous <jafo at tummy.com>
> tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python
> _______________________________________________
> Web Page:  http://lug.boulder.co.us
> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug