Motherboard query...

[Robert G. Brown]

On Fri, 1 Mar 2002, Jakob Østergaard wrote:

> Yep - for a naive implementation.  There are many performance tricks that can
> be pulled, and Linux currently pulls some of them.  Being smart about reads on
> RAID-5 is one of them.
> 
> > For instance, the 3ware 7410 card we get something like 75Mb/sec writes to 4
> > drives in raid 0 and 5.6MB/sec in raid 5.
> 
> Well, luckily one *implementation* with poor performance doesn't prove that efficient
> implementations are impossible   :)
> 
> ('implementation' here meaning both software and hardware)

Of course I didn't really want to start a RAID war;-), but it is always
lovely to listen when experts debate and very instructive to boot.  Just
for the record, as I said in the original message we use md (software)
RAID 5 already with SCSI disks and get excellent (worst case ~40 MB/sec,
best case more like 70) write performance.  As I understand it, the raid
code uses MMX instructions (if available) to stream a really efficient
XOR?  Mind you, I haven't looked.  

So we know/knew md works well with SCSI and is therefore not >>CPU<<
bound performance any worse than 40 or so MB/sec.  The real question was
whether there were IDE issues (DMA, managing multiple controllers,
whatever) that might limit either performance or the number of IDE
disks/controllers that could be used without md or the kernel
"breaking".

I'd like to thank everybody who has helped with this.  It has been
extremely useful.  I think that so far a reasonable summary of the
various contributions from many people (not all of whom I've had a
chance to independently thank) is:

  a) The Tiger 2466 is much better than the Tiger 2460 (which was/is
plagued with small problems in addition to having a narrower PCI bus).
The 2466 seems to work well but it is still early to have lots of hard
performance numbers from it.

  b) MSI's 760 MPX board has very similar features, appears to work, but
is even less tested.  In a week or two it should be better known, but it
is a reasonable choice to at least prototype as an equivalent to the
Tiger.

  c) Caveat Emptor where IDE-SCSI Raid boxes or controllers are
concerned.  RAID 5 can be done poorly or well, and some vendors do it
poorly (apparently Promise being one of them) and others do it well
(3ware being the most often mentioned and sometimes lavishly praised
example).  Performance measurements were cited that varied by an order
of magnitude (from 4-5 MB/sec through 60-70 MB/sec) in RAID 5 for
different implementations, and some vendors' systems you'd expect to do
well can actually have rotten performance.

  d) Linux's native software raid (md) has excellent performance with
either SCSI disks or IDE disks, and (according to e.g. Mark Hahn) works
well with 3 IDE controllers (two channels each) for up to six disks in
RAID 5.  However be careful -- one can have problems with particular
disks, particular controllers, or two many controllers.  A mailing list
exists for those who wish to try this and encounter problems, but if one
is willing to work a bit and spend as much as a couple of hundred
dollars swapping IDE controllers as necessary to find a combination that
works you should be able to succeed and build your own IDE RAID 5 server
for an easy $2K less than just about any commercial standalone box and a
price probably comparable to a 3ware solution.

Hopefully this is fair enough.  Any errors are obviously my own (I'm
doing this from memory and read most of the responses between 1 and 2 am
this morning:-).

Based on the above, we're going to prototype the MSI board (since our
vendor will loan us one) -- other folks in our cluster group are already
adopting the Tiger 2466 and speak of it enthusiastically although we
haven't tested it yet personally.  We are also going to keep the disks
and send back the RAID enclosure (if necessary accompanied by a testy
letter from a Duke lawyer:-) and recycle them into either one or two
md raids.

Thanks loads again to all who replied.  I learned a lot.

   rgb

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu