[Beowulf] Any recommendations for a good JBOD?

[Mark Hahn]

> I'm curious, what's the selling point for iSCSI then? The prices are
> quite ramped up and the performance not stellar. Do any of you in the
> HPC world buy i-SCSI at all?

ease, I suppose.  ethernet is omnipresent, so anything which uses 
ethernet has a big advantage.  the sticking point is really that 
the ethernet phenomenon has to some extent stalled at 1 Gb - 
that's where the mass market is, and it's not that clear whether
or how much drive there will be for adoption above 1Gb.

in homes, 1Gb is mostly overkill.  non-rich office settings 
(such as a lot of academia) still have 100bT to the desktop.
1Gb is plenty good enough for a lot of HPC - obviously almost
anything serial, small parallel or even large-loose parallel.

even though a single modern disk sustains greater BW than 1Gb,
I find that most users do not have an expectation of really 
high storage bandwidth.  sure, people who do good checkpointing
from large parallel apps cry for it.  some specialized fields 
want bandwidth even for serial jobs.  but a typical job in my
organization (disparate academic HPC) does IO infrequently and 
not very much.  when our clusters have IO problems, it's more 
often metadata for shared filesystems, rather than bandwidth.

however, I'm not actually claiming iSCSI is prevalent.  the protocol
is relatively heavy-weight, and it's really only providing SAN access,
not shared, file-level access, which is ultimately what most want...

>> procs+ram+nic can easily total less than $100; enclosures can be very cheap
>
> Really?! Hmm....What kind of "procs+ram+nic" combo can one get for
> less than $100. That is pretty surprising for me! That again makes me

atom motherboards (which include the CPU) start at $58 on newegg,
though the lowest-end items aren't all that appealing.  the first 
Gb models start at $80, so add a $20 dimm, and you're done.

(admittedly, small boards like those, being ITX, typically have 2x 
sata ports, not 6x.  but adding a cheap sata controller or just 
starting with a cheap uATX board doesn't really blow the budget...)

> wonder about the performance of these NAS boxes. Standard server + DAS
> seems safer than a low-end NAS.

the cheapest NAS boxes are build with arm/mips SOCs much like a 
wifi router.  (third party/free linux firmware for routers often
also includes such NAS boxes.)  recently a generation of Atom-based
NAS has come onto the market, which perform better, somewhat hotter, etc.
these appear to be the first cheap NAS to come close to saturating Gb - 
the embedded cpu models tended to poke along at 20-40 MB/s.

regards, mark hahn.