diskless nodes? (was Re: Xbox clusters?)
bogdan.costescu at iwr.uni-heidelberg.de
Thu Dec 6 04:28:01 PST 2001
On Thu, 6 Dec 2001, Velocet wrote:
> if you wanted it to be faster, just add more networks to the server...
> ... LVD 160 controllers are PRETTY damn fast. Put it all on a PCI-X
> board 64bit/133MHz pci ...
> ...then just start using multiple servers...
> Hell even put 2 or 3 NICs in each node...
Maybe I'm over-looking something, but you do all these to replace one
inexpensive IDE disk per node ? Here in Germany, I can get a resonably
fast and big IDE disk for 2-3 times the price of a (3Com, of course 8-))
100Mbit network card; for multiple network cards per node, you also need
to get multiple/big switches and a fat pipe (be it 100Mbit bonded or
Gigabit - bonded or not) for the NFS server, which needs to still provide
a pretty large storage capacity. Err, I think that I'll stick to local
[Maybe from another message] The fact that you are saving over NFS
temporary files which are only supposed to be accessed by the node which
created them is in fact a good thing. If other nodes would need to access
them as well, you get into coherency problems, want to disable client
caching and loose performance big time; however, in this case having the
temporary files on local disks or on a NFS server doesn't make much
> I'd go insane configuring things if I had disk-full nodes.
I think that you are sticking together 2 things: having local disks and
booting/using root FS from them. One of our clusters (which evolved from a
diskless state to an almost disk-full state) boots over network (RARP,
root FS on the main node), while the local disks are available exclusively
for scratch/temporary files. This way the network (one 100Mbit card per
node) is used almost exclusively for message passing; accesing the root FS
is just a negligible part of the traffic and input and output files (dealt
with only on the first node of a job) are not so big either. And BTW, we
are doing things not very different from what you do: mainly MM (CHARMM)
and QM/MM (CHARMM + GAMESS, NWChem).
However, I'm not saying that our solution is the best - just that it fits
our needs. That's all that matters, right ?
IWR - Interdisziplinaeres Zentrum fuer Wissenschaftliches Rechnen
Universitaet Heidelberg, INF 368, D-69120 Heidelberg, GERMANY
Telephone: +49 6221 54 8869, Telefax: +49 6221 54 8868
E-mail: Bogdan.Costescu at IWR.Uni-Heidelberg.De
More information about the Beowulf