[Beowulf] how large can we go with 1GB Ethernet? / Re: how large of an installation have people used NFS, with?

[psc]

I wonder what would be the sensible biggest cluster possible based on
1GB Ethernet network . And especially how would you connect those 1GB
switches together -- now we have (on one of our four clusters) Two 48
ports gigabit switches connected together with 6 patch cables and I just
ran out of ports for expansion and wonder where to go from here as we
already have four clusters and it would be great to stop adding cluster
and start expending them beyond number of outlets on the switch/s ....
NFS and 1GB Ethernet works great for us and we want to stick with it ,
but we would love to find a way how to overcome the current "switch
limitation".   ... I heard that there are some "stackable switches" ..
in any case -- any idea , suggestion will be appreciated.

thanks!!
psc

> From: Rahul Nabar <rpnabar at gmail.com>
> Subject: [Beowulf] how large of an installation have people used NFS
> 	with?	would 300 mounts kill performance?
> To: Beowulf Mailing List <beowulf at beowulf.org>
> Message-ID:
> 	<c4d69730909091040p3774581dmd50b460dc99e0a60 at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> Our new cluster aims to have around 300 compute nodes. I was wondering
> what is the largest setup people have tested NFS with? Any tips or
> comments? There seems no way for me to say if it will scale well or
> not.
>
> I have been warned of performance hits but how bad will they be?
> Infiniband is touted as a solution but the economics don't work out.
> My question is this:
>
> Assume each of my compute nodes have gigabit ethernet AND I specify
> the switch such that it can handle full line capacity on all ports.
> Will there still be performance hits as I start adding compute nodes?
> Why? Or is it unrealistic to configure a switching setup with full
> line capacities on 300 ports?
>
> If not NFS then Lustre etc options do exist. But the more I read about
> those the more I am convinced that those open another big can of
> worms. Besides, if NFS would work I do not want to switch.
>
>