[Beowulf] Cheap SDR IB
Shainer at mellanox.com
Thu Jan 31 10:09:39 PST 2008
> I thought the thrust of the original post was that you can
> build now a cheap IB cluster with up to 24 nodes. The
> subsequent discussion was around questioning whether you need
> IB for up to 16-24 nodes.
> The advantage you point to is for 32 nodes. There is no
> question that IB is much better at this scaling point for
> many codes, not just WRF.
> I used to think that the typical break point is about 16
> nodes. We have a lot of app data on our web site which
> confirm this and will be looking into how (if at all) quad
> cores change this.
With more cores on a single node, the IB benefits are seen in much lower number of nodes. I am testing some applications on a new cluster that I have (dual sockets quad core Barcelona), and my first results are with Fluent new benchmarks. I will have the numbers posted soon, so you all can take a look. For 2 nodes, IB shows an average of 15-20% higher performance then GigE, and this gap gets bigger with cluster size. At 4 nodes the difference was 40-50%. Even more important, 3 nodes results with IB were higher then 8 nodes with GigE, and GigE stop scaling after 3-4 nodes (performance numbers were flat after 3-4 nodes).
This is only one example, I know, but I am sure there will be many more. I personally going to check it on more applications, and would appreciate any suggestion on other applications people have interest to check.
> As far as the games are concerned never underestimate what
> people can do if sufficient and affordable resources are
> provided. If the opposite were true we'd be still staring at
> black and white/green screens and playing packman. Game
> developers have to offload a lot (too much in fact) to a
> client out of necessity, not because they want to (BTW, there
> are amazing exploits because of the lags and the way the
> clients are built). Having said that there is no question
> that IB is hardly applicable to MMORPGs on the scale they
> are, probably not even for LAN parties. However a small
> cluster for real time ray tracing might be a good proposition.
More information about the Beowulf