[Beowulf] Nvidia and IBM create GPU interconnect for faster supercomputing

[Prentice Bisbal]

On 03/26/2014 05:28 PM, Jason Riedy wrote:
> And Prentice Bisbal writes:
>> I haven't read the article in it's entirety, but it looks like
>> NVidia and IBM have just fired the first shot in Exascale race
>> (apologies for the mixed metaphors).
> More another shot towards fragmentation.  PCIe is something that
> (more or less) everyone can implement.  Not much margin.  Special
> interconnects (on board or between boards) are an open game.
> There are related announcements from Intel.  I'm kinda shocked
> (well, no, but I wish I were) at the lack of such an announcement
> from AMD.

In the rarefied market that is the race to Exascale, I don't think 
fragmentation is a big concern. It's not mentioned in the article, but I 
don't think this technology is going to be included in mainstream or 
commodity systems. I think this is the first step towards whatever IBM 
product is going to replace the Blue Gene, and in in that space 
fragmentation isn't really a concern.

This partnership makes total sense. Until the GPUs came onto the scene, 
IBM was pretty much unchallenged in the FLOPS/W game. Then GPUs came 
along and presented the first real challenge (SiCortex went under before 
it could be a contender). The BG/Q dominated the Green500 in June of 
2012, but then in November 2012 Green500 list came out and the BG/Q was 
knocked back to 5th place by 4 GPU systems (3 with Nvidia Teslas, one 
with AMD FirePro). The current list is dominated by systems with Tesla 
K20 GPUs. In these systems, I'm sure that the Xeon processors are 
putting a noticeable dent in the FLOP/W score and is a hindrance on the 
march to Exascale.

How do you fix this? You replace the Xeons with low power CPUS, 
something IBM has been very successful with with the Blue Gene systems. 
Or, depending on your point of view, you add GPUs to your Blue Gene. 
Adding NVLink will increase performance that much more. What would be 
interesting is if IBM could replace the existing proprietary network in 
the BG/Q with NVlink for internode communication, too. Probably doesn't 
scale beyond a few processors, but it's a thought.

--
Prentice