[Beowulf] Advice on 4 CPU node configurations
Craig.Tierney at noaa.gov
Tue Feb 8 19:40:04 PST 2011
On 2/8/11 11:54 AM, Gus Correa wrote:
> How about the wiring?
> Since it is switchless, how much cabling would it require?
> The cables don't seem to be fiber.
> How do you implement the torus (or other) topology?
> With daisy-chains?
> A 2D/3D (wraparound) mesh?
> I could only find a datasheet for the card and cables
> on their site, no pictures of a cluster or servers
I believe the cables are still copper. The connectors
looked similar at SC10 when I tried out Dolphinics
For a 3D card there are 6 ports, so you get two connections
in the X,Y, and Z directions. So the 3D torus is just basically
wiring the nodes serially along each axis. I don't recall
if any of the dimensions wrap around. But if you use
36 nodes (3x3x4) you can easily build a 864 core/1.7TB shared memory
node in a rack (24 cores/node, 48GB/node).
How it performs is the real question.
> Gus Correa
> Douglas Eadline wrote:
>> Although I wrote about the SMP and the Numascale hardware,
>> I have not yet had a chance to use it.
>> To me there are two issues worth considering. First,
>> if you need a big SMP this might be a low
>> cost solution. Second, should your application
>> not scale best in a node-based ccNuma system (ScaleMP
>> or Numascale), MPI is still an option. Indeed, no need
>> to rewrite the codes. And, management is probably
>> much easier.
>> Of course for large systems, clusters work best.
>>> On Thu, 2011-02-03 at 09:45 +0000, Hearns, John wrote:
>>>> Also look at ScaleMP and Numascale
>>>> Here's a damn good article from Doug Eadline:
>>>> I must admit though I don't know how far a budget of 30K takes you
>>> My understanding is that the Numascale cards cost "a bit more" than IB
>>> cards but you don't need a switch (it's a 3D taurus I think) so it would
>>> be worth looking into.
>>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
>>> To change your subscription (digest mode or unsubscribe) visit
>>> This message has been scanned for viruses and
>>> dangerous content by MailScanner, and is
>>> believed to be clean.
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
More information about the Beowulf