[Beowulf] Register article on Cray Cascade

Vincent Diepeveen diep at xs4all.nl
Fri Nov 9 11:36:12 PST 2012


On Nov 9, 2012, at 7:31 PM, atchley tds.net wrote:

> Modern switches need 100-150 ns per hop.

yeah that's BS when you have software that goes measure that with all  
cores busy.

I wrote a benchmark to measure that with all cores busy.

The SGI box back then that was having 50 ns switches which would have  
'in theory' a latency of 480 ns @ 500 cpu's,
so 960 for a blocked read, i couldn't get it down to less than 5.8 us  
on average.



> There are some things that do not scale per hp such as traversing  
> the PCIE link from socket to NIC and back. So, I see it as 1.2 to  
> go to the router and back and 100 ns per hop.
>
> Scott
>
>
> On Fri, Nov 9, 2012 at 11:17 AM, Vincent Diepeveen <diep at xs4all.nl>  
> wrote:
> The latency estimate taking 5 hops seems a tad optimistic to me
> except when i read the English wrong and they mean 1.7 microseconds a
> hop making it for a 5 hop 5 * 1.7 = 8.5 microseconds in total.
>
> "Not every node is only one hop away, of course. On a fully
> configured system, you are five hopes away maximum from any socket,
> so there is some latency. But the delta is pretty small with
> Dragonfly, with a minimum of about 1.2 microseconds for a short hop,
> an average of 1.5 microseconds on average, and a maximum of 1.7
> microseconds for the five-hop jump, according to Bolding."
>
> On Nov 8, 2012, at 7:13 PM, Hearns, John wrote:
>
> > Well worth a read:
> >
> >
> >
> > http://www.theregister.co.uk/2012/11/08/
> > cray_cascade_xc30_supercomputer/
> >
> >
> >
> > John Hearns | CFD Hardware Specialist | McLaren Racing Limited
> > McLaren Technology Centre, Chertsey Road, Woking, Surrey GU21  
> 4YH, UK
> >
> >
> > T:  +44 (0) 1483 262000
> >
> > D:  +44 (0) 1483 262352
> >
> > F:  +44 (0) 1483 261928
> > E:  john.hearns at mclaren.com
> >
> > W: www.mclaren.com
> >
> >
> >
> > The contents of this email are confidential and for the exclusive
> > use of the intended recipient. If you receive this email in error
> > you should not copy it, retransmit it, use it or disclose its
> > contents but should return it to the sender immediately and delete
> > your copy.
> >
> > _______________________________________________
> > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin
> > Computing
> > To change your subscription (digest mode or unsubscribe) visit
> > http://www.beowulf.org/mailman/listinfo/beowulf
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
> Computing
> To change your subscription (digest mode or unsubscribe) visit  
> http://www.beowulf.org/mailman/listinfo/beowulf
>



More information about the Beowulf mailing list