[Beowulf] fast interconnects, HT 3.0 ...

[Richard Walsh]

Eugen Leitl wrote:
> On Tue, May 23, 2006 at 08:49:08AM -0500, Richard Walsh wrote:
>
>   
>>    Speaking of Hyper Transport 3.0 and its AC chassis-to-chassis 
>> capabilities and 10 to 20
>>    Gbps performance maximums one-way (non-coherent, off chassis I 
>> believe), what do the
>>    people that know say about scalability.  Are we looking at coherency 
>>     
>
> I don't know (like it would stop me), but there are few-port HT switches,
> and if there are several ports on one chassis one could wire up
> some topology, which, hopefully, will match the problem. 
>
>   
    HT switches ... ?? ... can you point me to a reference?
>> within the board complex
>>    and basic reference ability off board or something else?
>>     
>
> I'm not sure how this is different from vanilla packet-switched
> MPI network. It's not about maintaining memory coherency.
>   
    Well, of course you can run MPI over it as you can on the Cray and 
Altix, but
    you are artificially separating memory in software that is in fact 
closer in hardware.
    That is where the pGAS programming models become more efficient.  Remote
    memory references expressed in the syntax and compiled to 
instructions for
    direct puts and gets without management or translation by a NIC.  It 
would seem
    that HT 3.0 supports this model across chassis as long as the 
programmer manages
    memory synchronization.
>>    Sounds like the Cray X1E pGAS memory model.  Is there a role for 
>>     
>
> I don't think there is any other model but message passing. It's not
> like this is a ccHT a la HORUS http://en.wikipedia.org/wiki/HORUS_interconnect
>   
    The inter-chassis, but non-coherent interface that HT 3.0 supports 
would seem to work
    very nicely with UPC and CAF.  They run very well on the Cray X1, 
which provides
    coherent memory on-board only as well. 

    Regards,

    rbw
> ------------------------------------------------------------------------
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>   


-- 

Richard B. Walsh

Project Manager
Network Computing Services, Inc.
Army High Performance Computing Research Center (AHPCRC)
rbw at ahpcrc.org  |  612.337.3467

-----------------------------------------------------------------------
This message (including any attachments) may contain proprietary or
privileged information, the use and disclosure of which is legally
restricted.  If you have received this message in error please notify
the sender by reply message, do not otherwise distribute it, and delete
this message, with all of its contents, from your files.
-----------------------------------------------------------------------