[Beowulf] [EXTERNAL] Re: anyone have modern interconnect metrics?

[Lux, Jim (US 3370)]

No big dollar demand, I suspect. Existing off the shelf optical stuff at 10G satisfies most needs.  And practically speaking, for cluster interconnects (relevant to here) – you also want low latency, good bisection bandwidth, and a bunch of other stuff.  I can see a “city to city” optical link wanting 100G or more (although the actual fiber is awfully cheap, why not just run N slower fibers).


From: Scott Atchley <e.scott.atchley at gmail.com>
Date: Thursday, January 18, 2024 at 12:01 PM
To: Jim Lux <james.p.lux at jpl.nasa.gov>
Cc: "beowulf at beowulf.org" <beowulf at beowulf.org>
Subject: Re: [Beowulf] [EXTERNAL] Re: anyone have modern interconnect metrics?

There is a lot of interest in lower-cost optics, but manufacturing costs for the alternatives to today's active optical cables have not provided the promised cost savings. Silicon photonics seems to be just a few years away just as fusion is just a decade away.

On Wed, Jan 17, 2024 at 6:16 PM Lux, Jim (US 3370) via Beowulf <beowulf at beowulf.org<mailto:beowulf at beowulf.org>> wrote:
To a certain extent, faster Ethernet is more likely to be a commodity – and at rates above 1 Gbps, there’s substantial “art” in making a PHY that works reliably.   At the 10G speed, there’s things like RapidIO and SRIO, but they

  1.  Only work for short distances (<<1 meter)
  2.  Are *very* board layout and other implementation sensitive.  Fine for getting in and out of a package, but not great for running any distance.
Then there’s XAUI (pronounced Zowie!) which is a multiwire wire interface between logic and 10G (or whatever) PHY.   But it’s got the same problems as SRIO/RapidIO (or for that matter, the venerable (now) TLK2711 SERDES).

10G and 40G Ethernet do actually work over distances of meters, and over some moderate range of temperatures, and are likely to meet EMI/EMC requirements.

It is interesting that there doesn’t seem to be the same commercial pressure for optical versions. They all exist, but typically as modules you’d slide into your switch, not components you’d solder to a board.  And there are plenty of XAUI->optical kinds of interfaces.  And optical cables are cheap and relatively rugged.


From: Beowulf <beowulf-bounces at beowulf.org<mailto:beowulf-bounces at beowulf.org>> On Behalf Of Scott Atchley
Sent: Wednesday, January 17, 2024 7:18 AM
To: Larry Stewart <stewart at serissa.com<mailto:stewart at serissa.com>>
Cc: Mark Hahn <hahn at mcmaster.ca<mailto:hahn at mcmaster.ca>>; beowulf at beowulf.org<mailto:beowulf at beowulf.org>
Subject: [EXTERNAL] Re: [Beowulf] anyone have modern interconnect metrics?

While I was at Myricom, the founder, Chuck Seitz, used to say that there was Ethernet and Ethernot. He tied Myricom's fate to Ethernet's 10G PHYs.

On Wed, Jan 17, 2024 at 9:08 AM Larry Stewart <stewart at serissa.com<mailto:stewart at serissa.com>> wrote:
I don't know what the networking technology of the future will be like, but it will be called Ethernet.
- unknown (to me)
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org<mailto:Beowulf at beowulf.org> sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit https://beowulf.org/cgi-bin/mailman/listinfo/beowulf<https://urldefense.us/v3/__https:/beowulf.org/cgi-bin/mailman/listinfo/beowulf__;!!PvBDto6Hs4WbVuu7!Ixq5D_bsl4Xvg1t1VWaYCWDx6D996js8p3ZYzDukhOAjJ13TNcEsw9PTrxQIVTKHI5LTjDrXMd4eCrVz-w_P3G62jVN8nSYT8Q$>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://beowulf.org/pipermail/beowulf/attachments/20240118/a76a019d/attachment-0001.htm>