[Beowulf] [EXTERNAL] IB vs. Ethernet

[Lux, Jim (US 430E)]

Most fast (>1 Gbps) mediums use some form of adaptive equalization which adds some delay (some number of bit times).
For FEC, it would depend on the code - I assume it's some sort of block code vs convolutional?
And, I'm pretty sure that the modulation is multiple bits/symbol on the wire.

Given the info in this article: https://www.signalintegrityjournal.com/articles/1286-gbs-ethernet-forward-error-correction-fec-analysis
And
https://www.ieee802.org/3/df/public/22_02/bruckman_3df_01a_220203.pdf

it looks like they are using a RS(544,514) code (RS = Reed-Solomon, 544 symbols transmitted for 514 information bits).  
And they translate 8 octets into 66 bit blocks then transcode those to 257 bit blocks. Which then get RS coded, etc. 

So, in round numbers, it looks like you'd need to have ~500 bits to be able to run the decoder. If those are all in one lane at 100 Gbps - that's 5 nanoseconds.
If it's split across 4 lanes at 25 Gbps, well, it's still 5 nanoseconds. 

That is comparable to the propagation delay in a 1 meter fiber.


-----Original Message-----
From: Beowulf <beowulf-bounces at beowulf.org> On Behalf Of Greg Lindahl
Sent: Sunday, March 15, 2026 11:15 PM
To: beowulf at beowulf.org
Subject: Re: [Beowulf] [EXTERNAL] IB vs. Ethernet

On Mon, Mar 02, 2026 at 09:08:31AM -0500, Scott Atchley wrote:

> How much does FEC add? I have been under the impression that it is now 
> mandatory ≥100Gbps.

You're correct that it is now required by ethernet, at 100Gbit and also 1 lane 25 Gbit. I have only worked with an FPGA implementation and it wasn't latency sensitive. It also seems to be the kind of thing that would be extremely implementation-dependent.

For a history lesson, one of the first things Qlogic wanted to do with Infinipath after buying us was to add FEC. Even below 100 gbits the error rates left all of us nervous.

-- greg


_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit https://urldefense.us/v3/__https://beowulf.org/cgi-bin/mailman/listinfo/beowulf__;!!PvBDto6Hs4WbVuu7!M9kalXASZv1aNdprCX-1wwMJmWA0qkE08oQGuLNUamsoCaO-LgGts6_uXOxP5NzYoi63FJts4v4U0g3lR_b3nw$