[Beowulf]Infrastruture planning for small HPC 40/100 gigabyet eyhernet or Infiniband?

[Robert G. Brown]

On Mon, 28 Jul 2008, Gerry Creager wrote:

> Back when the Earth was young, and the crust was still cooling, we ran serial 
> connections between computers, over long distances and sometimes between 
> power distributions.  It wasn't uncommon to see ground loops lead to arcing. 
> I don't see it as much now because I'm a little more careful about my 
> grounds, and I bridge such problems with glass rather than copper.
>
> The potential is still very real.

The potential is very real, and even if the wires at both ends are
"supposed" to not be touching anything even as "neutral" as the case
ground, given the number of machines with network interfaces made by
small shops in taiwan or the phillipines out of a stock chip but with
their own local design team, who can doubt that there are ones where
they do?  Ground loops are generally murphy's law objects, and since
they CAN happen, sooner or later they will.

    rgb

>
> gerry
>
> David Mathog wrote:
>> Jim Lux <james.p.lux at jpl.nasa.gov> wrote
>> 
>>> Quoting "Robert G. Brown" <rgb at phy.duke.edu>, on Mon 28 Jul 2008  06:15:44 
>>> AM PDT:
>>> 
>>>> On Mon, 28 Jul 2008, Eugen Leitl wrote:
>>>> 
>>>>> On Sun, Jul 27, 2008 at 07:19:56PM -0700, Jim Lux wrote:
>>>>> 
>>>>>> bear in mind that ordinary ethernet both coax and twisted pair is
>>>>>> galvanically isolated.
>>>>> This is strange, because I've seen (small) sparks and received (mild)
>>>>> shocks from both, in two different locations.
>>>> Ground loop.  Very dangerous.  You go first...;-)
>>>>
>>>>    rgb
>>> 
>>> Very odd.. I'd be looking for an outright short from the cables to 
>>> something (or, a LOT of capacitive coupling)...
>> 
>> Could this possibly be static electricity discharging?  Is the humidity
>> very low where this is being seen, and or, is the operator moving over
>> carpet shortly before the spark is observed?
>> 
>> I can't say that I've ever seen sparks leave an ethernet cable even here
>> in Pasadena when the winter humidity is close to zero, but I have had
>> sparks jump off my fingers as they passed near mounting screws on wall
>> plates.  In spark season I routinely get blasted by my car's door
>> handle, and there's definitely no ground loop going on there.
>> 
>> David Mathog
>> mathog at caltech.edu
>> Manager, Sequence Analysis Facility, Biology Division, Caltech
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org
>> To change your subscription (digest mode or unsubscribe) visit 
>> http://www.beowulf.org/mailman/listinfo/beowulf
>
>

-- 
Robert G. Brown                            Phone(cell): 1-919-280-8443
Duke University Physics Dept, Box 90305
Durham, N.C. 27708-0305
Web: http://www.phy.duke.edu/~rgb
Book of Lilith Website: http://www.phy.duke.edu/~rgb/Lilith/Lilith.php
Lulu Bookstore: http://stores.lulu.com/store.php?fAcctID=877977