[Beowulf] [jak at uiuc.edu: [Xgrid] Re: megaFlopsper Dollar? real world requirements]

[Eugen Leitl]

----- Forwarded message from "Jay A. Kreibich" <jak at uiuc.edu> -----

From: "Jay A. Kreibich" <jak at uiuc.edu>
Date: Sun, 15 May 2005 16:25:38 -0500
To: Eugen Leitl <eugen at leitl.org>
Cc: xgrid-users at lists.apple.com
Subject: [Xgrid] Re: megaFlopsper Dollar? real world requirements
Reply-To: jak at uiuc.edu
User-Agent: Mutt/1.4.2.1i

On Sun, May 15, 2005 at 06:48:20PM +0200, Eugen Leitl scratched on the wall:

> Don't forget that the highest data rate available is FedEx'ing a box of
> diskdrives.

  Although there is a grand history of comments like this, (I believe
  the original quote was "Never underestimate the bandwidth of a
  station wagon full of tapes") it often isn't actually true.  Sure, it
  wins "door-to-door", but having a box of hard drives next to my
  computer does not make the data usable.  For a valid look you need to
  look at memory-to-memory transfer.  In other words, include the time
  write the hard drives, disconnect them, mail them, reconnect them,
  and read all the data.  This always kills most tape arguments, and
  has slowly been eroding the hard drive argument.
  
  With OC-192 and 10GigE long-haul technology available (we're talking
  possible, not "possible on a budget"), for memory-to-memory
  operations networks usually win.  The simple fact that hard drive
  bandwidth isn't very high, and you have to push all that data through
  there twice (once to copy onto the disk, once to copy back off).

  There is also the obvious drawback that your bandwidth is very large,
  but your latency is "less than ideal."  On the other hand, it is
  usually dirt cheap next to long-haul high performance data circuits.

  It is a good reminder that sometimes the simplest solutions are the
  best, however.

> Typical 1394
> devices have 2 external ports plus an internal port, interconnected at a
> sort of "hub", but it's not a passive hub. It has significant smarts, maybe
> a "switch" might be a better conceptual model.

  The term "switch" in the networking world implies something very
  specific, mostly data isolation.  I've often wondered if FireWire
  "hubs" do this?  Given the name "hub", I've often assumed a repeater
  based technology, but I realize that's my networking bias.

> The other wrench in the works is that the raw bandwidth between two nodes is
> divided up into channels (just like buying a T-1 data wire... you can run
> one fat pipe, several 384 kbps H.320 links, or 64kbps voice channels, or any
> combination).  If you're just streaming sound for instance, you can allocate
> an isochronous channel to carry that bandwidth, or, if you need video, you
> allocate more bonded isochronous channels.  There's always a low rate
> channel available for "ad-hoc" messages.

  This is another reason I've always assumed FireWire "hubs" do not do
  bandwidth isolation.  If they did, each channel allocation would have
  to be per-link, rather then per-network, and it seems the resource
  allocation system would get quite complex (and that every interface
  would require a large amount of logic to perform such calculations).

   -j

-- 
                     Jay A. Kreibich | CommTech, Emrg Net Tech Svcs
                        jak at uiuc.edu | Campus IT & Edu Svcs
          <http://www.uiuc.edu/~jak> | University of Illinois at U/C

----- End forwarded message -----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20050517/7c992f2c/attachment.sig>