[Beowulf] Torrents for HPC

[Andrew Piskorski]

On Mon, Jun 11, 2012 at 02:02:43PM -0400, Jesse Becker wrote:

> I found that the time to build the torrent files--calculating checksums
> and such--was *far* more time consuming than the actual file
> distribution.  This is on top of the rather severe IO hit on the "seed"
> box as well.  

Hm, I wonder if zsync does better:  http://zsync.moria.org.uk/

Just now with zsync v0.6.1 (from 2009), running zsyncmake on a 696 MB
*.iso file took 9.7 seconds on my (rather pedestrian) desktop.  That
was reading from and writing to the same SATA disk, and it used one
cpu core at about 80% the whole time.  When I ran two zsyncmakes at
once, each one took twice as long and only used 40% cpu, so that 70
MB/s clearly seems to limited by disk-IO on this machine, not cpu.

-- 
Andrew Piskorski <atp at piskorski.com>
http://www.piskorski.com/