[Beowulf] Torrents for HPC
atp at piskorski.com
Mon Jun 11 23:37:35 PDT 2012
On Mon, Jun 11, 2012 at 02:02:43PM -0400, Jesse Becker wrote:
> I found that the time to build the torrent files--calculating checksums
> and such--was *far* more time consuming than the actual file
> distribution. This is on top of the rather severe IO hit on the "seed"
> box as well.
Hm, I wonder if zsync does better: http://zsync.moria.org.uk/
Just now with zsync v0.6.1 (from 2009), running zsyncmake on a 696 MB
*.iso file took 9.7 seconds on my (rather pedestrian) desktop. That
was reading from and writing to the same SATA disk, and it used one
cpu core at about 80% the whole time. When I ran two zsyncmakes at
once, each one took twice as long and only used 40% cpu, so that 70
MB/s clearly seems to limited by disk-IO on this machine, not cpu.
Andrew Piskorski <atp at piskorski.com>
More information about the Beowulf