[Beowulf] Announcing nettee 0.1.4

[Greg M. Kurtzer]

Can you elaborate on the features that you have added to Dolly?

I worked on a project recently to handle system provisioning of +1000 cluster 
nodes with 10GB file systems in under an hour. We used dolly in conjunction 
with the Warewulf cluster manager utilizing 16-32 node segments for the dolly 
ring and each ring having a "group lead"'s which pulls direct from the 
Warewulf's VNFS (virtual node file system) daemon and then dolly's to the
other members of the group.

As this was just a proposal, we didn't have the resources to scale to 1000
nodes, but based on our tests it is theoretically very favorable that we can
do it in substantially under 1 hour. Dolly rocks (Felix++)! :)

Thanks!

On Tue, May 03, 2005 at 01:35:32PM -0700, David Mathog wrote:
> nettee is a network "tee" program derived from Felix Rauch Valenti's
> dolly program.
> 
> nettee can create a network daisychain
> which passes a data stream along at full network bandwidth while
> (optionally) writing the data to each local node.  If your disks
> are slow and the data larger than disk cache then it will only
> move data as fast as the disks can be written.  Otherwise speeds
> up up to 11.3 Mb/sec were observed (100baseT).  So nettee
> is useful for distributing large files or in conjunction with
> SystemImager to clone nodes.
> 
> Additionally nettee can be configured to create a command chain
> that lets the same command run on every node in the chain
> with only a .01s delay from the time it is issued to the time
> it runs on all nodes (chain of length 20, 100baseT switched
> network.)  The downside - stdout and stderr are usually lost
> or unusable.
> 
> Example scripts showing a few uses for the program are
> provided in the distribution.  Instructions for using nettee
> with SystemImager are found in the README.TXT file.
> 
> Look here for more info:
> 
>   http://saf.bio.caltech.edu/nettee.html
> 
> The program is still in alpha.  It has been pretty reliable for me
> so far but your mileage may vary.
> 
> And yes, I did look at netcat.  In simple tests nettee moves
> large blocks of data about 25% faster than netcat does.  I have
> not attempted to figure out why nettee is faster.
> 
> Regards,
> 
> David Mathog
> mathog at caltech.edu
> Manager, Sequence Analysis Facility, Biology Division, Caltech
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
Greg Kurtzer
Berkeley Lab, Linux guy