[Beowulf] copying data between clusters
kyron at neuralbs.com
Fri Mar 5 08:30:44 PST 2010
On Fri, 05 Mar 2010 11:22:14 -0500, Mike Davis <jmdavis1 at vcu.edu> wrote:
> Michael Di Domenico wrote:
>> How does one copy large (20TB) amounts of data from one cluster to
>> Assuming that each node in the cluster can only do about 30MB/sec
>> between clusters and i want to preserve the uid/gid/timestamps, etc
> If the clusters are co-lo I wouldn't copy I would use shared storage. If
> they are not co-located I would use patience.
> Seriously though, for a one time copy, I would consider copying to an
> external system and then physically moving that system. To do this and
> preserve ownerships you will need to duplicate accounts and groups.
...and we are all assuming non-compressibility; otherwise, use pbzip2 ;)
More information about the Beowulf