[Beowulf] copying data between clusters

[Bill Rankin]

Umm, you have your network guys pull a fiber run (or two) from your
cluster's file server over to the other cluster's core network switch?

Alternately, you unbolt and pull the shelf of FC disks out of the rack, put
them on a cart and wheel them over to the other cluster's filer.
(1/2 :-)

It's an ill defined problem.

What's your network topology?  Per-node bandwidth is pretty meaningless if
you are oversubscribed (and most clusters are).  What's the biggest pipe
between cluster A and cluster B?

-bill




On Fri, Mar 5, 2010 at 10:23 AM, Michael Di Domenico <mdidomenico4 at gmail.com
> wrote:

> How does one copy large (20TB) amounts of data from one cluster to another?
>
> Assuming that each node in the cluster can only do about 30MB/sec
> between clusters and i want to preserve the uid/gid/timestamps, etc
>
> I know how i do it, but i'm curious what methods other people use...
>
> Just a general survey...
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>



-- 
Bill Rankin
wrankin1 at gmail.com



-- 
Bill Rankin
wrankin1 at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20100305/b1a51193/attachment.html>