[Beowulf] 10G and rsync

[Alex Chekholko]

Hi Michael,

I would recommend trying 'bbcp' before 'hpn-ssh' as the latter will really
only benefit you for high-latency links, e.g. across country.

Put the bbcp binary on both sides and try it out.  If you don't have a way
to install bbcp into a system $PATH, you can specify the absolute path to
the binary.  Random link with examples here:
https://www.nics.tennessee.edu/computing-resources/data-transfer/bbcp

Regards,
Alex

On Thu, Jan 2, 2020 at 8:32 AM Michael Di Domenico <mdidomenico4 at gmail.com>
wrote:

> just to further the discussion and for everyone's education i found
> this whitepaper, which seems to confirm what i see
>
>
> https://www.intel.com/content/dam/support/us/en/documents/network/sb/fedexcasestudyfinal.pdf
>
> maybe hpn-ssh is something i can work into my process
>
>
> On Thu, Jan 2, 2020 at 10:26 AM Michael Di Domenico
> <mdidomenico4 at gmail.com> wrote:
> >
> > does anyone know or has anyone gotten rsync to push wire speed
> > transfers of big files over 10G links?  i'm trying to sync a directory
> > with several large files.  the data is coming from local disk to a
> > lustre filesystem.  i'm not using ssh in this case.  i have 10G
> > ethernet between both machines.   both end points have more then
> > enough spindles to handle 900MB/sec.
> >
> > i'm using 'rsync -rav --progress --stats -x --inplace
> > --compress-level=0 /dir1/ /dir2/' but each file (which is 100's of
> > GB's) is getting choked at 100MB/sec
> >
> > running iperf and dd between the client and the lustre hits 900MB/sec,
> > so i fully believe this is an rsync limitation.
> >
> > googling around hasn't lent any solid advice, most of the articles are
> > people that don't check the network first...
> >
> > with the prevalence of 10G these days, i'm surprised this hasn't come
> > up before, or my google-fu really stinks.  which doesn't bode well
> > given its the first work day of 2020 :(
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://beowulf.org/pipermail/beowulf/attachments/20200102/6d845b42/attachment.html>