[Beowulf] 10G and rsync

[Jonathan Engwall]

The whitepaper was interesting. Single core VMs might be your best bet.

On Thu, Jan 2, 2020, 8:48 AM Michael Di Domenico <mdidomenico4 at gmail.com>
wrote:

> i'll check it, but keep in mind.  i'm not copying files between two
> servers, but rather between two directories on the same server.
>
> ideally if rsync is still using ssh under the covers in my scenario,
> i'm hopeful hpn-ssh might alleviate the bottleneck condition.  if it's
> not i'm back to square one.
>
> On Thu, Jan 2, 2020 at 11:42 AM Alex Chekholko <alex at calicolabs.com>
> wrote:
> >
> > Hi Michael,
> >
> > I would recommend trying 'bbcp' before 'hpn-ssh' as the latter will
> really only benefit you for high-latency links, e.g. across country.
> >
> > Put the bbcp binary on both sides and try it out.  If you don't have a
> way to install bbcp into a system $PATH, you can specify the absolute path
> to the binary.  Random link with examples here:
> > https://www.nics.tennessee.edu/computing-resources/data-transfer/bbcp
> >
> > Regards,
> > Alex
> >
> > On Thu, Jan 2, 2020 at 8:32 AM Michael Di Domenico <
> mdidomenico4 at gmail.com> wrote:
> >>
> >> just to further the discussion and for everyone's education i found
> >> this whitepaper, which seems to confirm what i see
> >>
> >>
> https://www.intel.com/content/dam/support/us/en/documents/network/sb/fedexcasestudyfinal.pdf
> >>
> >> maybe hpn-ssh is something i can work into my process
> >>
> >>
> >> On Thu, Jan 2, 2020 at 10:26 AM Michael Di Domenico
> >> <mdidomenico4 at gmail.com> wrote:
> >> >
> >> > does anyone know or has anyone gotten rsync to push wire speed
> >> > transfers of big files over 10G links?  i'm trying to sync a directory
> >> > with several large files.  the data is coming from local disk to a
> >> > lustre filesystem.  i'm not using ssh in this case.  i have 10G
> >> > ethernet between both machines.   both end points have more then
> >> > enough spindles to handle 900MB/sec.
> >> >
> >> > i'm using 'rsync -rav --progress --stats -x --inplace
> >> > --compress-level=0 /dir1/ /dir2/' but each file (which is 100's of
> >> > GB's) is getting choked at 100MB/sec
> >> >
> >> > running iperf and dd between the client and the lustre hits 900MB/sec,
> >> > so i fully believe this is an rsync limitation.
> >> >
> >> > googling around hasn't lent any solid advice, most of the articles are
> >> > people that don't check the network first...
> >> >
> >> > with the prevalence of 10G these days, i'm surprised this hasn't come
> >> > up before, or my google-fu really stinks.  which doesn't bode well
> >> > given its the first work day of 2020 :(
> >> _______________________________________________
> >> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin
> Computing
> >> To change your subscription (digest mode or unsubscribe) visit
> https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://beowulf.org/pipermail/beowulf/attachments/20200102/36b1ea19/attachment-0001.html>