[Beowulf] 10G and rsync

[Michael Di Domenico]

just to further the discussion and for everyone's education i found
this whitepaper, which seems to confirm what i see

https://www.intel.com/content/dam/support/us/en/documents/network/sb/fedexcasestudyfinal.pdf

maybe hpn-ssh is something i can work into my process


On Thu, Jan 2, 2020 at 10:26 AM Michael Di Domenico
<mdidomenico4 at gmail.com> wrote:
>
> does anyone know or has anyone gotten rsync to push wire speed
> transfers of big files over 10G links?  i'm trying to sync a directory
> with several large files.  the data is coming from local disk to a
> lustre filesystem.  i'm not using ssh in this case.  i have 10G
> ethernet between both machines.   both end points have more then
> enough spindles to handle 900MB/sec.
>
> i'm using 'rsync -rav --progress --stats -x --inplace
> --compress-level=0 /dir1/ /dir2/' but each file (which is 100's of
> GB's) is getting choked at 100MB/sec
>
> running iperf and dd between the client and the lustre hits 900MB/sec,
> so i fully believe this is an rsync limitation.
>
> googling around hasn't lent any solid advice, most of the articles are
> people that don't check the network first...
>
> with the prevalence of 10G these days, i'm surprised this hasn't come
> up before, or my google-fu really stinks.  which doesn't bode well
> given its the first work day of 2020 :(