<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body>
<div>
<div>
<div style="direction: ltr;">Aren’t you going to have issues with I/O contention if you are copying on the same machine but different directories? Is the transfer even going over the link</div>
</div>
<div><br>
</div>
<div class="ms-outlook-ios-signature">
<div style="direction: ltr;">Regards,</div>
<div style="direction: ltr;">Jonathan Aquilina</div>
<div style="direction: ltr;">Owner managing director</div>
<div><br>
</div>
<div style="direction: ltr;">Phone (356) 20330099</div>
<div style="direction: ltr;">Mobile (356) 79957942</div>
<div><br>
</div>
<div style="direction: ltr;">Email sales@eagleeyet.net</div>
</div>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Beowulf <beowulf-bounces@beowulf.org> on behalf of Jonathan Engwall <engwalljonathanthereal@gmail.com><br>
<b>Sent:</b> Thursday, January 2, 2020 5:56:25 PM<br>
<b>To:</b> Michael Di Domenico <mdidomenico4@gmail.com><br>
<b>Cc:</b> Beowulf Mailing List <Beowulf@beowulf.org><br>
<b>Subject:</b> Re: [Beowulf] 10G and rsync</font>
<div> </div>
</div>
<div>
<div dir="auto">The whitepaper was interesting. Single core VMs might be your best bet.</div>
<br>
<div class="x_gmail_quote">
<div dir="ltr" class="x_gmail_attr">On Thu, Jan 2, 2020, 8:48 AM Michael Di Domenico <<a href="mailto:mdidomenico4@gmail.com">mdidomenico4@gmail.com</a>> wrote:<br>
</div>
<blockquote class="x_gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
i'll check it, but keep in mind. i'm not copying files between two<br>
servers, but rather between two directories on the same server.<br>
<br>
ideally if rsync is still using ssh under the covers in my scenario,<br>
i'm hopeful hpn-ssh might alleviate the bottleneck condition. if it's<br>
not i'm back to square one.<br>
<br>
On Thu, Jan 2, 2020 at 11:42 AM Alex Chekholko <<a href="mailto:alex@calicolabs.com" target="_blank" rel="noreferrer">alex@calicolabs.com</a>> wrote:<br>
><br>
> Hi Michael,<br>
><br>
> I would recommend trying 'bbcp' before 'hpn-ssh' as the latter will really only benefit you for high-latency links, e.g. across country.<br>
><br>
> Put the bbcp binary on both sides and try it out. If you don't have a way to install bbcp into a system $PATH, you can specify the absolute path to the binary. Random link with examples here:<br>
> <a href="https://www.nics.tennessee.edu/computing-resources/data-transfer/bbcp" rel="noreferrer noreferrer" target="_blank">
https://www.nics.tennessee.edu/computing-resources/data-transfer/bbcp</a><br>
><br>
> Regards,<br>
> Alex<br>
><br>
> On Thu, Jan 2, 2020 at 8:32 AM Michael Di Domenico <<a href="mailto:mdidomenico4@gmail.com" target="_blank" rel="noreferrer">mdidomenico4@gmail.com</a>> wrote:<br>
>><br>
>> just to further the discussion and for everyone's education i found<br>
>> this whitepaper, which seems to confirm what i see<br>
>><br>
>> <a href="https://www.intel.com/content/dam/support/us/en/documents/network/sb/fedexcasestudyfinal.pdf" rel="noreferrer noreferrer" target="_blank">
https://www.intel.com/content/dam/support/us/en/documents/network/sb/fedexcasestudyfinal.pdf</a><br>
>><br>
>> maybe hpn-ssh is something i can work into my process<br>
>><br>
>><br>
>> On Thu, Jan 2, 2020 at 10:26 AM Michael Di Domenico<br>
>> <<a href="mailto:mdidomenico4@gmail.com" target="_blank" rel="noreferrer">mdidomenico4@gmail.com</a>> wrote:<br>
>> ><br>
>> > does anyone know or has anyone gotten rsync to push wire speed<br>
>> > transfers of big files over 10G links? i'm trying to sync a directory<br>
>> > with several large files. the data is coming from local disk to a<br>
>> > lustre filesystem. i'm not using ssh in this case. i have 10G<br>
>> > ethernet between both machines. both end points have more then<br>
>> > enough spindles to handle 900MB/sec.<br>
>> ><br>
>> > i'm using 'rsync -rav --progress --stats -x --inplace<br>
>> > --compress-level=0 /dir1/ /dir2/' but each file (which is 100's of<br>
>> > GB's) is getting choked at 100MB/sec<br>
>> ><br>
>> > running iperf and dd between the client and the lustre hits 900MB/sec,<br>
>> > so i fully believe this is an rsync limitation.<br>
>> ><br>
>> > googling around hasn't lent any solid advice, most of the articles are<br>
>> > people that don't check the network first...<br>
>> ><br>
>> > with the prevalence of 10G these days, i'm surprised this hasn't come<br>
>> > up before, or my google-fu really stinks. which doesn't bode well<br>
>> > given its the first work day of 2020 :(<br>
>> _______________________________________________<br>
>> Beowulf mailing list, <a href="mailto:Beowulf@beowulf.org" target="_blank" rel="noreferrer">
Beowulf@beowulf.org</a> sponsored by Penguin Computing<br>
>> To change your subscription (digest mode or unsubscribe) visit <a href="https://beowulf.org/cgi-bin/mailman/listinfo/beowulf" rel="noreferrer noreferrer" target="_blank">
https://beowulf.org/cgi-bin/mailman/listinfo/beowulf</a><br>
_______________________________________________<br>
Beowulf mailing list, <a href="mailto:Beowulf@beowulf.org" target="_blank" rel="noreferrer">
Beowulf@beowulf.org</a> sponsored by Penguin Computing<br>
To change your subscription (digest mode or unsubscribe) visit <a href="https://beowulf.org/cgi-bin/mailman/listinfo/beowulf" rel="noreferrer noreferrer" target="_blank">
https://beowulf.org/cgi-bin/mailman/listinfo/beowulf</a><br>
</blockquote>
</div>
</div>
</body>
</html>