[Beowulf] 10G and rsync

David Mathog mathog at caltech.edu
Thu Jan 2 09:44:39 PST 2020

On Thu, 2 Jan 2020 11:27:58 -0500 Michael Di Domenico wrote
>> 2) how many files are you xfering?  Are these generally large files or
>> many small files, or a distribution with a long tail towards small
>> files?
> small, 100-200 files, definitely not a MDT issue

1. Is a single large file transfer rate reasonable?

2. Ditto for several large files?

3. If you examine the transfer log for your 100-200 files and then feed 
those file names into tar to produce a local tar file is that rate 

4. Transfer of that single tar file reasonable?  (Presumably yes if 1. 
was OK.)

5. If you untar that single file on the remote system, is the rate 

What I'm getting at is that on one end or the other the transfer may be 
overwhelming the disk/RAID buffers and once the physical disk heads have 
to bounce around between different parts of the disk the rate goes down 
the tubes.

Are you sure there is not a patrol read ongoing on one system or the 
other?  That can cause this sort of disk head issue.

Also it might be this "hugepage" issue:


The fix for that was:

      echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag


David Mathog
mathog at caltech.edu
Manager, Sequence Analysis Facility, Biology Division, Caltech

More information about the Beowulf mailing list