[Beowulf] mysterious slow SATA on one machine

[David Mathog]

> David Mathog wrote:
>Raw results from various experiments here:
> 
>   http://saf.bio.caltech.edu/pub/pickup/bonnie++.rtf
>   http://saf.bio.caltech.edu/pub/pickup/sustained_write.rtf
> 

Some progress, see updated files above.

lspci showed there were two devices on the bus where the Sil 3114 was
located, it and the ATI Rage VGA controller.  It also showed the pci
latency for the former was 32 and the latter 66.  

The VGA controller is currently running in text mode, without the atyfb
module loaded, and with nothing happening on it (it just has the text
login prompt dislayed), and changing its latency up or down makes no
difference to disk speed (not shown in the files cited above).  The
machine was started once with the VGA jumpered off, but it didn't boot,
so it wasn't possible to completely remove it from the equation.  

However, increasing the pci_latency on the Sil 3114 as far as it would
go (to 144 as shown in lspci)  with 

 setpci -s '01:07.0' latency_timer=99

made a considerable difference.  For the older disk it brought it up to
approximately the same speed as on the nvidia ck804 controller.  For the
WD10EARS it sped things up about 50%, but didn't manage to match the
nvidia controller, possibly because of the absence of the ncq mentioned
previously in this thread.  Also on the 3114 it topped out at around
106MB/sec for the fastest bonnie++ applications, and that is pretty
close to the 132MB/s limit on the 32 bit PCI bus, whereas on the ck804
peaks were 140MB/s, which would be more than the bus can carry.  
Assuming the PCI on the Arima board is really 33 MHz, like the manual
says, and not 66 MHz, as lspci reports.

Vibration may still be playing some role, but apparently that wasn't the
primary problem.

I may still put in a PCI-X SATA controller though, as there is still
another 40% performance to go on the WD drive, and that will provide
enough bus bandwidth to support that.

Thanks,

David Mathog
mathog at caltech.edu
Manager, Sequence Analysis Facility, Biology Division, Caltech