Block Sizes

[Mark Hahn]

> > The question was about making I/O faster. What you suggest wastes less
...
> 
> Using a RAID 5 disk array with n disks will result in an (n-1)X I/O
> performance gain. With the price of disks today, it seems like a viable
> option for folks who need the speed.

raid5 usually does _not_ show that kind of nice speedup, 
especially if your workload involves writes.  it still works
VERY well, of course, and can scale to some pretty impressive
rates on mundane/cheap hardware.  IMO, for this kind of "scratch"
storage, you should crunch the MTBF numbers to see whether you
can just go with a simple raid0, since it's so much more efficient.
or, for that matter, whether you can afford raid1.

in MFFS (Mark's Fantasy File System), the filesystem itself is 
smart enough to keep some files striped, others replicated,
others in raid5, not to mention transparently migrating onto
the sdlt library.  oh, and for particular users, it just tosses
files into gnutella ;)

as for the original question, I recommend a superlative 
brute force tool: strace.  it'll tell you whether you're spending 
too much time in certain syscalls, and how big the chunks are 
that your runtime is reading.  for example, try running 
"strace -ttt dd if=somefile of=/dev/null bs=8k 3&>1 | less" 

WARNING: prepare to be disgusted by all the nonsense that glibc does 
whenever starting a program!