[Beowulf] since we are talking about file systems ...

Robert G. Brown rgb at phy.duke.edu
Tue Jan 17 15:13:50 PST 2006

On Tue, 17 Jan 2006, Douglas Eadline wrote:

>> A well designed and implemented file system shouldn't bog you down as
>> you scale out in size, even if you shouldn't.  Its sort of like your
>> car.  If you go beyond 70 MPH somewhere in the US that supports such
>> speeds, your transmission shouldn't just drop out because you hit 71 MPH.
> Joe,
> You obviously have not seen some of the cars I used to drive.

Just for the heck of it, I wrote the C benchmark.  On my 800 MHz laptop,
it creates 50000 files with 1024 bytes each (in fact, integers from 0 to
255 in binary format).  I used open/write/close instead of line based
I/O for maximal efficiency.  I get:

rgb at lilith|B:1180>disktest
50000 files created in 3.804779 seconds
rgb at lilith|B:1181>time ls file.* | wc
bash: /bin/ls: Argument list too long
       0       0       0

real    0m0.999s
user    0m0.928s
sys     0m0.068s

rgb at lilith|B:1182>llth
total 201516
drwxr-xr-x    2 rgb rgb 1421312 Jan 17 18:07 .
-rwx------    1 rgb rgb    1024 Jan 17 18:07 file.41592
-rwx------    1 rgb rgb    1024 Jan 17 18:07 file.41593
-rwx------    1 rgb rgb    1024 Jan 17 18:07 file.41594
-rwx------    1 rgb rgb    1024 Jan 17 18:07 file.41595
-rwx------    1 rgb rgb    1024 Jan 17 18:07 file.41596

This seems to be pretty fast on an ext3 filesystem.  Creating all the
files and writing them is less than 4 seconds, listing them takes about
a second.

Very shortly I'll reverse this and read in all these files in C.  I
expect this to take a lot longer because of the requisite stats and


Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu

More information about the Beowulf mailing list