You should also include large files. Particularly with the advent of media-servers and the like being able to perform efficiently with large files is important, and that\’s not covered here.

Last but not least I\’d like to see some comparison of storage efficiency for these different types of files. If you can move them fast, that\’s great, but if you can\’t store a particular type of file efficiently and I\’m going to lose say 20% of my storage because of it that\’s important to consider when making the choice.

@typhoidmary:
I would love to test different chipsets and different drives. I just need the money to buy it :)

BTW – thanks for all of your comments. I\’ve noticed you read my articles and post comments. That\’s always appreciated.

@mbainter:
I wanted to test xfs and the reiser\’s but I ran out of time and the article was getting a little long. I will try to do a follow-up at some point with those numbers (maybe next week).

I also didn\’t do large files (400 MiB+?) because of time, but I do want to do those runs.

For both of you – thanks for the comments.

Jeff

Is it possible to include some other, not so often used filesystems as well? e.g. GFS or GFS2 with various storage systems below like DRBD? And tuning opportunities for filesystems in typical scenarios would be a great article too :-)

Chris

Let me make some remarks which I hope are constructive criticism.
I have downloaded the fdtree source.
First off, it\’s a single thread
Your machine has 8 GB ram.

In my opinion, and from experience (also done some FS benchmarking), testing with a total file size <= 1.5 x amount of RAM can go to caches first. Your tests with small sizes amount to around 1.3-1.4 GB

While the tests with 4MB size the total ranges between 12-16GB.

I think the results of \”creation\” are not so bound by caches, while removal can be cached, and that\’s why I think removal of files and dirs, have timings which are quite small to the extent of not being able to draw conclusions in certain cases.

For metadata, and AFAIK, each file or dir has 4KB for the inode, (at least for the ext3), don\’t know for the others. One could imagine testing \”pure\” metadata with a \”touch\” *nix command, instead of dd, and being careful to make a total of 12GB/4KB = 3 million files+dirs for example.

Furthermore, you mention in the beginning why you are benchmarking metadata, but the fdtree misses completely one very important operation which is \”stat\” or a read of the inode, there are some workloads where you write once and read many (even if it\’s small files).

this leads to some sugestions:
I have used recently bonnie++ 1.03e, which does also metadata benchmarking including the \”stat\” operation.
The iozone tarball includes an exec called fileop (though I never tried it).

Though I imagine that you don\’t have a long time to do the testing (as some of us can), just to give you an example, one of my last tests with the above bonnie++ version, each run could take between 2 to 4 hours depending on the filesystem, and I also run it 10 times.

finally, if in your future tests you include the read/stat operation, just try out mount the FS with and without atime,diratime.

OK, that\’s it, sorry if I was too obvious in some things, or to strong, as I said I tried to be constructive, and continue your good work.

I read your other article at a time when I was starting some benchmarks, and I stopped to read first

regards

Mario David

I think you have some interesting points but let me explain a few things.

fdtree, while a simple bash benchmark, also uses all of the cores on my test box. While I didn\’t show the image, I have a picture of gkrellm while the benchmark is running. All 4 cores are being used. I\’m not entirely sure how this works but I think it\’s because of the recursion in the script. But this shows how little I know about bash.

Second, fdtree is not an all encompassing benchmark. It only tests file and directory create and removal in a specific order. I\’m hoping to test another benchmark named mdtree which also stresses other aspects of metadata performance.

Third, to be honest, I\’m not sure about the caching aspect of fdtree. Linux might cache the file operations, but since there are so many, I\’m not sure if it does or doesn\’t. Perhaps the recursion affects the caching. Something to look into. (thanks for pointing that out).

One thing I didn\’t do and should have done was to watch the CPU load during the runs. I sort of watched it using gkrellm but I didn\’t gather any statistics.

But you do correctly point out that almost any benchmark doesn\’t always stress all aspects that you are interested in. As you correctly point out, fdtree doesn\’t stress stat. Other benchmarks will stress the file systems in a different manner. For example, as you mention, Bonnie++ does stress metadata operations and is perhaps a reasonable benchmark to test.

Thanks for your comments. They are really appreciated. Don\’t hesitate to post.

Thanks!

Jeff