However like mark_w I find it amazing that there is not a clear perfromance benefit using ramfs. However I also note that there is no single mention of barriers (and particularly whether they were enabled or disabled in mount) in the whole article!

I would have expected the Ramdisk journal to always be faster, or, at the very least, as fast, and this is not always the case, as, on some workloads, it is substantially slower.

The one question that did occur to me was that it was possible that the metadata would all fit into cache on the second hard drive. This might increase performance of the \’metadata on a separate hard drive\’ solution more than anticipated, but it still wouldn\’t explain why the ramdisk solution was worse.

I was wondering which IO scheduler you had used. Going out of your way to provide elevator seeks wouldn\’t help a ramdisk and NOOP would probably be a better strategy for the ramdisk. As far as I know, though, you can only set one scheduler on the system (as opposed to one scheduler per device) so you might lose overall by setting noop for the system.

In these days in which an SSD is also a plausible choice, and can be used to good effect in, eg, ZFS in enhancing overall array performance, I wonder whether this is optimal, although it probably only matters in odd corner cases.

Chicken and egg.

BTW – you can change the IO scheduler on a per mount basis. If you look back at some of the IO scheduler articles you will see how to do it.

@timsmall: Amen! The only option I know of is the ACARD-9010 (http://www.acard.com/english/fb01-product.jsp?prod_no=ANS-9010&type1_title=%20Solid%20State%20Drive&idno_no=270). It\’s a little below $400 without the memory or flash card. It\’s not a bad solution but it\’s limited by the SATA connection (I\’m hoping they come out with a 6Gbps SAS or SATA connection – even better would be a PCIe connection!)

I think the reason you are not seeing the external journal improve performance is that fdtree.bash is single threaded and CPU bound rather than being IO bound. An external journal only improves performance when the workload is IO bound.

I ran fdtree.bash on XFS to see how it compared to your ext4 numbers, but I got the same numbers on XFS. That raised a red flag – ext4 should wipe the floor with XFS in these tests. Notably, though, my own test scripts that do very similar operations give an order of magnitude better performance on XFS than fdtree.bash.

Just to check, I spent an hour and rewrote fdtree.bash in C and this version produces numbers that match my own test scripts. e.g. directory creates measuring about 5,000/s instead of 250/s that fdtree.bash was measuring….

IOWs, it appears that the benchmark is the problem, not the filesystem you are testing or your methodology. This is one of the reasons I always monitor system performance (CPU, memory usage, etc) while benchmarks are running so that I can tell when performance is not what it should be… ;)

Cheers,

Dave.

Wow! You did a great deal of work! I was looking at the benchmark a bit more and one thing I\’ve noticed is that when I run it, all of the cores are busy. So if it\’s single threaded I should only see a single core loaded. The script uses recursion so I\’m wondering if bash spawns threads (or forks) when it does recursion. Any ideas?

I\’m looking for better metadata codes (I really like the word benchmark but that\’s the concept). Do you have any recommendations? Metarates has been recommended to me – any experience with this one?

BTW – I ran the same basic tests with a wider range of options for the journal partition (disk and ramdisk). I also ran IOZone for the same configurations. Look for an upcoming article series on the results I\’m still post-processing the results).

Dave – thanks again for your comments and insight. Greatly appreciated.

Jeff

1) It\’s not clear where you are benchmarking on a 32-bit or 64-bit kernel. Could you please repeat on the one that\’s missing?
1) If you created the partition sequentially, your journal partition, dev/sdb2, will reside on the inner tracks of the HD where it\’s much less efficient. If this is the case, can you rerun the benchmarks by moving the journal partition to the beginning of the HD?

Thanks