dcsimg

Metadata Performance of Four Linux File Systems

Using the principles of good benchmarking, we explore the metadata performance of four linux file systems using a simple benchmark, fdtree.

In a previous article, the case was made for how low file system benchmarks have fallen. Benchmarks have become the tool of marketing to the point where they are mere numbers and do not prove of much use. The article reviewed a paper that examined nine years of storage and file system benchmarking and made some excellent observations. The paper also made some recommendations about how to improve benchmarks.

This article isn’t so much about benchmarks as a product, but rather it is an exploration looking for interesting observations or trends or the lack thereof. In particular this article examines the metadata performance of several Linux file systems using a specific micro-benchmark. Fundamentally this article is really an exploration to understand if there is any metadata performance differences between 4 Linux file systems (ext3, ext4, btrfs, and nilfs) using a metadata benchmark called fdtree. So now it’s time to eat our dog food and do benchmarking with the recommendations previously mentioned.

Start at the Beginning – Why?

The previous article made several observations about benchmarking, one of which is that storage and file system benchmarks seldom, if ever, explain why they are performing a benchmark. This is a point that is not to be underestimated. Specifically, if the reason why the benchmark was performed can not be adequately explained, then the benchmark itself becomes suspect (it may just be pure marketing material).

Given this point, the reason the benchmark in this article is being performed is to examine or explore if, and possibly how much, difference there is between the metadata performance of four Linux file systems using a single metadata benchmark. The search is not to find which file system is the best because it is a single benchmark, fdtree. Rather it is to search for differences and contrast the metadata performance of the file systems.

Why is examining the metadata performance a worthwhile exploration? Glad that you asked. There are a number of applications, workloads, and classes of applications that are metadata intensive. Mail servers can be very metadata intensive applications because of the need to read and write very small files. Sometimes databases have workloads that do a great deal of reading and writing small files. In the world of technical computing, many bioinformatic applications such as gene sequencing applications, do a great deal of small reads and writes.

The metadata benchmark used in this article is called fdtree. It is a simple bash script that stresses the metadata aspects of the file system using standard *nix commands. While it is not the most well known benchmark in the storage and file system world, it is a bit better known in the HPC (High Performance Computing) world.

An Examination of fdtree

Before jumping into the results, it is appropriate and highly recommended to examine the benchmark itself. fdtree is a simple bash script that performs four different metadata tests:


  • Directory creation
  • File creation
  • File removal
  • Directory Removal

It creates a specified number of files of a given size (in blocks) in a top-level directory. Then it creates a specified number of subdirectories and then in turn subdirectories are recursively created up to a specified number of levels and are populated with files.

Directory Creation

This phase of the benchmark begins by creating the number of specified directories in the main directory using the simple “mkdir” command in a bash function “create_dir”.

mkdir $base_name"L"$nl"D"$nd"/"

The bash variables specify the details of the directory names. The next step is to call the “create_dir” function recursively with a different “base name” (directory) to create all of the required directories.

create_dirs $((nl-1)) $base_name"L"$nl"D"$nd"/"

File Creation

This step of the benchmark creates the required number of files using the “dd” command in a bash function, “create_files”.

dd if=/dev/zero bs=4096 count=$fsize of=$file_name > /dev/null 2>&1

To create files in the subdirectories, the bash function is called recursively. As part of the benchmark, the number of 4 KiB blocks is specified ($fsize).

File Removal

The third function in the benchmark is to remove the files that were created. This is done with the standard “rm” command in a function called “remove_files”.

rm -f $file_name

The function “remove_files” is called recursively to remove all of the files.

Directory Removal

The fourth and final function in the benchmark is to remove the directories. This is done in a bash function “remove_dirs” using the *nix command “rmdir $dir_names”

rmdir $dir_names

The function “remove_dirs” is called recursively to remove all of the directories.

Overall the script uses standard *nix commands for the benchmark. It does not use any recursive options for any of the *nix commands. It stresses the metadata capabilities of the file system because of the potentially large number of files and directories.

The one interesting thing that the test does, is round the results, time and rates, to integer values. So there could be times when the time to execute the test could be 0 seconds. That is, the test ran in less than 1 second.

Running the benchmark

In the benchmark exploration in this article, fdtree was used in 4 different approaches to stressing the metadata capability:


  • Small files (4 KiB)

    • Shallow directory structure
    • Deep directory structure

  • Larger files (4 MiB)

    • Shallow directory structure
    • Deep directory structure

The two file sizes, 4 KiB (1 block) and 4 MiB (1,000 blocks) were used to get some feel for a range of performance as a function of the amount of data. The two directory structures were used to stress the metadata in different ways to discover if there is any impact on the metadata performance. The shallow directory structure means that there are many directories but not very many levels down. The deep directory structure means that there are not many directories at a particular level but that there are many levels.

To create the specific parameters for fdtree used in the exploration, there were three overall goals:


  • Keep the total run time to approximately 10-12 minutes at a maximum

  • Keep the total data for the two directory structures approximately the same

  • Keep the run time for each of the four functions greater than 1 minute if possible

All four functions were not always run for 1 minute, sometimes only for a few seconds. These will be noted in the results.

The command lines for the four combinations are:

Small Files – Shallow Directory Structure

./fdtree.bash -d 20 -f 40 -s 1 -l 3

This command creates 20 sub-directories from each upper level directory at each level (“-d 20″) and there are 3 levels (“-l 3″). It’s a basic tree structure. This is a total of 8,421 directories. In each directory there are 40 files (“-f 40″) each sized at 1 block (4 KiB) denoted by “-s 1″. This is a total of 336,840 files and 1,347,360 KiB total data.

Small Files – Deep Directory Structure

./fdtree.bash -d 3 -f 4 -s 1 -l 10

This command creates 3 sub-directories from each upper level directory at each level (“-d 3″) and there are 10 levels (“-l 10″). This is a total of 88,573 directories. In each directory there are 4 files each sized at 1 block (4 KiB). This is a total of 354,292 files and 1,417,168 KiB total data.

Medium Files – Shallow Directory Structure

./fdtree.bash -d 17 -f 10 -s 1000 -l 2

This command creates 17 sub-directories from each upper level directory at each level (“-d 17″) and there are 2 levels (“-l 2″). This is a total of 307 directories. In each directory there are 10 files each sized at 1,000 blocks (4 MiB). This is a total of 3,070 files and 12,280,000 KiB total data.

Medium Files – Deep Directory Structure

./fdtree.bash -d 2 -f 2 -s 1000 -l 10

This command creates 2 sub-directories from each upper level directory at each level (“-d 2″) and there are 10 levels (“-l 10″). This is a total of 2,047 directories. In each directory there are 2 files each sized at 1,000 blocks (4 MiB). This is a total of 4,094 files and 16,376,000 KiB total data.

Comments on "Metadata Performance of Four Linux File Systems"

I have been surfing online more than three hours today,
yet I never found any interesting article like yours. It is pretty worth enough for me.
In my opinion, if all webmasters and bloggers made good content as you did, the internet will be much more useful than ever before.

I am so grateful for your blog article.Much thanks again. Much obliged.

I loved your article. Great.

Its like you read my mind! You appear to know so much about this, like you wrote the book in it or something.
I think that you can do with a few pics to drive the message home a little bit, but other
than that, this is wonderful blog. An excellent
read. I’ll definitely be back.

I visited multiple websites however the audio quality for audio songs current at
this web page is really fabulous.

The note pro?ided that all principal and
accrued interest was due in a single balloon payment at the end of the
note term, and neither ?r?ncipal nor interest could be prepaid.
Educating th?se customers in how to handle their tax affairs
in the futur? can sav? them many thousands of dollars. Avoid hype or gimmicks and let people know exactly ?hat to expect and what they
are required to do in order to receive wh?t they are being offered.

Al?? visit my weblog … Ft Lauderdale Cpa Firms

Thanks a lot for the blog article.Really looking forward to read more. Cool.

Thanks again for the blog.Really looking forward to read more. Cool.

Hello, the whole thing is going well here and ofcourse every one is sharing data, that’s in fact fine, keep up
writing.

Below you?ll obtain the link to some internet sites that we assume you ought to visit.

Every when inside a while we pick out blogs that we read. Listed beneath would be the most up-to-date websites that we decide on.

Muchos Gracias for your blog.Thanks Again. Will read on…

Very couple of internet sites that come about to become detailed beneath, from our point of view are undoubtedly nicely really worth checking out.

It rec?rds all the transactions taking ?lace in a company so that
you can always have a ch?ck on the losses if required and according to
generally accepted acco?nting principles, the basic framework of accounting shoul? always include finance and accounting records.
You need to seek ?ax ?iling Help ?f you would like stay ?way
from Tax Levy compli??tions like a Wage Levy or Bank Levy.
For conducting the additional analysis on th? ?pecific CPA network, which will assist you
to earn funds, all you want to do is you will need
to p?ace ?n the identify of that network al?ng with
some other app?icable keyword p?rases that could possibly assist you in discovering
out if there is any precise challenge with that individual or?anizati?n.

Also visit m? page; john costell? city of Boca Raton CPAt – Florian -

Here are a few of the web pages we suggest for our visitors.

Here are some hyperlinks to websites that we link to simply because we consider they are worth visiting.

Below you?ll obtain the link to some websites that we consider you need to visit.

We prefer to honor a lot of other net sites around the internet, even when they aren?t linked to us, by linking to them. Below are some webpages really worth checking out.

Always a major fan of linking to bloggers that I really like but do not get a great deal of link enjoy from.

We like to honor lots of other world-wide-web web-sites on the net, even though they aren?t linked to us, by linking to them. Beneath are some webpages worth checking out.

We came across a cool web site that you simply could love. Take a appear for those who want.

Always a huge fan of linking to bloggers that I appreciate but don?t get a whole lot of link enjoy from.

Wonderful story, reckoned we could combine some unrelated information, nonetheless truly really worth taking a search, whoa did 1 learn about Mid East has got far more problerms as well.

Here are a few of the internet sites we suggest for our visitors.

Ne?ertheless, whrn dealing with this kin of
agency, modest and huge ?usiness will be the concentration. Seek for
the full encouragement ?f the people aroud ?ou so that you’ll
be moti?ated to fulfi?l your targets. An acckuntant can preset you with
an accurate picture of yo?r casa? flow and places
which need attention, to ens?re that you know where you can f?c?s
your efforts.

Visit my webpage; boca Raton retirement planning Calculator

That is the finish of this write-up. Right here you will come across some websites that we feel you will value, just click the links.

We like to honor quite a few other net web sites around the internet, even though they aren?t linked to us, by linking to them. Underneath are some webpages really worth checking out.

The information mentioned within the report are a few of the best available.

Frajk Breinl?ng is a reco?nize? expert iin Aff?liat? Marketing.
Thhe entrepreneurial owner of, say, a $2M, $5M or $20M busine?s
will almost surel? be a very intriguing person. A CP? must complet? a certain ?mount of CPE (C?ntinuing Professional
Education) units each year.

Takee a look ?t my hom?page investment Advisor boca raton fl

We prefer to honor many other world-wide-web web-sites around the web, even if they aren?t linked to us, by linking to them. Under are some webpages really worth checking out.

We prefer to honor a lot of other net sites on the net, even when they aren?t linked to us, by linking to them. Beneath are some webpages worth checking out.

Although sites we backlink to beneath are considerably not associated to ours, we really feel they’re basically worth a go via, so possess a look.

Here are a number of the web sites we advocate for our visitors.

That will be the finish of this post. Right here you will find some websites that we assume you?ll appreciate, just click the hyperlinks.

That will be the finish of this write-up. Here you will find some internet sites that we feel you will appreciate, just click the hyperlinks.

The time to study or check out the subject material or web pages we have linked to beneath.

Hey, thanks for the article.Really thank you! Awesome.

We like to honor lots of other online web sites around the net, even though they aren?t linked to us, by linking to them. Underneath are some webpages really worth checking out.

Just beneath, are many entirely not associated sites to ours, nonetheless, they may be surely worth going over.

Just beneath, are a lot of totally not related websites to ours, however, they are surely really worth going over.

Magnificent goods from you, man. I’ve understand your stuff previous to and you are just extremely
fantastic. I actually like what you’ve acquired here, really like
what you are saying and the way in which you say
it. You make it enjoyable and you still take care of to keep it
smart. I cant wait to read much more from you. This is
really a tremendous website.

It’s wonderful that you are getting ideas from
this article as well as from our argument made here.

Very good site you have here but I was wanting to know if you
knew of any community forums that cover the same topics talked about in this article?
I’d really like to be a part of group where I can get feed-back from other knowledgeable people that
share the same interest. If you have any recommendations,
please let me know. Kudos!

Wonderful story, reckoned we could combine a handful of unrelated information, nevertheless definitely really worth taking a appear, whoa did 1 learn about Mid East has got much more problerms as well.

I used to be able to find good advice from your content.

Wow! This blog looks just like my old one! It’s on a totally
different topic but it has pretty much the same layout and design. Wonderful choice of colors!

That would be the finish of this post. Right here you will discover some web sites that we assume you?ll value, just click the links.

It’s an awesome article in support of all the online viewers; they will get benefit from it I am sure.

Feel free to surf to my webpage :: blanche

It’s hard to come by experienced people in this particular topic, but
you seem like you know what you’re talking about!
Thanks

Top grain leather is stronger leather than split leather but
both top grain and split leather will give you the security and strength needed on the road.
For the battery-optimized i – Pod, counter clockwise is your friend.

- With so many mobiles now allowing for web usage, your mobile website could appear slightly different on each handset.

Leave a Reply