Comments on: I Feel the Need for Speed: Linux File System Throughput Performance, Part 1 http://www.linux-mag.com/id/7525/ Open Source, Open Standards Sat, 05 Oct 2013 13:48:18 +0000 hourly 1 http://wordpress.org/?v=3.1 By: augthecaveman http://www.linux-mag.com/id/7525/#comment-7004 augthecaveman Wed, 30 Nov -0001 00:00:00 +0000 http://www.linux-mag.com/id/7525/#comment-7004 <p>This is all well and good, but what I\'d really like to see is an article about how to <strong>use</strong> such data to wring every last bit of throughput out of my already deployed drives. </p> This is all well and good, but what I\’d really like to see is an article about how to use such data to wring every last bit of throughput out of my already deployed drives.

]]> By: laytonjb http://www.linux-mag.com/id/7525/#comment-7005 laytonjb Wed, 30 Nov -0001 00:00:00 +0000 http://www.linux-mag.com/id/7525/#comment-7005 <p>@augthecaveman,</p> <p>I appreciate the question, I don\'t mean to pick on you but let me use your question to make a point. Everyone wants to know how to squeeze more performance out of their hardware. However, the fundamental question is, \"what does performance mean to you.\" Before you can start to squeeze the hardware, you need some measures of performance - more precisely, you need to define what applications and data you will use for testing.</p> <p>Defining these answers, believe it or not, is extraordinarily difficult. I\'ve been working with people for a number of years and I can honestly say that I haven\'t met a single one who could succinctly answer the question. It takes a great deal of work, effort, thought, preparation, etc. to define precisely by what you mean by performance. Again, not picking on you, but just saying \"throughput\" means very little. As you can see in the data in the article, using a single benchmark, IOZone, you can see up to a 100% difference in performance just by changing the parameters of the test! Before starting any performance tuning, you have to precisely define the test, the parameters, and what kind of results you want (or expect) to see. One question that many people struggle with is \"what benchmark(s) best define my application behavior.\" So a great starting point is to understand the IO patterns of your applications.</p> <p>Thanks for asking the question. I think answering the question can be a useful lesson for everyone.</p> <p>Jeff </p> @augthecaveman,

I appreciate the question, I don\’t mean to pick on you but let me use your question to make a point. Everyone wants to know how to squeeze more performance out of their hardware. However, the fundamental question is, \”what does performance mean to you.\” Before you can start to squeeze the hardware, you need some measures of performance – more precisely, you need to define what applications and data you will use for testing.

Defining these answers, believe it or not, is extraordinarily difficult. I\’ve been working with people for a number of years and I can honestly say that I haven\’t met a single one who could succinctly answer the question. It takes a great deal of work, effort, thought, preparation, etc. to define precisely by what you mean by performance. Again, not picking on you, but just saying \”throughput\” means very little. As you can see in the data in the article, using a single benchmark, IOZone, you can see up to a 100% difference in performance just by changing the parameters of the test! Before starting any performance tuning, you have to precisely define the test, the parameters, and what kind of results you want (or expect) to see. One question that many people struggle with is \”what benchmark(s) best define my application behavior.\” So a great starting point is to understand the IO patterns of your applications.

Thanks for asking the question. I think answering the question can be a useful lesson for everyone.

Jeff

]]> By: mdavid http://www.linux-mag.com/id/7525/#comment-7006 mdavid Wed, 30 Nov -0001 00:00:00 +0000 http://www.linux-mag.com/id/7525/#comment-7006 <p>Hi Jeff (again)<br /> from iozone options<br /> -e Include flush (fsync,fflush) in the timing calculations<br /> -o Writes are synchronously written to disk. (O_SYNC). Iozone will open the files with the O_SYNC flag. This forces all writes to the file to go completely to disk before returning to the bench-mark.</p> <p>and also check the -S<br /> when you make the statement<br /> \"For this article, cache effects will be limited as much as possible\"<br /> I think you should/could have added those options, believe me they make a great impact on the results.</p> <p>cheers</p> <p>Mario David </p> Hi Jeff (again)
from iozone options
-e Include flush (fsync,fflush) in the timing calculations
-o Writes are synchronously written to disk. (O_SYNC). Iozone will open the files with the O_SYNC flag. This forces all writes to the file to go completely to disk before returning to the bench-mark.

and also check the -S
when you make the statement
\”For this article, cache effects will be limited as much as possible\”
I think you should/could have added those options, believe me they make a great impact on the results.

cheers

Mario David

]]> By: laytonjb http://www.linux-mag.com/id/7525/#comment-7007 laytonjb Wed, 30 Nov -0001 00:00:00 +0000 http://www.linux-mag.com/id/7525/#comment-7007 <p>@mdavid,</p> <p>I thought about including those flags and did a little early comparison testing. But I decided not to include them. I was more interested in the general trends of the file systems than the absolute results (as long as I used the same options to all of the file systems tested). My primary goal in minimising the cache effects was the amount of memory used. This is still somewhat controversial - how large a file to use relative to the main memory. I haven\'t really seen any studies that vary the size of the file relative to main memory size.</p> <p>I will keep in mind your suggestion though. It might be interesting to include those options at some point and compare it to a run without them. I\'m wondering if the general overall observations will remain the same or not.</p> <p>Thanks!</p> <p>Jeff </p> @mdavid,

I thought about including those flags and did a little early comparison testing. But I decided not to include them. I was more interested in the general trends of the file systems than the absolute results (as long as I used the same options to all of the file systems tested). My primary goal in minimising the cache effects was the amount of memory used. This is still somewhat controversial – how large a file to use relative to the main memory. I haven\’t really seen any studies that vary the size of the file relative to main memory size.

I will keep in mind your suggestion though. It might be interesting to include those options at some point and compare it to a run without them. I\’m wondering if the general overall observations will remain the same or not.

Thanks!

Jeff

]]>