Jeff Layton is an Enterprise Technologist for HPC at Dell. He can be found lounging around at a nearby Frys enjoying the coffee and waiting for sales (but never during working hours).
Comments on "Short Stroking Hard Disks for Performance"
lbevanda
I think your disk’s short strokes affect your sound card…
Where one needs to use the entire disk, or most of it, I guess it would also help to use partitions on the outer cylinders for many small, frequently-accessed files? For example a partition for a squid web cache, or a mail server, and log files.
Thanks for contributing to a webzine that I read regularly.
And for taking the effort to make a good video (even if it did have a little static) ;-)
It seems to me that you are suggesting that you can reduce average read latency from 12ms to 8ms by using the outer 5% of the disk cylinders.
I don’t think you are correct. I think that your findings are artifacts of the testing method.
First your inner 25% of cylinders was 9.91 ms for ONE test.
And your outer 25% of cylinders was 9.53 ms for ONE test.
Less than 5% difference for inner/outer cylinders and with only one sample test that isn’t convincing. What if your system had some other process wake up for a fraction of one second during the test?
It would be much more convincing if you ran that test for a period of 3000 seconds on a in run level 1 with no kernel device drivers loaded except necessary to get to the console.
Also…. I think that the importance of any performance gains noted by this method are not as real as you might think. In the period of 30 seconds in the first test with 12.37ms seek time you had a total of somewhere around 2,500 random seeks on a drive with 60,801 cylinders. That means that there is about 24:1 odds that each random seek will involve a head seek and not just rotational latency.
Whereas… If you run the same 30 second test on a partition with only 15,000 cylinders, then the odds are only 6:1 that each random seek will involve a head seek and not just rotational latency.
Therefore I don’t think there is any substantial difference in the performance of any cylinders…. Just by a disk with less cylinders or stripe your data across two smaller disks instead of one. Much more cost effective and much bigger bang for your buck than buying standard size cheap disk and getting a “perceived” 25% improvement in a statistic.
I think that your evidence is much more in favor of making sure that you defragment your disk so that your DBF file or your
I really enjoy reading Linux Magazine articles and
I find them very useful in my work.
However, I think it would be a good idea for you to stick to
the text medium.
I don’t have time to watch an entire video, and I can’t
scan it for relevant information. I imagine the highest
value segment or your audience are all under similar time
constraints.
It’s very important for the magazine’s prospects that
Linux Magazine remain useful to technology professionals
working in a business IT context.
Case in point:
About six weeks ago, I found out about Colfax International
from a banner advertisement on one of your articles.
I clicked through from the banner ad on your article
to ColfaxIntl. website.
I then immediately ordered $16k worth of computer
hardware from them.
I told my sales rep at Colfax that I had found out about them from
an ad on your website.
So, it would seem that the advertising supported model
works and advertising supported content providers
should be supremely concerned with what their audience
finds useful.
This kind of attention to the audience’s practical needs
is THE reason that Google dominates the search engine world.
Yahoo was way out in front in 2000 and they sold their audience
like sheep to wolves. Lycos made big beautiful pages
that few users could load in a reasonable amount of time.
The audience left them for Google (where the
home page was under 5kB and results pages under 10kB) in
droves.
On the topic of the actual content rather than the medium….
Remember that you can also use ALL of the remaining capacity
of the drive for long term backup (of data for which the
primary copy is on a different device unless using RAID6).
As long as you don’t need high performance access to those
outer tracks during your backup process, you’ll never experience
any performance degradation from doing this.
I’m sure all of the downsides are obvious in retrospect. I can’t really imagine the upside unless there is something mechanical (i.e. how to tune up your skis, etc) involved. Even if a video is warranted, it should always be as an adjunct to text.
I disagree with you. I don’t think the time has much to do with it since we’re talking about latency. But in my opinion the change in latency is really due to the heads having to cover a smaller number of tracks. This is a fairly common known phenomenon but it is easily demonstrable. You see a lot of companies do this when running benchmarks using disks (this goes back to my diatribe about benchmarks). In addition, I wish I had the time to run the tests several times and report the averages and the standard deviations since that would give some indication of whether the times differences are real or in the noise.
@stevemadere
Are you talking about just my rather poor attempt at a video or the idea of using videos or other media for discussing all things Linux?
I will definitely admit that 15 minutes is too long to watch (I too would get bored). But I also admit I became curious while making the video about the impact of various capacities on latency so I started running more tests. Of course, I hit 15 minutes by then. :) Then I ran up against the deadline so I had to go with what I got.
So I apologize for my poor first attempt at a video. I’ve been chastised by the editor and publisher in a most thorough manner so I will get better (plus I need to find MUCH better tools for Linux – the ones I tried just stink).
While you’re reading this, do you have any suggestions for alternative media rather than just text? Is there something you like to use?
Honestly, I’m surprised file-systems are still not really dealing with this “external” fragmentation, which is what it’s all about.
Basically, the problem here is that the file-system aren’t really exploiting the “common” access-patterns of files to minimize seeking (remember, rotational latency ALSO contribute to seek, not just arm movements).
I think this could even be improved upon for the common use-cases, even while using the entire disk, by having a small collector service in the OS collecting normal usage-patterns, and then using idle-time for optimizing “inter-file” access for the most common patterns.
Above all, it would be a much-needed remedy of the “aging” problem all current filesystems suffers from.
I was suggesting that videos in general are not particularly
useful to people who are gathering information rather than
watching eye-candy for entertainment.
Unless you know of a video search engine that can fast-forward
to each occurrence of a specific word in a video and play that segment
instantly, text is a FAR more efficient medium for delivering
information about Linux.
It is true that for visual entertainment, scanning a video is typically
more efficient than text, but I don’t actual get any visual
entertainment from Linux.
Heck, I still use vi.
OK, if somebody made a video of a Linux system overheating and
catching fire, that might be visual entertainment on-topic
for LinuxMagazine but it’s a stretch…. ;)
The impact for me is that I read this zine in a place that blocks all videos. Hence, I never was able to actually determine what you were doing. Please stick to text in the future.
Sorry about this logging, but a month ago I had HD crash on 10G Maxtor
and curious as I’m still I opened it and could see physical scratch on
the surface of media, so I had to build up everything from old back up
Maxtor i(was) 91021U from 1999 and has been working flawless since so
this is not to complain. Actually I find an other one from my shelves
and bought a WD of 320G to have more room. I hope the maxtor will last
next decade as the first one. My machine is working 24/7 usually except
during my holidays. LXF Mag has helped me during years in many small
things. Thank You for it.
Great weblog right here! Additionally your web site quite a bit up fast! What web host are you the use of? Can I get your affiliate hyperlink to your host? I wish my site loaded up as fast as yours lol
Comments on "Short Stroking Hard Disks for Performance"
I think your disk’s short strokes affect your sound card…
@lbevanda,
LOL! Sorry about that. I’m not sure what happened. I guess you can blame Windows for that :)
Jeff
@laytonjb,
no matter, very interesting post…
Hi,
Very interesting post.
Where one needs to use the entire disk, or most of it, I guess it would also help to use partitions on the outer cylinders for many small, frequently-accessed files? For example a partition for a squid web cache, or a mail server, and log files.
Thanks for contributing to a webzine that I read regularly.
And for taking the effort to make a good video (even if it did have a little static) ;-)
It seems to me that you are suggesting that you can reduce average read latency from 12ms to 8ms by using the outer 5% of the disk cylinders.
I don’t think you are correct. I think that your findings are artifacts of the testing method.
First your inner 25% of cylinders was 9.91 ms for ONE test.
And your outer 25% of cylinders was 9.53 ms for ONE test.
Less than 5% difference for inner/outer cylinders and with only one sample test that isn’t convincing. What if your system had some other process wake up for a fraction of one second during the test?
It would be much more convincing if you ran that test for a period of 3000 seconds on a in run level 1 with no kernel device drivers loaded except necessary to get to the console.
Also…. I think that the importance of any performance gains noted by this method are not as real as you might think. In the period of 30 seconds in the first test with 12.37ms seek time you had a total of somewhere around 2,500 random seeks on a drive with 60,801 cylinders. That means that there is about 24:1 odds that each random seek will involve a head seek and not just rotational latency.
Whereas… If you run the same 30 second test on a partition with only 15,000 cylinders, then the odds are only 6:1 that each random seek will involve a head seek and not just rotational latency.
Therefore I don’t think there is any substantial difference in the performance of any cylinders…. Just by a disk with less cylinders or stripe your data across two smaller disks instead of one. Much more cost effective and much bigger bang for your buck than buying standard size cheap disk and getting a “perceived” 25% improvement in a statistic.
I think that your evidence is much more in favor of making sure that you defragment your disk so that your DBF file or your
I really enjoy reading Linux Magazine articles and
I find them very useful in my work.
However, I think it would be a good idea for you to stick to
the text medium.
I don’t have time to watch an entire video, and I can’t
scan it for relevant information. I imagine the highest
value segment or your audience are all under similar time
constraints.
It’s very important for the magazine’s prospects that
Linux Magazine remain useful to technology professionals
working in a business IT context.
Case in point:
About six weeks ago, I found out about Colfax International
from a banner advertisement on one of your articles.
I clicked through from the banner ad on your article
to ColfaxIntl. website.
I then immediately ordered $16k worth of computer
hardware from them.
I told my sales rep at Colfax that I had found out about them from
an ad on your website.
So, it would seem that the advertising supported model
works and advertising supported content providers
should be supremely concerned with what their audience
finds useful.
This kind of attention to the audience’s practical needs
is THE reason that Google dominates the search engine world.
Yahoo was way out in front in 2000 and they sold their audience
like sheep to wolves. Lycos made big beautiful pages
that few users could load in a reasonable amount of time.
The audience left them for Google (where the
home page was under 5kB and results pages under 10kB) in
droves.
On the topic of the actual content rather than the medium….
Remember that you can also use ALL of the remaining capacity
of the drive for long term backup (of data for which the
primary copy is on a different device unless using RAID6).
As long as you don’t need high performance access to those
outer tracks during your backup process, you’ll never experience
any performance degradation from doing this.
Array
WTF! – Posting a video is totally stupid.
I’m sure all of the downsides are obvious in retrospect. I can’t really imagine the upside unless there is something mechanical (i.e. how to tune up your skis, etc) involved. Even if a video is warranted, it should always be as an adjunct to text.
@benrussousa
I disagree with you. I don’t think the time has much to do with it since we’re talking about latency. But in my opinion the change in latency is really due to the heads having to cover a smaller number of tracks. This is a fairly common known phenomenon but it is easily demonstrable. You see a lot of companies do this when running benchmarks using disks (this goes back to my diatribe about benchmarks). In addition, I wish I had the time to run the tests several times and report the averages and the standard deviations since that would give some indication of whether the times differences are real or in the noise.
@stevemadere
Are you talking about just my rather poor attempt at a video or the idea of using videos or other media for discussing all things Linux?
I will definitely admit that 15 minutes is too long to watch (I too would get bored). But I also admit I became curious while making the video about the impact of various capacities on latency so I started running more tests. Of course, I hit 15 minutes by then. :) Then I ran up against the deadline so I had to go with what I got.
So I apologize for my poor first attempt at a video. I’ve been chastised by the editor and publisher in a most thorough manner so I will get better (plus I need to find MUCH better tools for Linux – the ones I tried just stink).
While you’re reading this, do you have any suggestions for alternative media rather than just text? Is there something you like to use?
Jeff
Honestly, I’m surprised file-systems are still not really dealing with this “external” fragmentation, which is what it’s all about.
Basically, the problem here is that the file-system aren’t really exploiting the “common” access-patterns of files to minimize seeking (remember, rotational latency ALSO contribute to seek, not just arm movements).
I think this could even be improved upon for the common use-cases, even while using the entire disk, by having a small collector service in the OS collecting normal usage-patterns, and then using idle-time for optimizing “inter-file” access for the most common patterns.
Above all, it would be a much-needed remedy of the “aging” problem all current filesystems suffers from.
I was suggesting that videos in general are not particularly
useful to people who are gathering information rather than
watching eye-candy for entertainment.
Unless you know of a video search engine that can fast-forward
to each occurrence of a specific word in a video and play that segment
instantly, text is a FAR more efficient medium for delivering
information about Linux.
It is true that for visual entertainment, scanning a video is typically
more efficient than text, but I don’t actual get any visual
entertainment from Linux.
Heck, I still use vi.
OK, if somebody made a video of a Linux system overheating and
catching fire, that might be visual entertainment on-topic
for LinuxMagazine but it’s a stretch…. ;)
The impact for me is that I read this zine in a place that blocks all videos. Hence, I never was able to actually determine what you were doing. Please stick to text in the future.
Stick to text. Video is blocked at my work.
Interesting… how to explain the following numbers. The /bacup directory is on sdb1
sdb1: ext4 1TB
sdb2: btrfs 1TB (not mounted)
Seek on sdb2: 16ms
Seek on loop0: < 1ms
Seek on sdb2 while seeking on loop0:
sdb2: 40 ms
loop0: 2.61
# dd if=/dev/urandom of=bigloop bs=1024 count=2048k
# losetup -f bigloop ; losetup -a
/dev/loop0: [0811]:60954319 (/backup/testing/bigloop)
# /home/jrobicha/seeker/seeker /dev/sdb2
Seeker v2.0, 2007-01-15, http://www.linuxinsight.com/how_fast_is_your_disk.html
Benchmarking /dev/sdb2 [883716MB], wait 30 seconds.............................
Results: 61 seeks/second, 16.25 ms random access time
# /home/jrobicha/seeker/seeker /dev/sdb2&
[1] 31296
# /home/jrobicha/seeker/seeker /dev/loop0
Seeker v2.0, 2007-01-15, http://www.linuxinsight.com/how_fast_is_your_disk.html
Benchmarking /dev/loop0 [2048MB], wait 30 seconds.........................................................
Results: 23 seeks/second, 43.10 ms random access time
..
Results: 383 seeks/second, 2.61 ms random access time
Sorry about this logging, but a month ago I had HD crash on 10G Maxtor
and curious as I’m still I opened it and could see physical scratch on
the surface of media, so I had to build up everything from old back up
Maxtor i(was) 91021U from 1999 and has been working flawless since so
this is not to complain. Actually I find an other one from my shelves
and bought a WD of 320G to have more room. I hope the maxtor will last
next decade as the first one. My machine is working 24/7 usually except
during my holidays. LXF Mag has helped me during years in many small
things. Thank You for it.
hm? what’s this?
the browser that you are using is?
is??
is???
!!INTERNET!EXPLORER!!
i feel sick
vomit
Maybe I should look in YouTube for magazine articles?
First page of each article is 0 value. Except this article that is not article but video.
Not watched.
Yeah, smaller partition = lower average seek time, but also means more fragmentation, think about it.
Great weblog right here! Additionally your web site quite a bit up fast! What web host are you the use of? Can I get your affiliate hyperlink to your host? I wish my site loaded up as fast as yours lol