Comments on: File System Evangelist and Thought Leader: An Interview with Valerie Aurora http://www.linux-mag.com/id/7416/ Open Source, Open Standards Sat, 05 Oct 2013 13:48:18 +0000 hourly 1 http://wordpress.org/?v=3.1 By: plougher http://www.linux-mag.com/id/7416/#comment-6704 plougher Wed, 30 Nov -0001 00:00:00 +0000 http://www.linux-mag.com/id/7416/#comment-6704 <p>Valarie Aurora, as usual, makes some interesting points about filesystems development. However, her comment that \"For many years, file systems on Linux was stalled\" is slightly misleading in that, perhaps inadvertently, it implies this is a Linux only phenomenon. Filesystems development everywhere for many years has been stalled. This is why operating systems text books still only describe the \"Sun-style VFS architecture and FFS-style on-disk format\". It is amazing that the Unix FFS a 20+ year old filesystem is still the default (or only) filesystem available on many Unixes.</p> <p>Why is this? In my opinion it is because the Unix FFS and FFS-like filesystems for many years was always \"good enough\" for companies in the position to fund filesystems development not to bother designing anything new. It may surprise some people, but the research community even in the 80s and 90s published many papers which showed that file sizes and file usage/access patterns were changing, which challenged many of the design assumptions in FFS-like filesystems. For instance it should still be possible to find these papers somewhere on the internet:</p> <p>Ousterhout J.K. et al, \"A Trace Driven Analysis of the Unix 4.2 BSD File System\", proceedings of the 10th Symposium on Operating Systems Principles, 15-24 December 1985.</p> <p>Ousterhout J.K., \"Why Aren\'t Operating Systems Getting Faster as Fast as Hardware\", Proceedings of the Summer 1990 Usenix Conference, Anaheim, CA, June 11-15, pp. 247-256.</p> <p>These and other papers led to research prototypes like the Bullet Fileserver (does anybody remember that?) and log structured filesystems (again proposed by Ousterhout in 1989), concepts which are only now appearing in development filesystems.</p> <p>I have to declare a vested disappointment in the lack of funding for filesystems development, as I did filesystems research in the late 80s/early 90s in the new area of multimedia filesystems. Disks and processors in those days were inadequate to properly support the demands of multimedia file access which led to new filesystem layouts and concepts, many of which I proposed. Again this paper should still be somewhere on the internet (shameless plug):</p> <p>Lougher, P. et al, \"The Design of a Storage Server for Continuous Media\", The Computer Journal, 36(1), 32-43 January 1993.</p> <p>Like now, in those days it was impossible to get research funding for filesystems (in the UK at least), but by piggy-backing onto the new area of multimedia I managed to get funding for a PhD and post-doc research in distributed multimedia filesystems. However, the failure of VOD (Video on Demand) trials in the 90s to gain sufficient ROI led to interest in this to disappear.</p> <p>Since then I have written and released SquashFS as an unpaid filesystem project for Linux. This is successful (much more than I anticipated), but being an unpaid niche project, it is regularly ignored as a source of filesystem innovation.</p> <p>The lack of interest in filesystems development over the last 20 years has been IMHO an industry and a personal career disaster. </p> Valarie Aurora, as usual, makes some interesting points about filesystems development. However, her comment that \”For many years, file systems on Linux was stalled\” is slightly misleading in that, perhaps inadvertently, it implies this is a Linux only phenomenon. Filesystems development everywhere for many years has been stalled. This is why operating systems text books still only describe the \”Sun-style VFS architecture and FFS-style on-disk format\”. It is amazing that the Unix FFS a 20+ year old filesystem is still the default (or only) filesystem available on many Unixes.

Why is this? In my opinion it is because the Unix FFS and FFS-like filesystems for many years was always \”good enough\” for companies in the position to fund filesystems development not to bother designing anything new. It may surprise some people, but the research community even in the 80s and 90s published many papers which showed that file sizes and file usage/access patterns were changing, which challenged many of the design assumptions in FFS-like filesystems. For instance it should still be possible to find these papers somewhere on the internet:

Ousterhout J.K. et al, \”A Trace Driven Analysis of the Unix 4.2 BSD File System\”, proceedings of the 10th Symposium on Operating Systems Principles, 15-24 December 1985.

Ousterhout J.K., \”Why Aren\’t Operating Systems Getting Faster as Fast as Hardware\”, Proceedings of the Summer 1990 Usenix Conference, Anaheim, CA, June 11-15, pp. 247-256.

These and other papers led to research prototypes like the Bullet Fileserver (does anybody remember that?) and log structured filesystems (again proposed by Ousterhout in 1989), concepts which are only now appearing in development filesystems.

I have to declare a vested disappointment in the lack of funding for filesystems development, as I did filesystems research in the late 80s/early 90s in the new area of multimedia filesystems. Disks and processors in those days were inadequate to properly support the demands of multimedia file access which led to new filesystem layouts and concepts, many of which I proposed. Again this paper should still be somewhere on the internet (shameless plug):

Lougher, P. et al, \”The Design of a Storage Server for Continuous Media\”, The Computer Journal, 36(1), 32-43 January 1993.

Like now, in those days it was impossible to get research funding for filesystems (in the UK at least), but by piggy-backing onto the new area of multimedia I managed to get funding for a PhD and post-doc research in distributed multimedia filesystems. However, the failure of VOD (Video on Demand) trials in the 90s to gain sufficient ROI led to interest in this to disappear.

Since then I have written and released SquashFS as an unpaid filesystem project for Linux. This is successful (much more than I anticipated), but being an unpaid niche project, it is regularly ignored as a source of filesystem innovation.

The lack of interest in filesystems development over the last 20 years has been IMHO an industry and a personal career disaster.

]]> By: davidgro http://www.linux-mag.com/id/7416/#comment-6705 davidgro Wed, 30 Nov -0001 00:00:00 +0000 http://www.linux-mag.com/id/7416/#comment-6705 <p>What about support for non-linux file systems?<br /> In particular I am worried about exFat (both read and write) because it is going to be the format of the next generation of memory cards (SDXC at least) and of course the gadgets that use them! It would also be valuable for partitions (which may be on portable drives) shared between OSs, with support for 4GiB+ files. </p> What about support for non-linux file systems?
In particular I am worried about exFat (both read and write) because it is going to be the format of the next generation of memory cards (SDXC at least) and of course the gadgets that use them! It would also be valuable for partitions (which may be on portable drives) shared between OSs, with support for 4GiB+ files.

]]>