Comments on: I Like My File Systems Chunky: UnionsFS and ChunkFS

By: typhoidmary

typhoidmary — Wed, 30 Nov -0001 00:00:00 +0000

Why would you not just use device mapper with LVM or EVM?

By: laytonjb

laytonjb — Wed, 30 Nov -0001 00:00:00 +0000

There are two reasons you don\'t want to use LVM.

1. File systems such as ext3 have limited sizes but there are people who want to use ext3 for larger file systems. LVM doesn\'t help in this context.

2. LVM doesn\'t help with the fsck times. Breaking up the file system into chunks can greatly reduce fsck time.

However, I am a proponent of using LVM if the file system is capable of growing to use added space. Emotionally I like the concept of shrinking a file system to gain back some space that I then can allocate somewhere else. But I have yet to do this myself and no one I\'ve spoken with has done it yet (I\'m sure there are people who would like to do - if so, let us know).

If you are referring to the \"homework\" of using LVM, then I think that is a good solution (another person has emailed me about that as well).

BTW - my email address in the original article is incorrect. It should be jlayton _at_ linux-mag.com. I fixed the article but the cache may trip up people.

Thanks for the post!

Jeff

By: caletronics

caletronics — Wed, 30 Nov -0001 00:00:00 +0000

For a long time I\'ve been using bind mounts (see below). I get the manageability of individual disks, one coherent /home, but, also (unlike chunkFS?) the ability to restrict the \"view\" to different NFS clients. I also share the pitfall of filling one disk while another may have lots of free space.

My question: what does chunkFS get me compared to bind mounts?

Thanks,
Chris D

For clarity I\'m just showing excerpts. It\'s worth pointing out that except for serval and ocelot other clients are unable to see my home directory and therefore the music directory inside that. But using bind I can also mount the music disk where all clients can see it.
/etc/fstab:
...
/dev/k01/01.3 /disk/01.3/ xfs rw 0 0
/dev/k02/02.3 /disk/02.3/ xfs rw 0 0
/dev/k03/03.1 /disk/03.1/ xfs rw 0 0
/disk/02.3/home/chrisd /home/chrisd none rw,bind 0 0
/disk/01.3/mythtv /home/mythtv none rw,bind 0 0
/disk/03.1/music /home/chrisd/music none rw,bind 0 0
/disk/03.1/music /home/mythtv/music none ro,bind 0 0
...

/etc/exports:
/home *(ro,fsid=0,no_root_squash,no_subtree_check,insecure)
/home/chrisd serval.zoo(rw,nohide,no_root_squash,no_subtree_check) \\
ocelot.zoo(ro,nohide,no_root_squash,no_subtree_check)
/home/chrisd/music serval.zoo(rw,nohide,no_root_squash,no_subtree_check) \\
ocelot.zoo(ro,nohide,no_root_squash,no_subtree_check)
/home/mythtv *(ro,nohide,no_root_squash,no_subtree_check,insecure) \\
serval.zoo(rw,nohide,no_root_squash,no_subtree_check)
/home/mythtv/music *(ro,nohide,no_root_squash,no_subtree_check,insecure)
...

By: drogo

drogo — Wed, 30 Nov -0001 00:00:00 +0000

I\'ve shrunken an LVM device before.

I wanted to backup a smallish RAID-5 array (3x200G drives) and came across the snapshot ability. Since I had originally used all the extents when I first created the array, I had to shrink the filesystem, then free up a few extents for the snapshot.

I was successful, but I did have a fresh backup sitting right next to the system. Heck, the backup was probably the voodoo I needed to ensure success. :D

By: typhoidmary

typhoidmary — Wed, 30 Nov -0001 00:00:00 +0000

I think my point about LVM was missed. The idea of a chunky FS is that you manage the fact that ext3 becomes less and less practical the bigger the span it has to cover. So a chunky FS system is really several smaller ext3 FS working \"seamlessly\" together. This is one of the things LVM does. While LVM is designed to grow and shrink and also span disks, there is nothing to stop it from spanning volumes on a disk.

So take that 1 TB drive, partition it in 10 GB (to take a size at random) sections, and combine these sections as 1 logical volume. ext3 then takes care of a FS section closer to it\'s \"comfort\" level, while LVM handles the issue of files spanning partitions.

The question remaining is whether or not fsck can run on the individual partitions, or if it must run on the logical volume. If it can\'t handle just the partition, then this is a great feature request for the LVM project.

By: laytonjb

laytonjb — Wed, 30 Nov -0001 00:00:00 +0000

@typhoidmary
I don\'t think I missed your point but maybe you don\'t see the difference between the two concepts. With your concept you combine partitions using LVM into a logical volume that you then use ext3. So for example, you could take five 1TB drives into a single 5TB LV. But when you run an fsck on the file system you are still running it across a single file system.

Using the principles of ChunkFS you can combine separate file systems into a single logical file system using UnionFS. In this approach, for example, you would create an ext3 file system on each of the 5 drives, then combine them using UnionFS into a seemingly single file system. If you need to run an fsck you can run it on one of 5 pieces without having to run it on all 5.

Note that you can still use LVM to create the LV\'s for each of the \"chunks\" and combine them with UnionFS.

So the big difference between your approach the approach in the article is that your approach creates a single ext3 file system and the article creates multiple ext3 file systems and combine them using UnionFS. Your approach allows you to have files that fill up the entire file system but the fsck is slow. In the approach in the article you can fill a chunk without filling the entire union, possibly causing problems. But the fsck is much faster than your approach.

Does this make sense or did I make it worse?

Jeff