Who knew that compression could be so useful in file systems? SquashFS, typically used for embedded systems, can be a great fit for laptops, desktops and, yes, even servers.
As we’ve demonstrated over the past several weeks, there are no shortage of new file systems in the latest version of Linux. (See NILFS: A File System to Make SSDs Scream, Linux Don’t Need No Stinkin’ ZFS: BTRFS Intro & Benchmarks and ext4 File System: Introduction and Benchmarks)
In keeping with the theme of new file systems, let’s take a look at SquashFS. SquashFS is a read-only compressed file system that has a variable block size. The primary intent of the file system is for constrained block device/memory systems. The classic example targets SquashFS for embedded systems but there are other uses for it that fall outside of the embedded world, and could surprise you.
Compressed File Systems
Data compression has been around for a very long time. As everyone knows, the concept behind data compress is to encode data using various techniques saving storage space. Compression also reduces the size of data that is to be transmitted. The most common example in Linux is gzip that is used to compress data files. Here’s a quick example illustrating the change in file size:
$ ls -lsah FS_scan.csv
3.2M -rw-r--r-- 1 laytonjb laytonjb 3.2M 2009-05-24 20:31 FS_scan.csv
$ gzip -9 FS_scan.csv
$ ls -lsah FS_scan.csv.gz
268K -rw-r--r-- 1 laytonjb laytonjb 261K 2009-05-24 20:31 FS_scan.csv.gz
The original file was 3.2MB and after using gzip, with the "-9" option that provides maximum compression, the file is 268KB. Also notice the the extension ".gz" to indicate that the file has been compressed with gzip. The compression ratio, which is the ratio of the original size to the compressed size, is 11.9:1. The compression ratio is very dependent on the uncompressed data (how compressible is the data?) and the compression algorithm.
There are generally two types of algorithms - lossless and lossy. Since this article is about data it will focus on lossless algorithms that have an encoding that be reversed to recreate the data exactly. There are a huge number of compression algorithms that take data and find a new encoding of the data that is much smaller. The difference between algorithms focuses on the techniques that create the new encoding. From the user perspective two of the biggest concerns are how much compression can be obtained and how much time and/or CPU usage it takes to perform the compression (or uncompression). However, the goal of all of them remains the same, to reduce the size of data to save space.
The primary trade-offs in compressed file systems is that it takes CPU cycles and time to compress and uncompress data in return for reduced storage space. If you have the cycles and don't need a fast file system then you can save space. Alternatively, if you are severely memory or storage space constrained, then compressed file systems may be the only choice. This is most common in embedded systems that have severe storage restrictions.
There is an older article that discussed the compressed file systems available for Linux at the time, how they could be employed, etc. To date it is one of the better articles around compressed file systems. Overall, there is a very large number of compressed file systems available for Linux. For example, there is, JFFS, JFFS2, LogFS, CramFS, compFUSEd, and SquashFS among others. The file system compFUSEd works in conjunction with existing file systems and FUSE to allow you to do compression with both read and write for your existing file system.
Btrfs also has a compression option. Compression can be turned on as a mount option and provides compression via the zlib function that is in the kernel. While this is a great feature and works on read as well as write (not all compressed file systems do this), the focus of btrfs is not just as a compressed file system. This article will focus on file systems that are primarily compressed file systems. In particular, it will focus on SquashFS, which is the newest compressed file system for Linux, offering lots of new features that can be be exploited by non-embedded devices (even your every day desktop!).
SquashFS has been under development outside the Linux kernel for a number of years. Phillip Lougher started SquashFS because the existing compressed file systems did not have the features that he wanted. He began development around the 2002 time frame and tried to get it merged into the kernel a couple of times. However, the Linux community wanted some changes and features which Phillip accomplished, while adding new features that put SquashFS above other compressed file systems in many respects. So version 4.0 of SquashFS went into the Linux kernel in version 2.6.29.
SquashFS is a read-only compressed file system which means that once you mount a SquashFS file system you can only read from it. However, as will be discussed later, you can combine SquashFS with a union mount file system UnionFS or Aufs to allow you to read and write to the file system (at least from the user perspective). The version of SquashFS currently in the kernel uses gzip for it's compression.
SquashFS is arguably one of the most feature-rich compressed file systems. In the documentation for SquashFS that is included in the kernel is a quick comparison with CramFS. The author has not checked the accuracy or validity of the comparison but it is reproduced here since it at least provides a list of SquashFS features.
|Max filesystem size:
|Max file size:
||~ 2 TiB
|Max entries per directory
|Max block size
|Sparse file support
|Tail-end packing (fragments)
|Exportable (NFS etc.):
|Hard link support:
|"." and ".." in readdir
|Real inode numbers:
|File creation time:
|Xattr and ACL support
The features in SquashFS, particularly the size of the file system and the size of the files and directories, are very appealing for compressed file systems that could use larger amounts of space. The kernel documentation also explains that SquashFS compresses the data as well as the inodes and the directories. The inode and directory data are highly compressed and packed on byte boundaries. Each compressed inode is 8 bytes in length (on average) with the exact length depending upon the file type. For example, a regular file, a directory, a symbolic link, and block/char device inodes all have different sizes.
SquashFS also has a unique feature in that it has a variable block size and the range of block sizes is rather large. By default the block size is 128Kb and the maximum block size is 1Mb. The larger block sizes can help with greater compression ratios (i.e. smaller size file systems). But using block sizes other than the typical 4Kb has created some difficulties for SquashFS.
At this time SquashFS doesn't decompress the blocks into the kernel pagecache. This means that SquashFS has its own cache, one for metadata and one for fragments (a total of two small caches). The cache isn't used for file data blocks which are decompressed and cached in the kernel pagecache in the typical fashion. But for fragment or metadata blocks that have been read as a result of a metadata access or a fragment access, the blocks are decompressed and temporarily placed into the SquashFS cache. SquashFS packs together metadata and fragments into blocks for maximum compression so that when a read access of a fragment or metadata happens, the retrieval also obtains other metadata and fragment data. Rather than discard these additional pieces of information, they are placed into the temporary cache in case a near future access is needed without having to retrieve them and decompress them.
SquashFS 4.0 has some other features that people have been wanting. During the creation of a SquashFS file system file duplicates can be removed. In addition, it can support both big endian and little endian architectures (very useful for embedded systems that use a wide range of processor architectures). Also, during the creation of the file system, SquashFS can create a file system for the endian architecture of the target platform.
To get more insight into SquashFS there is a nice video of the SquashFS developer, Phillip Lougher, discussing the file system. It is well worth the time to watch it.
There is a really well written HOWTO that describes how one goes about creating a SquashFS file system. The general approach is to create a file system image using the SquashFS tools. You can create an image of an entire file system, a directory, or even a single file. Then this image can be mounted directly (if it is a device) or mounted using a loopback device (if it is a file).
To get started with SquashFS you have to have either a 2.6.29 kernel or a patched kernel. You also need the user-space SquashFS tools that can be downloaded from the sourceforge site. Then you just build them and install them as directed in the tar file or at the SquashFS website.
Once the tools are installed and the kernel has the ability to use SquashFS file systems, then the next step is to create a file system using
mksquashfs. The general form of the command is,
# mksquashfs source1 source2 ... destination [option]
where source1 source2, etc. are files and directories to be added to the SquashFS image. They can be given with absolute or relative paths. There are a large number of options available. You can read the summary here or read the man pages once you have installed the SquashFS tools.
An example of creating a SquashFS file system is to take a subsection of a user's home directory that has data that won't be changed and create a SquashFS image and mount it in the user's home directory. In this case, the directory is
/home/laytonjb/Documents. If you are not writing any new data to the
Documents subdirectory, then mounting
/home/laytonjb/Documents as read-only will be fine for most systems. This directory is a little over 320 MB in size (prior to using SquashFS).
$ cd Documents
$ du -s -h
The hard drive is a Western Digital WD1600BEVE 2.5" EIDE hard drive connected via a USB 2.0 connection to a laptop with dual-core Intel Core2 Duo U7700 running at 1.33 Ghz.
The first step is to create a SquashFS image of
/home/laytonjb/Documents. This assumes you have enough space on the device that contains
/home/laytonjb/Documents and the resulting SquashFS image,
$ time sudo mksquashfs /home/laytonjb/Documents /squashfs/storage/Documents.sqsh
Parallel mksquashfs: Using 2 processors
Creating little endian 3.1 filesystem on /squashfs/storage/Documents.sqsh, block size 131072.
[===================================================] 3454/3454 100%
Exportable Little endian filesystem, data block size 131072, compressed data, compressed metadata, compressed fragments, duplicates are removed
Filesystem size 195683.71 Kbytes (191.10 Mbytes)
62.05% of uncompressed filesystem size (315374.80 Kbytes)
Inode table size 21033 bytes (20.54 Kbytes)
36.88% of uncompressed inode table size (57034 bytes)
Directory table size 16599 bytes (16.21 Kbytes)
48.76% of uncompressed directory table size (34041 bytes)
Number of duplicate files found 489
Number of inodes 1495
Number of files 1360
Number of fragments 164
Number of symbolic links 0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 135
Number of uids 1
Number of gids 0
Notice that mksquashfs discovers and uses both cores and recognizes the system is little endian and creates the image for that system (there is an option for creating images for specific endian systems). After the command is finished it gives you a summary of what it did along with the compression ratios. Notice that for this lowly hard drive and laptop it took a little over 40 seconds to build the image. However, as the file system grows in size and has more files, the longer it will take mksquashfs to create the image.