SquashFS: Not Just for Embedded Systems

Who knew that compression could be so useful in file systems? SquashFS, typically used for embedded systems, can be a great fit for laptops, desktops and, yes, even servers.

As we’ve demonstrated over the past several weeks, there are no shortage of new file systems in the latest version of Linux. (See NILFS: A File System to Make SSDs Scream, Linux Don’t Need No Stinkin’ ZFS: BTRFS Intro & Benchmarks and ext4 File System: Introduction and Benchmarks)

In keeping with the theme of new file systems, let’s take a look at SquashFS. SquashFS is a read-only compressed file system that has a variable block size. The primary intent of the file system is for constrained block device/memory systems. The classic example targets SquashFS for embedded systems but there are other uses for it that fall outside of the embedded world, and could surprise you.

Compressed File Systems

Data compression has been around for a very long time. As everyone knows, the concept behind data compress is to encode data using various techniques saving storage space. Compression also reduces the size of data that is to be transmitted. The most common example in Linux is gzip that is used to compress data files. Here’s a quick example illustrating the change in file size:

$ ls -lsah FS_scan.csv
3.2M -rw-r--r-- 1 laytonjb laytonjb 3.2M 2009-05-24 20:31 FS_scan.csv
$ gzip -9 FS_scan.csv
$ ls -lsah FS_scan.csv.gz
268K -rw-r--r-- 1 laytonjb laytonjb 261K 2009-05-24 20:31 FS_scan.csv.gz

The original file was 3.2MB and after using gzip, with the "-9" option that provides maximum compression, the file is 268KB. Also notice the the extension ".gz" to indicate that the file has been compressed with gzip. The compression ratio, which is the ratio of the original size to the compressed size, is 11.9:1. The compression ratio is very dependent on the uncompressed data (how compressible is the data?) and the compression algorithm.

There are generally two types of algorithms - lossless and lossy. Since this article is about data it will focus on lossless algorithms that have an encoding that be reversed to recreate the data exactly. There are a huge number of compression algorithms that take data and find a new encoding of the data that is much smaller. The difference between algorithms focuses on the techniques that create the new encoding. From the user perspective two of the biggest concerns are how much compression can be obtained and how much time and/or CPU usage it takes to perform the compression (or uncompression). However, the goal of all of them remains the same, to reduce the size of data to save space.

The primary trade-offs in compressed file systems is that it takes CPU cycles and time to compress and uncompress data in return for reduced storage space. If you have the cycles and don't need a fast file system then you can save space. Alternatively, if you are severely memory or storage space constrained, then compressed file systems may be the only choice. This is most common in embedded systems that have severe storage restrictions.

There is an older article that discussed the compressed file systems available for Linux at the time, how they could be employed, etc. To date it is one of the better articles around compressed file systems. Overall, there is a very large number of compressed file systems available for Linux. For example, there is, JFFS, JFFS2, LogFS, CramFS, compFUSEd, and SquashFS among others. The file system compFUSEd works in conjunction with existing file systems and FUSE to allow you to do compression with both read and write for your existing file system.

Btrfs also has a compression option. Compression can be turned on as a mount option and provides compression via the zlib function that is in the kernel. While this is a great feature and works on read as well as write (not all compressed file systems do this), the focus of btrfs is not just as a compressed file system. This article will focus on file systems that are primarily compressed file systems. In particular, it will focus on SquashFS, which is the newest compressed file system for Linux, offering lots of new features that can be be exploited by non-embedded devices (even your every day desktop!).


SquashFS has been under development outside the Linux kernel for a number of years. Phillip Lougher started SquashFS because the existing compressed file systems did not have the features that he wanted. He began development around the 2002 time frame and tried to get it merged into the kernel a couple of times. However, the Linux community wanted some changes and features which Phillip accomplished, while adding new features that put SquashFS above other compressed file systems in many respects. So version 4.0 of SquashFS went into the Linux kernel in version 2.6.29.

SquashFS is a read-only compressed file system which means that once you mount a SquashFS file system you can only read from it. However, as will be discussed later, you can combine SquashFS with a union mount file system UnionFS or Aufs to allow you to read and write to the file system (at least from the user perspective). The version of SquashFS currently in the kernel uses gzip for it's compression.

SquashFS is arguably one of the most feature-rich compressed file systems. In the documentation for SquashFS that is included in the kernel is a quick comparison with CramFS. The author has not checked the accuracy or validity of the comparison but it is reproduced here since it at least provides a list of SquashFS features.

Feature SquashFS CramFS
Max filesystem size: 2^64 256 MiB
Max file size: ~ 2 TiB 16 MiB
Max files: unlimited unlimited
Max directories: unlimited unlimited
Max entries per directory unlimited unlimited
Max block size 1 MiB 4 KiB
Metadata compression yes no
Directory indexes: yes no
Sparse file support yes no
Tail-end packing (fragments) yes no
Exportable (NFS etc.): yes no
Hard link support: yes no
"." and ".." in readdir yes no
Real inode numbers: yes no
32-bit uids/gids yes no
File creation time: yes no
Xattr and ACL support no no

The features in SquashFS, particularly the size of the file system and the size of the files and directories, are very appealing for compressed file systems that could use larger amounts of space. The kernel documentation also explains that SquashFS compresses the data as well as the inodes and the directories. The inode and directory data are highly compressed and packed on byte boundaries. Each compressed inode is 8 bytes in length (on average) with the exact length depending upon the file type. For example, a regular file, a directory, a symbolic link, and block/char device inodes all have different sizes.

SquashFS also has a unique feature in that it has a variable block size and the range of block sizes is rather large. By default the block size is 128Kb and the maximum block size is 1Mb. The larger block sizes can help with greater compression ratios (i.e. smaller size file systems). But using block sizes other than the typical 4Kb has created some difficulties for SquashFS.

At this time SquashFS doesn't decompress the blocks into the kernel pagecache. This means that SquashFS has its own cache, one for metadata and one for fragments (a total of two small caches). The cache isn't used for file data blocks which are decompressed and cached in the kernel pagecache in the typical fashion. But for fragment or metadata blocks that have been read as a result of a metadata access or a fragment access, the blocks are decompressed and temporarily placed into the SquashFS cache. SquashFS packs together metadata and fragments into blocks for maximum compression so that when a read access of a fragment or metadata happens, the retrieval also obtains other metadata and fragment data. Rather than discard these additional pieces of information, they are placed into the temporary cache in case a near future access is needed without having to retrieve them and decompress them.

SquashFS 4.0 has some other features that people have been wanting. During the creation of a SquashFS file system file duplicates can be removed. In addition, it can support both big endian and little endian architectures (very useful for embedded systems that use a wide range of processor architectures). Also, during the creation of the file system, SquashFS can create a file system for the endian architecture of the target platform.

To get more insight into SquashFS there is a nice video of the SquashFS developer, Phillip Lougher, discussing the file system. It is well worth the time to watch it.

Using SquashFS

There is a really well written HOWTO that describes how one goes about creating a SquashFS file system. The general approach is to create a file system image using the SquashFS tools. You can create an image of an entire file system, a directory, or even a single file. Then this image can be mounted directly (if it is a device) or mounted using a loopback device (if it is a file).

To get started with SquashFS you have to have either a 2.6.29 kernel or a patched kernel. You also need the user-space SquashFS tools that can be downloaded from the sourceforge site. Then you just build them and install them as directed in the tar file or at the SquashFS website.

Once the tools are installed and the kernel has the ability to use SquashFS file systems, then the next step is to create a file system using mksquashfs. The general form of the command is,

# mksquashfs source1 source2 ... destination [option]

where source1 source2, etc. are files and directories to be added to the SquashFS image. They can be given with absolute or relative paths. There are a large number of options available. You can read the summary here or read the man pages once you have installed the SquashFS tools.

An example of creating a SquashFS file system is to take a subsection of a user's home directory that has data that won't be changed and create a SquashFS image and mount it in the user's home directory. In this case, the directory is /home/laytonjb/Documents. If you are not writing any new data to the Documents subdirectory, then mounting /home/laytonjb/Documents as read-only will be fine for most systems. This directory is a little over 320 MB in size (prior to using SquashFS).

$ cd Documents
$ du -s -h
314M	.

The hard drive is a Western Digital WD1600BEVE 2.5" EIDE hard drive connected via a USB 2.0 connection to a laptop with dual-core Intel Core2 Duo U7700 running at 1.33 Ghz.

The first step is to create a SquashFS image of /home/laytonjb/Documents. This assumes you have enough space on the device that contains /home/laytonjb/Documents and the resulting SquashFS image, /squashfs/storage/Documents.sqsh.

$ time sudo mksquashfs /home/laytonjb/Documents /squashfs/storage/Documents.sqsh
Parallel mksquashfs: Using 2 processors
Creating little endian 3.1 filesystem on /squashfs/storage/Documents.sqsh, block size 131072.
[===================================================] 3454/3454 100%
Exportable Little endian filesystem, data block size 131072, compressed data, compressed metadata, compressed fragments, duplicates are removed
Filesystem size 195683.71 Kbytes (191.10 Mbytes)
	62.05% of uncompressed filesystem size (315374.80 Kbytes)
Inode table size 21033 bytes (20.54 Kbytes)
	36.88% of uncompressed inode table size (57034 bytes)
Directory table size 16599 bytes (16.21 Kbytes)
	48.76% of uncompressed directory table size (34041 bytes)
Number of duplicate files found 489
Number of inodes 1495
Number of files 1360
Number of fragments 164
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 135
Number of uids 1
	laytonjb (1000)
Number of gids 0

real	0m30.217s
user	0m51.955s
sys	0m1.924s

Notice that mksquashfs discovers and uses both cores and recognizes the system is little endian and creates the image for that system (there is an option for creating images for specific endian systems). After the command is finished it gives you a summary of what it did along with the compression ratios. Notice that for this lowly hard drive and laptop it took a little over 40 seconds to build the image. However, as the file system grows in size and has more files, the longer it will take mksquashfs to create the image.

Comments on "SquashFS: Not Just for Embedded Systems"

Hello. splendid job. I did not expect this. This is a excellent story. Thanks!

One more thing to say is that an online business administration diploma is designed for individuals to be able to effortlessly proceed to bachelor’s degree programs. The Ninety credit certification meets the other bachelor college degree requirements then when you earn your associate of arts in BA online, you will have access to the most recent technologies within this field. Several reasons why students would like to get their associate degree in business is because they are interested in the field and want to obtain the general training necessary prior to jumping right into a bachelor diploma program. Many thanks for the tips you provide with your blog.

There is noticeably a bundle to know about this. I feel you made certain good points in features also.

You can certainly see your skills in the work you write. The world hopes for more passionate writers like you who aren’t afraid to say how they believe. Always go after your heart.

Your home is valueble for me. Thanks!…

Throughout this awesome scheme of things you’ll get a B+ for hard work. Exactly where you misplaced me personally was on your details. You know, they say, details make or break the argument.. And that couldn’t be more correct at this point. Having said that, permit me say to you just what exactly did give good results. The text is actually extremely convincing and this is most likely why I am taking an effort to comment. I do not make it a regular habit of doing that. Secondly, whilst I can easily see a jumps in reasoning you make, I am not really certain of exactly how you seem to connect your details which in turn help to make your final result. For right now I shall yield to your point however trust in the near future you actually connect your dots much better.

This site is really a walk-via for the entire data you needed about this and didn’t know who to ask. Glimpse here, and you’ll definitely discover it.

Hi there! I simply would like to give a huge thumbs up for the great information you’ve right here on this post. I might be coming back to your blog for extra soon.

Great article! That is the type of info that are supposed to be shared across the net. Shame on Google for now not positioning this post higher!
trang diem co dau

We came across a cool site that you just may well delight in. Take a search in case you want.

I will immediately grasp your rss feed as I can not in finding your e-mail subscription hyperlink or e-newsletter service. Do you have any? Please let me understand so that I may just subscribe. Thanks.

“I value the blog.Really looking forward to read more. Much obliged.”

Please take a look at the websites we stick to, such as this one, as it represents our picks through the web.

Just beneath, are a lot of totally not connected sites to ours, having said that, they are certainly really worth going over.

I simply want to mention I am just beginner to blogging and certainly savored your blog. Likely I’m planning to bookmark your site . You definitely come with incredible stories. Regards for sharing your website.

Leave a Reply