dcsimg

Lighting the FUSE

Explore unusual filesystems support in Linux.

The Linux kernel supports a wide variety of filesystems. Some, such as ext2fs, ext3fs, and ReiserFS, are Linux-native filesystems — designed explicitly for Linux and commonly used to hold the Linux root (/), /home, /usr, and other directories. Other Linux filesystems are designed to support removable media or partitions belonging to other operating systems (for, say, a dual-boot system). Examples of other filesystems include ISO-9660, VFAT, and HPFS. Still other filesystems are network-enabled, allowing you to mount a remote computers’ filesystem (almost) as if it were local. Examples include the Network File System (NFS) and the Server Message Block/Common Internet File System (SMB/CIFS).

Despite this wide range of filesystem support in Linux, some users want more. Perhaps you want to access an FTP site as if it were a local filesystem. Maybe you’d like to access your mobile phone using ordinary Linux programs and file-manipulation commands. Or perhaps you’d like to cram more files on a disk without having to explicitly pipe everything through gzip or bzip2.

All of these latter tasks and many more can be accomplished via the Linux kernel’s Filesystem USErspace (FUSE) module and its support tools. FUSE enables Linux to use filesystems that are implemented as userspace code, which simplifies the creation of filesystems. The result is an explosion of special-purpose filesystems. The FUSE Web site at http://fuse.sourceforge.net includes a Projects link that summarizes the implementations that use FUSE, and I describe a few examples shortly.

Preparing to Use FUSE

FUSE relies on kernel support to work. In theory, this means you must compile FUSE support into your kernel, or at least as a kernel module. You can do this from the “File Systems” section of the 2.6.14 and later Linux kernel configuration area, as shown in Figure One.

Figure One: You can build FUSE support into your kernel in the usual way



In practice, you may not need to deal with kernel configuration tools. When you install FUSE, as described shortly, the software detects whether or not FUSE support is already present in the kernel. If it’s not, the FUSE package installer builds and installs the necessary kernel module. In fact, pre-2.6.14 kernels lack FUSE support entirely, so you must rely on the kernel module that ships with the FUSE package if you’re using one an older kernel.

Obtaining and Installing FUSE

You should recognize that FUSE is really three things:

1.A kernel module (fuse.ko) that serves as the interface between the kernel and user-mode tools.

2.The FUSE user-mode tools.

3.FUSE-based filesystems to mount remote FTP servers, uncompress compressed files, or whatever.

As noted earlier, you can obtain the kernel module either through a recent Linux kernel or through the FUSE package available from the FUSE Web site. Some distributions also ship with the FUSE user-mode tools, so you may be able to install them with rpm, apt-get, emerge, or some other system command or graphical tool. Aside from some trivial demo filesystems in the main FUSE package, actual filesystems must be obtained separately. The Projects link on the main FUSE Web site provides links to many FUSE filesystem projects.

The main FUSE package is installed in a typical way:

1.Extract the package into a directory by typing tar xvfz fuse-2.5.3.tar.gz in the directory that contains the tarball. (Of course, change the version number if you’re using a more recent package.)

2.Enter the newly-created directory by typing cd fuse-2.5.3 (again, you may need to change the version number).

3.Type ./configure to configure the package.

4.Type make to build the package.

5.As root, type make install to install the software.

This procedure installs the FUSE tools to the /usr/local directory tree and places the FUSE kernel module in your /lib/modules directory tree (if necessary), bypassing your distribution’s package management system. Thus, you should check to see if FUSE is available as a package for your distribution before attempting to install it in this way.

During the configure process, the build scripts look for existing FUSE kernel support. If it’s present, the tools configure themselves to not build the FUSE kernel module. If this support isn’t present, though, the build and install steps build and install the kernel module from the FUSE directory tree. Hence, you shouldn’t need to worry about whether or not your kernel includes FUSE support. If you subsequently rebuild your kernel, though, you should be sure to include FUSE support. If you don’t, your FUSE filesystems will stop working until you correct the matter, which you can do by reconfiguring and recompiling your kernel or kernel modules or by recompiling and re-installing FUSE.

Obtaining and Installing Filesystem Modules

The filesystem modules list on the FUSE Web site describes literally dozens of filesystems. A few examples include:

*EncFS. This is a pass-through encrypted filesystem, meaning that it encrypts data for storage on another filesystem. EncFS provides a sort of front-end that unencrypts the data. The intent of an encrypted filesystem is to protect your data in case of theft of the hardware. You can learn more at http://arg0.net/wiki/encf.

*SMB for FUSE. This filesystem, headquartered at http://www.ricardis.tudelft.nl/~vincent/fusesmb/, enables you to mount an entire SMB/CIFS network at once and traverse all the computers on that network, similar to Network Neighborhood or My Network Places in Windows. This is different from the Linux kernel’s SMB and CIFS drivers, which enable you to mount specific file shares individually. (Some GUI tools provided with many distributions provide an interface similar to that of SMB for FUSE, but without using FUSE.)

*RelFS. This filesystem combines the features of a filesystem and a relational database. Check its home page, http://relfs.sourceforge.net, to learn more.

*Gmail Filesystem. You can use a Gmail account as a filesystem with this tool, headquarted at http://richard.jones.name/google-hacks/gmail-filesystem/gmail-filesystem.html.

*CvsFS. Accessing a Concurrent Versions System (CVS) site as a filesystem becomes possible with this tool. Learn more at http://sourceforge.net/projects/cvsfs.

*SSH Filesystem. This filesystem, found on the Web at http://fuse.sourceforge.net/sshfs.html, provides access to a remote computer’s files via SSH. This can be very helpful if you want to protect remote file access with SSH encryption.

*CompFUSEd. Similar to EncFS in some ways, this filesystem adds compression to another filesystem. You can learn more at http://parallel.vub.ac.be/~johan/compFUSEd/.

*CurlFtpFS. You can access an FTP site as if it were a network filesystem using this software, which is based at http://curlftpfs.sourceforge.net.

*hierfs. This filesystem, found at http://hierfs.sourceforge.net, simulates the simultaneous availability of multiple removable media disks. When you try to access a file on a disk that’s not currently mounted, a dialog box asks you to insert the appropriate disk. This is intended for handling large numbers of CD-ROMs.

This list is just a summary of some of the available FUSE modules. Some concepts, such as those for SMB for FUSE and CurlFtpFS, aren’t unique to those projects; other filesystems exist to do similar jobs, and the preceding list only lists one for the sake of brevity. Other FUSE filesystems are unique and very specialized. You should check the complete list yourself. Even if you don’t think you need a FUSE filesystem, it’s possible that somebody’s come up with an unusual and creative FUSE tool to do something that you’d find useful.

For demonstration purposes, I’ll focus on CurlFtpFS, which enables you to treat an FTP server like an NFS or SMB/CIFS server, mounting the server and accessing files directly. This can be handy if you need to make extensive changes to files on a remote FTP server, or if you want to dispense with creating a local copy of a file to access it. For instance, if you use a Web hosting provider that lets you transfer files via FTP, you can mount your Web site via CurlFtpFS and directly edit the files with a text or HTML editor. You could also mount a public FTP site that holds many Portable Document Format (PDF) files and load them directly into xpdf, Adobe Acrobat Reader, or another PDF-reading program, without first copying the files to your hard disk.

The process for obtaining and installing CurlFtpFS is similar to that for FUSE itself: You download the package, extract it, run a configure script, type make, and then install it by typing make install as root. Before taking these steps, though, you might want to look for CurlFtpFS (or whatever other FUSE filesystem you want to use) in your distribution’s package collection. If you’re lucky, it’ll be present, simplifying installation.

Note that CurlFtpFS relies on the cURL library (http://curl.haxx.se). Thus, you may need to download and install cURL before you can compile CurlFtpFS. Most distributions include cURL, so look for a package called curl. If you need to compile CurlFtpFS from source, you may need to install a package called curl-devel, as well; many distributions split libraries’ development files off from the binary files in this way.

Accessing Remote FTP Servers

Once FUSE and CurlFtpFS are installed, you can try mounting a remote FTP server. Doing so is fairly straightforward; Use the curlftpfs program, which takes the URL of the FTP site and a mount point as arguments:

curlftpfs ftp://ftp.kernel.org/pub/linux/kernel \
/mnt/kernel

This command mounts the /pub/linux/kernel directory of the ftp://ftp.kernel.org site to the /mnt/kernel directory on your computer. (Of course, the mount point must exist.) You can then transfer the files from that site using cp, or you could extract the latest kernel source code directly, without first transferring it:

# cd /usr/src
# tar xvfj /mnt/kernel/v2.6/linux-2.6.16.tar.bz2
# fusermount -u /mnt/kernel

The result is that you needn’t clutter your computer with the kernel tarball, even temporarily. However, you may experience a delay of several seconds before the tarball starts extracting.

Unmounting the remote directory is done with the fusemount command, passed the –u option. You can also use the more conventional umount command, but normally only as root.

Naturally, CurlFtpFS supports a number of mount options. Many of these options are passed following a –o command-line parameter. Some of these options are similar to mount options for VFAT and other non-Unix filesystems; CurlFtpFS ignores ownership and permission information reported by the FTP server. It must therefore create such information for local use. The most important of these options is uid= UID, which sets the local user ID (UID) number to be applied to the files on the remote server. You can also use the umask= umask option to set the umask for files.

Note that the uid option pertains only to local file ownership assignment. If you need to specify a username and password for remote access (as opposed to relying on anonymous access), you can use the user= username:password option, which passes both the username and password. Combining this option with one to set the UID of mounted files, you can use a command like the following to access your personal files on a remote FTP server:

curlftpfs -o user=rodsmith:badpass,uid=500 \
ftp://ftp.example.com /mnt/example

This command mounts rodsmith ’s home directory on ftp.example.com to the /mnt/example directory, using badpass as the password. You can then transfer files back and forth, directly edit files, and so on, assuming rodsmith has full read/write access to the server. You can type this command either as root or as an ordinary user.

One serious drawback of this approach is that your password appears on the command line, where it might be read by an onlooker. It also appears in your command history. If you place such a command in a script, your password will appear there. One alternative is to use the ~/.netrc file to hold a username and password for a remote site:

machine ftp.example.com
login rodsmith
password badpass

Be sure to give the ~/.netrc file restrictive permissions, such as 0600 (-rw——-). When this file is in place, you can omit the user= option from your command-line. Even this approach isn’t ideal, but it’s usually better than including your username and password on the command line.

The curlftpfs command offers many additional options. Consult the man page for curlftpfs to learn more.

FUSE and /etc/fstab

Suppose you want to easily access a variety of FTP sites. Rather than typing long curlftpfs commands for each one, you can create /etc/fstab entries. These entries look something like this (split across two lines for publication):

curlftpfs#ftp.example.com /mnt/example
fuse rw,uid=500,user,noauto 0 0

They key point is the format of the device specification: It begins with the keyword curlftpfs, followed by a hash mark (#) and the FTP site’s address. The rest of the /etc/fstab entry is similar to any other, including the mount point, the filesystem code (fuse), mount options, and a pair of 0 entries for filesystem check codes. Chances are you’ll want to include mount options of user and noauto to enable ordinary users to mount and unmount the filesystem, and to prevent the system from attempting to mount the filesystem at boot time. This particular example assumes that the /etc/.netrc file includes a username and password for the remote server; if not, you’d need to include this information in the /etc/fstab entry or rely on ordinary (anonymous) access to the server.

With /etc/fstab entries like the preceding for your favorite FTP sites, you can easily access them as if they were remote NFS mounts:

$ mount /mnt/example
$ xpdf /mnt/example/foo.pdf
$ umount /mnt/example

Learrning More

Keep in mind that not every FUSE filesystem behaves exactly like CurlFtpFS. Details of mount options, capabilities, and so on vary from one filesystem to another. Features such as methods of authentication for network-based filesystems are particularly likely to vary from one filesystem to another. Before using a particular FUSE filesystem, read its documentation to learn about any important peculiarities.

Fatal error: Call to undefined function aa_author_bios() in /opt/apache/dms/b2b/linux-mag.com/site/www/htdocs/wp-content/themes/linuxmag/single.php on line 62