A filesystem doesn’t just store the contents of files. It also stores a great deal of file metadata, or data about the files. In addition to the file’s name, file metadata includes attributes such as access rights (permissions) and the type of the file (whether it’s a directory, a symbolic link, a device or something else).
Extended filesystem attributes are a fairly new addition to the Linux kernel that allow you to create your own per-file attributes.[ Extended filesystem attributes were first mentioned in the September 2003 issue of
Linux Magazine, available online at
http://www.linux-mag.com/2003-09/acls_01.html, and were described in more detail in the June 2004 “Compile Time” column, available online at
http://www.linux-mag.com/2004-06/compile_01.html.] Because extended attributes are user-defined
(name, value) pairs, there’s vitually no limit to the kinds of applications you can create.
Setting and Retrieving Extended Attributes
For example, if you have a collection of photographs from a digital camera, you can use extended attributes to record when the photo was taken, the contents of the picture (people, sights, and so on), and your camera settings. With extended attributes, keeping such metadata in a separate database isn’t necessary; each set of attributes is “attached” to the file.
On Linux, extended attributes are manipulated using the setfattr and getfattr commands to set and retrieve extended attributes, respectively. (See the sidebar “Enabling Extended Attributes” if your system or filesystem does not yet support extended attributes.)
Continuing with the photographs example, to store the year a picture was taken, use setfattr:
$ setfattr –n user.year –v 1990 oldpicture.jpg
The user prefix is a name space, a mechanism that provides for different classes of extended attributes. As shown above, attribute names are always specified in the full namespace.attribute form. The user name space is intended for any user-defined attributes. (Other name spaces, such as trusted may exist, but special permissions may be needed to access them, so it’s best to use user.)
To retrieve the information stored in a file’s extended attribute, use getfattr:
$ getfattr oldpicture.jpg
# file: oldpicture.jpg
user.year=1990
A word of caution before you store a lot of valuable information in extended attributes: many backup programs such as tar are not yet aware of extended file attributes. If you archive your files with such a” na&# 149;ve” utlity, you will lose all extended attributes. To maintain the data in the file and its extended attributes, use programs like xfsdump or star or include a dump of the attributes from getfattr –d.
Custom zsh Qualifiers
Storing and reading extended attributes is useful, but as SQL is to a relational database, the real power of the feature lie in queries — being able to choose the file or set of files that meet certain extended attributes criteria.
This is where the
Z Shell or
zsh comes in.
zsh has many powerful features to choose files based on conventional filesystem attributes (see the March 2004 article “Wildcards Gone Wild” at
http://www.linux-mag.com/2003-12/power_01.html).
For example, the following command lists directories:
zsh% echo *(/)
Here the * (“asterisk”) is the usual wildcard for matching all files, and the following parentheses delimit the zsh qualifier, / (“slash”), which selects matching files (everything in this case) that are directories.
There are many other qualifiers for matching all sorts of different filesystem attributes.
*The zsh command echo*(@) shows those files that are symbolic links.
*The command echo*(m1) shows files modified within the last 24 hours. (The m selects the last modified time. Because the default unit for m is days, m1 is one day.)
*echo /*(/u:root:f:o=rwx:) combines qualifiers to list subdirectories of root (/) that are owned by root and have world readable, writable and executable permissions.
zsh doesn’t have a specific qualifier for handling extended attributes, but it does have a more powerful qualifier that you can use to achieve the result. The e qualifier allows you to insert shell code that’s evaluated, effectively making it is possible to write your own qualifier.
Let’s start with the syntax to write qualifiers and return to extended attributes shortly. If you’ve written shell scripts before, writing qualifiers will look very familiar; if not, don’t worry: the examples are all fairly simple.
Including shell code in a list of glob qualifiers involves first specifying the letter e. The e is followed by a delimited string that contains the shell code to be evaluated. The shell code is evaluated once for each file, with the REPLY variable set each time to the filename in question. The shell code returns true or false to determine whether or not the file should be matched. (For now, ignore the fact that REPLY seems like a strange name for the variable: it is also possible to modify the variable to change what the wildcards are expanded to.)
For example, suppose that you want to know which files in the current directory also exist (or don’t exist) in another directory. The conventional way to do this is to write a shell for loop:
for f in *; do
if [[ –e /other/directory/$f ]]; then
print – $f
fi
done
(You may be wondering about the dash in the command print – $f. A lone dash ends the list of command options; any parameters that follow the – are arguments that must be processed by the command. For example, although file names rarely begin a dash, providing the leading dash separates the options from the file names and prevents file names from being interpreted as options.)
With the e qualifier, the equivalent would be:
print –l – *(e:’[[ –e /other/directory/$REPLY ]]’:)
The print command is a built-in zsh command, similar to echo. With the –l (“l” for “line”) option, print displays each argument passed to it on a separate line.
In addition to being shorter, the latter zsh command has the advantage that you can use it as an argument to any command. To pass all the files together to a command, you would otherwise need to make the script more complicated or use xargs.
Quoting can become a little awkward. In the example above, it was necessary to add single quotes to preserve the $(” dollar”) symbol. (Without the quotes, the REPLY variable would be expanded too early instead of when the qualifier is evaluated.) Double quotes do not work because variables are expanded inside double quotes. To avoid problems with quoting, it is a good idea to define short shell functions for pieces of shell code that you use frequently.
For example, the shell function in() defined by…
zsh% in() { [[ –e $1/$REPLY ]] }
… can simplify the previous form of the command to:
zsh% print –l – *(e:in /other/directory:)
By using a function, the awkward $ character need not be quoted. (In all shells, the exit status of the last command in a function determines the return status of the function, hence the lack of an explicit return statement.)
Querying Extended Attributes
With that background, let’s look now at how to check extended attributes from a zsh qualifier. To begin, let’s write a function that just checks for the existence of a particular attribute. The command to get the attribute is:
getfattr –n user.$1 $REPLY
This assumes that the name of the attribute is a parameter to the function.
The exit status of getfattr isn’t particularly helpful, but the command’s output is. You can capture the output of getfattr using command substitution. The most commonly used syntax for command substitution is to enclose the command in backquotes (` `), but in modern shells the use of the $(…) construct is preferred. To test if the output of getfattr is non-empty, use the test –n:
fattr() {
[[ –n $(getfattr –n user.$1 $REPLY 2>/dev/null) ]]
}
Now, you can use *(e:fattr year:) to pick out any files with the user.year attribute set.
Taking this further, let’s extend the fattr() function to query the value of the attribute (and not just its existence). To do this, you need to save the value of the attribute in a local variable and then perform a comparison:
fattr() {
local val=$(getfattr –n user.$1 ––only-values $REPLY 2>/dev/null)
[[ –n $val && ( –z $2 || $val = $~2 ) ]]
}
The first line of this function gets the value of the attribute. The ––only-values option makes getfattr only output the actual value of the attribute. By default the filename and attribute key are also output.
The second line of the function determines the return status and makes use of the logical and (&&) and logical or (||) operators in a condition test. If no value is found, the function should return failure. The first part of the condition test handles this by checking that the val variable is non-empty. If a value is found but no value was supplied for comparison — that is, $2 is empty — it returns success. This allows the new version of the fattr() function to be used in the same way as the previous version to test whether a particular attribute is set. The final component of the condition test does the actual comparison of the attribute value against the second parameter.
You may have noticed that the second parameter is actually written $~2 here. The tilde (~) turns on zsh’ s glob_subst option for this single expansion. This means that zsh treats the result of expanding $2 as a shell pattern. This allows us to do more powerful searches. (The sidebar “Variable Expansions in zsh” has a little more explanation of this.)
With this new version of the function, you can now perform simple queries in your directory of photographs and have the results expanded in-place right on the command-line, just as you’d use * and other wildcards.
For example, if you want to view all pictures tagged as taken in Salzburg, you could run the command:
xv *(e:fattr place Salzburg:)
(xv is a common image viewing application.) You can even chain together more than one test.
xv *(e:fattr place Salzburg:e:’fattr year 200[23]’:)
This views all photographs taken in Salzburg during 2002 or 2003. The command combines a traditional shell pattern (200\[23\]) to produce a range of years (although the special pattern characters ([ and ]) did have to be quoted).
Better Than a Database
As you can see, querying metadata in extended attributes from the command-line is a lot more convenient than using a real database. It’s especially convenient that the results of a query can be directed straight to the command-line.
Extended attributes are also a lot more convenient than embedding metadata in the file itself, which is quite a common practice. For example, JPEG images include EXIF data and ID3 tags are a common way to embed track information in MP3 music files. The trouble with embedded metadata is that separate tools are then needed to query or manipulate this metadata.
There are a surprising number of different uses for custom qualifiers, accessing extended attributes being just one. And if you dig into zsh, you’ll find a wealth of interesting and powerful features. Let’s look at three.
Suffix Aliases
In a graphical file manager, you can typically open any file by double clicking on it. The file manager maintains an association between file types and programs that can open each file type. zsh 4.2 adds a similar feature to the shell: suffix aliases.
For example, the following associates PDF files with the acroread program:
zsh% alias –s pdf=acroread
Like normal aliases, suffix aliases provide a shorthand form for expanding whole command lines. So, with the association above, you can now type the name of a PDF file and the shell automatically expands the command-line to run acroread.
This can also be useful with program files that can not be run directly. For example, you can define a suffix alias to run .exe files with wine and .class files to run in Java.
For convenience, zsh also includes a function to setup some suffix aliases for you automatically. The function is named zsh-mime-setup(), and it uses the common mime.types and mailcap files to find associations between file types and programs.
To run the function, type:
zsh% autoload –U zsh-mime-setup zsh% zsh-mime-setup
If this works, run alias –s to see the list of suffix aliases that have been defined. (For more information, read the zshcontrib man page.)
Reusing Command Sequences from the History
One useful shell feature is history: past commands that have been executed are saved in a list. Typically the UP ARROW key is used to retrieve previous commands for reuse. There are also search operations that allow you to find previous commands. Sometimes, however, you will find that you don’t just want to reuse one previous command but a whole sequence of commands.
For example, compiling and testing a program might involve a few steps such as the following:
./configure ––prefix=$HOME
make
make test
make install
A few minutes later you make a change to the program and want to build
it again. You could type each of the commands again, but using the shell
history can save a lot of effort.
First, find the configure command. Type CONTROL-R to start a backward search. zsh displays a second prompt below your initial prompt. You can now type a search string and zsh fills in the first matching command for you. Pressing CONTROL-R again makes zsh search again for another command.
zsh% ./configure ––prefix=$HOME
bck-i-search: configu_
Having retrieved the command from the history, you could now just press RETURN. When it finished, you could search again for the next command in the sequence– make.
However, there is an even better option. Instead of pressing RETURN, press CONTROL-O. Like RETURN, this immediately runs the command currently on the command line, but when the command finishes, the shell automatically selects the next command from the history. So, in this case, the command make is automatically inserted at the next prompt. Pressing CONTROL-O again runs make and then inserts make test at the command-line.
Disappearing Prompts
The shell prompt is the text that appears just to the left of where you type a command. In most shells, including zsh, the shell prompt is highly configurable, and some people fill the prompt with a huge variety of information — username, hostname, current working directory, among other tidbits — while others prefer simple and unobtrusive prompts.
In zsh, you can embed information in your prompt without adding extra clutter with the right prompt. The right prompt works much the same as the normal prompt, except that they appear on the right side of the terminal.
For instance, the following command puts the current time in a right prompt:
zsh% RPS1=[%T]
RPS1 is the variable to define to create a the right prompt. The %T is special syntax in zsh prompt strings: it causes the current time to be inserted.
An interesting feature of right prompts is what happens when a long command is typed, such that the cursor reaches the prompt. If you try typing a long command, you’ll see: just as the cursor is about to reach it, the prompt disappears.
As a new feature of zsh 4.2, it’s possible to make right prompts automatically disappear when you press RETURN to execute the command. This is enabled with the transient_rprompt option. You can enable that option with setopt:
zsh% setopt transient_rprompt
One distinct advantage of this is that if you need to copy-and-paste commands from your terminal, the prompts don’t get mixed in with the commands. This is especially useful with commands that span multiple lines.
For example, zsh uses a different prompt for secondary lines. By default, this prompt lists the shell constructs, so you typicallly see prompts like the following:
zsh% for f in *; do
for> while
for while>
…
It is impossible to the select the commands from this using the mouse without also selecting the prompts. However, by moving the prompts to the right and having them disappear, this problem is averted.
Secondary prompts are defined using the PS2 and RPS2 variables. To disable secondary prompts and the left and make them appear at the right, type:
zsh% unset PS2
zsh% RPS2=”<%^”
Now, if you type multiline constructs such as for loops and if statements, you should see the shell constructs in the right prompt. And with the transient_rprompt option, the prompts will be cleared up each time you press RETURN.
Loading ...
No Comments
No comments yet.