For the past few months, we've been learning how the compiler and linker work together to take the programs you write and convert them into executables that the operating system can run. We've followed the process from source code to object module to executable, with static and shared libraries thrown in as well.
For the past few months, we’ve been learning how the compiler and linker work together to take the programs you write and convert them into executables that the operating system can run. We’ve followed the process from source code to object module to executable, with static and shared libraries thrown in as well.
But when it’s time to actually run your executable, what does Linux do for you?
What The Shell Does
Let’s say you’ve just finished compiling a program to run. Use the code found in Listing One and Listing Two as our examples. It’s a simple executable that loads a shared library and calls a function in that shared library. It then calls the sleep() system call to suspend itself for a while (so we can find out information about it while it’s running) before exiting. Compile the shared library by running:
void b_printer ()
printf (“Printing something from a shared library.\n”);
This generates an executable file named a.out. So what exactly happens when you run a.out at your prompt?
After checking for shell functions or aliases (and finding none for this command), the shell calls fork() so that now two copies of the shell are running (for more on the fork() system call, see the Spring 1999 Compile Time column online at http://www.linux-mag.com/1999-05/compile_01.html). After the fork() returns, the resulting child shell process calls the exec() system call, which causes the operating system to load and run the a.out program in place of the child shell, effectively killing the shell (but not the process). The parent shell process waits for a.out to finish, then issues another prompt.
The exec() system call (short for execute), is one of many different system calls that control program execution. Each one works a little bit differently, so the calling program needs to carefully choose which one to use. The prototypes for the different versions of exec() are listed in Figure One.
Figure One: Prototypes for the ‘exec’ system call found in <unistd.h>
int execl (const char *path, const char *arg, …);
int execlp (const char *file, const char *arg, …);
int execle (const char *path, const char *arg , …, char* const envp);
int execv (const char *path, char *const argv);
int execvp (const char *file, char *const argv);
int execve (const char *filename, char *const argv , char *const envp);
The functions that end in a “p” (execlp() and execvp()) will search your path for the given file. For example, if the program calls execlp (“gcc”, …), the system will look for the file “gcc”. For the functions that do not end in a “p”, you must specify the entire path to the program, including the filename, as the path argument.
Functions that have an “l” in their name (execl(), execlp(), and execle()) have the program’s arguments passed as arguments to the specific exec function. There can be as many arguments as you like.
Functions that have a “v” in their name (execv(), execvp(), and execve()) have the program’s arguments passed via a single pointer to an argv-like structure (a pointer to an array of char *‘s).
Functions that end in an “e” (execle() and execve()) allow you to pass an environment to the executable in addition to the program’s arguments. This lets you adjust environment variables like HOME or PATH, or add or remove any environment variables you want. The environment is always passed via a pointer to an array of char *‘s (just as the arguments are passed in the “v” functions).
What the Kernel Does
Once the child shell calls exec() to start running a.out, the kernel has to do some work before before the program can begin execution. Let’s take a closer look at the a.out program.
Figure Two shows an edited sample of the output generated by running objdump -s a.out. After printing the type of executable a.out is, it lists its various sections (that’s what the -s flag controls).
Figure Two: Output of the command objdump -s a.out
The exact contents within each section aren’t that important; we’re interested in the section names right now. The sections that the kernel cares most about are .text, .data, and .bss. In FigureTwo, objdump shows the .bss section as .sbss which can be considered the same as the .bss section. More on what these sections contain in a bit.
Figure Three: A process’s virtual address space
In the March issue (http://www.linux-mag.com/2002-03/compile_01.html), we showed how a process virtual address space is divided into various pieces, more properly called “segments.” Figure Three shows a more detailed map of the virtual address space.
Below the heap, there are three separate segments (text, data, and bss) that match the sections in a.out that objdump reported, which, as you might have guessed, isn’t a coincidence.
Copying Sections to Segments
If you were to follow the Linux kernel as it ran through one of the exec() functions, it would eventually lead to a function in the kernel called load_elf_binary() (located in the kernel source file fs/binfmt_elf.c) that copies the data from the .bss section of the executable into the bss segment in the new process’s virtual address space. It also copies the .data section into the new data segment, and the .text section into the new text segment.
You may be asking what’s in these sections and/or segments? The text segment contains the instructions to be executed, the data segment contains the static and global data that is initialized, and the bss segment contains global data that is uninitialized. “Global data” corresponds to variables or constants declared outside the scope of any function, such as the main() function. “Static data” corresponds to variables or constants that have been declared as static, even those inside functions. The compiler will automatically initialize all variables, so the bss segment is often empty.
After the segments are set up and the arguments to the program and its environment are copied into the address space, load_elf_binary() calls the start_thread() function, and the program begins executing the code in its text segment.
What About Shared Libraries?
Last month, we explained how you can dynamically load object files into your programs. But how does this work once the operating system has already set up the processes address space as described above? Each new library that is dynamically loaded gets its own text, bss, and data segments in the process address space. The operating system stacks them up as they are loaded. You can see this best by looking at a process while it’s running.
You can get a map of a running process’s virtual address space by looking in the file /proc/pid/maps, where pid is the process ID of the one we’re interested in. To get the process ID, use the ps command:
Figure Four shows the (edited) map of the process space of the executable (this is why we added the call to sleep() in Listing One). The numbers in the first column show the virtual address ranges for each of the segments. The first two are for the text and data segments of a.out. There’s no bss segment in the process map since there was no data in the bss section of a.out.
You can identify the text segment by the “x” in the second column (it stands for “executable,” just like in the output from ls -l). The “w” means the segment is writable and identifies the data segment. Executable segments cannot be written on because Linux doesn’t allow self-modifying code. All segments must be readable (the “r” in the second column).
Below the segments for our executable are the segments for the “ld” library. In this case, “ld” doesn’t stand for the ld linker, but rather what’s called the program interpreter (yes, that’s a very poor name for it). The program interpreter analyzes the executable, figures out which shared libraries are necessary for it to run, and locates and loads them into the virtual address space for the executable. In this case, it brought in libc, libdl, and libsafe.
The libsafe library (along with libc) is always linked into executables by the ld linker. It ensures that you don’t overflow your stack or try to write past what you are currently allowed. (For more on libsafe, see http://www.gnu.org/directory/libsafe.html).
Also in the map, we can see the shared library we built, b.so. However, this library wasn’t loaded into the virtual address space by the program interpreter. That’s because b.so wasn’t specified on the command line that built a.out. Instead, it was the dl library (and the dlopen() function) that loaded b.so.
In a sense, the dl library duplicates some of what the ld program interpreter does. The difference is when these two libraries load other shared libraries into the process’s virtual address space: the program interpreter does it when the process starts up, the dl library does it any time after the process has begun executing.
But That’s Not All!
Believe it or not, there’s still a lot that hasn’t been covered. The Linux kernel (in those exec() functions) does a lot of other things to make your executables run. You can consult the kernel source file fs/exec.c or the last chapter of the O’Reilly book Understanding the Linux Kernel if you’d like to learn more.
An interesting thing you’ll find if you look in the source is that the Linux kernel is not limited to running ELF executables. You can also run executables of the old a.out format (not to be confused with the program in our example) on your system.
So hopefully you can now see that although it might not seem like it, Linux is hard at work “under the hood” to make your programs run.
In the meantime, happy hacking!
Benjamin Chelf is an author and engineer at CodeSourcery, LLC. He can be reached at firstname.lastname@example.org.
Fatal error: Call to undefined function aa_author_bios() in /opt/apache/dms/b2b/linux-mag.com/site/www/htdocs/wp-content/themes/linuxmag/single.php on line 62