If you’ve developed code for any length of time, chances are that you have a” favorite” bug. Maybe it was that hard-to-find bug lurking in seemingly perfect code. Or perhaps it was that pesky little error buried within fifteen layers of function calls. Or perhaps it was the most dreadful kind of bug, the intermittent crash, that only occurred when Jupiter and Saturn were aligned just so– you earned your Jolt that day!
Whatever the bug, you no doubt reached for the GNU Project Debugger (GDB) to track the bugaboo down. If you’re an occasional GDB user, you probably run your program inside GDB until it crashes and then perform a post mortem in situ. That’s not a bad approach– it’s usually effective– but it’s like owning a Porsche just to pick up groceries. The little German gem gets you there (in a hurry), but you’re missing out on so much more.
In this article, let’s open the throttle a little and look at some GDB features that can help you squash bugs faster.
Without a doubt, one of the most common GDB commands you’ll use is backtrace. backtrace outputs your program’s current call stack and shows information about each stack frame. It’s a quick way to get your bearings in a running program, and is likely the first command to run when your program stops executing (especially after a segmentation fault).
If you have a large stack though, the output of backtrace can quickly become unwieldy. For example, in the test program shown in Listing One, there is a recursive call that can easily create a stack trace of several thousand frames. Most of the time, though, you’re only interested in the most recent five to ten frames, if even that much. To focus on recent calls, you can limit the default backtrace size. For instance, to limit backtrace to the first five frames, use the command set backtrace limit 5.
Listing One: A simple program used throughout the article 01#include <stdio.h> 02 03int f(int x) 04{ 05 if (x > 1) return x + f(x-1); 06 return(1); 07} 08 09int main(int argc, char** argv) 10{ 11 int i, x = 0, y; 12 13 for(i = 0; i <= 20000; i++) { 14 x += i; 15 } 16 17 y = f(20000); 18 19 printf(”x = %i, y = %i\n”, x, y); 20 21 return 0; 22}
Or, if you just want to limit the backtrace output for a single stack trace, you can supply a limit on the command line like this:
(gdb) backtrace 3 #0 f (x=1) at test.c:6 #1 0×0804839c in f (x=2) at test.c:5 #2 0×0804839c in f (x=3) at test.c:5 (More stack frames follow…)
GDB stack traces show you basic information about each stack frame, but that information is fairly compact. Once you find the frame you’re interested in, you can use the info command to see more detailed information.
For example, let’s say you’re interested in the fiftieth f() frame from the top of the stack in the previous stack trace. To find out more information about the frame, you can move up forty-nine frames and run info frame.
(gdb) up 49 #49 0×0804839c in f (x=50) at test.c:5 5 if(x > 1) return x + f(x-1);
(gdb) info frame Stack level 49, frame at 0xbffd7fb0: eip = 0×804839c in f (test.c:5); saved eip 0×804839c called by frame at 0xbffd7fc0, caller of frame at 0xbffd7fa0 source language c. Arglist at 0xbffd7fa8, args: x=50 Locals at 0xbffd7fa8, Previous frame’s sp is 0xbffd7fb0 Saved registers: ebp at 0xbffd7fa8, eip at 0xbffd7fac
You can also get a listing of all the current function’s arguments with the info args command, or all of the local variables using info locals.
(gdb) info args x = 50 (gdb) info locals No locals.
Print Smarter
In addition to the call stack, the values of program variables are typically of great interest. The basic command for examining variables is print, which outputs the value of a variable or expression. But print has many features that allow you to fine tune what’s printed. Modifying just a couple of settings can greatly improve the output of print.
For example, the way that GDB prints arrays is not always the most useful. By default it uses a compact format that saves space, but makes it difficult to visualize all of the information in an array. If you’d rather see a more readable, space-consuming, output format, you can tell GDB to pretty-print an array by running set print array on. Similarly, if you’re debugging an object-oriented language such as C++, you can have GDB pretty-print an object with set print pretty on. (The same command also enables pretty-printing for C struct s.)
Pretty-printing static arrays is great, but it doesn’t do much good if you have a pointer to a dynamically-allocated array: if you have a pointer to a dynamically allocated array there is no way for GDB to know how long the array actually is. Fortunately, GDB allows you to create an artificial array to specify how many elements to print, given the first element as a starting point. To create an artificial array, use the @ operator in your print statement after the variable that represents the first element in your array followed by the length of the array.
For example, assume you have a pointer to an array of doubles, named my_dynamic_array. Furthermore, also assume that you know the length of this array to be 256 elements. To print out all 256 elements, you could run the command print*my_dynamic_array@256. You must dereference the my_dynamic_array variable, because artificial arrays take the actual first value, not a pointer to the first value of the array.
Artificial arrays can also be used to print subsections of arrays. For instance, if you want to print the last half of the same array of doubles, run print my_dynamic_array[128]@128.
The x command is yet another way to get at the data in an array or any other arbitrary region of memory inside your program’s memory space. x allows you to examine the memory at a given address in memory, printed in a format of your choosing. The basic format for x is to provide an address; given an address, GDB outputs the word at that spot in memory (a word is four bytes). Then, if you want to have GDB print out subsequent words, continue running x without an address. This also works if you just used print to examine a value in memory. You can also use x without an argument after running info breakpoints; in this case, x starts printing from the address of the last breakpoint. Try x after info line: it starts printing from the address of the current line.
By default, x prints memory a word at a time in hexadecimal format. You can change the defaults by giving x a different format when you run it. New formats are specified by following the command with a / and the format you’d like to use.
For example, to print three bytes of memory in binary, run x/3tb. Or, to print a region of memory as a null-terminated string, run x/s. Every time you give a format, that becomes the new default. If you’d like to see all of the formatting options that are available, run help x inside GDB.
A Convenient Store
Deep into debugging, with values flying everywhere, it would be handy to store some of them for later use. You can do that in GDB by setting convenience variables, which allow you to store the results of an expression for later reference, without affecting your program’s memory space. For example, you can call a function and store its return value to pass it to another function later on in your debugging session.
Creating convenience variables is easy. All you have to do is set them to a value using the set command; no other declaration is necessary. You can name your convenience variables with any characters you’d like (except for a few reserved register names) as long as the name starts with a $. For instance, to save the result of a call to the function add(), do the following:
(gdb) set $sum = add(2,2) (gdb) print $sum $1 = 4
You’ve probably noticed that the output of print always precedes the output value with $ N=, where N is some number. What are those, exactly? GDB stores output results in a value history, and each history value can later by referenced using its name, just as you would use a convenience variable.
For instance, to print the fifth history value, use the command print $5. You can also reference history values relative to the most recent value. For example, giving $$0 (or just $) references the last history value, and $$1 (or just $$) references the second-to-last history value. To reference the third-to-last, give $$2 (but $$$ doesn’t work) and so on.
Creating Your Own Commands
What if you want to find the value of i at the point where x becomes greater than 50? The obvious approach is to step through the execution of the program and keep printing the value of x to see when the threshold is reached. That” works,” but is a decidedly uninteresting way to spend five or ten minutes. Mindless, repetitive work is best left to the computer, and that’s where user-defined commands come into play.
GDB lets you create user-defined commands to execute a list of other GDB (or user-defined) commands. You can also use loops and conditionals in your user-defined commands, and you can pass a command up to ten arguments. Commands can be re-used and provide a powerful way for you to reduce your workload as you’re debugging a program.
Creating a user-defined command is easy: just type define mycommand and then type in the instructions to execute, followed by end. For example, here’s a command that steps through the program until x is greater than 50, at which point it stops and prints the value of i.
(gdb) define mystep >while x >= 50 >step >end >print i >end
You can also use conditionals in your commands. For instance, to print all of the values of i that are a multiple of 5, you could modify the previous command like this:
(gdb) define mystep >while x >= 50 >if i%5 == 0 >print i >end >step >end >print i >end
Now you have a handy little command that saves a bunch of typing. Let’s face it though, this is not a command that’s likely to be very useful at other points. In fact, it only works if you have variables named x and i and want to find when x is greater than fifty. However, with a few small changes, you can make mystep take arguments, making it a much more more useful command:
Since GDB uses text substitution for its arguments, you can call this command with the names of the variables you’d like to use. To get mystep to work exactly like the original example, you could run mystep x 50 i.
Once you have a couple of general-purpose commands created, odds are that you’re not going to want to retype them every time GDB starts– that would defeat the purpose of the command. To make your commands a little more long-lived, you can enter them offline in a text file, one command per line. Then, when you load GDB, run source commandfile to load all of your custom commands.
Automating Your Debugging
Let’s say that you have the following function (yes, it’s better to use a constant for the 5 in the summation):
int g(int a, int b) { int x = 5; x = x + a + b; return x; }
Now, let’s say that g() gets called about a thousand times inside a loop with different values for a and b. As you’re debugging your program, though, you decide that 5 might not be the correct value to use as the constant. Instead, you’d like to try 8.
One option would be to change the constant in the source code and recompile, but recompiling can be time consuming though, and you’d have to do it for every value that you’d like to try. A better option is to take advantage of GDB’s ability to let you assign values to variables in your program as it is running. After x has been declared, but before the summation has been executed, you can just run set x=8. (See the sidebar” Impromptu Programming” for more GDB commands that can change the behavior of running programs.)
Impromptu Programming
Most of the time, GDB is most useful as a tool for tracing your program’s execution. But GDB need not just be a passive observer. GDB can alter your program’s behavior in the middle of execution to make it sit, stand, jump through hoops, or even shake hands.
For example, let’s say that you have a function f()…
50 int f(int a, int b) 51 { 52 int x; 53 if (a == b) { 54 x = a*b; 55 } 56 else { 57 x = a+b; 58 } 59 return x; 60 }
… with a breakpoint immediately prior to the evaluation of the if statement on line 53.
If a and b are equal, the program always executes x=a*b. What if you’d like to see how the program behaves if a+b is returned when the two values are equal? One option would be to modify the program and recompile, but that’s not always practical and rarely convenient. Instead, you can use the jump command to skip right to line 57, as if the if statement had evaluated to false.
(gdb) jump 57
Now, what if you’d like to see what happens if f() returns a/b? Surely, that requires a recompile, right? In fact, no. When your program stops inside f(), you can tell GDB to return from the function with a custom return expression by running return a/b. (return doesn’t continue execution though, so run continue after the return).
You can even call other functions in the return expression. So, to see what happen if f() were called recursively, you could stop execution at line 59 and call return f(x,x).
Functions can also be called at other places by using the call command. For instance, to see the result of calling f(5,2) from any point in your program, just run call f(5,2). One word of caution, though: When you call a function, it’s called just as if your program had called it at the current point in the program. If the function you call has side effects, continuing the program may produce unexpected results.
But you don’t have to run set x=8 a thousand times, once for each invocation of g(). Instead, take advantage of GDB’s breakpoint commands. When you set a breakpoint, you can give GDB a list of commands to run when it hits that breakpoint. Even better, if you end the command list with continue, GDB never even stops execution of your program.
So, for the function above, you could set a breakpoint to change the value of x to 8 like this (assuming x=x+a+b; is on line 55 of main.c):
The commands directive is always issued immediately after the call to break and indicates the beginning of a command list. The command list then continues until you terminate it with end. In this case, the command list consists of three commands: silent, set, and continue.
silent squelches the output when the breakpoint is hit. Normally, when GDB hits a breakpoint, it prints out a bunch of information about the current location, like this:
Breakpoint 1, f (a=1, b=2) at main.c:55 55 x = x + a + b;
However, if you’re running continue at a breakpoint, there’s no reason to emit anything. silent tells GDB that you don’t want to see that information when the breakpoint is hit.
Another way to automatically execute commands is through the use of command hooks. With a command hook, you can give GDB a sequence of commands to execute when execution of your program stops or starts, or when any other command is run.
For example, to print out a stack trace of the first few stack frames every time execution of your program stops, set a stop hook to backtrace with the following:
(gdb) define hook-stop >backtrace 5 >end
Both command hooks and breakpoint commands are user-defined commands. In other words, you are free to use loops and conditionals inside these specialized, custom commands. You are also free to call other user-defined commands as you need them.
Watchpoints and Catchpoints
Breakpoints stop execution of your program at a certain, well-known point, but what if you want to find the location where the value of a variable changes or an exception occurs. Use watchpoints and catchpoints. With a watchpoint you can tell GDB to watch a variable and stop as soon as that variable’s value changes. Similarly, if you’re debugging a C++ program, you can use a catchpoint to tell GDB to stop whenever an exception is thrown or caught.
To set a watchpoint, give GDB the watch command, followed by the variable or memory address that should be watched. Be careful though: GDB can only set watchpoints on variables that currently exist. If you want to watch a local variable in a function, you need to be inside that function to set the watchpoint, and as soon as the local variable goes out of scope the watchpoint is deleted. (If you want to set the watchpoint every time you execute the function, use a breakpoint command to set it on entry to the function.) On the other hand, if the variable is global, your watchpoint will stick around for the life of your program.
As an example, take a look back at Listing One. Once execution reaches main(), you could set a watchpoint on the variable x with the following statement:
(gdb) watch x Hardware watchpoint 2: x
Notice that watch says that the watchpoint is a hardware watchpoint. This is an important detail: it means that the computer’s CPU has support for controlling watchpoints at the hardware level. If that feature isn’t available (and it isn’t for every CPU), GDB has to check the value of the watched variable in software after every single instruction. As you can imagine, that makes programs run very, very slowly.
Watchpoints are useful to detect “writes,” or when a variable changes. What if you’d like to know when your variable is “read”? You’re in luck. GDB also lets you set watchpoints that detect when a variable is read. To set a read watchpoint, use the rwatch command instead of watch. You can also use awatch to set a watchpoint that stops on both reads and writes. There is one caveat though: read watchpoints can only be set if you have support for hardware watchpoints.
Setting catchpoints is very similar to setting a breakpoint or watchpoint. Instead of giving a variable or program location, though, you give the type of event that you’d like to catch. For example, if you’d like to have GDB stop when exceptions are thrown, run catch throw; or if you’d like to stop when exceptions are caught. run catch catch.
Go Forth and Squash
Hopefully, you’ll now be more productive in GDB and experience less stress when you’re perfect program turns out to be less-than-perfect. If you’d like to reduce your debugging stress even further, check out the GDB User Manual, which is available from the GNU web site at http://www.gnu.org/software/gdb/documentation/. Although a bit terse, the documentation is quite comprehensive and has oodles of information about GDB’s many powerful features.
William Nagel is the Chief Software Engineer at Stage Logic, and the author of the book Subversion Version Control: Using the Subversion Version Control System in Software Development Projects. He can be reached at
class="emailaddress">bill@williamnagel.net.
Loading ...
No Comments
No comments yet.