dcsimg

Using Perl/Tk for Simple Graphing

With the recent multiple and varied outbreaks of Windows-based worms generating ever-increasing loads of spam, I've been taxed as the system administrator for the company server to maintain a vigil against the attacks. While the actual worms can't infect my box, the onslaught of worm payloads (and the inevitable increase in spam from infected machines) has threatened an ongoing denial of service attack. At one point recently, I was accepting and attempting to process over 2,000 worm payloads per hour (including generating RFC-mandated bounce messages for those), as well as handling the 2,000 extremely false "you have a virus" messages thoughtfully (not!) generated by the antivirus blockers.

With the recent multiple and varied outbreaks of Windows-based worms generating ever-increasing loads of spam, I’ve been taxed as the system administrator for the company server to maintain a vigil against the attacks. While the actual worms can’t infect my box, the onslaught of worm payloads (and the inevitable increase in spam from infected machines) has threatened an ongoing denial of service attack. At one point recently, I was accepting and attempting to process over 2,000 worm payloads per hour (including generating RFC-mandated bounce messages for those), as well as handling the 2,000 extremely false “you have a virus” messages thoughtfully (not!) generated by the antivirus blockers.

On my machine, the most limited commodity during a heavy mail storm is the CPU. I’ve got plenty of RAM and disk space, but when I’m handling twenty to fifty simultaneous SMTP transactions and sending them all over to amavisd to be processed, my load average can start climbing up to 5 or 10 quite rapidly, which pretty much shuts down everything else that the system is trying to do. For a while, I was even running a script that watched the load average at fifteen second intervals, and if the load exceeded six, automatically shut down my mail system (gracefully) until the load returned to two or less.

Now that I’ve gotten things a bit more under control, I knew that I wanted to monitor load average changes for at least the times that I was sitting at my laptop. I looked at xload, but didn’t like the output format, and xload either has to run on the server or connect to the whod daemon for a remote probe. Well, I’m not running rwhod, so I’d have to run it on the server, causing lots of extra net traffic to update a graphic in real time.

But I remembered someone recently talking about how Perl-Tk can create nice graphical user interfaces. I’d never spent much time learning Perl-Tk, so I considered this the perfect opportunity to tackle the learning problem via a practical problem. I cracked open my recently acquired Mastering Perl/Tk from O’Reilly, and started reading.

The Tk toolkit was originally created as an extension to the TCL command language, but was ported early-on to have a general interface, including bindings for Perl. (When I’m feeling bold, I often claim that Tk and Expect are the only two things that has kept TCL from disappearing completely.) A Tk program draws widgets on an X11 display. These widgets range from simple text labels to complicated graphs and more, allowing for a complex, dynamic user interface to be created in short order — once you get the basics down. In fact, there’s even a web browser (with clickable links, embedded images, and form support) built just with Perl-Tk.

Programming for Tk requires a bit of rethinking if you’re used to traditional command-line programming. A Tk program is event-loop driven, meaning that you set everything up for your interface, and then just say “go.” Once the main loop has started running, any further actions of your program are triggered by responding to events. These events can come from the mouse and keyboard, from timers, and even from watching for a filehandle to become ready to read.

For example, you can set up a button on the screen with an event handler callback when the button is pressed. If the user presses the button, your callback gets called. These callbacks are single-threaded: while your callback is executing, the rest of the interface is locked out.

This simplifies the design of the application (over say, a threading or forked application), because you never have to worry that some variable is changing out from under you. However, this strategy also complicates things a bit: you should never place a long-running action in a callback. While I don’t have the space here to get much more into the design strategies to help with this issue, let me say that it does take a bit of familiarization before it comes clear.

My application was actually quite simple, thanks to the Tk::Graph module (found in the CPAN). This module creates a Graph widget, ready to accept data values to display them. The only other thing I needed in my application was a way to fetch the load average values. I realized that I could execute a command such as…


% while uptime; do sleep 5; done

… on my server, and this would give me a line such as…


9:36AM up 3 days, 55 mins, 2 users,
load averages: 0.58, 0.51, 0.59 >

… updated every five seconds. Through an ssh command, I could run this loop remotely, and I’d get a low-bandwidth data stream of my desired data. And, the appearance of new data is also the event that causes my application to update the display, so that’s all there is to it.

Let’s take a look at the program in Listing One. Lines 1 and 2 are the standard Perl preamble, turning on warnings and compiler restrictions. (For those of you that are long-time readers of my work, I left out the traditional $|++ because this program doesn’t produce output on stdout, and I was saving about six keystrokes that way.)




Listing One: A program to monitor and graph load averages


1 #!/usr/bin/perl -w
2 use strict;
3
4 use Tk; # CPAN
5 use Tk::Graph; # CPAN
6
7 my $HOST = “blue.stonehenge.comm”;
8
9 my $mw = MainWindow->new;
10
11 ## create the Graph widget:
12
13 my $graph = $mw->Graph
14 (
15 -type => ‘LINE’,
16 -linewidth => 3,
17 -look => 60,
18 -sortnames => ‘alpha’,
19 -legend => 0,
20 -headroom => 10,
21 -ylabel => ‘load’,
22 -xlabel => ’5 sec units’,
23 -yformat => ‘%.2f’,
24 -config => {
25 av_01 => {
26 -title => ‘One’,
27 -color => ‘red’,
28 },
29 av_05 => {
30 -title => ‘Five’,
31 -color => ‘orange’,
32 },
33 av_15 => {
34 -title => ‘Fifteen’,
35 -color => ‘yellow’,
36 },
37 },
38 );
39 $graph->pack(-expand => 1, -fill => ‘both’);
40
41 ## setup the uptime process
42 open UPTIME, “ssh $HOST ‘sh -c \”while uptime;
do sleep 5; done\”‘|”

43 or die “child: $!”;
44
45 $mw->fileevent(\*UPTIME, readable =>
\&uptime_ready_to_read);

46
47 BEGIN {
48 my $buffer = “”;
49
50 sub uptime_ready_to_read {
51
52 sysread(\*UPTIME, $buffer, 1024, length $buffer);
53 while ($buffer =~ s/^(.*)\n//) {
54 my $input = $1;
55 my ($one, $five, $fifteen) =
56 $input =~ /load av[^0-9.]+([0-9.]+)
[^0-9.]+([0-9.]+)[^0-9.]+([0-9.]+)$/

57 or die “cannot grok $input”;
58 ## warn “saw $one $five $fifteen”;
59 $mw->configure(-title => “$HOST: $one,
$five, $fifteen”);

60 $graph->set({ av_01 => $one, av_05 =>
$five, av_15 => $fifteen });

61 }
62 }
63 }
64
65
MainLoop;

Lines 4 and 5 pull in the Tk and Tk::Graph module, respectively. Although Tk will dynamically pull in most widget module definitions, it has this annoying habit of spitting out trace messages to stderr about additional things being pulled in on demand, so it seems to be better to add it directly to keep the program quiet.

Line 7 is the only configuration constant: the host to which we’ll ssh to get the load average data. Line 9 creates the MainWindow widget object. This object represents the “application” to the user. When this object disappears, it takes everything else down with it, so we treat it very specially. Note however that nothing has actually happened on the screen, and won’t until we start the main loop at the end of the program.

Lines 13 to 39 create the Graph widget. We create this as a child of the main window to indicate that it logically belongs there. Widgets are always created in a hierarchical fashion, and almost always displayed in such a way that children are within their parent widgets.

Lines 15 to 37 define the configuration parameters for this widget, as defined by the widget specification. Most of these were tweaking various things to get the graph to “look right.” For example, I’m asking for a line graph, with lines 3 pixels wide, automatically scrolling the last 60 values, labeled appropriately. The -config element subhash defines the three lines with their colors. I’m not much of a graphics guy, but I thought red for the most important one (the one-minute load average) was the best.

Line 39 defines the visual relationship of this graph widget to its parent. Tk comes with a few different geometry managers, and probably the simplest to use and understand is the pack manager. In this case, we’re telling pack to manage this widget and to have it fill up the entire space of the parent widget. This means that my graph will always be as big as the enclosing window, so I can make it small when I want it in the corner of my laptop display, or take up a full screen when I want to see it across the room.

Had the main loop been running already, this would also have the effect of actually popping the graph onto the display from what was formerly an empty main window. But, since we’re doing all this work before starting the main loop, we won’t be seeing anything just yet.

The next thing to do (in lines 42 to 43) is to wire up the remote uptime loop. A simple pipe-process-open suffices here. Note that I have to force /bin/sh syntax for this loop, because my default login shell is /bin/tcsh on the server, resulting in ugly triple-quoting.

Line 45 connects any “ready-to-read” condition on the filehandle to an event callback, which calls my uptime_ready _to_read() subroutine defined a few lines down. When the main loop is waiting for something to do, it will now watch if there’s any data in the unbuffered handle of UPTIME. If so, my subroutine is called to process that data. The key here is the unbuffered handle: I must be very careful never to use the normal buffered I/O with this handle or the event may not trigger later at the right time.

Lines 47 to 63 set up the response to data being available on the handle. I’m creating a static local variable called $buffer, initially an empty string, in line 48.

The problem I’m solving here is that the handle may be ready to read when only a part of a line has been seen across the socket connection, or even when two lines are available (if there’s been a network lag). So, I read whatever I can, appending it to my own maintained buffer and then process as many whole lines as I can from the front of the buffer. Line 52 reads whatever data is available on the unbuffered handle (up to 1 K), and puts it at the of $buffer. Line 53 tries to repeatedly remove an entire line from the beginning of the buffer and if successful, this line is placed in $input in line 54.

The 1024-byte limit in line 52 is just an arbitrary number. If you set this to “1 byte,” it just means that the subroutine would get called to read each of the 60-ish characters of a line and then immediately return, triggering yet another “ready to read” event again and again until a complete line had been seen. If the number is set too big, the code still only gets what’s ready.

Lines 55 to 57 extract the three load-average numbers from the end of the string into the three named variables. Line 58 was a debugging trace I inserted to ensure that everything was being seen properly. I left it in, because if I was changing the regex, I’d want to ensure proper parsing.

Line 59 changes the X11 window title on the application’s main window as we see each new value. This is a nice feedback item, because I can mouse-over the window when it is iconified and still get the load averages.

Line 60 is really the crux of the program. Take the three load averages, and plot them as the next item of data. The graph widget does all the hard work from there, figuring out the maximum scale, maintaining the last sixty data points of the five-second intervals, giving me a nice five-minute window of activity display. And once that’s done, we’re done.

Line 65 is the invocation of the MainLoop, which puts Tk’s event manager in charge. This call returns only when the main window goes away. Since we didn’t provide any explicit buttons or actions to do that, the only way the main window goes away is when the user clicks on the standard close-window box. Once MainLoop starts, window management events (like resize or iconify requests) are handled automatically, and the “ready to read” condition on the filehandle triggers more data to be added to the graph. It’s all that simple.

Hopefully, you’ve seen how easy it is to create an arbitrary management tool in a few simple lines of code. Until next time, enjoy!



Randal Schwartz is the chief Perl guru at Stonehenge Consulting and the author of many books on Perl. He can be reached at merlyn@stonehenge.com. Download this month’s Perl code from http://www.linux-mag.com/downloads/2004-05/perl.

Comments are closed.