Logs: Your Linux System’s Lovable Worker Bees

Can't bring yourself to love logs? You should take a second look.

The lowly and lonely log files sit there day after day gathering dust and events as your system purrs along without issue. That is, until something bad happens. Then you scramble to find out why the system rebooted or had a memory problem. Maybe it was a network denial of service attack. Or was it a runaway process? Or worse still, a hacker after your MP3 collection. How will you know? If you said, “Look at the logs”, then you’re halfway to a resolution.

In most cases, those lowly log files are your best friends. Disasters, system anomalies, user error and careless hackers all leaves tracks in the logs. If you know where to look and what to look for, you’re that much better off.

The Basics

Log files (logs) are text files, owned by root or the application’s daemon process user account, that receive entries whenever some significant event occurs. And, significant can mean anything from an HTML page view to an attempt to crack your root user account. Events can range from informational, in the example of an HTML page view, to critical for sudo failures, often indicative of root account hack attempts.

Processes associated with specific applications, services or daemons often have their own log files. The logging level or log detail is a configurable parameter. To save space and avoid log “madness,” system administrators will alter logging levels to capture only higher priority events and filter out informational notices. When you only log negative events, your logfiles stay smaller and reduce complexity when searching for a root cause of a system or application failure.

Perhaps the only exception to that statement is web logging. Webmasters and statistics programs collect every event, informational or otherwise, and use them for producing reports on site usage.

Each log file has a corresponding configuration file that governs its location, log level and other configurable parameters. Some log files capture events for multiple processes, as is the case for system events. The Apache web server has two log files, by default, the access log and the error log. Most programs and applications send all of their events to a single log file.

Location, Location, Location

You’ll find most of your log files located in the /var/log directory. Some administrators and webmasters change the default locations for their logfiles. Apache administrators, on shared systems, will create log directives so that each hosted web site has its own access and error log.

Figure 1 shows you a Debian 5 system’s /var/log directory listing. This is a standard (default) Debian 5 x86_64 installation with no additional software.

Debian System's /var/log directory
Figure 1: Debian System’s /var/log directory

In Figure 2, you can examine a CentOS 5.x system, that has a standard installation plus several additional applications including the X Window Desktops GNOME and KDM.

CentOS 5 System's /var/log Directory
Figure 2: CentOS 5 System’s /var/log Directory

The /etc directory is home for most configuration files on your Linux system but certain applications (Apache, for example) either keep their configuration file(s) in an /etc subdirectory or in an application-specific configuration directory.

On the CentOS system, the Apache configuration installed under /etc/httpd. The logs directory is a symbolic link that points to /var/log/httpd where you saw it in Figure 2.

Rotation, Rotation, Rotation

Log rotation is one of those automated system processes that we, as system administrators, take for granted. We shouldn’t. Log rotation is highly configurable but most of us lazy administrators just leave everything as is, where is. As for the “where” part, the logrotate.conf, like other configuration files, resides in the /etc directory. The following shows you the default logrotate.conf from the example CentOS system.

$ cat /etc/logrotate.conf
# see "man logrotate" for details
# rotate log files weekly

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones

# uncomment this if you want your log files compressed

# RPM packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp -- we'll rotate them here
/var/log/wtmp {
    create 0664 root utmp
    rotate 1

# system-specific logs may be also be configured here.

From this listing, you can see that your logs rotate on a weekly basis and the system retains four previous copies of your rotated logs. That’s why, when you look at Figure 2, you see messages, messages.1, messages.2, messages.3 and messages.4 files. The messages file is the current syslog. The other messages.X files are the rotated logs from previous weeks.

$ ls -la /var/log/messages*
-rw------- 1 root root   3798 May 23 18:56 /var/log/messages
-rw------- 1 root root 186528 May 23 04:02 /var/log/messages.1
-rw------- 1 root root 130014 May 16 04:02 /var/log/messages.2
-rw------- 1 root root 340531 May  9 04:02 /var/log/messages.3
-rw------- 1 root root  60222 May  2 04:02 /var/log/messages.4

On May 30, you’ll see a new log file appear as messages.1, the current messages.1 renames to messages.2, messages.2 becomes messages.3. The messages.4 file drops off and the oldmessages.3 replaces it. The logs rotate according to the rules in /etc/logrotate.conf.

If you need to force a log rotation, you can by entering the logrotate command with the -f (force) switch and the configuration file for the logs you want to rotate. To force a rotation for the system logs on the CentOS system, issue the command:

$ sudo logrotate -f /etc/logrotate.conf
$  ls -la messages*
-rw------- 1 root root    563 May 23 20:09 messages
-rw------- 1 root root   5013 May 23 20:09 messages.1
-rw------- 1 root root 186528 May 23 04:02 messages.2
-rw------- 1 root root 130014 May 16 04:02 messages.3
-rw------- 1 root root 340531 May  9 04:02 messages.4

Note that the system log (messages) rotated as predicted.

Graphical, Shmaphical

For those of you who must live in a graphical environment, here’s a log viewer tool for you: The GNOME System Log Viewer. From the GNOME menu, find this log viewer under System->Administration->System Log. See the GNOME System Log Viewer in Figure 3.

GNOME System Log Viewer Application
Figure 3: GNOME System Log Viewer Application

The entries in bold, shown in Figure 3, are bold because new entries have been added since you opened the viewer and viewed those logs.

Logs. Aren’t you glad you have them. Exciting? Maybe not. Helpful? Always. It’s true that logs aren’t the celebrities of a Linux system but they’re the “behind the scenes” heroes and you’d do yourself a favor by familiarizing yourself with them.

Next week, you’ll continue to log some time with log files, when you take a closer look at log contents.

Comments on "Logs: Your Linux System’s Lovable Worker Bees"


Honestly this article lacks any sort of detail.

I can imagine that a person who just started with system administration _and_ never saw linux before will find the single bit of information usefull that tells us to look in /var/log/.

Apart from that…I\’ll be waiting for next week. Hoping for an in-depth writeup about (somewhere in this series) about log servers, event correlation, different products on the market to analyze logs (splunk,…. – not working for them but so far the best I found. I\’m open to alternatives)


What a \’fluff\’ piece!!
I was hoping to at least revisit log purpose, processing, management, etc., but reached the end of the article and wondered where the missing pages were! This article is absolutely ridiculous. If the intention was to merely expose a newbie to Linux logs, then it would be appropriate to say so, at least as a disclaimer.
Where was the discussion of myriad other log viewing / searching tools?
Where was any discussion about CONTENT type in various logs?
What about the directory structure often present in modern log directories?
What about log file aggregation?
What about remote logging?
What about log digesting and triggering based on log contents?
What about log monitoring?
Heck, this \’article\’ could have even been a short series based on many \’log\’ related topics!

… and I suppose you want me to continue reading future articles too?

Bill Pier


It\’s an introductory piece. Next week\’s goes more in depth but I had to establish a baseline so that everyone would have an even starting place. I didn\’t want to jump in to an advanced logs discussion and alienate new people from the article. It\’s an important topic and newbies need to learn it too.


Thanks for the article. I hope your going to make a series out of it. I need to have knowledge of logs. Hes right. You have to start low for some of us and work your way up.


There sure are some rude people about. If the article is too basic for you, perhaps you should go and have a look at the docs in the OS – that\’s the place that was intended for an expert audience to go. Then the rest of us can enjoy the intro article in peace.

I thought that the article was a good start to a series aimed at helping people learn how to use logs. Thanks for a great article.


I\’d never bothered to look at the Gnome System Log Viewer and now I have. So thanks for that!


I echo mdiehn\’s comment.
We\’re not all Linux gurus yet. I\’ve been on Ubuntu for around 2 years now & never looked at the logs unless I absolutely had to — which isn\’t that often (thanks Linux!). Even then I\’m looking for that one file that solves my specific problem. I\’m so focused on finding that one tree that I haven\’t yet fully grokked how the file structure of the whole forest works.

I did not know that I could rotate the logs that easily! I usually employ the Windows method of emptying the file out by hand to track changes. This is much better.

And that\’s why the intro article was a good idea.

they don’t want to turn in logs for pircavy or other reasons and the first comments out accuse the person of cheating. Or hiding something that must be bad. Questioning their character. Wow.

This is often a cause of problems in the database world where multiple clients may be executing queries on desparent data.

I think that what you said was very logical.

However, think about this, suppose you composed a catchier title?
I am not suggesting your content isn’t solid., however what if you added something that grabbed people’s attention?
I mean Logs: Your Linux System?s Lovable Worker Bees | Linux Magazine
is kinda boring. You might look at Yahoo’s home page and see how they create article headlines to grab people interested. You might try adding a video or a picture or two to grab people excited about what you’ve
got to say. Just my opinion, it might make your website a little livelier.

Hello to all, the contents present at this
web site are truly awesome for people knowledge, well, keep
up the good work fellows.

Looking forward to reading more. Great article post. Really Great.

Hi there! Do you use Twitter? I’d like to follow you if that would be ok. I’m definitely enjoying your blog and look forward to new posts.

Thanks a lot for the blog article.Much thanks again. Will read on

fgkSm2 pretty helpful material, overall I imagine this is worth a bookmark, thanks

We came across a cool internet site which you may enjoy. Take a appear for those who want.

That would be the finish of this post. Right here you?ll obtain some web-sites that we think you will value, just click the hyperlinks.

Here is an excellent Blog You might Uncover Fascinating that we encourage you to visit.

Please stop by the sites we adhere to, which includes this 1, because it represents our picks through the web.

The facts mentioned in the post are some of the top obtainable.

Muchos Gracias for your blog article.Much thanks again. Will read on…

Appreciate you sharing, great blog article. Great.

It’s an remarkable piece of writing in favor of all the internet users; they will obtain benefit from it I am sure.

Please visit the sites we adhere to, including this one, as it represents our picks in the web.

Usually posts some pretty interesting stuff like this. If you?re new to this site.

I simply want to mention I am just very new to blogging and really liked this website. More than likely I’m want to bookmark your blog . You actually come with good article content. Bless you for sharing your website page.

It’s not my first time to go to see this web site, i am
browsing this website dailly and take good
data from here every single day.

my webpage :: JaeYKapsos

Adn1sP wzwxabmttvzt, [url=http://grwhzxxtqxia.com/]grwhzxxtqxia[/url], [link=http://wrpfemqkkxtm.com/]wrpfemqkkxtm[/link], http://xpppqopwmbos.com/

I do consider all the ideas you’ve presented
for your post. These are very convincing and will certainly work.
Nonetheless, the posts are too quick for starters. May you please lengthen them a bit from subsequent time?
Thanks a lot for your post.

Feel free to visit my blog :: TracyKHulsey

The time to read or visit the subject material or internet sites we’ve linked to below.

Please let me know if you’re looking for a article author for your blog.
You have some really good articles and I believe I
would be a good asset. If you ever want to
take some of the load off, I’d love to write some articles
for your blog in exchange for a link back to mine.
Please blast me an e-mail if interested. Thank you!

Feel free to visit my site – LucioHMikola

Leave a Reply