ONLamp.com
oreilly.comSafari Books Online.Conferences.

advertisement


Making Apache httpd Logs More Useful

by Rich Bowen
02/01/2007

No doubt you're already aware of the standard logfiles that Apache httpd creates for you. There's the access log, which tells you every time a request is made to your server. There's also the error log, which makes a note every time something goes wrong or something of interest happens that you should know about.

There are a few things that you can do to make your access log more useful, such as using the combined, rather than the common, logfile format--but that's another article. Look at the documentation for mod_log_config for more information on that.

You may not know that there are several additional logging modules that provide information about certain types of things that happen on your server.

The modules discussed here are available in 2.0 and 2.2, but not in 1.3.

mod_logio

When mod_log_config makes a log entry, the number of bytes transfered can be (and usually is) logged using the %b variable in the LogFormat. This number is less useful than you might wish, as it logs the size of the body of the response, and does not include the headers. Because a significant percentage of the data transferred to the client is comprised of headers, this doesn't provide the whole picture of how much data you're transferring. It also doesn't include the request at all, so on a site where file uploads are permitted, you end up seeing only a part of your total bandwidth usage.

mod_logio adds two new variables to those available to the LogFormat directive, which allows you to log the total bytes transferred, including headers, both input and output.

These two variables are %I and %O, which will log the size of the input--the request, including headers and request body--and the output, including all the headers.

For example, you might have a LogFormat directive like:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio

You'll find this line in your default configuration file if you install 2.2 from source. I don't know whether your particular distro included this line for you.

On the end, you'll see the two additional variables that I mentioned. These result in logfile entries that look like:

    192.168.200.105 - - [24/Nov/2006:11:23:30 -0500] "GET / HTTP/1.1" 200
        8054 "http://wooga.drbacchus.com/index.php?" "Mozilla/5.0 (X11; U; Linux
        i686; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0" 935 8522

This shows a request for the front page of my website, and it shows that the bytes transferred, not including the headers, came to 8054. However, if you include the headers in that figure, it comes to 8522 bytes. There were 935 bytes in the request.

This gives you a much better picture of how much bandwidth your site is actually using and includes what's coming in, as well as what's going out.

mod_log_forensic

mod_log_forensic, added in 2.0, gives a little additional data that may help you troubleshoot problems on your server.

In particular, mod_log_forensic logs a fixed-format logfile that tells you if and when your requests complete. It makes a log entry when the request is initiated, and another when the request is completed.

To start logging, add this directive to your configuration file:

    ForensicLog logs/forensic_log

The initial log entry looks like:

    +5fb1:45671e25:0|GET /wordpress/index.php?feed=rss2category_name=podcasts
    HTTP/1.1|Host:wooga.drbacchus.com|Accept:*/*|Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.7|Accept-Encoding:gzip,identity|User-Agent:Feedreader
    3.07 (Powered by Newsbrain)

That is, it logs all of the headers of the request and assigns a unique identifier to the log entry. If you're using mod_uniqueid, the mod_uniqueid identifier will be used. In Apache 2.0, mod_uniqueid is required to use mod_log_forensic; In 2.2 and later, it is optional.

When the request is completed, another log entry is made:

    -5fb1:45671e25:0

Pages: 1, 2

Next Pagearrow





Sponsored by: