Sunday, October 25, 2009

Apache:Piped logging program to rotate Apache logs

There was a requirement from IT Infrastructure Team that the Apache Logs has always been cumbersome to maintain as it gets full due to heavy rush at the server end. They needed a utility to rotate the logs as per their requirement.

I explored tools for apache logs maintenance and finally got a chance to try out something.Here is my finding:

rotatelogs is a simple program for use in conjunction with Apache's piped logfile feature. For example:

CustomLog "|rotatelogs /var/logs/logfile 86400" common

This creates the files /var/logs/logfile.nnnn where nnnn is the system time at which the log nominally starts (this time will always be a multiple of the rotation time, so you can synchronize cron scripts with it). At the end of each rotation time (here after 24 hours) a new log is started.

CustomLog "|rotatelogs /var/logs/logfile 5M" common

This configuration will rotate the logfile whenever it reaches a size of 5 megabytes.

ErrorLog "|bin/rotatelogs /var/logs/errorlog.%Y-%m-%d-%H_%M_%S 5M"

This configuration will rotate the error logfile whenever it reaches a size of 5 megabytes, and the suffix to the logfile name will be created of the form errorlog.YYYY-mm-dd-HH_MM_SS.

■ Synopsis
■ Options
■ Portability
rotatelogs [ -l ] logfile [ rotationtime [ offset ]] | [ filesizeM ]

-l (2.0.51 and later)
Causes the use of local time rather than GMT as the base for the interval. Note that using -l in an environment which changes the GMT offset (such as for BST or DST) can lead to unpredictable results!
The path plus basename of the logfile. If logfile includes any '%' characters, it is treated as a format string for strftime(3). Otherwise, the suffix .nnnnnnnnnn is automatically added and is the time in seconds. Both formats compute the start time from the beginning of the current period.
The time between log file rotations in seconds.
The number of minutes offset from UTC. If omitted, zero is assumed and UTC is used. For example, to use local time in the zone UTC -5 hours, specify a value of -300 for this argument.
The maximum file size in megabytes followed by the letter M to specify size rather than time. Use this parameter in place of both rotationtime and offset.
The following logfile format string substitutions should be supported by all strftime(3) implementations, see the strftime(3) man page for library-specific extensions.

%A full weekday name (localized)
%a 3-character weekday name (localized)
%B full month name (localized)
%b 3-character month name (localized)
%c date and time (localized)
%d 2-digit day of month
%H 2-digit hour (24 hour clock)
%I 2-digit hour (12 hour clock)
%j 3-digit day of year
%M 2-digit minute
%m 2-digit month
%p am/pm of 12 hour clock (localized)
%S 2-digit second
%U 2-digit week of year (Sunday first day of week)
%W 2-digit week of year (Monday first day of week)
%w 1-digit weekday (Sunday first day of week)
%X time (localized)
%x date (localized)
%Y 4-digit year
%y 2-digit year
%Z time zone name
%% literal `%'

Note: Do create the logs directory under /var directory.
For testing, Just restart the httpd service and you will see these directories:

[root@localhost logs]# ls -la
total 28
drwxr-xr-x 2 root root 4096 2009-10-26 01:45 .
drwxr-xr-x. 23 root root 4096 2009-10-26 01:13 ..
-rw-r--r-- 1 root root 632 2009-10-26 01:43 errorlog.2009-10-25-20_13_59
-rw-r--r-- 1 root root 632 2009-10-26 01:44 errorlog.2009-10-25-20_14_15
-rw-r--r-- 1 root root 0 2009-10-26 01:13 logfile
-rw-r--r-- 1 root root 310 2009-10-26 01:21 logfile.1256500257
-rw-r--r-- 1 root root 381 2009-10-26 01:39 logfile.1256501364
-rw-r--r-- 1 root root 81 2009-10-26 01:44 logfile.1256501673
[root@localhost logs]#

1 comment:

  1. [root@localhost logs]# cat errorlog.2009-10-25-20_13_59
    [Mon Oct 26 01:43:59 2009] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
    [Mon Oct 26 01:43:59 2009] [notice] Digest: generating secret for digest authentication ...
    [Mon Oct 26 01:43:59 2009] [notice] Digest: done
    [Mon Oct 26 01:43:59 2009] [notice] mod_python: Creating 4 session mutexes based on 256 max processes and 0 max threads.
    [Mon Oct 26 01:43:59 2009] [notice] mod_python: using mutex_directory /tmp
    [Mon Oct 26 01:43:59 2009] [notice] Apache/2.2.13 (Unix) DAV/2 PHP/5.2.9 mod_python/3.3.1 Python/2.6 mod_ssl/2.2.13 OpenSSL/0.9.8k-fips mod_perl/2.0.4 Perl/v5.10.0 configured -- resuming normal operations
    [root@localhost logs]#