Previously posted on blog.labrat.info on March 4, 2010
Trying to find a way to make awstats and newsyslog work well together has been a problem.
newsyslog is a great way to take care of rotating files. It’s got some great knobs that can be tweaked. Logs can be rotated by date (different ways of determining date as well) or by size. There’s a great explanation of what can be done with newsyslog here.
awstats is a great way to get some very useful information from log files. I mostly use it on Apache log files but it can also many more. If you haven’t checked it out, you should.
The problem with trying to get these two programs to work is I haven’t been able to find a way to not lose data when a log gets rotated. If a log gets rotated before awstats gets a chance to parse it that information is lost. As far as I know newsyslog doesn’t have the capability of calling script before it does the log rotation. This is something that was trivial to do in Linux with logrotate. But I don’t have it so I can’t use it. The line in /etc/newsyslog.conf for me looks like this:
/var/log/http/apache.log 644 6 100 * B /var/run/httpd.pid 30
The first parameter is the log file to be rotated. The second is the permissions of the file to be created (r-wr–r–). The third is the number of copies to keep around (6). The forth is the size in KB before the file gets rotated (100). The fifth is a date or time for the file to be rotated. In this case I just care about the size so no time is given. The sixth is a set of predefiend letters that can do different things. In this case the “B” is there so no comments are introduced into the log file. The seventh and eith field are the path to the PID file for Apache and the signal to send to it to rotate the logs. A much better explanation than I will ever be able to give on why the signal is 30 can be found here.
I did find that awstats has a very cool program called logresolvemerge.pl that can merge a bunch of files together. Even better is the fact that it can be called in-line from awstats configuration in the LogFile variable. My awstats config now looks like this:
LogFile="/var/www/awstats/tools/logresolvemerge.pl /var/log/http/apache.log* |"
This now uses the star as a wildcard and logresolvemerge.pl will just merge all the files it can find and return them to awstats. The pipe at the end of the string is very important. It doesn’t work without it. Now, even if a file got rotated between runs of awstats, they still will be parsed!