Complete Virtual Mail Server/awstats

From Gentoo Wiki
Jump to:navigation Jump to:search
This page contains changes which are not marked for translation.
Other languages:
This article is part of the Complete Virtual Mail Server series, and may require previous parts to have been read or followed.

Even though postfix provides logs, having them graphically displayed and analyzed can give much more insight. www-misc/awstats is a popular log analyzer that can parse logs and setup proper results.


AWStats is a web-application but no longer relies on webapp-config. If it hasn't emerged already, it needs to be emerged:

root #emerge --ask www-misc/awstats

Once installed, a config file needs to be created, either per domain, or one that handles all domains:

root #cp /etc/awstats/awstats.model.conf /etc/awstats/

AWStats isn't installed via webapp-config, but rather a simple symlink is used:

root #ln -s /usr/share/awstats/wwwroot /var/www/



AWStats comes with reasonable defaults, but some need to be changed nevertheless.

For one, awstats assumes that vhosts aren't used. When using apache's default combined vhost logs for example, awstats will fail to run.

Assume apache's combined LogFormat is setup as follows:

CODE Apache's LogFormat
LogFormat "%v %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" VLOG=%{VLOG}e" vhost

The following changes then need to be made:

FILE /etc/awstats/ Logformat to apache's.
LogFormat = "%virtualname %host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot"

Next, awstats needs to know about the domains and aliases to filter from the log file:

FILE /etc/awstats/ awstats listen to the domains
HostAliases="localhost REGEX[example\.com$] REGEX[example\.(org|net)$]

Also, awstats needs to store its database somewhere. Gentoo has created /var/lib/awstats for this use, but it can be stored anywhere. Make sure the permissions are set so that the apache user can write to it:

FILE /etc/awstats/ database storage

Any other changes to the configuration file are optional, but interesting to look into:


awstats needs to process the apache log file to build its database. Once confirmed that it is working manually it can be automated.


First, awstats should be run from the console, to spot any initial errors:

user $ -update -showdropped

This should list any issues and missing domain names from the config.


If everything is working perfectly, it can then be added to cron.hourly:

FILE /etc/cron.hourly/awstatsAWstats cronjob
#!/bin/sh -update > /dev/null 2>&1

Remember to make the script executable, if needed.


Awstats will process the log file every hour, but when logrotate rotates apache's log, some entries may be missing. This is easily solved however:

FILE /etc/logrotate.d/apache2Diff of pre-init script
# Apache2 logrotate snipet for Gentoo Linux
# Contributes by Chuck Short
/var/log/apache2/*log {
+  prerotate
+  /etc/cron.hourly/awstats > /dev/null 2>&1
+  endscript
  /etc/init.d/apache2 reload > /dev/null 2>&1 || true


For awstats to be used from apache, the webhost needs to properly setup. In the alias section, the following needs to be added:

FILE /etc/apache2/vhosts.d/stats.example.comAliases for awstats
Alias /awstats/classes "/usr/share/awstats/wwwroot/classes"
Alias /awstats/css "/usr/share/awstats/wwwroot/css"
Alias /awstats/icon "/usr/share/awstats/wwwroot/icon"
Alias /awstats/js "/usr/share/awstats/wwwroot/js"
ScriptAlias /awstats/ "/usr/share/awstats/wwwroot/cgi-bin/"

Finally, awstats needs the correct permissions to be accessible:

FILE /etc/apache2/vhosts.d/stats.example.comAliases for awstats
<Directory "/usr/share/awstats/wwwroot">
        Options None
        Require all granted

After a restart of apache, awstats should be available via If no config option is passed to awstats, it uses the current hostname, which means in this case could have been omitted.

Awstats for mail log

AWStats is known for being an apache log parser. However it can also be used to parse mail logs.


After copying to quite a few changes are required to turn awstats into a mail log parser:

FILE /etc/ mail.log instead of access.log (diff -u, only additions and deletions)
+LogFile="perl /usr/bin/ standard < /var/log/mail.log |"
+LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd"
-HostAliases="localhost REGEX[myserver\.com$]"
+HostAliases="localhost REGEX[example\.(net|org)$]"
-LevelForBrowsersDetection=2         # 0 disables Browsers detection.
+LevelForBrowsersDetection=0         # 0 disables Browsers detection.
                                     # 2 reduces AWStats speed by 2%
                                     # allphones reduces AWStats speed by 5%
-LevelForOSDetection=2               # 0 disables OS detection.
+LevelForOSDetection=0               # 0 disables OS detection.
                                     # 2 reduces AWStats speed by 3%
-LevelForRefererAnalyze=2            # 0 disables Origin detection.
+LevelForRefererAnalyze=0            # 0 disables Origin detection.
                                     # 2 reduces AWStats speed by 14%
-LevelForRobotsDetection=2           # 0 disables Robots detection.
+LevelForRobotsDetection=0           # 0 disables Robots detection.
                                     # 2 reduces AWStats speed by 2.5%
-LevelForSearchEnginesDetection=2    # 0 disables Search engines detection.
+LevelForSearchEnginesDetection=0    # 0 disables Search engines detection.
                                     # 2 reduces AWStats speed by 9%
-LevelForKeywordsDetection=2         # 0 disables Keyphrases/Keywords detection.
+LevelForKeywordsDetection=0         # 0 disables Keyphrases/Keywords detection.
                                     # 2 reduces AWStats speed by 1%
-LevelForFileTypesDetection=2        # 0 disables File types detection.
+LevelForFileTypesDetection=0        # 0 disables File types detection.
                                     # 2 reduces AWStats speed by 1%
 LevelForWormsDetection=0            # 0 disables Worms detection.
                                     # 2 reduces AWStats speed by 15%

With those changes in place, a manual run should work without any issues:

user $ -showcorrupted -showdropped


To scan the mail log every hour, the existing awstats script in cron.hourly can be appended with the following:

FILE /etc/cron.hourly/awstatsAdd parsing of the mail log -update > /dev/null 2>&1

Also syslog is getting rotated and thus awstats needs to parse the mail log file before the mail log is being rotated:

FILE /etc/logrotate.d/syslog-ngModify mail log entry in syslog
# Mail system
/var/log/mail.log /var/log/ /var/log/mail.err /var/log/mail.warn {
        /etc/init.d/syslog-ng reload &gt; /dev/null 2&gt;&1 || true
The /var/log/mail.log file does not have to be on the same server. AWStats will have to have access to it. This could be via NFS, or having syslog do remote logging.

If logging of apache files is not desired, or webmail resides on a different server, the webserver log parsing can be removed from cron jobs.