Security Handbook/Logging
System administrators should choose between different system loggers in order to gain better viability into attacks.
Additional verbosity on logging should be added to catch warnings or errors that might indicate an ongoing attack or a successful compromise. Attackers often scan or probe before directly attacking a targeted system.
It is also vital that log files are easily readable and manageable. System administrators are able choose between system loggers and should choose a logger that best fits their need for system security.
See the logging meta article about available logging software on Gentoo.
Sysklogd
Sysklogd is the most common logger for Linux and Unix in general. It has some log rotation facilities, but using /usr/sbin/logrotate in a cron job (logrotate is configured in /etc/logrotate.conf) might prove to be more powerful as logrotate has many features. How often log rotation should be done depends on the system load.
See the Sysklogd article for more information.
Below is the standard syslog.conf with some added features. The following cron and tty lines have been uncommented and a remote logging server had been added. To further enhance security could add logging to two places.
/etc/syslog.conf# /etc/syslog.conf Configuration file for syslogd.
#
# For more information see syslog.conf(5) manpage.
#
# Define standard logfiles. Log by facility.
#
auth,authpriv.* /var/log/auth.log
*.*;auth,authpriv.none -/var/log/syslog
cron.* /var/log/cron.log
daemon.* -/var/log/daemon.log
kern.* -/var/log/kern.log
lpr.* -/var/log/lpr.log
mail.* /var/log/mail.log
user.* -/var/log/user.log
uucp.* -/var/log/uucp.log
local6.debug /var/log/imapd.log
# Logging for the mail system. Split it up so that it is easy to write scripts to parse these files.
#
mail.info -/var/log/mail.info
mail.warn -/var/log/mail.warn
mail.err /var/log/mail.err
# Logging for INN news system
#
news.crit /var/log/news/news.crit
news.err /var/log/news/news.err
news.notice -/var/log/news/news.notice
# Some `catch-all' logfiles.
#
*.=debug;\
auth,authpriv.none;\
news.none;mail.none -/var/log/debug
*.=info;*.=notice;*.=warn;\
auth,authpriv.none;\
cron,daemon.none;\
mail,news.none -/var/log/messages
# Emergencies and alerts are sent to everybody logged in.
#
*.emerg *
*.=alert *
# I like to have messages displayed on the console, but only on a virtual
# console I usually leave idle.
#
daemon,mail.*;\
news.=crit;news.=err;news.=notice;\
*.=debug;*.=info;\
*.=notice;*.=warn /dev/tty8
#Setup a remote logging server
#
*.* @logserver
# NOTE: adjust the list below, or you'll go crazy if you have a reasonably
# busy site..
#
#daemon.*,mail.*;\
# news.crit;news.err;news.notice;\
# *.=debug;*.=info;\
# *.=notice;*.=warn |/dev/xconsole
local2.* --/var/log/ppp.log
Attackers will most likely try to erase their tracks by editing or deleting log files. System administrators can make covering up evidence more difficult by logging to one or more remote logging servers. Get more info about sysklogd by executing man syslogd.
Metalog
See the Metalog article for more information.
Metalog by Frank Dennis is not able to log to a remote server, but it does have advantages when it comes to performance and logging flexibility. It can log by program name, urgency, facility (like sysklogd), and comes with regular expression matching with which you can launch external scripts when specific patterns are found. It is very good at taking action when needed.
The standard configuration is usually enough. To be notified by email whenever a password failure occurs use one of the following scripts.
For postfix:
/usr/local/sbin/mail_pwd_failures.shPostfix#!/bin/sh echo "$3" | mail -s "Warning (program : $2)" root
For netqmail:
/usr/local/sbin/mail_pwd_failures.shNetqmail#!/bin/sh echo "To: root Subject:Failure (Warning: $2) $3 " | /var/qmail/bin/qmail-inject -f root
Remember to make the script executable by issuing chmod +x /usr/local/sbin/mail_pwd_failures.sh
Then uncomment the command line under "Password failures" in /etc/metalog/metalog.conf like:
/etc/metalog/metalog.confMetalogcommand = "/usr/local/sbin/mail_pwd_failures.sh"
Syslog-ng
See the Syslog-ng article for more information.
Syslog-ng provides some of the same features as sysklogd and metalog with a small difference. It can filter messages based on level and content (like metalog), provide remote logging like sysklogd, handle logs from syslog (even streams from Solaris), write to a TTY, execute programs, and it can act as a logging server. Basically it is the best of both loggers combined with advanced configuration.
Below is a classic configuration file slightly modified.
/etc/syslog-ng/syslog-ng.confSyslog-ng@version: 3.17 #mandatory since Version 3, specify the version number of the used syslog-ng
options {
chain_hostnames(no);
# The default action of syslog-ng is to log a STATS line
# to the file every 10 minutes. That's pretty ugly after a while.
# Change it to every 12 hours for a nice daily update of
# how many messages syslog-ng missed (0).
stats_freq(43200);
};
source src {
unix-stream("/dev/log" max-connections(256));
internal();
};
source kernsrc { file("/proc/kmsg"); };
# define destinations
destination authlog { file("/var/log/auth.log"); };
destination syslog { file("/var/log/syslog"); };
destination cron { file("/var/log/cron.log"); };
destination daemon { file("/var/log/daemon.log"); };
destination kern { file("/var/log/kern.log"); };
destination lpr { file("/var/log/lpr.log"); };
destination user { file("/var/log/user.log"); };
destination mail { file("/var/log/mail.log"); };
destination mailinfo { file("/var/log/mail.info"); };
destination mailwarn { file("/var/log/mail.warn"); };
destination mailerr { file("/var/log/mail.err"); };
destination newscrit { file("/var/log/news/news.crit"); };
destination newserr { file("/var/log/news/news.err"); };
destination newsnotice { file("/var/log/news/news.notice"); };
destination debug { file("/var/log/debug"); };
destination messages { file("/var/log/messages"); };
destination console { usertty("root"); };
# By default messages are logged to tty12...
destination console_all { file("/dev/tty12"); };
# ...if you intend to use /dev/console for programs like xconsole
# you can comment out the destination line above that references /dev/tty12
# and uncomment the line below.
#destination console_all { file("/dev/console"); };
# create filters
filter f_authpriv { facility(auth, authpriv); };
filter f_syslog { not facility(authpriv, mail); };
filter f_cron { facility(cron); };
filter f_daemon { facility(daemon); };
filter f_kern { facility(kern); };
filter f_lpr { facility(lpr); };
filter f_mail { facility(mail); };
filter f_user { facility(user); };
filter f_debug { not facility(auth, authpriv, news, mail); };
filter f_messages { level(info..warn)
and not facility(auth, authpriv, mail, news); };
filter f_emergency { level(emerg); };
filter f_info { level(info); };
filter f_notice { level(notice); };
filter f_warn { level(warn); };
filter f_crit { level(crit); };
filter f_err { level(err); };
filter f_failed { message("failed"); };
filter f_denied { message("denied"); };
# connect filter and destination
log { source(src); filter(f_authpriv); destination(authlog); };
log { source(src); filter(f_syslog); destination(syslog); };
log { source(src); filter(f_cron); destination(cron); };
log { source(src); filter(f_daemon); destination(daemon); };
log { source(kernsrc); filter(f_kern); destination(kern); };
log { source(src); filter(f_lpr); destination(lpr); };
log { source(src); filter(f_mail); destination(mail); };
log { source(src); filter(f_user); destination(user); };
log { source(src); filter(f_mail); filter(f_info); destination(mailinfo); };
log { source(src); filter(f_mail); filter(f_warn); destination(mailwarn); };
log { source(src); filter(f_mail); filter(f_err); destination(mailerr); };
log { source(src); filter(f_debug); destination(debug); };
log { source(src); filter(f_messages); destination(messages); };
log { source(src); filter(f_emergency); destination(console); };
# default log
log { source(src); destination(console_all); };
Syslog-ng is very easy to configure, but it is also very easy to miss something in the configuration file since it is huge. The author still promises some extra features like encryption, authentication, compression and MAC (Mandatory Access Control) control. With these options it will be a perfect for network logging, since the attacker cannot spy on the log.
And syslog-ng does have one other advantage: it does not have to run as root!
Log analysis with Logcheck
Of course, keeping logs alone is only half the battle. An application such as Logcheck can make regular log analysis much easier. logcheck is a script, accompanied by a binary called logtail, that runs from the cron daemon and checks the system logs against a set of rules for suspicious activity. It then mails the output to root's mailbox.
logcheck and logtail are part of the app-admin/logcheck package.
Logcheck uses four files to filter important log entries from the unimportant:
- logcheck.hacking - Contains known hacking attack messages.
- logcheck.violations - Contains patterns indicating security violations.
- logcheck.violations.ignore - Contains keywords likely to be matched by the violations file, allowing normal entries to be ignored.
- logcheck.ignore - matches those entries to be ignored.
Do not leave the logcheck.violations.ignore file empty. logcheck uses the grep utility to parse logs, some versions of which will take an empty file to mean wildcard. All violations would thus be ignored.