Полноценный виртуальный почтовый сервер/amvisd, spamassassin и clamav

From Gentoo Wiki
Jump to:navigation Jump to:search
This page is a translated version of the page Complete Virtual Mail Server/amavisd spamassassin clamav and the translation is 35% complete.
Outdated translations are marked like this.
Other languages:
Заметка
This article is part of the Complete Virtual Mail Server series, and may require previous parts to have been read or followed.

Spam is becoming more and more of an issue on the Internet and a robust and solid solution is required. There are paid services for even more spam protection but that should not be required and will not be discussed in this article.

Postfix

The first line of defense, is postfix itself. Postfix offers a few basic means to block spam, or rather spammers. Using smtpd_client_restrictions it is possible to use public DNS blacklists. There are 3 popular DNS blacklists of which one is incorporated into the other. These two lists are also the most accurate ones. The most important one is zen.spamhaus.org and as a backup bl.spamcop.net can be used. Using them with postfix is very simple. Add these domains as reject_rbl_client in main.cf:

ФАЙЛ /etc/postfix/main.cfИспользование черных списков DNS
# Block spam using DNS blacklists
smtpd_client_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_rbl_client zen.spamhaus.org, reject_rbl_client bl.spamcop.net
Заметка
permit_mynetworks and permit_sasl_authenticated are added for good measure. If other restrictions are required or already in use, keep them in place.

Amavis

Введение

Spam-assassin and ClamAV are the tools to block spam and viruses, however amavis is required to tie this all together. Amavis will actually behave as a mail server in itself, accept mail, filter it, and send it onwards again. For this to work, postfix will need to actually listen for mail twice. The default port 25 is where mail initially is received on. From there on it is sent to amavis, which will be listening on port 10024. When amavis is done with the message, it will be sent to postfix on a different port, 10025. The reason for this should be obvious. If mail would be offered again on port 25, it would be passed to amavis again and thus in an endless loop. Obviously, postfix on port 10025 would only be listening to known hosts, like localhost and not check for spam anymore.

Заметка
Because of this setup, it is quite easily possible, to have these 3 mail 'handlers' on 3 different hosts. It could be possible to have the primary mailserver, that listens on port 25 on the firewall, have it forward to a safe isolated server running only amavis and have amavis sent mail to an internal postfix server where it delivers mail to the mail storage.

Установка

Amavis should have been installed already, if not, emerge it. This should pull in spam-assassin and clamav as its dependencies:

root #emerge --ask mail-filter/amavisd-new

Основная конфигурация

Amavisd offers an enormous amount of options and going over all them will take some time. The configuration file /etc/amavisd.conf however is well documented and divided into clear sections. Each section will be examined as needed. Only options that will be changed will be mentioned to cut down the text for readability.

Заметка
The configuration file is actually perl code and proper precautions should be taken when editing this file.

For this example amavisd will be running on host foo but this could be any other host as well, amavisd does not require to run on the same host as postfix. Also the domain used is only used to identify the server itself with, not the domains amavisd will be scanning.

Заметка
With amavis being quite complex, troubleshooting can be difficult enough as it is.

The first step, is to disable all actual checks and to enable logging. Also some default values should be setup:

ФАЙЛ /etc/amavisd.confDisable anti-spam, enable logging
@bypass_virus_checks_maps = (1);  # controls running of anti-virus code
@bypass_spam_checks_maps  = (1);  # controls running of anti-spam code
# $bypass_decode_parts = 1;         # controls running of decoders&dearchivers
 
$mydomain = 'example.com';
$myhostname = 'foo.example.com';
 
$log_level = 5;              # verbosity 0..5, -d
Заметка
virusalert@example.com and spam.police@example can be renamed to undef if not wanting to receive any information on breaches. If not, these e-mail addresses should exist.

Normally, restarting postfix should restart amavisd as well. For now, only amavisd should be started to see if there are any initial problems:

root #/etc/init.d/amavisd start

Linking amavisd to Postfix

With amavisd working in bare skeletal mode, it should theoretically just pass mail through. Perfect for testing the postfix -> amavisd -> postfix binding.

First, a second postfix transport, where amavis will inject its mail, is added. A lot of options are defaulted to empty, since either they have been checked already, or interfere otherwise:

ФАЙЛ /etc/postfix/master.cfFilter mail through amavisd
localhost:10025 inet n  -       n       -       2       smtpd
  -o smtp_dns_support_level=enabled
  -o content_filter=
  -o myhostname=foo.example.com
  -o local_recipient_maps=
  -o relay_recipient_maps=
  -o smtpd_restriction_classes=
  -o smtpd_client_restrictions=
  -o smtpd_helo_restrictions=
  -o smtpd_sender_restrictions=
  -o smtpd_recipient_restrictions=permit_mynetworks,reject
  -o mynetworks=127.0.0.0/8
  -o strict_rfc821_envelopes=yes
  -o smtpd_error_sleep_time=0
  -o smtpd_soft_error_limit=1001
  -o smtpd_hard_error_limit=1000
  -o smtpd_client_connection_count_limit=0
  -o smtpd_client_connection_rate_limit=0
  -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
  -o smtpd_authorized_xforward_hosts=127.0.0.0/8

With this transport in place, it should only listen on localhost and only accept mail from localhost. This should be extended if amavis is run elsewhere, but keep in mind anything is accepted.

Заметка
maxproc' is set to 2 above. This can be increased if more listening daemons are required. However this number needs to match amavis's $max_servers.

Next another transport is added for, which could be considers 'being' amavis in a sense:

ФАЙЛ /etc/postfix/master.cfamavis transport
amavis    unix  -       -       n       -       2       lmtp
  -o disable_dns_lookups=yes
  -o lmtp_send_xforward_command=yes
  -o lmtp_data_done_timeout=1200

After the transport for amavis has been added, smtp should be told to route all mail through amavis. For this two option need to be added to smtpd and change the maxproc to match amavis's:

ФАЙЛ /etc/postfix/master.cfrelay to amavis
smtp       inet  n       -       n       -       2       smtpd
  -o content_filter=amavis:[127.0.0.1]:10024
  -o receive_override_options=no_address_mappings
smtps     inet  n       -       n       -       2       smtpd
  -o smtpd_tls_wrappermode=yes
  -o content_filter=amavis:[127.0.0.1]:10024
  -o receive_override_options=no_address_mappings

Restarting both amavisd and postfix then should pass all mail through amavisd:

root #/etc/init.d/amavisd restart
root #/etc/init.d/postfix restart

Тестирование

Sending a message to testuser@example.com remotely and locally should work fine. After the message has arrived, the headers should be checked.

Заметка
When doing this test, either use webmail or a remote client using a different mailserver or mail through port 25 or 465. Do not test using mail submission on port 587 as that bypasses amavisd.

Looking at the mail headers, it should be noticed that it was sent through amavisd and re-delivered to postfix:

КОД Mail header
Received: from localhost (localhost [127.0.0.1])
    by foo.example.com (Postfix) with ESMTP id 03ABA22E4C
    for <testuser@example.com>; Wed, 28 Dec 2011 11:41:05 +0100 (CET)
Received: from foo.example.com ([127.0.0.1])
    by localhost (foo.example.com [127.0.0.1]) (amavisd-new, port 10024)
    with LMTP id 6N9l4nIQa620 for <testuser@example.com>; Wed, 28 Dec 2011 11:41:04 +0100 (CET)
Received: from ext.example.net (ext.example.net [1.2.3.4])
    (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
    (No client certificate requested)
    by foo.example.com (Postfix) with ESMTPS id C136021F97
    for <testuser@example.com>; Wed, 28 Dec 2011 11:41:04 +0100 (CET)

Examining the above it is clearly visible that the mail was received by postfix via SMTPS even. It was then forwarded to amavisd on port 10024 via LMTP and finally redelivered to postfix using SMTP again.

ClamAV

Введение

ClamAV is the de facto open source virus scanner for linux. Amavis can be linked to many different free and commercial virus scanners, but here clamav will be used. ClamAV is specifically designed for scanning e-mail. It consists of two parts, clamav itself, and freshclam, the clamav updating service. By default it updates every two hours, which should be enough for anyone.

Установка

ClamAV should have been installed already, if not it should be emerged:

root #emerge --ask app-antivirus/clamav
Заметка
The clamdtop flag lets clamav build the clamdtop utility, a nice monitoring tool for the viruscanner.

Конфигурация

ClamAV will be configured to run in daemonized mode, e.g. it will be listening for connections (from amavisd). The other option (and the default fallback in amavisd) is to have amavisd use the commandline scanner, which is much slower and much much more resource intensive.

ClamAV does not have to be run on the same host, however it is recommended for performance reasons to keep it on the same host, depending on resource usage.

To be able to communicate, clamav needs to be part of amavisd's group:

root #gpasswd -a clamav amavis

It always helps to allow clamd to output some debug information:

ФАЙЛ /etc/clamd.confВключение Debug параметра
# Enable debug messages in libclamav.
# Default: no
Debug yes

Also clamd needs some settings setup in its configuration file so that amavis can talk to it:

ФАЙЛ /etc/clamd.confПозволим amavis подключиться к clamd
# Path to a local socket file the daemon will listen on.
# Default: disabled (must be specified by a user)
LocalSocket /var/run/clamav/clamd.sock
 
# Initialize supplementary group access (clamd must be started by root).
# Default: no
AllowSupplementaryGroups yes
Заметка
These options appear to be the default nowadays.

When running clamav on a hardened kernel, there will be warnings about certain operations not being permitted:

КОД Пример ошибки LibClamAV
[LibClamAV] Bytecode: disabling JIT because SELinux is preventing 'execmem' access.

This is expected and okay. ClamAV can run fine without JIT.

Before starting clamav for the first time, the virus database needs to be downloaded. Freshclam is responsible for downloading and keeping the virus database up to date. Freshclam gets automatically started by the clamd startup script, but clamd will fail to start due to a missing database:

root #/etc/init.d/clamd start

Now monitor the clamav log file to see freshclam download the initial virus database:

ФАЙЛ /var/log/clamav/freshclam.logЖурнал запуска Freshclam
freshclam daemon 0.98.6 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64)
ClamAV update process started at Sat Mar 21 16:30:16 2015
Downloading main.cvd [100%]
main.cvd updated (version: 55, sigs: 2424225, f-level: 60, builder: neo)
Downloading daily.cvd [100%]
daily.cvd updated (version: 20219, sigs: 1354642, f-level: 63, builder: neo)
Downloading bytecode.cvd [100%]
[LibClamAV] Bytecode: disabling JIT because SELinux is preventing 'execmem' access.
Run  'setsebool -P clamd_use_jit on'.
ERROR: During database load : LibClamAV Warning: RWX mapping denied: Can't allocate RWX Memory: Operation not permitted
WARNING: Database successfully loaded, but there is stderr output
bytecode.cvd updated (version: 247, sigs: 41, f-level: 63, builder: dgoddard)
Database updated (3778908 signatures) from database.clamav.net (IP: 200.236.31.1)
WARNING: Clamd was NOT notified: Can't connect to clamd through /var/run/clamav/clamd.sock: No such file or directory

Теперь база данных обновлена, перезапустите clamd:

root #/etc/init.d/clamd restart
ФАЙЛ /var/log/clamav/clamd.logЖурнал запуска ClamAV
Sat Mar 21 16:35:18 2015 -> clamd daemon 0.98.6 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64)
Sat Mar 21 16:35:18 2015 -> Running as user clamav (UID 105, GID 206)
Sat Mar 21 16:35:18 2015 -> Log file size limited to 1048576 bytes.
Sat Mar 21 16:35:18 2015 -> Reading databases from /var/lib/clamav
Sat Mar 21 16:35:18 2015 -> Not loading PUA signatures.
Sat Mar 21 16:35:18 2015 -> Bytecode: Security mode set to "TrustSigned".
Sat Mar 21 16:35:29 2015 -> Loaded 3773304 signatures.
Sat Mar 21 16:35:30 2015 -> LOCAL: Unix socket file /var/run/clamav/clamd.sock
Sat Mar 21 16:35:30 2015 -> LOCAL: Setting connection queue length to 200
Sat Mar 21 16:35:30 2015 -> Limits: Global size limit set to 104857600 bytes.
Sat Mar 21 16:35:30 2015 -> Limits: File size limit set to 26214400 bytes.
Sat Mar 21 16:35:30 2015 -> Limits: Recursion level limit set to 16.
Sat Mar 21 16:35:30 2015 -> Limits: Files limit set to 10000.
Sat Mar 21 16:35:30 2015 -> Limits: MaxEmbeddedPE limit set to 10485760 bytes.
Sat Mar 21 16:35:30 2015 -> Limits: MaxHTMLNormalize limit set to 10485760 bytes.
Sat Mar 21 16:35:30 2015 -> Limits: MaxHTMLNoTags limit set to 2097152 bytes.
Sat Mar 21 16:35:30 2015 -> Limits: MaxScriptNormalize limit set to 5242880 bytes.
Sat Mar 21 16:35:30 2015 -> Limits: MaxZipTypeRcg limit set to 1048576 bytes.
Sat Mar 21 16:35:30 2015 -> Limits: MaxPartitions limit set to 50.
Sat Mar 21 16:35:30 2015 -> Limits: MaxIconsPE limit set to 100.
Sat Mar 21 16:35:30 2015 -> Archive support enabled.
Sat Mar 21 16:35:30 2015 -> Algorithmic detection enabled.
Sat Mar 21 16:35:30 2015 -> Portable Executable support enabled.
Sat Mar 21 16:35:30 2015 -> ELF support enabled.
Sat Mar 21 16:35:30 2015 -> Mail files support enabled.
Sat Mar 21 16:35:30 2015 -> OLE2 support enabled.
Sat Mar 21 16:35:30 2015 -> PDF support enabled.
Sat Mar 21 16:35:30 2015 -> SWF support enabled.
Sat Mar 21 16:35:30 2015 -> HTML support enabled.
Sat Mar 21 16:35:30 2015 -> Self checking every 600 seconds.

Подключение amavisd к clamav

Amavisd should connect to the socket of clamd and thus clamav needs to be enabled as one of the main antivirus scanners. The fallback of invoking clamav from the commandline should not be changed. Also the virus check bypass needs to be disabled to be effective:

ФАЙЛ /etc/amavisd.confВключение clamav
# @bypass_virus_checks_maps = (1);  # controls running of anti-virus code
 
['ClamAV-clamd',
  \&ask_daemon, ["CONTSCAN {}\n", "/var/run/clamav/clamd.sock"],
  qr/\bOK$/m, qr/\bFOUND$/m,
  qr/^.*?: (?!Infected Archive)(.*) FOUND$/m ],

After restarting amavisd, viruses should be able to detected and blocked:

root #/etc/init.d/amavisd restart

Тестирование

To test whether the virus filter works, an anti-malware testfile exists, sending an e-mail using this string should trigger the virus scanner:

КОД EICAR-STANDARD-ANTIVIRUS-TEST-FILE
X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*
Заметка
Again, it should be ensured that this is message is passed through the sending mailserver, and only be blocked by the receiving server (foo.example.com). Using a client that connects to port 25 and is allowed to relay should suffice.

Looking at the mail.log file, the following should be revealed:

ФАЙЛ /var/log/mail.logТест на заражение вирусом
Dec 28 14:05:33 7of9 amavis[7554]: (07554-01) Blocked INFECTED (Eicar-Test-Signature) {DiscardedInternal,Quarantined}, MYNETS LOCAL [10.0.0.2]:41144 [10.0.0.2] <root@test.example.net> -> <testuser@example.com>, quarantine: virus-WYHuLpwdzPVr, Queue-ID: 7FC7B22DF6, mail_id: WYHuLpwdzPVr, Hits: -, size: 407, 166 ms
Dec 28 14:05:33 7of9 postfix/lmtp[15452]: 7FC7B22DF6: to=<testuser@example.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=0.41, delays=0.21/0.02/0.01/0.16, dsn=2.7.0, status=sent (250 2.7.0 Ok, discarded, id=07554-01 - INFECTED: Eicar-Test-Signature)

In theory, the virus scanner should be fully functional now.

Spam Assassin

Введение

Spam Assassin is an excellent spam filter. It has become quite complex throughout the years and requires some effort to configure correctly.

Установка

There really should be no need to install spamassassin separately. The spamassassin USE flag should have pulled it in as a dependency of amavisd-new.

root #emerge --ask mail-filter/spamassassin

Конфигурация

The default configuration suffices for standard use.

Заметка
If users don't e-mail via the submission port, the trusted_networks variable should be setup properly to bypass spam filtering from known users.

Обновление

A key feature of Spam Assassin is its ability to self-update. Updates are handled via a so-called update channel.

Spam Assassin comes with the sa-update tool so updates can be fully automated. Spam Assassin updates can be done by using the --nogpg flag to ignore gpg keys, but should really only be done as a last resort. Adding the spamassassin GPG key is a simple 2 step process.

root #sa-update --import GPG.KEY
root #rm GPG.KEY
Предупреждение
Possibly due to bug #396307 it is required to first create a directory for the keyring and set proper permissions prior to importing the keys.
root #mkdir -p /etc/mail/spamassassin/sa-update-keys; chmod 700 /etc/mail/spamassassin/sa-update-keys

After adding the spamassassin update channel, it needs to be updated. After running this command, check for any errors:

root #sa-update -D

Once these updates have completed they need to be compiled for use with Spam Assassin. Also any errors should be spotted here:

root #sa-compile

Unlike clamav, there is no 'freshassassin' and a cronjob is required to do updates. To keep Spam Assassin up to date, a cronjob should be created for the task:

ФАЙЛ /etc/cron.daily/sa-updatesОбновление правил фильтрации Spam Assassin
#!/bin/sh
sa-update --allowplugins --gpgkey D1C035168C1EBC08464946DA258CDB3ABDE9DC10  --channel saupdates.openprotect.com --channel updates.spamassassin.org
sa-compile

Making the cronjob executable ensures it runs regularly:

root #chmod +x /etc/cron.daily/sa-updates

Подключение amavisd к Spam Assassin

Actually, Spam Assassin does not need to be linked to amavisd, it is an integral part of amavisd. Enabling the spamfilter in amavisd does spam filtering:

ФАЙЛ /etc/amavisd.confВключение Spam Assassin
# @bypass_spam_checks_maps  = (1);  # controls running of anti-spam code

With this change, amavisd needs to be restarted:

root #/etc/init.d/amavisd restart

Тестирование

Testing is done again via a client connecting to port 25 that does not have its own spamfilter. For content GTUBE can be used. On that site there is also a suitable mail message in RFC-822 format.

КОД Пример тела спама
XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X

Checking in the testusers inbox, or junkbox more likly, the message can be found and its header examined:

КОД Заголовок спама
Received: from localhost (localhost [127.0.0.1])
    by foo.example.com (Postfix) with ESMTP id B7EDA22F62
    for <testuser@example.com>; Wed, 28 Dec 2011 16:02:46 +0100 (CET)
X-Quarantine-ID: <BukBLFBE3OWo>
X-Virus-Scanned: amavisd-new at example.com
X-Spam-Flag: YES
X-Spam-Score: 1000.907
X-Spam-Level: ****************************************************************
X-Spam-Status: Yes, score=1000.907 required=6.2 tests=[ALL_TRUSTED=-1,
    FH_FROMEML_NOTLD=0.18, GTUBE=1000, MISSING_HEADERS=1.207,
    MISSING_MID=0.14, NO_DNS_FOR_FROM=0.379, TVD_SPACE_RATIO=0.001]
    autolearn=no

Finetuning Amavisd

Amavis has a few more settings that can be changed to do some fine-tuning.

Recipient delimiter

When using postfix with the recipient_delimiter amavisd can be told to make use of this feature:

ФАЙЛ /etc/amavisd.confrecipient_delimiters for amavis
# Delimiter must match the equivalent (final) MTA delimiter setting.
$recipient_delimiter = '+';		# (default is undef, i.e. disabled)

It might be interesting to add the following to /etc/postfix/main.cf, otherwise the user+foo@domain might not be delivered:

ФАЙЛ /etc/postfix/main.cfrecipient_delimiters for postfix
# ADDRESS EXTENSIONS (e.g., user+foo)
#
# The recipient_delimiter parameter specifies the separator between
# user names and address extensions (user+foo). See canonical(5),
# local(8), relocated(5) and virtual(5) for the effects this has on
# aliases, canonical, virtual, relocated and .forward file lookups.
# Basically, the software tries user+foo and .forward+foo before
# trying user and .forward.
#
recipient_delimiter = +

Disperse quarantine

It is possible to disperse the quarantine over several sub-directories. For this directories need to be created first:

root #
mkdir /var/amavis/quarantine/virus;
mkdir /var/amavis/quarantine/spam;
mkdir /var/amavis/quarantine/banned;
mkdir /var/amavis/quarantine/badh;
mkdir /var/amavis/quarantine/clean;
chown -R amavis:amavis /var/amavis/quarantine/*;
chmod -R gu+rwX /var/amavis/quarantine/*;
chmod -R o-rwx /var/amavis/quarantine/*
ФАЙЛ /etc/amavisd.confDisperse quarantine
#$clean_quarantine_method          = 'local:clean/%m';
$virus_quarantine_method          = 'local:virus/%m';
$spam_quarantine_method           = 'local:spam/%m.gz';
$banned_files_quarantine_method   = 'local:banned/%m';
$bad_header_quarantine_method     = 'local:badh/%m';

Also, setting a spam cutoff level helps in reducing stored spam. The cutoff level makes it that no spam is stored above a certain spam-score:

ФАЙЛ /etc/amavisd.confset a cutoff level for spam
$sa_quarantine_cutoff_level = 25; # spam level beyond which quarantine is off
Заметка
25 is the default. This value, aswell as the other levels can and should be tuned to organizational needs.

Spam Delivery

$final_spam_destiny is by default set to D_PASS, meaning that even with a high score at which it gets marked as spam, it is still delivered to the users mailbox. Modern mail-clients, which trust Spam Assassin, can then automatically move it to their SPAM folder.

Bayes database path

Set the bayes_path option in SpamAssassin's configuration file so tools such as sa-learn write to the correct database location:

ФАЙЛ /etc/spamassassin/local.cfset bayes_path to be within Amavisd's home directory
bayes_path /var/amavis/.spamassassin/bayes

Виртуальные хосты

If this server handles more than one domain, telling amavis can help here:

ФАЙЛ /etc/amavisd.confAdd additional aliases to amavis
@local_domains_maps = ( [".$mydomain", "mail.example.net", "mail.example.org", "mail2.example.com"] );

Очистка

With Spam Assassin and ClamAV working as expected, debugging information can be reduced to the normal minimal:

ФАЙЛ /etc/amavisd.confDisable debugging in amavsd
# Section III - Logging
# true (e.g. 1) => syslog;  false (e.g. 0) => logging to file
$do_syslog = 1;                   # (defaults to 0)
 
#NOTE: levels are not strictly observed and are somewhat arbitrary
$log_level = 0;		   # (defaults to 0), -d
 
# Turn on SpamAssassin debugging (output to STDERR, use with 'amavisd debug')
#$sa_debug = '1,all';  # defaults to false
ФАЙЛ /etc/clamd.confОтключение Debug параметра в clamav
# Enable debug messages in libclamav.
# Default: no
#Debug yes