Virtual Mail with Ubuntu, Postfix, Dovecot and ViMbAdmin

As part of pushing our new release of ViMbAdmin, I wrote up a mini how-to for setting up a virtual email system on Ubuntu where the components are:

  • Postfix as the SMTP engine;
  • Dovecot for IMAP. POP3, Sieve and LMTP;
  • ViMbAdmin as the domain / mailbox / alias management system via web interface.

It supports a number of features including mailbox archival and deletion, quota support and display of mailbox sizes (as well as per domain totals).

Find the how-to at:

Adventures with LDAP (OpenLDAP) – SSL, Multi-Master Replication and Monitoring

In my career to date, I successfully managed to avoid all but the periphery engagement in OpenLDAP. Until recently that is – we had to build a Microsoft Exchange like environment with open source software in a way that was closely integrated and easily managed. But, more on that another time. For anyone else diving into OpenLDAP, here are some articles on my experiences that I have penned:

ViMbAdmin :: New Release 2.0.6

Today, we’re pleased to announce the immediate availability of 2.0.6 which has a number of incremental fixes and improvements.

Just over a week ago, we released V2 of ViMbAdmin which was a complete UI refresh.

Thanks for all the feedback and bug reports since then.

Today, we’re pleased to announce the immediate availability of 2.0.6 which has a number of incremental fixes and improvements including:

  • Domain is now ‘sticky’ when moving between mailboxes, aliases and logs making it much easier to browse a single domain;
  • A cookie is now used to remember the page length for individual users;
  • We now use grouped icons with tooltips rather than labelled buttons throughout;
  • The horrible your IP address has changed message is gone.

As usual, a full change log is available here and the packaged release can be downloaded directly here.

Introducing ViMbAdmin – Virtual Mailbox Administration

Open Solutions are pleased to announce the immediate availability of our latest free and open source web application, ViMbAdmin, a web based interface which will allow you to manage mailboxes, virtual domains and aliases.

Open Solutions are pleased to announce the immediate availability of our latest free and open source web application, ViMbAdmin (vim-be-admin). ViMbAdmin is a web based interface which will allow you to manage mailboxes, virtual domains and aliases.

ViMbAdmin is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License version 3, or (at your option) any later version.

ViMbAdmin was entirely funded by Open Solutions and developed by our staff. If you find this application of value, please consider making a donation to our chosen charity.

Do you want to see it in action? We have a live demo which you can access here. You can also browse screenshots by clicking the image on this page.

ViMbAdmin was written in PHP using our own web application framework which includes the Zend Framework, the Doctrine ORM and the Smarty templating system with JQuery on the frontend.

ViMbAdmin is hosted on its own Google Code project page where you can find documentation, browse the source code and access our Subversion repository. We have set up a Google Groups discussion group and you can read our ViMbAdmin blog posts.

ViMbAdmin can work as a slot in replacement for Postfix Admin with a few MySQL ALTER statements.

Features

  • Super admin(s) user level with full access;
  • Admin(s) user level with access only to assigned domains and their mailboxes and aliases;
  • Super admins can create and modify super admins and admins;
  • JQuery Datatable throughout for quick in browser searching and pagination;
  • Create, modify and purge domains including limited the number of mailboxes and aliases a non-super admin can create per-domain;
  • Activate / deactivate admins, domains, mailboxes and aliases at the click of a button;
  • Full logging;
  • Facility for users (mailbox owners) to change their password;
  • Forgotten Password / Password Reset function for admins;
  • Very configurable including:
    • set default values for quotas, number of mailboxes and aliases for domain creation;
    • templated welcome and settings email for users;
    • either plain or MD5 mailbox password support;

 

Combatting SPAM in ISP Mail Servers

Fighting SPAM is not easy. Here’s an example of how we configured an ISP mail server cluster to reduce the SPAM load…

Introduction

ISPs must provide a mail relay service to their end users. However, mail relay isn’t a revenue generating service and ISPs can’t allocate unlimited staff resources to these services. The problem administrators face is trying to find a balance between unobstructively allowing their clients to send legitimate mail but to stop them from spamming.

The main source of SPAM from clients (SMTP clients in the ISPs IP space) is typically computers infected with bots / malware behind the users router. It may be their own or even a neighbor taking advantage of an unsecured wireless network. There is a smaller secondary source als0 – spammers who have someone gotten hold of a customers SMTP authentication details and are logging in from outside of the ISPs IP space to relay mail through.

Now, the problem this causes is that when the ISP mail servers pass on this SPAM to its destination such as a Yahoo! or GMail recipient, these companies examine the ratio of SPAM to HAM coming from the ISPs servers. When this ratio goes the wrong way, they legitimately block the mail servers meaning that customers of the ISP cannot get their mail delivered to these destinations. This is a big problem.

In this article, I’ll describe a new mail cluster set-up for an ISP to try and ensure their SPAM:HAM ratio stays on the right side.

Solution  Overview

The mail cluster comprises a number of SMTP servers all configured identically (in fact all booting from the same image using PXE and an NFS root file system with dedicated local /var partitions). They are running Debian Stable (Lenny at the time of writing) and the mail server is the excellent Postfix.

Postfix is configured to accept authentication using SASL with Dovecot (configured only for authentication, no POP3 or IMAP services) over TLS on port 25 or SSL on port 465. Users on the ISP’s address space do not need to authenticate to relay mail. This is actually a legacy configuration and quite common among ISPs. If we were to go back many years starting all over again, I’d certainly try and evaluate the merits of only allowing relay after authentication irrespective of your ISP.

Mail server server selection is made via DNS round robin but this could easily be changed to a load balancer. As part of this retrofit, we took the opportunity to IPv6 enable all the servers.

We now need to look at securing these boxes to ensure the level of SPAM is controller. There are three points at which we examine incoming mail:

  1. before it hits the SMTP daemon at all;
  2. after the SMTP headers (MAIL FROM, RCPT TO) have been passed to the daemon but before the mail has been accepted (i.e. before the SMTP DATA command);
  3. after the daemon has accepted the mail.

We’ll examine those in each of the sections below.

Rate Limiting Connections

In order to limit repeated connections to the mail server – such as from a SPAM bot that creates a new SMTP session for every mail (rather than sending many mails in one session), we use iptables recent match module.

This module keeps count of the number of connections in a given time period allow us to take alternative action if the number is above a threshold. In our case, we just drop additional connections until the number of connections drops below that threshold. For example:

-A INPUT -p tcp -m state --state NEW -m multiport
        --dports 25,465 -j SMTP_IN

-A SMTP_IN -p tcp --syn -m recent --name SMTP_RATE_LIMIT --set
-A SMTP_IN -p tcp --syn -m recent --name SMTP_RATE_LIMIT --update
        --seconds 60 ! --hitcount 10 -j SMTP_IN_LIMIT
-A SMTP_IN -j ACCEPT

-A SMTP_IN_LIMIT -j LOG --log-prefix "SMTP_IN: LIMIT EXCEEDED:: "
-A SMTP_IN_LIMIT -j DROP

We have some additional rules where, for example, we have whitelisted core infrastructure IP addresses and also mail from non ISP IP addresses (the purpose here is to stop SPAM bots from within the network).

Checks Before Accepting the Mail

Once the client connects to Postfix we check a number of other items.

We of course use some of Postfix’s own policy checks (such as reject_non_fqdn_sender, reject_non_fqdn_recipient, etc) but that’s pretty par for the course and not discussed below.

The ISP uses a commercial subscription to the Spamhaus data feed service. This is a very reliable database of IP addresses that are known to be sending SPAM (and other classifications). This is the first thing we check – customer or not. If you’re on this list, we do not accept mail from you.

If you’re not coming from the ISP address space, we then initiate gray listing using a Debian package called postfix-gld (see this site) which uses a MySQL database as its storage engine. See greylisting.org for more details on this useful anti-SPAM measure.

Now, if you’ve gotten this far, we use a package called postfix-policyd (see this site) to initiate rate limiting. Clients are limited in an hourly window on how many mails they can send and on how many recipients in total these mails can reach. The exact numbers are a moving goal post for now while we tweak it.

We could of course have used policyd for gray listing also but this posed a significant problem. Our Postfix configuration looks like this:

smtpd_recipient_restrictions =
    ...
    check_policy_service inet:127.0.0.1:10031,
    permit_mynetworks,
    permit_sasl_authenticated,
    check_policy_service inet:127.0.0.1:2525
    ...

As you can see, the issue is that we do not want to gray list our own IP range or people with a valid username and password but we do want to rate limit these. Using policyd for both is not possible unfortunately.

At this point, your mail will be accepted for relay. We not start the next batch of checks.

Content Filtering

We now need to examine these mails to discard SPAM and mails infected with a virus. For this we use the excellent amavisd-new package with SpamAssassin and ClamAV installed on the servers.

We’re still tweaking the rules for best effect but as it stands, if a virus is detected in the mail it’s silently discarded. A mail with a reasonable SPAM score is marked as SPAM in the subject but mails with a high score are also just discarded. DSNs are not sent.

If a mail passes all these checks, it’ll be relayed onwards.

Fighting SPAM is far from easy but the above will make a big dent. Unfortunately we can’t stop just there. We also have scripts to analyse the logs and rate limiting database to pull out the bad guys so the ISP’s tech support team can chase these up retroactively and instruct them on protecting their machines.