recfilter

• Introduction
• Download
• Usage
• Options
• Workflow

Introduction

recfilter is a somewhat special purpose mail filter. I designed it to allow me to automatically recognize mail that originated outside my local network because my email address is only used within the company and 99.9% of all mail originating outside the company network is spam.

As new mail shows up, this filter reads the Received: headers, looking for the one where the company email server initially gets the mail from somewhere outside the company. If it recognizes an evil mail source it adds an X-External-Spam: header showing the IP address of the external mail source.

It keeps a database of the IPs it recognized, and looks for new IPs coming from that same IP range (I noticed that spam often comes in bursts from IP N.N.N.1 then N.N.N.2, etc.)

Download

You can download the C++ source for the tool here:

recfilter.tar.gz

The code depends on the sqlite libraries, which are normally available in most linux distro repos.

Usage

recfilter [options...] < infile > outfile

The default mode of operation for recfilter is to act as a filter, reading an email message on stdin and writing a modified message to stdout which may have an X-External-Spam header added with the IP address of the source of the external mail.

The options described below can be used to change this mode of operation or run various database maintenance operations.

Options

--help Print a summary of the options and exit.

--database=file Name the database file to use for any operations. The default file is $HOME/.recfilter. The database is created if it does not already exist.

--serverpat=wildcard Specify a pattern recfilter can use to recognize the Received: header that records where the mail entering your local mail system came from. This always starts with the string "by " (since the headers is saying the mail cam from somewhere and was received by your server. Examples might be "by mail.example.com" or "by server*.example.com" if there are lots of possible mail servers named server01.example.com, server02.example.com. etc.

--delserver=wildcard Remove a wildcard previously added by serverpat.

--delip=ipaddr Remove an IP address which recfilter automatically added to the database. You may decide it made a mistake treating that source as spam, and you can remove the individual IP entries.

--clean=days Remove any IP address entries not seen for this many days (designed to be used in a cron job to keep the database from filling up). Spammers often move around a lot so spam IPs can become un-used.

--filter This is just an option that can be explicitly given to make recfilter run as a filter, reading a mail message on stdin and writing it to stdout after any database updates to add IP addresses and add the X-External-Spam: header to the message. This is the default if no options are given.

--list List the contents of the database in human readable form. If you want to delete entries from the database, this listing will tell you what is there to be deleted.

--addwhite=wildcard If perfectly legitimate mail originates from many different systems inside your local network, you can use this to whitelist IP addresses which might otherwise be treated as spam. An example might be "192.168.0.*". If a matching IP shows up as the "from" address in a Received: header matching a serverpat, then that mail will not be marked as external spam.

--delwhite=wildcard Delete a whitelist entry previously added by addwhite.

--help Print a summary of the usage and options.

For all the options above that take an argument, you can give the string "help" as an argument and get an expanded description of that argument.

Workflow

You can include recfilter in a series of mail filters specified in a ~/.forward file (as one example), or if you are using fetchmail, you can use the mda option to filter incoming mail.

The trollfilter tool includes the source for dannyboy which is a helpful tool to pipe many filters from one to the next.

I run dovecot and postfix on my machine, so the final filter in the list is deliver which winds up adding the mail to my dovecot IMAP server.

The sieve filter provided by the dovecot pigeonhole project now has the X-External-Spam: header available in the mail as it arrives, so a ~/.dovecot.sieve filter like this will sort the troll messages into a special inbox:

require "fileinto";
if exists "X-External-Spam" {
   fileinto "INBOX.externalspam";
   stop;
}

I have lots more filter rules than that, but this is an example of how I use recfilter.

(By the way, this is the main reason I run my own IMAP server - to get consistent mail client independent mail filtering done by the server before the client even sees it. If I want to change clients or I have to use a different client on my android phone I still get a consistent view of my mail. By consolidating all the mail from my different mail accounts on the same IMAP server, I also get consistent filtering rules regardless of where the mail came from.)