ScrollOut F1 Email Gateway 3

Posted by JD 06/25/2012 at 23:00

I don’t know that email firewall is the right term for the ScrollOut F1 toolset, but that’s what the tool claims. It is definitely good at blocking spam. Most people would call this an inbound email gateway.

Setting it up was pretty easy on a stock Ubuntu 12.04 JeOS virtual machine following the instructions over at HowToForge . Understanding the settings was a little harder. A few minor issues were also solved.

Download

Grab it from the Scrollout Sourceforge Project Page.

Overview

This suite of applications sits between the internet AND your current email server. It doesn’t matter what that email server runs. Exchange, Zimbra, postfix, sendmail, exim, or something else. I didn’t have to change any settings on our email server at all for this suite to work. I did have to change a router setting to forward port 25 to the new virtual machine instead of the old email server, but that was it. I didn’t bother changing any DNS MX records at all.

If anything bad happened, I could just switch back the router port-forward rule and there won’t be any problems.

Further, since this box is now between the internet and our main email server, the backup window for the main server doesn’t force all email to our backup MX server. Messages will be received and stored on the gateway until the main server is available again.

Outbound email does not traverse through this new box, just inbound and only pure SMTP traffic.

Settings

After following the installation instructions – basically, download this tar file and run this script, we are left with a URL, ID and password. Loading all the packages took about 10 minutes and there were a bunch. The base OS wasn’t used for anything else, so I wasn’t worried about any extra programs. Actually, I’d cloned a virtual machine with a 4G HDD meant to be a jumping off point for stuff like this.

There are 2 main settings pages to get correct before we can enable the gateway. Ok, there are really 5 tabs, but only 2 are critical.

Connect

Settings on this tab are:

  • Network Configuration: Auto / Manual
  • Local IP address
  • Subnet Mask
  • Gateway
  • DNS Servers
  • DNS Suffixes Search

Manual network configuration – after all, this is a server for email. Knowing exactly which IP address this server uses is critical.

Local IP address – this is the internal IP for the email gateway box. I had a few issues with the PHP scripts setting the IP to be x.×.×.234 until I found the script and hard coded my desired static IP, x.×.×.62

Subnet mask – standard 255.255.×.x – whatever your LAN uses

Gateway – this is the IP of whatever device routes traffic not on your local LAN. Often it is a router IP.

DNS Servers – x.×.×.200 is common in corporate environments, but for home users, x.×.×.1 is common.

DNS Suffix Search – whatever your domain is – not the hostnames, just the domain.

When I pressed Save and Apply the php took the inputs and updated the static IP for the system, set the /etc/resolve.conf and a few other files. Like I mentioned earlier, it set the static IP to be x.×.×.234 a few times, regardless of what was in the webGUI. I found where that was hard coded in the PHP /var/www/connection.cfg and that stopped the issue. Using a console instead of ssh was necessary, since the IP address was changing with every Apply before I figured out this issue.

I did enter a bug about this IP address problem, but all the bug reports have been removed from SourceForge. The project team prefers email reports, I guess. A new version was pushed to the project page on 6/24, so it is actively under development. I looked inside the current package and saw that 192.168.1.234 is still the default IP address. Be prepared for this when you do the install.

Route

Settings on this tab are:

  • Organization
  • Hostname
  • Domain names
  • Replicate Domains to

These settings are for the way email gets routed to the back-end email servers. What really happens is the /etc/postfix/transport file is updated based on what you type in here. Multiple email domains are supported. I know 3 work.

Organization doesn’t seem to matter.

Hostname is the hostname you want this machine to have. It will be placed into the /etc/hosts file. These changes are destructive to any other hosts in there. There is a file scrolloutf1/www/cfg/hosts that is used to seed this file, so we can add any other hosts for our LAN to here and have those properly added to the /etc/hosts on the gateway.

Domains are in the form of a pair of entries per domainname – the domainname and the IP address for the back-end email server inside brackets. This format is significant to postfix, the MTA used by ScrollOut. It is possible at add more email domains or to remove excess domains that aren’t desired.

jdpfu.com		x.×.×.43
points to the internal email server here.

Replicate Domains to setting isn’t used in our situation.

That’s it. There are tabs for Secure, Collect and Monitor, but they aren’t immediately critical. Setting up your spam-collector later will be something you want.

Just be certain that HTTP port-80 traffic to this email gateway cannot be accessed from outside your LAN. We didn’t change the default password yet. Be certain you do.

Logs

Before I switched the router port forward, a tail -f /var/log/mail.log was started on the email gateway to watch the email come in and see it handled properly. Messages arrived almost immediately after the router change. Most were marked as spam and refused. To ensure that valid email would get through, I used an outside email provider and sent a TEST-FOO message. It worked. Thunderbird received the email almost immediately. This is good. I replied to that message and as expected, it was received on the other side.

Seeing rejected servers in an mail log is a wonderful thing, especially from countries with which we do not normally correspond.

It is also nice to take down the main email server in the middle of the day without worry that emails will be dropped. Holding email is definitely working.

I’m not certain all the greylists are working as I like, yet. A few domains needed to be added so those emails were not rejected. A few others, I think we will just reject going forward.

Issues

There were a few issues during the installation. Creating software to handle every possible server configuration is hard. Here’s just a list of the things I didn’t think were quite correct:

  • Setup ignored the already configured IP address and used one that didn’t seem connected to anything .234.
  • Setup overwrote the existing /etc/hosts file.
  • Setup overwrote the existing /etc/hostname file.
  • Setup overwrote the existing /etc/mailname file.
  • Postfix .db files were soon older than the source files. This happened for sender, virtual, relay_recipients, and transport files. Each was manually updated with a postmap run.

Each of these caused issues for me that had to be sorted.

Some of these issues have to be my fault. The setup program is pretty clear that it will take over the entire server, so my expectations may have been off. There isn’t a README or INSTALL file included with the package, just a install.sh that we need to trust.

Domains like buy.com, LinkedIn and a few paid services were added to the allowed domains, since all their email was being blocked. Some domains run by a few friends and email lists had to be added too.

Conclusion

Setting up this gateway was pretty painless for all the capabilities that it provides. I’m happy we did it and generally happy with the results. It is clear that some legitimate email systems behave in a spammer-like way and will be blocked unless I do something. I’ll need to watch for these bounced messages for a while. I did relax some of the rejection rules on the gateway, since our full email system is still running AV and AS systems too.

  1. Marius Gologan 07/21/2012 at 16:53

    Hello,

    The bugs identified and highlighted here are the most useful and some of them have been revised.

    I did not find a contact on this website and wanted to thank you for this nice article, the time and effort invested in escalating everything from the bottom to the top.

    Regards,
    Marius Gologan.

  2. JD 07/21/2012 at 18:37

    Hi @Marius,

    Thanks for stopping by and leaving a comment. FWIW, I did attempt to provide bug reports at the sourceforge support page and I thought that I’d sent an email to your gmail account too. Perhaps not. No bugs are showing up on the project page anymore.

    BTW, you (and anyone else) can probably guess a valid email address for me on this domain if they’d like to talk privately.

    I’m very willing to help this project and feel free to grab the text from this article as a starting point for the README.txt or other document to be included with the distro.

    Some of the screens could do with additional explanation. For example, the Security Levels. I can’t tell what any of those actually do based on the current information in the GUI. A detailed README really would be helpful.

    Documentation is always the last thing, for a good reason. Let me know if I can help further.

    I have seen issues with postgrey not working as expected. The whitelist_recipients file doesn’t appear to work at all, while the whitelist_clients does work, but lacks clear examples for commonly desired whitelists. These are hardly issues for Scrollout, but users that choose Scrollout instead of using the other tools directly will need a little more help than average.

    I did see another ScrollOut article over at HowToForge this week, but haven’t looked at it in detail to see what’s new of if an Upgrade method has been provided for the tool.

    I’m not a python guy, so I probably won’t be providing any patches, I’ll try to provide feedback …

    Thanks again for this tool and stopping by and lots of thanks for setting up the Scrollout blog.

  3. Marius 07/21/2012 at 23:17

    Hello John,

    I noticed you mentioned, in your article, the issue with the reported bugs on the forum. I’m not sure, but maybe you posted on “Get Help”, not on “Bugs”. The forum was inactive for 1-2 weeks during June, nothing was deleted. In a challenging way, objective criticism and complains should always be constructive.

    Documentation, instructions on the gui, fixing manual whitelist for postgrey (btw: currently relays on auto-whitelist), manual updates and rollback, backup and restore are all on the todo list. But first things first: lets trick some tricky-spam and fish some phishing. :)

    Currently I’m focusing on a method of catching junk using MD5 signatures, in parallel to clamav. I didn’t test its efficiency yet.

    As you also suggested, I will update the (exaggerated) minimum requirements left from the initial open virtual appliance file that was published in the beginning. That was a 20GB thin provisioned disc space covering the time-frame when the server is down – highlighted very well in your article.

    This project (or any other) should be connected fast, set on an optimum level and give results. Otherwise, simply put, will be a failure like other commercial or non-commercial attempts that take money, training, support and time from our lives.

    Again, thank you.
    I look forward to get it done in a complete and simple form very soon.

    PS: English is not my first language.

    Marius.