FrontlineMail is a network-based content filter designed to help businesses and other domain administrators deal with the growing problem of spam and viruses. FrontlineMail works by intercepting unfiltered Internet e-mail via SMTP (Simple Mail Transfer Protocol), passing the e-mail through a number of local and external tests to determine whether the message is spam or a virus, and then delivering filtered, spam-free and virus-free e-mail via SMTP to a recipient mail server. Identified spam and viruses can either be delivered to the recipient mail server for further processing or they can be quarantined. Because the entire transaction is done via SMTP, which all mail servers must support to send/receive Internet e-mail, FrontlineMail supports all commercial and non-commercial mail server software and does not require any end-user or mail server configuration. The filtering can be enabled with a simple DNS (domain name service) MX record change. Once messages are properly identified, the customer has a number of choices regarding message handling and delivery. For example, identified viruses can be quarantined into a special quarantine account (residing on or off the customer's network) and spam can be automatically filtered to a junk or bulk folder for end-user review. The system supports a variety of configurations based on customer needs and requirements, including the distinct handling of spam and viruses and the ability to forward quarantined messages to one or more destinations on a per domain or per user basis.
Filtering Methodology The filtering mechanism of FrontlineMail relies on the SMTP protocol for accepting unfiltered Internet e-mail and delivering filtered, properly classified e-mail to a recipient mail server. In the absence of filtering, Internet e-mail works by having the sending mail server identify the valid mail server for a domain and then transferring messages to that mail server using the SMTP protocol. The valid mail server is identified using the DNS MX record for the domain. Figure 1 shows how e-mail works without filtering:
Figure 1. With FrontlineMail filtering, the basic model of Internet e-mail does not change. The difference is that a server dedicated to content filtering is inserted between the sending server and the receiving server. Figure 2 shows how this works.
Figure 2. The content filter communicates with both the sending mail server and the receiving mail server via SMTP, ensuring seamless operation in a variety of mail server setups. The insertion of the filtering server between the sending and the receiving mail server as outlined above is accomplished using DNS MX records. Normally, the receiving mail server is identified as the mail exchanger (MX) for a domain in DNS. This MX record might look like the following:
domain.com. IN MX 10 mail.domain.com.
This record says that for domain.com, all e-mail should be sent to the host mail.domain.com. To enable FrontlineMail filtering for this domain, all one would need to do is replace the host mail.domain.com with the hostname of the filtering server. The modified MX record might look like this:
domain.com. IN MX 10 mx1.aspextra.net.
Now, mx1.aspextra.net has become the valid mail server for this domain. All e-mail for domain.com will now be sent to the filtering server. Once the filtering server has completed its work, mail is tranferred directly to the receiving mail server via SMTP.
Tests Performed
The FrontlineMail filtering server performs a number of tests to determine whether the message is spam, a virus or a good e-mail. These tests are performed in real-time and do not delay the transmission of e-mails beyond a few milliseconds. The following diagram provides a general overview of these tests:
Figure 3. Connection validation Connecting mail servers are first checked to determine whether they are valid sources of Internet e-mail. These checks include such things as mail host spoofing (improper HELO responses), relay authorization, return address validation and RFC compliance. Rejects known spammers This step looks at the IP address and sender domain of the connecting mail server to determine whether the sender is a known spammer, open relay or other recognized source of spam and viruses. This step is optional though it is highly recommended to reduce load on the recipient mail server and minimize the impact of dictionary attacks.
Virus Scan
The message is scanned for viruses. Virus definitions are updated every hour, ensuring that new viruses are identified quickly and without user intervention. If the message is deemed to contain a virus, the message can either be deleted or quarantined at the user's discretion.
Spam Filter
Check whitelists User specified whitelists will bypass spam filtering.
Heuristics Tests
Heuristics tests look for various characteristics commonly found in spam messages. For example, multiple instances of the word “Viagra”, heavy use of html and remote-loading images increase the likelihood that the message is spam. The results of these tests are scored and added to the aggregate spam score for the message.
Bayes Probability Analysis The Bayes filter uses probability analysis to determine whether messages are likely to be spam or not spam. By looking at the contents of identified spam messages and identified good messages, the Bayes filter keeps track of various identifiers or “tokens” found in each type of message. The presence of these tokens (both good and bad) are used to calculate the probability that the message is spam or not spam.
Checksum Tests
The checksum or “fingerprint” tests try to identify spam messages by calculating a unique checksum for various parts of the message, i.e. certain headers and/or the body, and looking up that calculated checksum in an online database of known spam. Since checksums for digital data are generally unique (like a fingerprint), it can be used very effectively to identify messages that have been previously determined to be spam. For example, if a spammer is sending a particular spam message to 10,000,000 recipients, it may take some time for the message to be delivered to all recipients. If after the 10,000th delivery, the checksum of the message is added to the checksum database as spam, the remaining recipients can reference the database and reject the message.
RBL/RHSBL Tests
RBL stands for Real-time BlockList and RHSBL stands for Right-Hand Side BlockList. RBLs are databases maintained by various organizations that list IP addresses of known spammers, open relays, open proxies, compromised systems and other sources of spam. RHSBLs do the same thing but instead of using IP addresses, they use domain names commonly associated with spam. The “right-hand side” refers to the right-hand side of the email address of the envelope sender. Messages are not blocked outright when they are found on one of these lists; rather the overall spam score of the message is adjusted upwards to reflect the greater likelihood of the message being spam.
Calculate aggregate spam score
The results of the four test types outlined above are used to determine an overall spam score for the message. If the message exceeds the threshold score for spam, it is determined to be spam. Messages that score below the threshold are determined to be good e-mail.
Tagging
All filtered messages are tagged to help mail administrators determine whether the message has been filtered or not. This tagging is done by inserting special headers into the source of the e-mail. These headers are not visible normally and can only be seen by viewing the source code of the message. By tagging filtered messages, mail administrators have a great deal of flexibility in managing the filtered mail once it is received at the receiving mail server.
All filtered messages receive the following header:
X-Virus-Scanned: by aspextra.net
This header confirms that the message has been filtered by ASPextra.net. In some cases, spammers will try to bypass the filtering server and send mail directly to the receiving mail server. In such a scenario, the mail administrator can setup a rule to reject all messages that don't contain this header, which means unfiltered mail will be rejected immediately. Legitimate mail servers would never bypass DNS to send mail directly to a particular mail server or ip address so the likelihood of blocking real mail with this rule is close to nil.
Identified spam messages receive additional headers. Here's an example of the headers inserted into a spam message:
The “X-Spam-Status” header shows how the message scored and lists the various tests that contributed to its spam score.
The “X-Spam-Flag: YES” header is only inserted into messages that are determined to be spam. Messages are determined to be spam when the spam score for the message exceeds a pre-determined threshold score. These headers can be used effectively for additional filtering on the receiving mail server. For example, some mail administrators may want to provide their users with Bulk or Junk Mail folders in their individual e-mail accounts. Such administrators can easily setup a server-side filtering rule that moves messages with the “X-Spam-Flag: YES” header to a user's Bulk folder.
Quarantine Options
FrontlineMail supports a number of quarantine options for identified spam and viruses.
Viruses Identified viruses are by default deleted by the system. This removes the threat of most e-mail-borne viruses from entering a company's network. For businesses that would prefer to have identified viruses quarantined, FrontlineMail can optionally send viruses to an e-mail account of the customer's choosing (i.e., viruscheck@customerdomain.com). This quarantine account can reside on the customer's mail server or ASPextra.net can provide a secure quarantine account on behalf of the customer. In most cases, we do not recommend quarantining viruses as certain virus breakouts (such as MyDoom or Klez) can adversely impact the availability of the receiving mail server, especially for large domains. Spam FrontlineMail supports a number of different scenarios for handling identified spam.
No Quarantine
With this model, all mail is passed to the receiving mail server and there is no special handling of messages identified as spam. Identified spam is still tagged as such (via the insertion of headers into the source of the message) but the FrontlineMail service does not change the destination of these messages. This model provides the most flexibility for domain administrators as all mail (except viruses) is delivered to the receiving mail server. Once messages are received, it is up to the domain administrator to determine the fate of identified spam.
Most mail servers support the filtering of messages based on the presence of various headers in the source of the message. For example, most mbox-based mail systems support the special handling of messages via a tool called Procmail. The mail administrator can setup a set of Procmail rules to filter incoming messages with the “X-Spam-Flag: YES” header to a user's Bulk folder (a mail folder in the user's account specifically created to store spam). Other mail servers, such as Cyrus IMAP, support server-side filtering using Sieve rules. Sieve rules work similarly to client-side message filters, allowing end-users to filter incoming mail based on particular message characteristics. Sieve rules can also be used to filter incoming messages into users Bulk folders. If the mail server does not support server-side message filters, end-users can setup their own filtering rules from within their e-mail application. Two of the most popular e-mail clients in use today, Outlook and Mozilla, both support client-side message filters based on headers that appear in the source of the e-mail. These mail clients can be easily setup to filter identified spam to a folder of the user's choice, i.e. Junk.
Central Quarantine
Some domains may prefer to centralize identified spam messages into a central quarantine account that resides either on the receiving mail server or offsite on a third-party mail system, i.e. a quarantine account hosted by ASPextra.net. Centralization of the quarantine account removes the burden of reviewing spam messages away from the end-user and into the hands of the IT department. In our experience, companies that have IT departments seem to prefer this model as it allows greater control of what employees see and read in their e-mail accounts.
With this model, the FrontlineMail filtering server redirects all identified spam for a domain to a particular e-mail account, i.e. spamcheck@customerdomain.com. This account can be a regular e-mail account on the receiving mail server or if the customer chooses, it can be hosted on behalf of the customer on an ASPextra.net server.
User-based Quarantine
Another quarantine option is to have user-based quarantine accounts. This model allows domain administrators to setup separate quarantine accounts for each user. These accounts can be regular e-mail accounts residing on the receiving mail server or offsite. For example, identified spam for the user 'john@domain.com' might be redirected to a separate account called 'john-spam@domain.com.'
Security Considerations
FrontlineMail has been designed to be a secure extension of a company's network. The contents of e-mail represent company confidential information and should be managed with the same care as any company data asset. FrontlineMail supports TLS (SSL) encrypted connections throughout the SMTP transaction. This encryption helps protect your e-mail from being intercepted by would-be hackers.
Figure 4.
For organizations that require the utmost security in message delivery, we also support SSL-key based authentication between the FrontlineMail filtering server and the receiving mail server.
ASPextra.net's privacy policy also ensures that a customer's data is always theirs and will never be subject to review by any third-parties without the customer's explicit permission. Unless specifically requested by the customer, FrontlineMail does not store the contents of e-mail at all. The filter is a simple pass-through that maintains the integrity of your communications infrastructure. ASPextra.net does log certain SMTP transaction information to help troubleshoot problems in mail delivery should they arise but this information is only kept for 7 days and we do not archive this information. ASPextra.net's log files are also subject to our privacy policy and will not be used by ASPextra.net for any purpose other than to provide service to its customers.
Redundancy and Failover
FrontlineMail has been architected for maximize performance, redundancy and failover protection. All filtering servers are mirrored on multiple servers for optimal performance and load balancing. In the event that one server goes down, another server will quickly take its place. This provides uninterrupted mail service for all customer domains. If a customer's mail server goes down, we queue all mail for that customer until the mail server is brought back online, providing additional failover protection for a company's e-mail infrastructure. We also maintain backup mail relays on separate distinct networks in the event that our primary network goes offline. Ed Buck is a founder & principal with ASPextra.net. Ed also founded and manages www.selfhelplinux.com & www.oooforum.org.
Related Articles
Accepting Payments Online If you want to sell online, you need to be able to accept credit card payments. The traditional way... Written by: Mario Sanchez
Hosting
Control Panel Benefits Control Panel Benefits A dedicated server is only as good as the software used to manage it.... Written by: AskWebHosting
Hosting
Control Panel Explained and Test Every Web host provides you with one, and if you do a lot of work on your Web site it could be... Written by: Mitch Keeler
Hosting
Control Panel What is it My Hosting Provider Offer Control Panel, What Is It? A control panel is additional software,... Written by: AskWebHosting
Hosting
Domain Names Explained So you want a domain name... You want your very own "yourcompany.com" Internet identity. You want... Written by: Mario Sanchez
Hosting
Effectiveness of Web Hosting Directories John is very happy today, for the first time in his life he has created his very own website. Now... Written by: Priyanka Agarwal
Hosting
FrontlineMail Spam and Virus Gateway OverviewFrontlineMail is a network-based content filter designed to help businesses and other... Written by: Ed Buck
Hosting
Gain More Web Site Control With htaccess There are many tools of the trade in the world of Web site development, but very few have as many... Written by: Mitch Keeler
Ed Buck is a founder & principal with ASPextra.net. Ed also founded and manages www.selfhelplinux.com & www.oooforum.org.
