• Print

Junk E-mail (spam) Filtering (Advanced)

This documentation has been written for technical support staff, the technically advanced and the curious. Those looking for an overview and basic instructions, please visit AntiSpam.

Here's How It Works -- In Detail

Anti-spam filtering is a two step process. Each incoming message is given a "spam score" and then either placed in your Inbox or placed in your junk-mail folder depending on the spam score and the customer set risk level.

	When messages arrive at the University of Toronto UTORmail postoffice, they are scanned with a system called PureMessage and given a spam score ranging between 0 (probably not spam) and 100 (almost certain to be spam). The spam score is added to the message in a special line in the message header. This line is usually not shown, but with most email programs you can see it if you want to.
	The message then goes to the email server.
	If you have spam filtering turned OFF, the message goes to your Inbox.If you have spam filtering turned ON, any message with a sufficiently high spam score (defined by your chosen risk level) is placed in a special email folder named junk-mail.Any messages in junk-mail older than seven days are regularly DELETED.

Identifying Junk Email (spam)

PureMessage looks for many message characteristics common to spam including certain phrases or words; inconsistencies or errors in the message header; use of color, bolding, or capitalization; the use of certain routes to deliver the message; whether the message already appears in a large catalogue of known spam, etc.

The scoring system is regularly updated by Pure Message to improve its effectiveness and deal with changes in the characteristics of spam over time.

Many aspects of the message are considered in coming up with the final spam score. In practice, the scoring system has been found to be quite effective at identifying what most people consider to be spam.

Still, just because a message looks like spam to PureMessage does not mean that it is spam. You may choose to receive marketing email messages from a company you purchased a new computer or a book from, for example. Also, some responsible organizations you may want to hear from use the same email distribution programs as spammers or might like formatting their messages in spam-like ways.

To be sure that you do not lose messages you want, it is important to regularly check your junk-mail folder. Any message in your junk-mail folder that is more than seven days old will be regularly deleted!

Viewing the spam score of a message

When PureMessage decides a message looks like spam, it adds a line to the message header. With most email programs, this small change will not be evident when you view the message because they usually show only a few header lines, like Date:, To:, From: and Subject:.

If you would like to see the header line containing the spam score, you'll need to have your email client program display the full details of the message headers. How that is done depends on which email client you use.

Here is an example of what you might find:

X-PMX-Spam: Gauge=XXXXXXIIIIII, Probability=66%, Report="MAILTO_TO_SPAM_ADDR, NO_MX_FOR_FROM, ONLY_COST, RAZOR2_CHECK, SPAM_PHRASE_02_03, SUPERLONG_LINE"

The spam score is indicated by the "Probability=66%".

The "Report" attribute lists keywords for the message characteristics that determined the spam score. Some characteristics are good (characteristic of messages that are not spam) and some are bad (common to messages people consider to be spam). The score is an expression of the aggregate of all the relevant characteristics.

For Technical Support Staff

The risk levels work as follows:

For Technical Support Staff or Technically Advanced Customers

Client Side Filtering

Customers with special filtering needs (e.g. someone who wants all spam with the word "mortgage" to nevertheless go to their Inbox, but doesn't want to see other spam), can turn off server-side filtering, and create their own filter rules for the "X-PMX-Spam" header in the email client software (e.g. Netscape Messenger, Netscape Mail, Pine, etc.) This is called "client-side filtering". Doing this is more complex and less efficient. We expect that only support staff or technically advanced customers will be interested in doing this.

To do client side filtering, you need to create a rule that looks for X-PMX-Spam: Gauge=XXXXXX. We recommend you use six "X", which will match a probability of 60% or more. More spam will be caught if you use five "X" (matching 50% or higher), but the rate of "false positives" is then higher.

Please refer to the client side filtering page for specific instructions on configuring the following email clients:

Supported: Netscape Messenger 4.79, Netscape Mail 7.x, Outlook Express

Unsupported: Eudora, Mulberry, Outlook XP

Acknowledgements

We wish to thank the University of Washington and the University of Ohio for sharing their anti-spam documentation. We also appreciate the technical expertise that the Unversity of Washington shared with us.