Email Security and Spam Reduction

Introduction

SMTP email is one of the 3 most heavily used protocols over the Internet, the others are HTTP and BitTorrent. The theoretical purpose of SMTP is to enable anyone to be able to send an email message which will be read by anyone else. The reality is somewhat different.

Other than securing email programs from technical attacks the other purpose of email security is to prevent misuse of email which occurs outside certain defined boundaries. For example there is no point in Alice sending Bob an email in a language Bob does not understand. Alice's effort in sending it and Bob's in deleting it unread is wasted and counterproductive. The same applies if Alice sends a message to Bob that Bob is not interested in. For occasional one to one emails this isn't a problem, but if Alice is sending out very many emails to recipients who are not interested in reading these this activity becomes a significant problem.

When automated and distributed means are used to send out unwanted emails to very many people the total wasted time becomes a significant resource. Someone who lives for 80 years has a lifetime of 2,524,608,000 seconds. If it takes just 2 seconds to identify and delete each unwanted email, sending out 631,152,000 unwanted emails (less than the number of people who now use the Internet) in one sense is equivalent to killing someone who had a life expectancy of another 40 years.

Technical attacks against message user agents (MUAs)

An MUA is a program you use directly to read and send email. Examples include Evolution, Mutt, Outlook Express and Mozilla Thunderbird. For webmail users the MUA is the interaction between the web browser when connected to the webmail host and the web application.

Malware affecting MUAs directly

If the MUA executes programs contained in the email directly these programs are very likely to be malicious. Poor security results from mixing of data (messages sent and received) and executable content and any confusion or ambiguity between these. A number of email worms have targeted Outlook which was found to be scriptable to give active content access to its address book, enabling worms to propagate themselves to everyone in the infected MUA's address book.

This also provided a replication mechanism for programs able to spread both as worms and as viruses - by infecting other executables as well as by using email transport. At one time, versions of MUAs were capable of automatically executing content on opening - the user did not need to click on an attachment to exectute it, but this extreme kind of insecurity is now less likely. Having an extra dialog when a user clicks on an executable attachment to make sure they are aware it is executable is now more common.

A number of vulnerabilities, e.g. cross-site scripting arise from the expectation that the MUA directly renders HTML, automatically running any Javascript embedded within the HTML document.

Web bugs and remotely monitored reading of email

Microsoft Outlook has a feature enabling the person sending the email to request that an automatic receipt be returned when the person receiving the email opens it. This feature is considered by some to involve an invasion of their privacy. In connection with conventional mail, recorded delivery involves extra expense for the sender and so is not used very often. Delivering a legal summons requires a bailiff who will identify the recipient on giving them the summons. The bailiff is therefore able to testify that someone answering to the name of the recipient, or otherwise identified as them has received the summons. This is expensive and occurs rarely in practice, but the law requires this delivery method when the recipient of a message has a legal obligation to respond to it, or when important property interests are concerned. The issue is that under normal circumstances the recipient of a message is not obliged immediately to let the sender know that they have received it and drop other priorities in order to respond.

Technical attacks against email clients include the "web bug", e.g. taking the form of an invisible pixel image loaded from a remote website, where the image URL contains a cookie or randomly generated number which keys the identity of the email that was sent. Notify.com advertise their ability to exploit this vulnerability. If you are responsible for email administration you might consider blacklisting email coming from or relayed through the servers of companies that advertise intrusion services of this kind.

Defeating this privacy attack either requires use of a indirectly HTML capable MUA (e.g. Mutt where the user has to choose to open the HTML attachment in a separate web browser), or a directly HTML capable MUA which does not automatically open remote linked content.

Evolution configured not to open web bugs screenshot

If you want to be very confident that your act of reading a message is not monitored remotely, your safest choice is probably to use a plain text email client e.g. Mutt or Pine to avoid reading email in HTML altogether, unless you know that your HTML MUA avoids all loading of linked HTML components, frames, attachments or cookies without user authorisation. For more information see the Wikipedia web bug article.

Role of Message Transport Agents (MTAs)

Before discussing technical attacks intended to compromise this type of email relay program and the spam problem, let's consider what MTAs do.

Email is sent from a MUA to an outgoing MTA relay using the SMTP protocol. The message is then typically relayed, also using the SMTP protocol between a number of MTAs until the final MTA responsible for the recipient's incoming email receives it and spools it on a server ready for download. Downloading typically uses IMAP or POP3 protocols, which may possibly be combined with HTTP for web email. SMTP relaying between MTAs was, along with the Usenet news system, an early example of a Peer to Peer protocol .

Relaying of email between MTAs, from a security point of view, is classified as follows:

  1. Sending: The MUA sending email submits it to a Mail Submission Agent (MSA), (which is usually a program which also acts as a MTA, but may be separate). Further MTA to MTA relaying may occur. There is an expectation of common administration or contractual relationship between MTA domains relaying within phase 1. Mail should be relayed only for users of the ISP or mail administration operating the outgoing MTA.

  2. Cross domain: The MTA responsible for outgoing mail from the sender domain connects to a MTA acting for the mail recipient domain, in most situations identifying this using a DNS Mail eXchanger MX record. In most cases these 2 MTAs will be under different administration and there is no expectation of any prior relationship. In situations where one MTA is acting legitimately as a forwarder or email list host, more than one cross domain transfer may occur. MTAs must always be configured to prevent promiscuous relaying - in the sense that there should be a relationship either with the email sender or the email recipient or both, but not with neither.

  3. Receiving: The MTA receiving incoming email for the recipient domain may relay this through other MTAs, e.g. with higher priority DNS MX records or inside the firewall protecting the final recipient's LAN. Within phase 3 there is also an expectation of common administration or contractual relationship between MTA domains relaying within phase 3.

RFC 4409 Message Submission for Mail.

RFC 2505 Anti-Spam Recommendations for SMTP MTAs

SMTP flow diagram

SMTP flow diagram picture

MTA technical attacks

Sendmail, Postfix and Exim are considered relatively mature and stable programs. Until a few years ago, the MTA thought responsible for most high volume mail installations,Sendmail had a history of remotely exploitable buffer overflows. Since the last of these in 2003 , the most recent potentially exploitable vulnerability is thought unlikely to have resulted in any compromised systems in practice.

Spam is another matter entirely. This threat generally isn't intended to compromise individual mail programs but by exploiting the almost free and universal connectivity provided by SMTP email it threatens the open nature of email use.

SPAM

Definition - Unsolicited Bulk Email

If you are going to filter something, or create useful legislation making something illegal or encourage good practice, you need to know your enemy. The Internet Service Provision (ISP) and email development community generally defines spam as:

Another definition, Unsolicited Commercial Email is occasionally used, but those with a non-commercial agenda who send out 1,000,000 unsolicited email messages are as much a nuisance and cost to the recipients collectively as anyone else who sends the same number of UBEs. (The University of Central England also gains no benefit from association between unwanted email and UCE.)

Where does a Spam come from ?

Each MTA in the transmission chain normally inserts a Received: header above any previous ones. Exceptions to this include sending MTAs inside a firewall where administrators wish to avoid divulging details concerning private networks. Spammers are expected to falsify all headers, but a legitimate MTA receiving and relaying a message from a spammer will normally insert a correct header and this should include the IP address of the computer used by the spammer to send the offending message. If your message deliverer doesn't let you view the full message headers then you won't be able to trace the origin.

From nobody  Tue Mar 13 01:45:59 2007
Return-Path: <lvderhooihlid@xcelenergy.com>
X-Original-To: rich-filter@localhost
Delivered-To: rich-filter@localhost
Received: from localhost.localdomain (localhost.localdomain [127.0.0.1])
	by localhost.localdomain (Postfix) with ESMTP id 317D640035
	for <rich-filter@localhost>; Tue, 13 Mar 2007 01:45:55 +0000 (GMT)
Received: from coppssewood.net [80.68.90.112]
	by localhost.localdomain with POP3 (fetchmail-6.3.4)
	for <rich-filter@localhost> (single-drop);
	Tue, 13 Mar 2007 01:45:55 +0000 (GMT)
Received: from xcelenergy.com ([60.11.226.111])
	by coppssewood.net (8.13.4/8.13.4/Debian-3sarge3) with SMTP id
	l2D1iRSe000980; Tue, 13 Mar 2007 01:44:40 GMT
Message-ID: <5a6b01c764b2$26ed9850$54ef1101@lvderhooihlid>
From: "Ciera" <lvderhooihlid@xcelenergy.com>
To: "Collin" <manda@coppssewood.net>
Cc: "Neil Payne" <mailer-daemon@copsseewood.net>
Subject: Feeling Down
Date: Mon, 12 Mar 2007 14:24:32 -1100
MIME-Version: 1.0
Content-Type: multipart/related; type="multipart/alternative";
	boundary="----=_NextPart_296_ECB7_9DBFBDCF.BE338C8B"
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2600.0000
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000
X-Spam-Status: No, score=3.8 required=6.0 tests=EXTRA_MPART_TYPE, HTML_MESSAGE,
	TVD_FW_GRAPHIC_NAME_LONG,WEIRD_QUOTING autolearn=no version=3.1.4
X-Spam-Level: ***
X-Spam-Checker-Version: SpamAssassin 3.1.4 (2006-07-26) on saturn
X-Virus-Scanned: ClamAV 0.90.1/2826/Mon Mar 12 23:21:57 2007 on copsseewood.net
X-Virus-Status: Clean
X-Presumed-Origin: 60.11.226.111 xcelenergy.com 2007:03:13 13585
X-CSV-Result: 60.11.226.111 xcelenergy.com none (550, '?.?.?',
	"No CSV1 record for '_client._smtp.xcelenergy.com'.")
Status: RO
Content-Length: 22779
Lines: 363

(spam body deleted)

The spam was received from 60.11.226.111. A traceroute suggested this host is in China (more likely it was relayed by a compromised PC there and that none of the sending routers responsible for putting it onto the Internet backbone had DNS PTR records in the in-addr.arpa domain).

rich@saturn:~$ traceroute 60.11.226.111
traceroute to 60.11.226.111 (60.11.226.111), 30 hops max, 40 byte packets
 1  * * *
 2  10.254.68.1 (10.254.68.1)  7.236 ms  9.847 ms  8.944 ms
 3  brhm-t2cam1-b-v118.inet.ntl.com (213.106.228.237)  8.788 ms  8.935 ms  7.798 ms
 4  brhm-t3core-1b-ge-010-0.inet.ntl.com (195.182.176.65)  8.444 ms  9.566 ms  9.586 ms
 5  bir-bb-b-so-100-0.inet.ntl.com (213.105.174.5)  9.008 ms  45.800 ms  50.796 ms
 6  nth-bb-a-so-300-0.inet.ntl.com (62.253.185.105)  34.801 ms  50.385 ms  24.355 ms
 7  gfd-bb-b-so-010-0.inet.ntl.com (62.253.185.98)  15.366 ms  15.842 ms  15.105 ms
 8  sl-gw11-lon-11-0.sprintlink.net (213.206.156.101)  16.863 ms  15.977 ms  16.229 ms
 9  sl-bb20-lon-11-0.sprintlink.net (213.206.128.56)  16.784 ms  15.973 ms  15.858 ms
10  sl-bb27-nyc-9-0.sprintlink.net (144.232.9.164)  84.579 ms  83.804 ms  82.597 ms
11  sl-bb24-nyc-8-0.sprintlink.net (144.232.13.173)  83.501 ms  88.375 ms  83.683 ms
12  sl-bb21-nyc-6-0.sprintlink.net (144.232.13.186)  82.989 ms  84.510 ms  83.915 ms
13  sl-bb26-pen-4-0-0.sprintlink.net (144.232.20.142)  86.824 ms  107.486 ms  86.446 ms
14  sl-bb21-stk-13-0.sprintlink.net (144.232.20.163)  185.041 ms  182.400 ms  156.797 ms
15  sl-gw27-stk-8-0.sprintlink.net (144.232.4.246)  155.472 ms  156.486 ms  155.498 ms
16  sl-china7-7-0.sprintlink.net (160.81.232.194)  582.017 ms sl-china7-5-0.sprintlink.net (160.81.7.82)  564.912 ms  560.169 ms
17  219.158.3.177 (219.158.3.177)  574.902 ms  577.271 ms *
18  * 219.158.7.50 (219.158.7.50)  608.780 ms  576.770 ms
19  * 61.138.38.70 (61.138.38.70)  595.674 ms  600.936 ms
20  61.138.38.130 (61.138.38.130)  642.099 ms  633.122 ms  635.071 ms
21  221.210.238.50 (221.210.238.50)  630.514 ms  627.206 ms  627.668 ms

Use of the whois command was more helpful in identifying the ISP responsible for the connection used to send the spam.

rich@saturn:~$ whois 60.11.226.111
% [whois.apnic.net node-1]
% Whois data copyright terms    http://www.apnic.net/db/dbcopyright.html

inetnum:      60.11.0.0 - 60.11.255.255
netname:      CNCGROUP-HL
descr:        CNCGROUP Heilongjiang Province Network
descr:        China Network Communications Group Corporation
descr:        No.156,Fu-Xing-Men-Nei Street,
descr:        Beijing 100031
country:      CN
admin-c:      CH444-AP
tech-c:       BG63-AP
status:       ALLOCATED PORTABLE
mnt-by:       APNIC-HM
mnt-lower:    MAINT-CNCGROUP-HL
mnt-routes:   MAINT-CNCGROUP-RR
changed:      hm-changed@apnic.net 20041231
changed:      hm-changed@apnic.net 20050218
source:       APNIC

route:        60.11.0.0/16
descr:        CNC Group CHINA169 Heilongjiang Province Network
country:      CN
origin:       AS4837
mnt-by:       MAINT-CNCGROUP-RR
changed:      abuse@cnc-noc.net 20060118
source:       APNIC

person:       CNCGroup Hostmaster
nic-hdl:      CH444-AP
e-mail:       abuse@cnc-noc.net
address:      No.156,Fu-Xing-Men-Nei Street,
address:      Beijing,100031,P.R.China
phone:        +86-10-82993155
fax-no:       +86-10-82993144
country:      CN
changed:      abuse@cnc-noc.net 20041220
mnt-by:       MAINT-CNCGROUP
source:       APNIC

person:       Binghui Gao
nic-hdl:      BG63-AP
e-mail:       gaobh@mail.hl.cn
address:      Communication Corporation Internet Enterprise Division of HLJ
phone:        +86-451-2804465
fax-no:       +86-451-2804442
country:      CN
changed:      gaobh@mail.hl.cn 20030221
mnt-by:       MAINT-CNCGROUP-HL
source:       APNIC

Current Anti-Spam technologies

Misguided attempts at spam solutions

The FUSSP - Final and Ultimate Solution to the Spam Problem

Unfortunately despite very many such proposals, there isn't a FUSSP. Imagining they have the FUSSP doesn't seem to exclude those whom you might imagine to be able to bring a great deal of influence to bear upon the subject.

Quote

"Two years from now, spam will be solved," he [Bill Gates] told a select group of World Economic Forum participants at this Alpine ski resort. "And a lot of progress this year," he added at the event late Friday, hosted by U.S. talk show host Charlie Rose.

Bill Gates speaking in DAVOS, Switzerland, Jan. 24, 2004

More Realistic approaches

More realistic approaches are likely to involve limiting the scale and scope of the problem.

Only some in the above list are technical or partly technical approaches. All the above approaches also depend to an extent on social, legal and educational changes. The reason there is no FUSSP is because spam is as much a human behaviour problem as it is a technical one.