[Read an edited version of this on the BBC News online, as ever]
About two weeks ago I started getting a lot of bounced emails. Most of them were notifications from the ‘postmaster’ somewhere that my email could not be delivered because the recipient didn’t exist, but quite a lot were from spam filters to tell me that I’d sent messages that they weren’t willing to accept.
It seems I’ve been pushing stocks in dodgy companies, offering pharmaceuticals without prescription and even sending virus-laden images to unwitting users.
Except it wasn’t me. Honest.
Most of the messages that came into my mailbox, at the rate of one or two hundred an hour, were originally sent by people with names like [email protected], and although that’s my domain, it certainly isn’t one of the few email addresses I send from.
And closer examination of the headers reveals that they didn’t come from the andfinally.com domain or any of the servers run by my network provider.
But of course, I had nothing to do with these messages, or the thousands that presumably get through to unwitting recipients instead of being bounced.
A spammer, or group of spammers, has picked on my domain to use as the fake ‘from’ and ‘return-to’ field in the headers of the emails they send out, hoping to fool a few more filters into letting them through.
Because of the way email works, and the lack of any built-in authentication, this is a trivially easy thing to do. As a result I have to cope with thousands of these messages coming in, and I face the danger that my domain, on which I depend, will be blacklisted and my real emails will stop getting through.
It’s a complete mess, and it’s getting on my nerves.
There are a few things I can do, and I’m going to have to waste my time persuading my network provider to set up one or more of the cobbled-together authentication systems like Sender-ID or Sender Policy Framework. At least then recipients can choose to check whether I really did send an email about Goldmark Industries to non-existent users in Poland!
It won’t fix the real problem, of course. It won’t stop people around the world forging emails that seem to come from me or injecting them into the network. And it won’t stop the damage to my reputation that results.
Earlier this week I gave a talk to a customer conference organised by Lyris, whose business relies on getting email delivered to people. They make ListManager software, and had asked me along to talk about the fuuture of email – and whether
it even has one.
Despite the many problems, and even despite my own current experiences and unhappiness, email isn’t going to go away. It fills an important space in the ‘information ecosystem’, since messages are persistent and asynchronous, unlike instant messaging, and the recipient is notified of their arrival in a simple and usually convenient way.
Of course we need to sort out the big issues which make spam and forged headers possible. We need to think about how trusted computers and modern network standards like version six of the Internet Protocol, IP, can be used to authenticate messages and their senders. And we may want to review the operation of core email protocols, like the Simple Message Transport Protocol, STMP instead of relying on add-ons like Sender-ID to do the work.
For example, at the moment messages are sent along with their headers, so when I send a note to my friend Simon about the latest cool tech gadget – or, more often, when he points something out to me – a copy is made and transmitted over the network.
That copy sits on his server until he either downloads it using the Post Office Protocol, or reads it remotely using IMAP, the Internet Message Access Protocol. In either case he looks at his local copy and not the one sitting on my server.
But in an always-on world surely there’s no real need to send those bits over the network until they are actually needed?
Millions of people work happily with Gmail and Hotmail accounts, only able to access their messages when they have a connnection, so why not extend the model, at least for those of us who can get online pretty much at will?
Instead of SMTP we could have SMIP – the simple message indication protocol.
Instead of chucking bits across the network we could send only the headers, leaving the message itself to be retrieved by the recipient when they choose to.
And then we could do some proper checking before messages were accepted.
Because apart from solving the problem of how to know whether someone has received or read an email, this would also make it generating spam with fake headers a lot less useful for spammers.
When an email client received a message that appeared to come from me at andfinally.com, the first thing it would do would be to ask my mail server if there really was a message waiting to be collected.
It’s a lot harder to hijack a domain’s DNS entry than it is to forge a header, so most of the spam would never be accepted. And there would be no bounced messages since even the dumbest corporate email server would realise that a forged header which didn’t relate to a message sitting on the real andfinally.com server didn’t come from me.
Checking the originating domain is basically how SPF and Sender-ID work anyway but they are not yet in widespread use and take some effort to set up – as I am currently discovering.
They rely on having a network provider who is willing to respond to technical support queries and make changes to the mail server configuration, and so far my provider hasn’t bothered to get back to me.
I know I could run my own mail server on my own box, but while the technical aspects don’t worry me I already have too little time for real work, and taking direct control of even more aspects of my online existence is not really an option.
It would be much better to have an email architecture that actually made forged headers an exceptional technical achievement instead of something that any two-bit spammer can do in seconds.
Bill’s Links
It isn’t just me;
http://www.speed.net/support/forgery/
Sender Policy Framework/Sender ID: http://www.maawg.org/about/whitepapers/spf_sendID/
Sender Policy framework
http://en.wikipedia.org/wiki/Sender_Policy_Framework
Not the perfect solution, though:
http://www.theregister.co.uk/2004/09/03/email_authentication_spam/
I had similar problems some time ago – my hosting company contacted to say their server was being hit by 2GB of mail per hour.
Eventually it became so bad, I had to abandon the domain name because our reputation (and this was a church website) was so tarnished.
I can say confidentally that things do get much better when you implement SPF/SenderID – most ISPs do check the records on incoming email even if they don’t set it up for their own users.
One other thing – if you move the domain name to http://www.123-reg.co.uk, it’s easy to setup your own TXT records for SPF
There are some probably wonderful simple ways of securing email – ranging from a more secure protocol to digital signatures. The reason these don’t take off is corporate greed.
Companies want to use any new feature to their competitive advantage. In this world of software patents and “Intellectual Property” this means that any new feature can be used to ensure that only your mail system, or the mail systems of people that pay your license fees, are allowed on the Internet. That rules out open source and makes competition expensive.
Microsoft control the mailer that a lot of people use. They have the power to block a competing technology by making their mailer incompatible with it or making it hard to use. PGP suffered this. Their behaviour suggests that they wish they had the power to force a technology onto other users, forcing them to go out and buy Windows. Fortunately they don’t.
We need a solution that is developed in a truly collaborative method that is free for anyone to implement and is not used. Email and the Internet themselves are so successful because they were developed in such a way. Only a truly open solution can solve this problem.
Although I cannot completely disagree with Richard’s comments on corporate inertia, I don’t think that things are helped by the people tasked with solving the problem.
A few years ago I put together some ideas I had developed while completing an MSc (see http://www.onlinecounsellors.co.uk/esmtp/esmtp.htm). These ideas include the use of headers and the subsequent retrieval of the email (if the server address is real) and how these changes could work alongside the current SMTP environment. My motivation was to reduce the amount of Spam I receive rather than any monetary gain.
I then went looking for someone to run these ideas past and was directed to the Anti-Spam Research Group (ASRG), part of the Internet Research Task Force (IRTF) (see http://asrg.sp.am/). Unfortunately, in the 4 or 5 months I subscribed to their mailing list, it seemed to consist primarily of a lot of angry and touchy people who were far more interested in un-constructively criticising each other than they were in cooperating in order to come up with a workable solution.
It seemed to me that any spammer worried about his future would probably also subscribe to the mailing list, if only to reassure themselves that there was no real threat to their business model likely to arrive any time soon.
I had exactly the same problem a couple of months ago and was just as annoyed as you are that anyone can do this to anyone else and we – the victims – are powerless to do anything.
My ‘fix’ was to alter my domain mail redirection facility to turn off the ‘catch all’ option so that email addresses with [email protected] would be bounced and only mail sent to valid addresses on the domain would be received.
It reduced the amount of bounced e-mail I was receiving but I agree wholeheartedly that more needs to be done to prevent this abuse.
Keep going Bill. As you mentioned to me before, keep throwing pebbles and maybe the small ripples you are making might actually lead to something being done.
First, I’d like to offer my now and future comiserations.. ‘Future’ because the ‘[email protected]’ addresses will now be settling into peoples mail folders and address books (for newbies who try and respond) etc.. These fake addresses will then be harvested by the spammers, and their buddies in the malware trade, and added to new spam and virus lists.
This happened to me four years ago and I still get spam and viruses sent to those addresses, I’ve passed through the annoyance, anger, and vigilante phases and now just filter them. They still arrive, half a dozen each day..
[Finally, I don’t understand your comment about SPF needing your mail server admins to take action, they do not usually need to do anything. SPF is part of the Domain Name System entry for ‘andfinally.com’, it does not require chanegs to the configuration of the mail server. Unless your DNS provider is also your mail server provider, you should be able to bypass them when setting this up, just talk to your DNS provider. I was able to do it online via the Web interface of my DNS provider, using the ‘spf wizard’ on http://www.openspf.org/ .
I have the same problem although I’m guessing its not on quite the same scale – I get about three or four hundred bounced spam messages every day. I use a catch-all email account on my domain as I like to use a unique email address for each website I sign up for (so I can track where spammers are getting my address from). I don’t actually get much spam myself, maybe 1 or 2 messages a day – however I get about 300 emails a day which are spam which originated elsewhere but was sent with an address ending in my domain. This has then bounced back as undeliverable from the intended recepients email server when they have blocked it or changed their address. This wouldn’t be such a big deal – I have the bandwidth and space to cope with them – but I can’t filter them easily as every email server seems to use a different format for bouncing them. Roll on authentication of senders before accepting an email, and some standards for error codes. There must be hundreds of millions of these sorts of messages bouncing back and forth clogging things up.
A life without spam would be like a life without junk mail flying through your letter box each day. Or a life without random text messages from Boz asking what dress I’ll be wearing to the prom (I actually received this text).
Now after the initial realisation that ‘Hey I’m not popular, it’s just spam’ passes I agree that spam gets rather annoying.
Yet, any form of modern communication, whether it be by post, text message, email etc will always produce unwanted communication and I really can’t see how we can entirely rule this out.
I like the SMIP concept though.
Right, where’s that dress for the prom?
I reckon, instead of trying to tackle the spammers, we should impose heavy penalties on anyone found clicking a link found in spam, or actually buying “Miracle H3r8al V1a6ra!”. Kill off the target market, and the spammers will go somewhere else. Maybe…
I imagine the internet as an organic system, with all of the so-called programming ‘flaws’ as niches. Your idea about altering protocals to prevent such niche-filling as spamming or viruses sidesteps the fundamental fact that the more sophisticated any system gets, the more anomalies there will be, whether they are planned anomalies (like viruses) or serendipitous(like errors). If you do your little STMP switcheroo, the first people who will fill the unforseen niches will be spammers and hackers by virtue of the fact that commercial programmers only react to problematic programming like spam and viruses.
So what I mean is this:
the more you try to change the internet, the more little flies will be buzzing around your electronic ears. Besides, the open nature of emails and the internet that allow spam to exist is the very reason the internet is special. Take away the spam and you take away all of the ‘electronic civil liberties’ that we enjoy. Like mothers with no baby milk on planes at Heathrow. Catch?