Tag Archives: Challenge-response spam filtering

Didtheyreadit’s Response To Privacy Issues Part II

More on Alastair Rumpell’s response to my privacy concerns about his new email monitoring service, didtheyreadit.  (Here’s the first one.)

I wondered how the email addresses harvested by Rampell would be used (These would include all emails sent from and to recipients via the service since as far as I can understand it didtheyreadit, unlike MSGTAG, would work via tagging the email address, not the email itself. This would involve collecting the email address of sender and recipient). Alastair’s response: “We don’t harvest any e-mail addresses—I wasn’t sure to which e-mail addresses you are referring. We can send you e-mails to the account you register with, but we also allow you to opt-out at any time. We do not send any commercial e-mail or e-mail for any other companies to our customer list.” That’s not quite the complete denial I was looking for, but perhaps I wasn’t specific enough in my original post.

Another question I raised: How will Rampell prevent this service being used by spammers and other mass-mail marketers? Alastair’s response:
“We limit you to 750 messages per month. Very few individuals will ever exceed this number…whereas all mass-mail marketers would.” Fair enough.

Although Alastair takes pains to address my general privacy concerns, however, I’m not sure I can agree with his arguments. He candidly writes, “I had a discussion with somebody last week who was offended and repulsed by the idea of our service; the reason why is because a criminal could use our service to tell if somebody was at home. (Although she recognized that a telephone call could be used for the same purpose).” I can agree with that: Privacy is a long tunnel that can suck you in if you’re not careful — where everything is a threat — but while I don’t think didtheyreadit and MSGTAG represent threats to one’s physical safety, there are still some serious issues out there.

Alastair, for example says in response to my question “Why is the service invisible by default?” (In other words, why is there no default notice in the email informing the recipient the email they are reading is being tracked). Alastair’s response: “I believe it is what the market demands.” He later goes on: “We are planning on doing a free version (like msgtag) that automatically places the disclosure there, as it is a form of marketing. In
our initial tests, though, people who were trying the service were very concerned about having it disclosed to the recipients that the messages were being tracked.” I think that pretty much defines the problem. If someone sends a message to someone but doesn’t want them to know they know their message is being monitored, you’ve pretty much got yourself trapped in a privacy quagmire. If I do something to know something about you, but I don’t want you to know I am doing something to know something about you, then I would submit that as a default definition of snooping, or invasion of privacy.

What’s more, what kind of user would want to monitor their sent emails so invisibly? It’s hard to imagine they’re sending something to Aunt June or their son Bobcat. Given the other elements of didtheyreadit — monitoring exactly when, how long, where and how many times an email has been read — I’d say a consumer who demands the service be invisible may not be the kind of customer you’d be proud of having. What’s more, Alastair’s response to the issue of informing the recipient the email is being tracked is a rather strange one, in my view: Including a message informing the recipient might deter customers. “Even if it is an option,” he writes, ”it will confuse a good deal of people who might avoid using our service as a result.” I can hardly agree with that. Including an option to address a serious privacy issue is only likely to deter folk who aren’t great respecters of privacy.

I had some other issues with Alastair’s company, not least because it sells products that inhabit a grey privacy area. They include a keyboard logger called Spector, and ViewRemote (“record everything that happens on your computer and watch it from any other computer in the world!”). Alastair’s response: “I realize that some of our other products are often considered invasions of privacy. However, we take great pains to make sure that the products (ViewRemote and Spector) are only used by authorized people. For example, you cannot install ViewRemote or Spector without entering your computer’s administrative password—so it can’t be installed without your permission. Installing Spector or ViewRemote on somebody else’s computer is not only a gross violation of privacy—but it’s also illegal. I feel that this is immoral and unethical, and thus we do not support it. But “spying” on your own computer, for lack of a better word, is sometimes necessary. Our products have been used to catch an employee stealing, identifying a pedophile, etc.”

I’m sure there are legitimate uses of such programs. But it leaves an uncomfortable taste that the company whose main products are what I would call stealth software is now selling a service that invisibly and remotely monitors the fate of emails. Alastair, who says his academic background is on the other side of privacy, via cryptography research, is at least discussing the issues, which is a good sign. But I am not sure I agree with him when he concludes that ”I believe that DidTheyReadIt is relatively harmless. Yes, you can use it to catch somebody in a lie…but there are a wealth of legitimate purposes that give the sender more information (such as if the message was even received) without necessarily infringing upon the privacy of the recipient.”

My response: Yes, in the midst of spam’s deluge there’s definitely a legitimate market here for checking whether your email got to where it was supposed to go safely. But it shouldn’t be necessary to go beyond that, to check about aspects of its fate that should really be the private property of the recipient: How long the message was read, where it was opened, whether it was forwarded to others. Furthermore, didtheyreadit (and MSGTAG) need to address the issue of allowing the recipient to easily and definitively opt out of having the emails they receive tagged by such services; if possible, before the first email they receive from either service. If such companies don’t address these issues before they get successful, they may find themselves caught up the full glare of privacy advocates, and end up destroying what is in essence a useful and benign service.

Didtheyreadit’s Response To Privacy Issues Part I

Further to my posting about Didtheyreadit, a service which allows the sender to know whether/when/where the recipient opened their email (and even how long they read it), here’s a response from the company’s owner and founder, Alastair Rampell, addressing my concerns about the serious privacy issues it raises.

Alastair acknowledges “you are right in that the service raises some privacy issues” but says there are legitimate uses for it, not least ensuring that your emails have not been trapped in some spam filter: ”I can say that I use didtheyreadit for all the e-mail that I send, but I really don’t use it for any sort of nefarious purpose, but rather just to tell that my e-mail went through. Spam has become such a big problem that quite often legitimate (non-commercial) mail gets blocked, too—but the sender never knows. Didtheyreadit addresses this problem.”

I wondered why didtheyreadit did not include some clear message (unlike its competitor, MSGTAG) alerting the recipient to the fact the sender is using the service. Alastair’s reply: “You can just add a line to your signature, saying “this message was sent with didtheyreadit.com.” We don’t try to force people to disclose that they are using the service.” Fair point, but I think a message should be the default setting, with users able to remove it or change the wording if they want. Firstly, the company providing the service bears some responsibility about how the service is used, so they should be influencing users about good practices (such as not concealing from recipients that the opening of their emails is being monitored). Secondly, the more standard any such message informing users their emails are being monitored, the more consumer awareness of the service. The more aware consumers are, the more a reasoned decision can be made about whether they consider it acceptable behaviour.)

This is perhaps the most important issue about email tagging, and will, I think, determine whether it’s deemed acceptable by the market. MSGTAG have it more or less right in that they inform the recipient, and, in theory, give them the change to opt out. However, this is after the event (i.e. after the first email they receive) so it’s not prior consent, and nor is it particularly prominent (the message is stuck at the bottom of an email). Correcting this might be tricky, but would be the fairest resolution, in my view. A recipient should be offered a choice of rejecting or accepting the tag and subsequent tags first.

More on Alastair’s response in the next posting.

Stopping Spammers and Scammers By Patrolling Their Shopfront

America’s new anti-spam CAN-SPAM Act is a great way to stop spam, so long as the spammer is legit. The problem is, most spammers aren’t.

Mass.-based software company Ipswitch Inc. estimate that more than two-thirds of all spam is deceptive, meaning that spammers disguise the links to their website “behind unrelated graphics and pictures, or by camouflaging their site as a commonly used consumer e-tail site”. Some of this, of course, is real business (however sleazy) but a lot of it is scamming. From Ipswitch’s press release it’s not quite clear whether their software is aiming at the former, the latter or both.

“Over two-thirds of all spam messages include deceptive content intended to trick the recipient into believing the sender represents a legitimate business,” said John Korsak, messaging product marketing manager at Ipswitch. “Because of their legitimate look and feel, recipients do not associate these types of messages as spam when they appear in their email in-box. To protect people from unknowingly sharing private financial details, it is critical email providers employ a URL Domain Blacklist to verify the sender’s true identity.” That kind of sounds like most spam is scam, which can’t be right. It’s bad, but it is not yet that bad.

Anyway, the URL Domain Blacklist is one filter in 20 in Ipswitch’s IMail Server — the others are Bayesian Statistical filtering, Reverse DNS Lookups, SMTP filters, and whathaveyou — which “unmasks illegitimate spam messages by looking at the actual underlying link and comparing it to a growing list of more than 18,000 repeat spammers”.

It’s not a bad idea. Links are the one things all spams and scams have in common, and they’re relatively easy to identify, unlike text (which can be disguised by clever use of HTML, the language used to create webpages, or by images). But there are still problems, and the press release (and website) are maddeningly imprecise about what, exactly, is being targetted here: Spam or scam?

If it’s the latter, I don’t think URL blacklists are going to be much help. From what we know of phishing scams, the main email-based scam, the website addresses that scammers want us to go to don’t last very long — sometimes only a few hours — meaning that you need to have a very long and rapidly updating list of known scammers. And while Ipswitch is probably right in arguing that they don’t get many false positives — good email mistaken for spam — I don’t think that’s the problem here. The problem is you’re chasing the one element in your average scam email that’s changing most: The scammer’s Internet shopfront. That can be set up and pulled down in a matter of minutes.

Could Social Clustering Be Used To Kill Off Spam?

We can relax: Boffins are now grappling with spam.

Nature reports that P. Oscar Boykin and Vwani Roychowdhury of the University of California, Los Angeles, have come up with a way to tackle at least half the emails we get, namely those we get from friends, colleagues, and anyone else either we know or the people we know know (I’ve always wanted to write that sentence).

It works like this: If Alice knows and e-mails Bob and Chris, for example, then Bob and Chris are far more likely to know and e-mail each other than if they didn’t share a friend in common. E-mails radiating from a spam source don’t share this clustering property – the vast majority of recipients don’t know each other. “The method,” Nature says, “effectively turns the spammers’ weapon on themselves. The very fact that they can send out so many messages secures their low overall degree of clustering – it’s what gives them away.”

This is all done by inspecting the ‘from’, ‘to’ and ‘cc’ fields in a user’s inbox. An automated system can quickly build up a blacklist of spammers, as well as a ‘whitelist’ of approved sources. E-mails above a certain ‘clustering threshold’ are always friendly, and those below a lower threshold are always spam.

Boykin and Roychowdhury acknowledge this may only apply to about 50% of email. But those would have been filtered without any errors, and it would have required no user intervention at all. The remaining e-mail would have to be filtered by other means, but as the authors say, ”our algorithm may be used as a platform for a comprehensive solution to the spam problem when used in concert with more sophisticated, but more cumbersome, content-based filters.”

It’s not a bad idea at all. By looking at header fields rather than content the filtering process would be much quicker. Furthermore, the only way I could see the spammers getting around it would be to spoof header fields so they somehow anticipated the social clusters of the recipients: In other words, they’d have to try to figure out who was on someone’s white list for their message to get through. (Although I suppose spoofing the actual recipient’s email address as the sender field might be enough.)

What’s intriguing is how this might feed into social networks like Friendster. Could these groups be mobilised as automatic whitelists for users, so that, for example, I could, with a mouse click, ensure that everyone on my Friendster list is automatically on my whitelist? If this sort of thing caught on, it might give an added incentive to join such networks for folk like me who find places like Friendster a bit too, er, youthful and places like LinkedIn a bit too, er, business-oriented.

Subject Fields – A Way To Foil Spam?

What to put in the Subject field these days to avoid spam filters?

Clive of collision detection (who, incidentally, wrote a first class piece about European virus writers for the NYT) points out that the spam “battle has now claimed its first linguistic casualty. It occurred to me yesterday that you can no longer send an email to anyone with the sole word “hello” in the message header.” That’s because the recipient (and almost certainly any spam utility) would regard it as spam, and so chuck it out.

His conclusion: “If you want to appear human-like, put a human-like message header on your email. And that means you can’t say just “hello” or “hi,” because that no longer qualifies as a human-like message.”

True. Here’s my tuppence’ worth: What I’d like to see are some creative methods to communicate with each other. Here are some suggestions:

  • Members of the same family could use a code word in the subject field like [rabbit] which should get it past Bayesian spam filters, once those filters have been educated a bit.
  • Friends and colleagues communicate with one another by more elaborate subject fields: [Meeting] postponed, goof off until three pm or [Wedding] Er, it’s off. The idea is that the bit in parentheses stays the same, as an indicator, while the rest of the subject field changes.
  • Users can then set up filters which funnel emails containing those [] codes directly to certain folders.
  • The overriding principle is that the Header field should a) provide some seriously useful information about what’s in the email, and b) carry with it some sort of [category]. I’m no expert on Bayes, but I figure this would really help the filtering process.

The bottom line is this: Subject headers should be another line of defence against spam. If we used them better than ‘hi!’ or ‘hello’ we could frustrate spammers who would themselves have to put a lot more effort into generating credible headers. That just might make it uneconomic enough.

No Sign Of Letup On Spam So Far

Unsurprisingly, the new U.S. anti-spam law has had no effect whatsoever.

Commtouch, a provider of anti-spam solutions, said it saw no significant change in the number of spam attacks in the first week of 2004, and that less than 1% of all bulk email complied with the new CAN-SPAM regulations.

Although Commtouch notes it is too early to tell, as spammers are still on holiday, I’ve noticed no slowdown at all. This is not unexpected, since most spammers operate outside the law – when was the last time you had a legitimate-looking junk email that was not trying to disguise itself?

But it’s not just the really sleazy guys still doing it. MX Logic, another anti-spam provider, looked at a random sample of over 1,000 unsolicited commercial emails during the course of a seven day period beginning New Year’s Day and found only three of the messages complied with the CAN-SPAM Act. “Calling this a high rate of non-compliance would be a gross understatement,” said Scott Chasin, MX Logic’s chief technology officer. “It is no surprise that rogue spammers would fail to comply, but the non-compliant messages we saw appeared to be from all types of companies.”

This could be just reputable (I use the term loosely) email marketers not getting up to speed on something that was only signed into law on December 16. If you are an email marketer and you do want to comply, here’s a checklist of what you should do, courtesy of Intermark Media, itself a an email marketer (the list is somewhat revealing to us normal folk, in that it shows what kind of tricks spammers tend to do to give the impression everything is hunky dory and that, at some point of personal weakness, we actually agreed to receive spam from them):

— Collect this information on every member of your opt-in database: IP address, date and time of opt-in, and source URL of sign-up.
— Be wary of any list managers who do not require this sensitive information from you as it is of crucial importance that all parties involved have it.
— Provide a clear opt-in process for the consumer.
— State your intentions in your privacy policy.
— Make your privacy policy easily accessible to the consumer.
— Upon receiving a customer’s permission to send offers you should notify them of their consent. This also allows the consumer to become double opt-in or unsubscribe from receiving any offers.
— Upon receiving a database to manage always run a permission email to the database in order to notify the consumers that you are the source of the emails they will be receiving and this will allow them to unsubscribe from your mailings or become double or even triple opt-in.
— Never change the headers that you send emails from.
— Use valid and relevant from and subject lines for all campaigns.
— Do not use misleading subject lines for any purposes, including creating new responsive lists from recipients that open or click on a campaign.
— If you receive a subject line you feel is questionable ask the advertiser to provide another one.
— Make sure the email address you are sending campaigns from is valid and working.
— In the footer, provide an explanation of why the consumer is receiving the ad.
— In the footer, provide your company’s valid postal address. If you are managing a client’s list, make sure their address appears as well.
— Make sure every campaign has a valid, working and obvious unsubscribe mechanism that easily removes the consumer from your database.
— Keep a real-time update of unsubscribes and remove them from your database and the databases of all parties involved.
— Do not email to consumers who unsubscribe from your database.
— Do not allow others to email to consumers who have unsubscribed from your database.

News: How Not To Fight Spam

 From the How Does This Work Again Dept? comes news of a company that pays spammers to take your name off their list. But the whole thing depends on trusting spammers, which is too early in the morning to find a suitable analogy for. Wired reports that Global Removal charges subscribers a $5 lifetime fee to have their e-mail addresses put on a permanent do-not-spam list. Addresses on the list are then compared with, and removed from, mailing lists maintained by Global Removal’s partnering businesses — more than 50 known spammers and an equal number of legitimate e-mail marketers. The idea: unlike other attempts at creating do-not-spam lists, this will work because it gives spammers an incentive to cooperate. Money.
It’s not a terrible idea, but it rests on a fallacy: that spammers are not interested in email addresses of folk who don’t want to receive spam. I just don’t buy that. Spammers usually work for other people — they’re just a delivery mechanism — and they need to be able to deliver in bulk — in other words, send the pitch to as many email addresses as possible. They’ll be happy to take Global Removal’s money as extra cash on the side, and remove a few email addresses, but they are not going to stop harvesting — scouring the web for email addresses — or guessing (obtaining an ISP’s address, for example gormless.com, and then testing a telephone book full of regular names, from andy@gormless.com to zob@gormless.com to see whether they get through). So it means that you have no guarantee any other email addresses you have won’t get harvested in this way. Unless all spammers sign up for the service, and agree to stop harvesting new email addresses, it won’t work.
Lastly, the way spammers increasingly work is not through spam lists but by open proxy servers — other people’s computers, which are tricked into sending on spam, and in many cases, hosting the websites respondents visit — meaning that it’s very, very hard to trace where the spam came from. Global Removal will either have to offer forensic monitoring of the spammers signed up to its service to ensure compliance, or else it will only work with the (very, very small) number of spammers who are halfway legitimate, in that they do not disguise where their spam comes from, let alone comply with various state and country laws governing spam. Sadly, spammers are getting sleazier, and a service like Global Removal just adds another financial incentive for spammers to get into the game.

Software: Another Spam Service…

 Once more reinforcing the impression this is spamblog central, here’s another product that promises to rid you of spam (99.5% of it to be precise). PrismEmail.com was launched today by Vault Information Services (VIS) — it can be used with any operating system or email program, doesn’t require that anything be installed on the user’s computer, and works with the user’s
existing email address such that no change in email address is necessary.
Spam is filtered by the server before being downloaded by the user. PrismEmail offers a 30-day free trial. Oh, and it also uses my favourite, Bayesian statistical filtering. Might be worth a try.

Mail: MSGTAG Replies

Good software always seems to be controversial. That’s not to say there’s not two sides to the debate: Those who think Plaxo is a scam to get you to give up your private data aren’t exactly right, but they may not be exactly wrong, either: time will tell whether it becomes a great service or an intrusive nag. Similarly, another product I’ve taken to, MSGTAG, has its critics, who say allowing folk to check whether their emails have been opened is an unacceptable invasion of privacy, not least because most folk who receive such ‘tagged’ emails don’t know their email program has just sent a message home advising the sender they’ve just opened an email. (See a recent email from an outraged user.) All this is true, but it doesn’t undermine the idea that in principle, it’s a great idea. We would all be a lot more productive — not to mention safe — if we knew the emails we were sending out to friends, colleagues, customer service departments, actually reached their intended recipient.

Anyway, for those of you who are interested in hearing MSGTAG’s side of the debate, here’s their recent response to the letter I mentioned above. Original complaints in purple. I’ve cut it back a bit.

The sender has no real right to know when and if I read his email, where will this go next…tracking how often the email is open, tracking to whom I on forward the email…the possibilities are endless and tantamount to spying and invasion of privacy.

The MSGTAG read receipt process is not designed to be invasive. We feel that it is more than reasonable for a person to know if and when their mail has been read by the intended recipients. There are many situations where this benefits both the sender and the recipient. If an email hasn’t been read before a critical time, a sender can know to contact the recipient to give them the information by another means.

Our view on the subject of mail notification is that at the moment email is an unbalanced exchange. The recipient gets to read the email, but the sender doesn’t get to know if they have. If you send something via a courier service, for example, if you refuse to sign for it, you can’t open it. If you do sign for it, the sender knows straight away.

With MSGTAG we are trying to make it as fair as possible. There are some services that offer to give out all sorts of information about the recipient, such as how long the email was viewed for, how many times, who it was forwarded to, etc. Though we know how to implement this type of functionality, we have chosen a different path of fixing what we see as a broken process, without making the cure worse than the disease by adding privacy-invading features. The negative “possibilities are endless” for all sorts of technologies: we ask that we are judged by what we do, not by what can be done.

MSGTAG tells the sender only the time a message was first opened. It does not provide the sender with the IP address or geographical location of their recipients, nor does it embed tags into attachments to track forwarding or printing behaviour.

However, I do appreciate that not all Internet users wish to receive MSGTAG tagged emails. We respect the business decisions of companies such as yours that wish to implement firewall or proxy technology to prevent MSGTAG tags from being triggered. Furthermore, we have implemented a system within MSGTAG Status that allows users to disable tagging for certain recipients who have asked not to be tagged.

MSGTAG also collects the recipient’s email address, email ID, IP address and email headers without the recipient’s authorisation or knowledge.

It is true that we collect the recipient’s email address and the email ID – this is provided to us by the sender of the email. As I pointed out in the previous paragraph, we don’t collect the recipient’s IP address and we don’t have access to the header information except for:

The subject line – this is used in the notification email so that users know which e-mail has been read, without it they would only know that one of their emails has been read, but they wouldn’t know which one.

The message ID generated by the sender’s e-mail client – this is a unique code attached to all emails by most email clients so that the clients can reliably tell e-mails apart. We use it for the same purpose.

The address the e-mail was sent to – we use this for the same reason as the subject line – so the user knows which e-mail the notification is about.

We also record when the tag was added, and when it was triggered so that we can tell the users when it was triggered, and what the elapsed time was. That is all that we collect from the email.

I agree that what we do with the small amount of information we collect is a serious privacy issue. That is why we have a privacy policy publicly posted on our site. There are several prominent links to it, including within the application itself. I refer to the following relevant section of our Privacy Policy:

“MSGTAG facility
The Software uses the MSGTAG service to determine whether an e-mail that has been tagged by the Software has been received by the intended recipient. In order to achieve this, MSGTAG must store the subject, message ID, message recipient, date sent, and MSGTAG account name of the sender for each e-mail tagged by the Software. If tagging is disabled in the application, MSGTAG does not store this information. MSGTAG will not sell, share or rent this information to any other parties.”

At present, there is only one person in our organisation who has access to the email addresses used in MSGTAG – a System Administrator. As General Manager of MSGTAG, I do not have access. Tech support staff must ask the system administrator for this information on a case by case basis, in order to address specific problems raised by our customers.

We publicly state what happens to email addresses collected. They are only valuable to spammers. They are not valuable to us, because we abide by our Privacy Policy, and cannot exploit them. It would be commercial suicide for us to misuse the email addresses stored on our servers. The integrity of our brand is more valuable than a list of email addresses. Besides, we hate spam with a passion.

“This is in direct contravention to the privacy act and the rules governing the collection of personally identifiable information.”

We also feel that MSGTAG’s email tracking service is not only an invasion of our privacy but is also an infringement of the “Information Access” and “Computer Equipment Access” laws as their service provides “back-flow” traffic, without the recipient’s knowledge or consent, directly from their computer software and hardware.”

We are unaware of any infringement as per your suggestions. Fisher Young Group takes its obligations and allegations of this nature extremely seriously. If you can provide us with more information about the specific areas of law that are at dispute, we will investigate your concerns thoroughly.

Matthew Miller

Interesting stuff. Let us know how you feel.