Earlier this week, Felten made the observation that the government eavesdropping on Lavabit could be considered as an insider attack against Lavabit users. This leads to the obvious question: how might we design an email system that’s resistant to such an attack? The sad answer is that we’ve had this technology for decades but it never took off. Phil Zimmerman put out PGP in 1991. S/MIME-based PKI email encryption was widely supported by the late 1990’s. So why didn’t it become ubiquitous?
Usability. It’s a huge pain to set up and manage PGP encrypted email, even among a small group. See, for example, Whitten and Tygar’s paper on this exact topic. Closer to home for me, every member of the 2007 California Top to Bottom Review of electronic voting systems used OpenPGP for our internal communications. We even cheated and had a shared private key for our shared mail alias, rather than per-user private keys. While in the end it did work, particularly making it easy to clean things up when our work was finished — everybody just deleted their copy of the private key and we no longer had to worry about all the copies of the ciphertexts floating around — we had all sorts of weird hiccups along the way, and we’re talking about a group of security professionals. The S/MIME universe with hierarchical PKI seems to work reasonably well in closed, centrally administrated domains (i.e., internal email for a single company), but cross-domain secure communication again never really took off.
Features. I was an early beta user of Google’s Gmail and I was immediately hooked. Having instantaneous search over all my email was a powerful feature. Now, with a decade of my email all indexed and available, it’s invaluable. If I was using PGP or something like it, then I’d be giving up on all of Gmail’s search features. And then there’s Gmail’s truly effective spam filtering. Prior to Gmail, I spent a lot of effort training a local Bayesian filter, and it never came close to what Gmail does. In an encrypted world, the servers in the middle can’t help you as much with reducing spam. The more data they’ve got, the better they can protect you against spam, phishing, malware, etc. (These same benefits apply to many other webmail services; I’m not trying to argue for the superiority of Gmail relative to other webmail services.)
Speed. A big part of why Gmail is fast, even when you’re using a slow crappy connection, is that a lot of the data stays on the server. If somebody emails you a big attachment and you forward it on to somebody else, it’s never downloaded to your browser. It just gets passed along. Everything about webmail (or custom smartphone email apps) is built around speed. Conversely, with encrypted email, your client would need to download everything, assemble the new email, encrypt it, then push it back upstream. On a crappy connection, this would be unacceptably slow.
If you want to have communications where a man-in-the-middle attacker can’t read your messages, then you need to have local cryptographic secrets. There’s just no way around it. Even if you try to be clever, supporting features like search over encrypted data on the server, you’ll never approach the efficiency and features available when the server can see the plaintext of your email.
Is there an alternative? It might be possible to build a distributed social network that does the right thing. (Diaspora isn’t dead yet, but clearly isn’t ready for prime time. Daniel Sandler and I came up with something similar in 2008/2009 called FETHR. FETHR has very strong integrity but no privacy features. Frientegrity adds privacy.) One clever part of trying to build traditional public key cryptography into a social network is that the social network effectively implements a PGP-style web of trust. If Alice comments on one of Bob’s posts, and everything is digitally signed and hash-chained together, then you now have a path from Alice that implicitly endorses Bob’s public key. This effectively solves one of the hardest parts of the puzzle: scaling up the public key discovery and validation problem. Unfortunately, once you start properly encrypting messages, the server again can’t help you with spam and malware (although it’s easier to ignore people who you’re not “following”, which is a partial win). Neither the Diaspora, FETHR, nor Frientegrity designs make any attempt to unlink the sender from the recipient. Users could potentially follow one another through Tor, although that would put a lot of stress of Tor if the system got too big; it also wouldn’t help when you want to comment on a post with your real identity rather than anonymously. Potentially worse, every user in these systems is effectively divulging the volume of data they’re publishing; any post-facto observer of your published/encrypted timeline can see when you’re active and when you’re not.
Suffice to say, that at least for the foreseeable future, you won’t be abandoning your favorite webmail service, at least for the bulk of your emailing needs. If you anticipate needing to communicate securely, even in the face of compromised email servers, then you’re going to want to dive into the world of OpenPGP. If you want to defend against an observer reconstructing your social graph, off-the-shelf tools aren’t going to help you and you’ll need to cobble things together on your own.
A couple of tangental issues that should be considered but people
are either not thinking about or are ignoring.
Firstly is the problem of “roles”. A person has many seperate and often unrelated roles in life.
Having just on digital identity via a single Public Key as is being encoraged/mandated by many
goverments is dangerous. It is a single point of failure, but also it ties together roles in ways
that are distinctly undesirable. Most people keep a clear seperation from their personal life
and work life from their financial life, and even in their personal life many people segregate
different aspects, such as being secretary of the kniting club a member of a dance club/group
and their diet group or AA group etc.
Few is any web browsers or email clients alow for multiple roles in an easy to use and fool
proof way, we are not “prisoners” of our client software rules and we are definatly not a
number.
The second thing to consider is the email “inbox”. The first thought about “an inbox” is why?
and the second can we do without it altogether?
Look at it this way the inbox is not for the users conveniance but the mail systems conveniance
and in some respects it’s a single point of failure or target for the likes of snoopers and also a
choke point. In effect it forces what are many one-to-one corespondance relationships into a
many-to-one relationship into the “inbox” which then becomes a one-to-many relationship
with folders used to store the one-to-one messages in. Rather than a single inbox on a vulnerable
server how about many folders stored of nodes in an anonymous mix-net?
Anonymous folders on nodes in a mix-net are going to be hard to use NSLs etc against, and is
The first step in decoupaling the two communicating parties from any external observation.
Though the shutdown of Lavabit received a lot of attention, there may have been a precedent about eight years ago. In August 1999, Bruce Schneier mentioned the ZipLip e-mail service which could be accessed over SSL. Among other features was the ability to send a message for which the recipient would require a password to access. In 2005, the ZipLip service posted the following message:
—————
Ziplip Secure Services
Thank you for using ZipLip’s free secure mail
service. We appreciate your patronage and wish
to inform you that we will be discontinuing our
service on June 30th, 2005. For various reasons,
including new U.S. legislation which significantly
impacts the individual’s privacy rights, ZipLip is
no longer able to provide its free secure email
services with any reasonable assurance of privacy
and security, particularly in the context of a hosted
service. We will revisit the service issue when our
legislature reinstates our privacy rights.
Please make the necessary arrangements to use another
Webmail service before June 30th. We are unable to offer
any data migration services. We sincerely apologize for
any inconvenience caused.
—————
Though it is by no means clear, it seems reasonable that they did not want the situation of being required to secretly violate the privacy of users.
Mr. Wallach
I am a student in an ethics class that focus on using ethics with technology and am required to reply to a blog. This week yours has sparked my interest so to speak and I have a few comments for you.
In class we have started discussing the ethical frameworks used to determine wither something is ethical or not. In reading your blog here it sounds to me (correct me if I’m wrong) like you are using the “Consequentialist” framework in waging the cost vs benefit of using encrypted email vs unencrypted email. Thus stating that because even though encrypted email protects your privacy better than unencrypted email the difficulty of setting up encrypted email has made it less ethical to use.
My question for you is two fold. First am I correct in my deduction and second should we, as users, expect these programs to encrypt our data to protect our privacy because protecting our privacy is ethically right even if it hinders our use of the program (Deontology)?
Mr. Wallach
I am a student taking a class on philosophy and technology and I am assigned to comment on a blog posting of my choice and this week I have read yours.
The question you posed at the beginning of your post about creating a email server that can avoid the government looking at the information being shared between two users is interesting to me. In actuality I don’t think the government would even allow something to exist in this day and age like that. With technology progressing so fast I think its going to be hard for our government to keep up with the amount of access they are granted or not granted on the internet. Thats why when people want to create a emailing system or site that is completely private I still question whether or not certain details are being left out. I believe our government is going to have to start putting up more laws that will help them adapt to the on coming technological serge. The question I have for you is, do you believe that there is or will be a truly secure site on the web?
We have been discussing a lot about privacy and technology in my class and whether the accounts (like email) that we have on the internet are actually private. I wanted to also ask, do you believe this new system you posted about is more secure then others in the past? Is it possible to have a private account on the internet?
Hello, although I agree in general, I’d like to criticise a few points.
The PKI-driven x509 world is probably only easier to use, if you have a strictly enforced security policy in a tight hierachical organisation, eg. a company, with centrally configured PC and such. It is much more awkward, and I have only my own anecdotical evidence, to bring the various different platforms of different students towards using the same PKI, the same LDAP and have them register for signed certificates and create and properly install their own private keys with x509 certifcates than it is with PGP. Currently PGP interfaces very easily with Thunderbird/Enigmail and it takes you much less time to have everybody up to the point where they can communicate with each other (and you have explained a great deal about signed certificates along the way, as they are experiencing this step too for themselves).
As for the features section: It might be astonishing, but there are email clients out there that provide acceptable indexing functions and you will find that you receive even less spam than through google, if you throw away all messages that are not signed (by someone you “know” or who is “related” to you or your friends probably). In your (well, ideal) encrypted world, I assume all your private keys are kept within secured smart cards and a signature would be integrity protection as well as proof of work. Zero spam and 97% less email traffic within the Internet.
Aside from that, and from the slight google advertisement I think I agree somewhat, albeit I like to mention, that I have seen something similar in the struggle between telco and internet standardisation where one party argued in favour of having all intelligence within the network and the other wanted the smarts embedded in the endpoints. I am slightly leaning to the latter and am in favour of short trust-paths.
If you had the email store locally (think thunderbird) you could search it too.
Gmail is good, but it would be better if there was a way to force PFS DH for SSL SMTP. And it didn’t give the SSL private keys to the NSA.
There was a book about government written by Milton Freedman called “Tyranny of the Status Quo”. Geeks have noted “The Tyranny of the default”, so if people have to opt out, few will.
The problem creating secure email is not technical, but social.
Google – if it wanted to – could create GMail apps that would be open about the metadata and send only cleaned information to its servers, and still display ads without beign able to see the messages itself, just keywords. Yet it won’t do so. It wants to make it worse. If the robobigot thinks your name looks funny, it will lock out your G+ account until you produce your driver’s license. But you can’t review anything on Google Play without it. How long until you need to have your driver’s license name and picture on your google profile or GMail won’t work?
[IMAP vs web] “On a crappy connection, this would be unacceptably slow.”
It’s only “unacceptably” slow if you _decide_ that it is. And really, what fraction of your email involves large attachments? (That’s rhetorical; I guess everyone is going to see people-not-like-them as exceptional.)
The fact that the decryption needs to be done locally, is what makes it “the fastest thing you can get” rather than “unacceptably slow.” And the same argument pretty much applies to the features paragraph too.
Sure, by ditching basic security, we could have less spam in our inboxes. But we could also have even _less_ spam in our inboxes, if we ceased to use email altogether, and simply lived life without ever getting to communicate with anyone. Relaxing the constraints on the problem, saying “if we don’t need it to work right, then we can get benefit x,” seems like a strange choice, IMHO.
Excellent article Dan. I am very glad you published this so I can point to it when I try to explain why encrypted email is not ubiquitous and is quite clumsy even for experts, except in an enerprise situation where there is IT support covering all of the senders and receivers. Thanks.
PGP doesn’t protect against metadata collection.