When I teach computer security, one of the first lessons is on the need to have a clear threat model, that is, a clearly defined statement of which harms you are trying to prevent, and what assumptions you are making about the capabilities and motivation of the adversaries who are trying to cause those harms. Many security failures stem from threat model confusion. Conversely, a good threat model often shapes the solution.
The same is true for security research: the solutions you develop will depend strongly on what threat you are trying to address.
Lately I’ve noticed more and more papers in the computer security research literature that include subpoenas and/or search warrants as part of their threat model. For example, the Vanish paper, which won Best Student Paper (the de facto best paper award) at the recent Usenix Security symposium, uses the word “subpoena” 13 times, in passages like this:
Attackers. Our motivation is to protect against retroactive data disclosures, e.g., in response to a subpoena, court order, malicious compromise of archived data, or accidental data leakage. For some of these cases, such as the subpoena, the party initiating the subpoena is the obvious “attacker.” The final attacker could be a user’s ex-husband’s lawyer, an insurance company, or a prosecutor. But executing a subpoena is a complex process involving many other actors …. For our purposes we define all the involved actors as the “adversary.”
(I don’t mean to single out this particular paper. This is just the paper I had at hand — others make the same move.)
Certainly, subpoenas are no fun for any of the parties involved. They’re costly to deal with, not to mention the ick factor inherent in compelled disclosure to a stranger, even if you’re totally blameless. And certainly, subpoenas are sometimes used to harass, rather than to gather legitimately relevant evidence. But are subpoenas really the biggest threat to email confidentiality? Are they anywhere close to the biggest threat? Almost certainly not.
Usually when the threat model mentions subpoenas, the bigger threats in reality come from malicious intruders or insiders. The biggest risk in storing my documents on CloudCorp’s servers is probably that somebody working at CloudCorp, or a contractor hired by them, will mess up or misbehave.
So why talk about subpoenas rather than intruders or insiders? Perhaps this kind of talk is more diplomatic than the alternative. If I’m talking about the risks of Gmail, I might prefer not to point out that my friends at Google could hire someone who is less than diligent, or less than honest. If I talk about subpoenas as the threat, nobody in the room is offended, and the security measures I recommend might still be useful against intruders and insiders. It’s more polite to talk about data losses that are compelled by a mysterious, powerful Other — in this case an Anonymous Lawyer.
Politeness aside, overemphasizing subpoena threats can be harmful in at least two ways. First, we can easily forget that enforcement of subpoenas is often, though not always, in society’s interest. Our legal system works better when fact-finders have access to a broader range of truthful evidence. That’s why we have subpoenas in the first place. Not all subpoenas are good — and in some places with corrupt or evil legal systems, subpoenas deserve no legitimacy at all — but we mustn’t lose sight of society’s desire to balance the very real cost imposed on the subpoena’s target and affected third parties, against the usefulness of the resulting evidence in administering justice.
The second harm is to security. To the extent that we focus on the subpoena threat, rather than the larger threats of intruders and insiders, we risk finding “solutions” that fail to solve our biggest problems. We might get lucky and end up with a solution that happens to address the bigger threats too. We might even design a solution for the bigger threats, and simply use subpoenas as a rhetorical device in explaining our solution — though it seems risky to mislead our audience about our motivations. If our solution flows from our threat model, as it should, then we need to be very careful to get our threat model right.
Considering the case of subpoenas does provide a useful rhetorical model for partitioning data against internal threats, but it doesn’t seem terribly disingenuous to focus some effort on a subpoena case versus an insider threat.
Your first argument that it is in society’s interest to have users’ private information stored in a way that is accessible to a service provider, “Our legal system works better when fact-finders have access to a broader range of truthful evidence.” echoes arguments against the freedom of members of ‘our’ society to use cryptography in general, while later pointing out that ‘other’ societies may have better arguments for wanting that freedom.
The second point is certainly valid, excess consideration of any one characterization of unwanted data access is not a solid way to design an entire system that is secure. In the case of the Vanish paper, I don’t think they’re proposing an entire system, merely a subsystem that addresses a more specific cryptographic task.
When discussing a distributed, expiring key system like that proposed in the Vanish paper, using the subpoena as a rhetorical device is simply useful, and could be a potential selling point to users of that system. However, I might propose that a cryptographic key witch hunt metaphor or perhaps a spanish crypto-inquisition device where a service provider is compelled to confess their master unlock code amongst imagery of thumbscrews and a rack may be what you’re looking for in this discussion. Are we really so far from those chapters of ‘our society’ that they need not be considered in that balance of the “very real cost imposed on the subpoena’s target and affected third parties, against the usefulness of the resulting evidence in administering justice”?
In my opinion, classifying subpoenas and search warrants as threats to email security is a good eye-opener. No matter how light a threat is, it should be taken seriously because you don’t know what bigger damage they might lead to. Most cases of security breaches happen at a time you don’t expect them to happen. But seriously though, there is no security system in the world that could be considered absolute. There can only be minimum security, medium security and maximum security whose components are aligned appropriately to the the threats each level of security is facing. The only thing that separates these levels of security is the degree of difficulty in breaching them. Obviously, maximum security levels are the most difficult ones to breach and are usually the most expensive to implement and put in place. But an adversary who has the resources and sheer determination may find a loophole and exploit the system to his advantage. Computer security practitioners always strive to formulate security systems which cover a wide array of threats while trying to minimize the costs involved by the very security systems being put in place. This would be a case of a good balancing act which is not easy to do. Going back to subpoenas and search warrants, I cannot think of any security system that cannot be penetrated by those. Hmm…should we go back to the good old days of communicating through papers? Just my thoughts. Vince at registry cleaners station
“…if any member violates a local law.”
and how do you know before trial that they’ve violated a local law?
Here’s a recent case where a subpoena seems to be pretty chilling: http://wikileaks.org/wiki/Gmail_may_hand_over_IP_addresses_of_journalists
I’m still not convinced that “rubber-stamped” subpoenas shouldn’t be ruled out as a threat… yes, there are mechanisms to discourage and penalize abusive subpoenas but that’s often too late.
What if someone either threatens or bribes an insider? They may not be malicious in the traditional sense. But if the data is there and can be released by some third party, it is a risk. Or negligence like social engineering attacks. Anything I can get you to release under subpoena may be something I can get you to do under other conditions. And I would probably trust you, but what about some new hire whose wife might have a gambling problem?
A subpoena is something which can get you to breach security and/or privacy against your will and with your full knowledge. If you cannot do so under these conditions, it is far less likely that the data would be released even if you wanted to and knew about it.
Ed, unless I’m misreading your post, you seem to have switched horses in midstream here. You go from:
Lately I’ve noticed more and more papers in the computer security research literature that include subpoenas and/or search warrants as part of their threat model.
(my emphasis), to:
But are subpoenas really the biggest threat to email confidentiality? Are they anywhere close to the biggest threat? Almost certainly not.
(my emphasis again)
Am I missing something here? Is there a reason a paper that “includes” subpoenas as “part of” its threat model is somehow implying that they are “the biggest threat” to email security?
Subpoena’s become a very interesting threat when dealing with outsourcing be it traditional outsourcing or cloud. In many cases, there does not need to be any notification to the data owner when the provider receives a subpoena. So that is certainly something to consider when using these services, particularly if you are the data you “own” falls under compliance regulations of one sort or another.
is to the ongoing business of the organization. At least some of the methods used to “protect” against subpoenas (e.g. autodeletion of old email) damage ongoing work and/or require extensive workarounds (documented or undocumented) to keep projects going. (I once worked for a company that was trying to implement a 30-day deletion policy — the only problem was that most of our section’s business involved projects lasting from 3 to 12 months, conducted almost entirely by email. Some people ignored the policy, others just copied the contents of their mail to other files not subject to mandatory deletion.)
I agree with your basic point, Ed, that subpoenas are rarely an accurate characterization of the most likely or most serious threat. However, they have one property which can make them an attractive threat model component, under some circumstances: they represent, in a sense, the limiting case of a certain type of attack. Intruders or malicious insiders can be constrained by internal security mechanisms, but the strength, consistency and effectiveness of these mechanisms is rarely easy to characterize with any precision. A subpoena, on the other hand, corresponds to a maximally powerful intruder or insider, who’s completely unconstrained by those internal security mechanisms. A proposed security tool that fails to defend against the “subpoena attack” may still be useful against more conventional attacks. But a security tool that does defend against subpoenas clearly works against both insiders and intruders as well.
Dan,
I see your point that subpoenas can be a limiting case for some classes of threats. But there are important threat classes for which they are not a limiting case.
Subpoena threats have two important characteristics. First, they apply to information from a single individual user — so they’re not a limiting case for attacks that gather information from many users at once. Second, subpoenas are backward-looking, wanting to retrieve information from the past — so they’re not a limiting case for attacks that gather or log information in real time, in the hope that it will be useful (to the attacker) later.
The Vanish system is a good example here. Vanish is designed to prevent backward-looking information gathering attacks against an individual user. That’s a good match for the subpoena threat, and other threats with the same characteristics, but it’s a poor match for other kinds of attacks.
Are you sure it’s accurate to say that subpoenas can only demand information about a single user? That doesn’t sound right to me. Think about a Google who stores data on many users; a subpoena could demand information from Google about multiple of its users.
I do see your point about how subpoenas shouldn’t be your main focus, but I’m not sure that I agree that the subpoena shouldn’t be a major concern.
Too often subpoenas are used as attack methods by litigation happy lawyers. If you don’t have a strategy for how you are going to deal with an attack coming from that vector, you are going to be left in a bad situation. A subpoena is one of those things you have no real choice about complying with. How many of the big tech companies have been sued or attacked in some other manner through the court system this year alone?
Many corporations regard managing what can be acquired with a subpoena as one of its IT department’s tasks. This sounds more unsavory than it is. During litigation, the court is generally loath to suppress a subpoena, and therefore the litigants can get license to troll through everything they can get their hands on. Given that litigation is the hardest of hardballs, they’ll be happy to find anything embarrassing or damaging. Given the frequency with which corporations are involved in litigation, it is worthy of thinking about.
Sometimes corporations don’t want to be left open to a subpoena for the same reason that people who aren’t doing anything illegal can still be worried about privacy.
All that being said, I agree with you that including subpoenas in the “threat model” sounds strange.