Yesterday I started a thread on new wiretapping technologies, and their policy implications. Today I want to talk about how we should deal with the ability of governments to record and store huge numbers of intercepted messages.

In the old days, before there were huge, cheap digital storage devices, government would record an intercepted message only if it was likely to listen to that message eventually. Human analysts’ time was scarce, but recording media were relatively scarce too. The cost of storage tended to limit the amount of recording.

Before too much longer, Moore’s Law will enable government to record every email and phone call it knows about, and to keep the recordings forever. The cost of storage will no longer be a factor. Indeed, if storage is free but analysts’ time is costly, then the cost-minimizing strategy is to record everything and sort it out later, rather than spending analyst time figuring out what to record. Cost is minimized by doing lots of recording.

Of course the government’s cost is not the only criterion that wiretap policy should consider. We also need to consider the effect on citizens.

Any nontrivial wiretap policy will sometimes eavesdrop on innocent citizens. Indeed, there is a plausible argument that a well-designed wiretap policy will mostly eavesdrop on innocent citizens. If we knew in advance, with certainty, that a particular communication would be part of a terrorist plot, then of course we would let government listen to that communication. But such certainty only exists in hypotheticals. In practice, the best we can hope for is that, based on the best available information, there is some known probability that the message will be part of a terrorist plot. If that probability is just barely less than 100%, we’ll be comfortable allowing eavesdropping on that message. If the probability is infinitesimal, we won’t allow eavesdropping. Somewhere in the middle there is a threshold probability, just high enough that we’re willing to allow eavesdropping. We’ll make the decision by weighing the potential benefit of hearing the bad guys’ conversations, against the costs and harms imposed by wiretapping, in light of the probability that we’ll overhear real bad guys. The key point here is that even the best wiretap policy will sometimes listen in on innocent people.

(For now, I’m assuming that “we” have access to the best possible information, so that “we” can make these decisions. In practice the relevant information may be closely held (perhaps with good reason) and it matters greatly who does the deciding. I know these issues are important. But please humor me and let me set them aside for a bit longer.)

The drawbacks of wiretapping come in several flavors:
(1) Cost: Wiretapping costs money.
(2) Mission Creep: The scope of wiretapping programs (arguably) tends to increase over time, so today’s reasonable, well-balanced program will lead to tomorrow’s overreach.
(3) Abuse: Wiretaps can be (and have been) misused, by improperly spying on innocent people such as political opponents of the wiretappers, and by misusing information gleaned from wiretaps.
(4) Privacy Threat: Ordinary citizens will feel less comfortable and will feel compelled to speak more cautiously, due to the knowledge that wiretappers might be listening.

Cheap, high capacity storage reduces the first drawback (cost) but increases all the others. The risk of abuse seems particularly serious. If government stores everything from now on, corrupt government officials, especially a few years down the road, will have tremendous power to peer into the lives of people they don’t like.

This risk is reason enough to insist that recording be limited, and that there be procedural safeguards against overzealous recording. What limits and safeguards are appropriate? That’s the topic of my next post.

The revelation that the National Security Agency has been wiretapping communications crossing the U.S. border (and possibly within the U.S.), without warrants, has started many angry conversations across the country, and rightly so. Here is an issue that challenges our most basic conception of the purposes of government and its relation to citizens.

Today I am starting a series of posts about this issue. Most discussions of the wiretap program focus on two questions: (1) Is the program legal? and (2) Regardless of its legality, does the program, as currently executed, serve our national interest (bearing in mind the national interest in both national security and citizens’ privacy)? These questions are surely important, but I want to set them aside here. I’m setting aside the legal question because it’s outside my expertise. I’m setting aside any evaluation of the current program for two reasons. First, we don’t know the exact scope of the current wiretap program. Second, most people – on both sides – think the second question is an easy one, and easy questions lead to boring conversations.

I want to focus instead on the more basic questions of what the extent of national security wiretapping should be, and why. The why question is especially important.

The first thing to realize is that this is not your parents’ wiretap debate. Though the use (and sometimes misuse) of wiretapping has long been a contentious issue, the terms of the debate have changed. I’m not referring here to the claim that 9/11 changed everything. What I mean is that wiretapping technology has changed in ways that ought to reframe the debate.

Two technology changes are important. The first is the dramatic drop in the cost of storage, making it economical to record vast amounts of communications traffic. The second technology change is the use of computer algorithms to analyze intercepted communications. Traditionally, a wiretap would be heard (or read) immediately by a person, or recorded for later listening by a person. Today computer algorithms can sift through intercepted communications, looking for sophisticated patterns, and can select certain items to be recorded or heard by a person.

Both changes are driven by Moore’s Law, the rule of thumb that the capability of digital technologies doubles every eighteen months or, equivalently, improves by a factor of 100 every ten years. This means that in 2016 government will be able to store 100 times more intercepted messages, and will be able to devote 100 times more computing capability to its analysis algorithms, compared to today. If the new world of wiretapping has not entirely arrived, it will be here before long.

So government will have greater eavesdropping capabilities and, more interestingly, it will have different capabilities. How should we respond? Surely it is not right simply to let government do whatever it wants – this has never been our policy. Nor can it be right to let government do no wiretapping at all – this has not been our policy either. What we need to understand is where to draw the line, and what kind of oversight and safeguards we need to keep our government near the line we have drawn. I hope that the next several posts can shed some small amount of light on these questions.