February 21, 2018

Archives for February 2018

(Mis)conceptions About the Impact of Surveillance

Does surveillance impact behavior? Or is its effect, if real, only temporary or trivial? Government surveillance is back in the news thanks to the so-called “Nunes memo”, making this is a perfect time to examine new research on the impact of surveillance. This includes my own recent work, as my doctoral research at the Oxford Internet Institute, University of Oxford  examined “chilling effects” online, that is, how online surveillance, and other regulatory activities, may impact, chill, or deter people’s activities online.

Though the controversy surrounding the Nunes memo critiquing FBI surveillance under the Foreign Intelligence Surveillance Act (FISA) is primarily political, it takes place against the backdrop of the wider debate about Congressional reauthorization of FISA’s Section 702, which allows the U.S. Government to intercept and collect emails, phone records, and other communications of foreigners residing abroad, without a warrant. On that count, civil society groups have expressed concerns about the impact of government surveillance like that available under FISA, including “chilling effects” on rights and freedoms. Indeed, civil liberties and rights activists have long argued, and surveillance experts like David Lyon long explained, that surveillance and similar threats can have these corrosive impacts.

Yet, skepticism about such claims is common and persistent. As Kaminski and Witov recently noted, many “evince skepticism over the effects of surveillance” with deep disagreements over the “effects of surveillance” on “intellectual queries” and “development”.  But why?  The answer is complicated but likely lies in the present (thin) state of research on these issues, but also common conceptions, and misconceptions, about surveillance and impact on people and broader society.

Skepticism and assumptions about impact
Skepticism about surveillance impacts like chilling effects is, as noted, is persistent with commentators like Stanford Law’s Michael Sklansky insisting there “little empirical support” for chilling effects associated with surveillance or Leslie Kendrick, of UVA Law, labeling the evidence supporting such claims “flimsy” and calling for more systematic research on point. Part of the problem is precisely this: the impact of surveillance—both mass and targeted forms—is difficult to document, measure, and explore, especially chilling effects or self-censorship. This is because demonstrating self-censorship or chill requires showing a counterfactual state of affairs: that a person would have said something or done something but for some surveillance threat or awareness.

But another challenge, just as important to address, concerns common assumptions and perceptions as to what surveillance impact or chilling effects might look like. Here, both members of the general public as well as experts, judges, and lawyers often assume or expect surveillance to have obvious, apparent, and pervasive impact on our most fundamental democratic rights and freedoms—like clear suppression of political speech or the right to peaceful assembly.

A great example of this assumption, leading to skepticism about whether surveillance may promote self-censorship or have broader societal chilling effects—is here expressed by University of Chicago Law’s Eric Posner. Posner, a leading legal scholar who also incorporates empirical methods in his work, conveys his skepticism about the “threat” posed by National Security Agency (NSA) surveillance in a New York Times “Room for Debate”  discussion, writing:

This brings me to another valuable point you made, which is that when people believe that the government exercises surveillance, they become reluctant to exercise democratic freedoms. This is a textbook objection to surveillance, I agree, but it also is another objection that I would place under “theoretical” rather than real.  Is there any evidence that over the 12 years, during the flowering of the so-called surveillance state, Americans have become less politically active? More worried about government suppression of dissent? Less willing to listen to opposing voices? All the evidence points in the opposite direction… It is hard to think of another period so full of robust political debate since the late 1960s—another era of government surveillance.

For Posner, the mere existence of “robust” political debate and activities in society is compelling evidence against claims about surveillance chill.

Similarly, Sklansky argues not only that there is “little empirical support” for the claim that surveillance would “chill independent thought, robust debate, personal growth, and intimate friendship”— what he terms “the stultification thesis”—but like Posner, he finds persuasive evidence against the claim “all around us”. He cites, for example, the widespread “sharing of personal information” online (which presumably would not happen if surveillance was having a dampening effect); how employer monitoring has not deterred employee emailing nor freedom of information laws deterred “intra-governmental communications”; and how young people, the “digital natives” that have grown up with the internet, social media, and surveillance, are far from stultified and conforming but arguably even more personally expressive and experimental than previous generations.  In light of all that, Sklansky dismisses surveillance chill as simply not “worth worrying about”.

I sometimes call this the “Orwell effect”—the common assumption, likely thanks to the immense impact Orwell’s classic novel 1984 has had on popular culture, that surveillance will have dystopian societal impact, with widespread suppression of personal sharing, expression, and political dissent. When Posner and Sklansky (and others that share these common expectations) do not see these more obvious and far reaching impacts, they then discount more subtle and less apparent impacts and effects that may, over the long term, be just as concerning for democratic rights and freedoms. Of course, theorists and scholars like Daniel Solove have long interrogated and critiqued Orwell’s impact on our understanding of privacy and Sklansky is himself wary of Orwell’s influence, so it is no surprise his work also shapes common beliefs and conceptions about the impact of surveillance.  That influence is compounded by the earlier noted lack of systematic empirical research providing more grounded insights and understanding.

This is not only an academic issue. Government surveillance powers and practices are often justified with reference to other national security concerns and threats like terrorism, as this House brief on the FISA re-authorization illustrates. If concerns about chilling effects associated with surveillance and other negative impacts are minimized or discounted based on misconceptions or thin empirical grounding, then challenging surveillance powers and their expansion is much more difficult, with real concrete implications for rights and freedoms.

So, the challenge for documenting, exploring, and understanding the impact of surveillance is really two-fold. The first is one of research methodology and design: designing research to document the impact of surveillance, and a second concerns common assumptions and perceptions as to what surveillance chilling effects might look like—with even experts like Posner or Sklansky assuming widespread speech suppression and conformity due to surveillance.

New research, new insights
Today, new systematic empirical research on the impact of surveillance is being done, with several recent studies having documented surveillance chilling effects in different contexts, including recent studies by  Stoycheff [1], Marthews and Tucker [2], as well as my own recent research.  This includes an empirical legal study[3] on how the Snowden revelations about NSA surveillance impacted Wikipedia use—which received extensive media coverage in the U.S. and internationally— and a more recent study[4], which I wrote about recently in Slate, that examined among other things how state and corporate surveillance impact or “chill” certain people or groups differently. A lot of this new work was not possible in previous times, as it is based on new forms of data being made available to researchers and insights gleaned from analyzing public leaks and disclosures concerning surveillance like the Snowden revelations.

The story these and other new studies tell when it comes to the impact of surveillance is more complicated and subtle, suggesting the common assumptions of Posner and Sklansky are actually misconceptions. Though more subtle, these impacts are no less concerning and corrosive to democratic rights and freedoms, a point consistent with the work of surveillance studies theorists like David Lyon[5] and warnings from researchers at places like the Citizen Lab[6], Berkman Klein Center[7], and here at the CITP[8].  In subsequent posts, I will discuss these studies more fully, to paint a broader picture of surveillance effects today and, in light of increasingly sophisticated targeting and emerging automation technologies, tomorrow. Stay tuned.

* Jonathon Penney is a Research Affiliate of Princeton’s CITP, a Research Fellow at the Citizen Lab, located at the University of Toronto’s Munk School of Global Affairs, and teaches law as an Assistant Professor at Dalhousie University. He is also a research collaborator with Civil Servant at the MIT Media Lab. Find him on twitter at @jon_penney

[1] Stoycheff, E. (2016). Under Surveillance: Examining Facebook’s Spiral of Silence Effects in the Wake of NSA Internet Monitoring. Journalism & Mass Communication Quarterly. doi: 10.1177/1077699016630255

[2] Marthews, A., & Tucker, C. (2014). Government Surveillance and Internet Search Behavior. MIT Sloane Working Paper No. 14380.

[3] Penney, J. (2016). Chilling Effects: Online Surveillance and Wikipedia Use. Berkeley Tech. L.J., 31, 117-182.

[4] Penney, J. (2017). Internet surveillance, regulation, and chilling effects online: A comparative case study. Internet Policy Review, forthcoming

[5] See for example: Lyon, D. (2015). Surveillance After Snowden. Cambridge, MA: Polity Press; Lyon, D. (2006). Theorizing surveillance: The panopticon and beyond. Cullompton, Devon: Willan Publishing; Lyon, D. (2003). Surveillance After September 11. Cambridge, MA: Polity. See also Marx, G.T., (2002). What’s New About the ‘New Surveillance’? Classifying for Change and Continuity. Surveillance & Society, 1(1), pp. 9-29;  Graham, S. & D. Wood. (2003). Digitising Surveillance: Categorisation, Space, Inequality, Critical Social Policy, 23(2): 227-248.

[6] See for example, recent works: Parsons, C., Israel, T., Deibert, R., Gill, L., and Robinson, B. (2018). Citizen Lab and CIPPIC Release Analysis of the Communications Security Establishment Act. Citizen Lab Research Brief No. 104, January 2018; Parsons, C. (2015). Beyond Privacy: Articulating the Broader Harms of Pervasive Mass Surveillance. Media and Communication, 3(3), 1-11; Deibert, R. (2015). The Geopolitics of Cyberspace After Snowden. Current History, (114) 768 (2015): 9-15; Deibert, R. (2013) Black Code: Inside the Battle for Cyberspace, (Toronto: McClelland & Stewart).  See also

[7] See for example, recent work on the Surveillance Project, Berkman Klein Center for Internet and Society, Harvard University.

[8] See for example, recent work: Su, J., Shukla, A., Goel, S., Narayanan, A., De-anonymizing Web Browsing Data with Social Networks. World Wide Web Conference 2017; Zeide, E. (2017). The Structural Consequences of Big Data-Driven Education. Big Data. June 2017, 5(2): 164-172, https://doi.org/10.1089/big.2016.0061;MacKinnon, R. (2012) Consent of the networked: The worldwide struggle for Internet freedomNew YorkBasic Books.; Narayanan, A. & Shmatikov, V. (2009). See also multiple previous Freedom to Tinker posts discussing research/issues point.

 

Software-Defined Networking: What’s New, and What’s New For Tech Policy?

The Silicon Flatirons Conference on Regulating Computing and Code is taking place in Boulder. The annual conference addresses a range of issues at the intersection of technology and policy and provides an excellent look ahead to the tech policy issues on the horizon, particularly in telecommunications.

I was looking forward to yesterday’s panel on “The Triumph of Software and Software-Defined Networks”, which had some good discussion on the ongoing problem surrounding security and privacy of the Internet of Things (IoT); some of the topics raised echoed points made on a Silicon Flatirons panel last year. My colleague and CITP director Ed Felten made some lucid, astute points about the implications of the “infiltration” of software into all of our devices.

Unfortunately, though (despite the moderator’s best efforts!), the panel lacked any discussion of the forthcoming policy issues concerning Software-Defined Networking (SDN); I was concerned with some of the incorrect comments concerning SDN technology. Oddly, two panelists stated that Software Defined Networking has offered “nothing new”. Here’s one paper that explains some of the new concepts that came from SDN (including the origins of those ideas), and another that talks about what’s to come as machine learning and automated decision-making begin to drive more aspects of network management. Vint Cerf corrected some of this discussion, pointing out one example of a fundamentally new capability: the rise of programmable hardware. One of same panelists also said that SDN hasn’t seen any deployments in the wide-area Internet or at interconnection, a statement that has many counter-examples, including projects such as SDX (and the related multi-million dollar NSF program), Google’s Espresso and B4, and Facebook’s Edge Fabric to name just a few of the public examples.

Some attendees commented that the panel could have discussed how SDN, when coupled with automated decision-making (“AI” in the parlance du jour) presents both new opportunities and challenges for policy. This post attempts to bring some of the issues at the intersection of SDN and policy to light. I address two main questions:

  1. What are the new technologies around SDN that people working in tech policy might want to know about?;
  2. What are some interesting problems at the intersection of SDN and tech policy?

The first part of the post summarizes about 15 years of networking research in three paragraphs, in a form that policy and law scholars can hopefully digest; the second part of the post are some thoughts about new and interesting policy and legal questions—both opportunities and challenges—that these new technologies bring to bear.

SDN: What’s New in Technology?

Software-defined networking (SDN) describes a type of network design where a software program runs separately from the underlying hardware routers and switches can control how traffic is forwarded through the network. While in some sense, one might think of this concept as “nothing new” (after all, network operators have been pushing configuration to routers with Perl scripts for decades), SDN brings several new twists to the table:

  1. The control of a collection of network devices from a single software program, written in a high-level programming language. The notion that many devices can be controlled from a single “controller” creates the ability for coordinated decisions across the network, as opposed to the configuration of each router and switch essentially being configured (and acting) independently. When we first presented this idea for Internet routing in the mid-2000s, this was highly controversial, with some even claiming that this was “failed phone company thinking” (after all, the Internet is “decentralized”; this centralized controller nonsense could only come from the idiots working for the phone company!). Needless to say, the idea is a bit less controversial now. These ideas have taken hold both within the data center, in the wide area, and at interconnection points; technology such as the Software Defined Internet Exchange Point (SDX) makes it possible for networks to exchange traffic only for specific applications (e.g., video streaming), for example, or to route traffic for different application along different paths.
  2. The emergence of programmable hardware in network devices. Whereas conventional network devices relied on forwarding performed by fixed-function ASICs, the rise of companies such as Barefoot Networks have made it possible for network architects to customize forwarding behavior in the network. This capability is already being used for designing network architectures with new measurement and forwarding capabilities, including the ability to get detailed timing information of individual packets as they traverse each network hop, as well as to scale native multicast to millions of hosts in a data center.
  3. The rise of automated decision making in network management (“AI Meets Networking”). For years, network operators have applied machine learning to conventional network security and provisioning problems, including the automated detection of spam, botnets, phishing attacks, bullet-proof web hosting, and so forth. Operators can also use machine learning to help answer complex “what if” performance analysis questions, such as what would happen to web page load or search response time if a server was moved from one region to another, or if new network capacity was deployed. Much of this work, however, has involved developing systems that perform detection in an offline fashion (i.e., based on collected traces). Increasingly, with projects like Google Espresso and Facebook Edge Fabric, we’re starting to see systems that close the loop between measurement and control. It likely won’t be long before networks begin making these kinds of decisions based on even more complex inputs and inferences.

SDN: What’s New for Tech Policy?

The new capabilities that SDN offers presents a range of potentially challenging questions at the intersection of technology, policy, and law.  I’ve listed a few of these interesting possibilities below:

  • Service Level Agreements. A common contractual instrument for Internet Service Providers (ISPs) is the Service Level Agreement (SLA). SLAs typically involve guarantees about network performance: packet loss will never exceed a certain amount, latency will always be less than a certain amount, and so forth. SDN presents both new opportunities and challenges for Service Level Agreements. On the opportunity side, SDN creates the ability for operators to define much more complex traffic forwarding behavior—sending traffic along different paths according to destination, application, or even the conditions of individual links along and end-to-end path at a particular time.

    Yet, the opportunity to create these types of complex SLAs also presents challenges.When all routing and forwarding decisions become automated, and all interconnects look like Google Espresso, where an algorithm is effectively making decisions about where to forward traffic (perhaps based on a huge list of features ranging from application QoE to estimates of user attention, and incorporated into an inscrutable “deep learning” model), how does one go about making sure the SLA continues to be enforced?What new challenges and opportunities do the new capabilities of programmable measurement bring for SLAs? Some aspects of SLAs are notoriously difficult to enforce today.

    Not only will complex SLAs become easier to define, the rise of programmable measurement and advancements in network telemetry will also make SLAs easier for customers to validate. There are huge opportunities in the validation of SLAs, and once these become easier to audit, a whole new set of legal and policy questions will arise.

  • Network Neutrality. Although the Federal Communication Commission (FCC)’s release of the Restoring Internet Freedom order earlier this year effectively rescinds many of the “bright line rules” that we have come to associate with net neutrality (i.e., no blocking, no throttling, no paid prioritization), the order actually leaves in place many transparency requirements for ISPs, requiring ISPs to disclose any practices relevant to blocking, throttling, prioritization, congestion management, application-specific behavior, and security. As with SLA definition and enforcement, network neutrality—and particularly the transparency rule—may become more interesting as SDN makes it possible for network operators to automate network decision-making, as well as for consumers to audit some of the disclosures (or lack thereof) from ISPs. SDX allows networks to make decisions about interconnection, routing, and prioritization based on specific applications, which creates new traffic management capabilities that raise interesting questions in the context of net neutrality; which of these new capabilities would constitute an exception for “reasonable network management practices”, and which might be viewed as discriminatory?

    Additionally, the automation of network management may make it increasingly difficult for operators to figure out what is going on (or why?), and some forwarding decisions may be more difficult to understand or explain if they are driven by a complex feature set and fully automated. Figuring out what “transparency” even means in the context of a fully automated network is a rich area for exploration at the intersection of network technology and telecommunications policy. Even things seemingly as simple as “no blocking” get complicated when networks begin automating the mitigation of attack traffic, or when content platforms begin automating takedown requests. Does every single traffic flow that is blocked by a network intrusion detection system need to be disclosed? How can ISPs best disclose the decision-making process for each blocking decision, particularly when either (1) the algorithm or set of features may be difficult to explain or understand; (2) doing so might aid those who aim to circumvent these network defenses?

Virtualization: A Topic in Its Own Right. The panel moderator also asked a few times about policy and legal issues that arise as a result of virtualization. This is a fantastic question that deserves more attention. It’s worth pointing out the distinction between SDN (which separates network “control plane” software from “data plane” routers and devices) from virtualization (which involves creating virtual server and network topologies on a shared underlying physical network). In short, SDN enables virtualization, but the two are distinct technologies. Nonetheless, virtualization presents many interesting issues at the intersection of technology and policy in its own right. For one, the rise of Infrastructure as a Service (IaaS) providers such as Amazon Web Services, as well as other multi-tenant data centers, introduce questions such as how to enforce SLAs when isolation is imperfect, as well as how IaaS providers can be stewards of potentially private data that may be subject to takedown requests, subpoenas, and other actions by law enforcement and other third parties. The forthcoming Supreme Court case, Microsoft vs. United States, concerning law enforcement access to data stored abroad. Does the data actually live overseas, or this this merely a side effect of global, virtualized data centers? As virtualization is a distinct topic from SDN, the policy issues it presents warrant a separate (future) post.

Summary. In summary, SDN is far from old news, and the policy questions that these new technologies bring to bear are deeply complex and deserve a careful eye from experts at the intersection of policy, law, and technology. We should start these conversations now.

Making Sense of Child Protection Predictive Models: Tech-Soc Reading Group Feb 20

How are predictive models transforming how we think about child protection, and how should we think about the role of such systems in a democracy?

If you’re interested to ask these questions, join us at 2-3pm on Tuesday, Feb 20th at Sherrerd Hall room 306 for our opening Technology and Society Reading group meeting. The conversation is open to anyone who’s interested, and who’s willing to do the reading in advance.  (if you’re interested, come for our lunch talk that day as well, which focuses on privacy & safety concerning intimate partner violence)

In 2016, Allegheny County, which includes the city of Pittsburgh, became the first region in the US to use a predictive model to “identify families most in need of intervention” for their department of  Children, Youth and Families. These models create a risk score for children who might be at risk of violence and guide decisions about followup by government employees. In January 2018, three very different perspectives  were published widely about this system: a New York Times Magazine article, a chapter in Virginia Eubanks’s new book Automating Inequality (blog post here), and a new academic paper by the creators of the system that outlines the work they’ve done to make the system transparent and fair.

About the Technology and Society Reading Group

At the Center for Information Technology Policy, we bring together people with expertise in technology, engineering, policy, and the social sciences to ask those questions and develop new approaches together.  You can sign up for the mailing list here, where you will receive reading material for discussion before the event.

This semester, conversations will be organized by CITP fellows J. Nathan Matias and Ben Zevenbergen.

If you have ideas for a conversation you would like to see us discuss, please email Nathan () and Ben () with suggestions, and some references that you think might be interested

Time: every other Tuesday from 2pm-3pm, starting Feb 20th