July 19, 2018

When the business model *is* the privacy violation

Sometimes, when we worry about data privacy, we’re worried that data might fall into the wrong hands or be misused for unintended purposes. If I’m considering participating in a medical study, I’d want to know if insurance companies will obtain the data and use it against me. In these scenarios, we should look for ways to preserve the intended benefit while preventing unintended uses. In other words, achieving utility and privacy is not a zero-sum game. [1]

In other situations, the intended use is the privacy violation. The most prominent example is the tracking of our online and offline habits for targeted advertising. This business model is exactly what people object to, for a litany of reasons: targeting is creepy, manipulative, discriminatory, and reinforces harmful stereotypes. The data collection that enables targeted advertising involves an opaque surveillance infrastructure to which it’s impossible to give meaningfully informed consent, and the resulting databases give a few companies too much power over individuals and over democracy. [2]

In response to privacy laws, companies have tried to find technical measures that obfuscate the data but allow them carry on with the surveillance business as usual. But that’s just privacy theater. Technical steps that don’t affect the business model are of limited effectiveness, because the business model is fundamentally at odds with privacy; this is in fact a zero-sum game. [3]

For example, there’s an industry move to replace email addresses and other personal identifiers with hashed versions. But a hashed identifier is nevertheless a persistent, unique identifier that allows linking a person across databases, devices, and contexts, as well as targeting and manipulation on the basis of the associated data. Thus, hashing completely fails to address the underlying privacy concerns.

Policy makers and privacy advocates must recognize when privacy is a zero-sum game and when it isn’t. Policy makers like non-zero sum games because they can simultaneously satisfy different stakeholders. But they must acknowledge that sometimes this isn’t possible. In such cases, laws and regulations should avoid loopholes that companies might exploit by building narrow technical measures and claiming to be in compliance. [4]

Privacy advocates should recognize that framing a concern about data use practices as a privacy problem is a double-edged sword. Privacy can be a convenient label for a set of related concerns, but it gives industry a way to deflect attention from deeper ethical questions by interpreting privacy narrowly as confidentiality.

Thanks to Ed Felten and Nick Feamster for feedback on a draft.


[1] There is a vast computer science privacy literature predicated on the idea that we can have our cake and eat it too. For example, differential privacy seeks to enable analysis of data in the aggregate without revealing individual information. While there are disagreements on the specifics, such as whether de-identification results a win-win outcome, there is no question that the overall direction of privacy-preserving data analysis is an important one.

[2] In Mark Zuckerberg’s congressional testimony, he framed Facebook’s privacy woes as being about improper third-party access to the data. This is arguably a non-zero sum game, and one that Facebook is equipped to address without the need for legislation. However, the much bigger privacy problem is Facebook’s own data collection and business model, which is inherently at odds with privacy and is unlikely to be solved without legislation.

[3] There are research proposals for targeted advertising, such as Adnostic, that would improve privacy by drastically changing the business model, largely cutting out the tracking companies. Unsurprisingly, there has been no interest in these approaches from the traditional ad tech industry, but some browser vendors have experimented with similar ideas.

[4] As an example of avoiding the hashing loophole, the 2012 FTC privacy report is well written: it says that for data to be considered de-identified, “the company must achieve a reasonable level of justified confidence that the data cannot reasonably be used to infer information about, or otherwise be linked to, a particular consumer, computer, or other device.” It goes on to say that “reasonably” includes reasonable assumptions about the use of external data sources that might be available.

Comments

  1. Footnote [3] seems to contradict the main theme of the post.
    Why don’t you think companies are using Adnostic (or something simliar) if it is possible to keep the value of personalized advertising but without compromising privacy? Is it just a little more expensive and there aren’t that many people who care enough about privacy? If so, then yes there is a trade-off but things are not zero-sum.

    • Arvind Narayanan says:

      To clarify, the claim is not that targeted advertising + privacy is zero-sum, but that the current targeted advertising business model + privacy is zero sum. Adnostic drastically changes the business model; in particular transfers power from the current ad tech companies to browser vendors and users. In other words, even somewhat addressing the privacy concerns requires a big change to the business model. So I think the footnote supports the theme of the post.

      (For full disclosure, I’m a coauthor of the Adnostic paper.)

  2. Yuhong Bao says:
  3. Richard Stallman says:

    The article’s point is correct, but is stated to narrowly because
    it uses the term “business model” rather than “purpose”.

    Consider, for instance, cameras looking at the street that recognize
    the faces or license plate numbers of passers-by. For the most part,
    they have no business model, but they do have a purpose: to track
    people. The fact that the purpose is not profit makes little difference
    to the danger of surveillance.

    If the state generally knows where each person goes, or who
    communicates with whom, that magnifies its capacity for repression.
    Therefore, we need to drastically reduce the amount of data that is
    collected about people. See
    https://gnu.org/philosophy/surveillance-vs-democracy.html.

    Even when there’s a business model and it is not based purely and
    totally on surveillance, and not entirely zero-sum, the surveillance
    can be so dangerous to society that we need to put an end to it.

    Consider Uber: it identifies customers and tracks where they go. It
    profiles its customers to increase its profit but isn’t dependent on
    doing so. Nonetheless, the collection of data about people’s
    movements is such a threat to political freedom that we absolutely
    must put an end to it.

    https://www.theguardian.com/commentisfree/2018/apr/03/facebook-abusing-data-law-privacy-big-tech-surveillance
    proposes a legal approach to put an end to dangerous surveillance.

    See https://stallman.org/uber.html for many reasons to refuse to do
    business with Uber.

    • Lawrence D’Oliveiro says:

      Perhaps because the purpose of the article *was* to discuss “business models” rather than “purposes”?

      If you want to write your own article about the “purposes” of business models, by all means feel free to do so. Don’t come criticizing others for not conforming with your ideology.