November 24, 2024

Lessons from Facebook's Beacon Misstep

Facebook recently beat a humiliating retreat from Beacon, its new system for peer-based advertising, in the face of users’ outrage about the system’s privacy implications. (When you bought or browsed products on certain third-party sites, Beacon would show your Facebook friends what you had done.)

Beacon was a clever use of technology and might have brought Facebook significant ad revenue, but it seemed a pretty obvious nonstarter from users’ point of view. Trying to deploy it, especially without a strong opt-out capability, was a mistake. On the theory that mistakes are often instructive, let’s take a few minutes to work through possible lessons from the Beacon incident.

To start, note that this wasn’t a privacy accident, where user data is leaked because of a bug, procedural breakdown, or treacherous employee. Facebook knew exactly what it was doing, and thought it was making a good business decision. Facebook obviously didn’t foresee their users’ response to Beacon. Though the money – not to mention the chance to demonstrate business model innovation – must have been a powerful enticement, the decision to proceed with Beacon could only have made sense if the company thought a strong user backlash was unlikely.

Organizations often have trouble predicting what will cause privacy outrage. The classic example is the U.S. government’s now-infamous Total Information Awareness program. TIA’s advocates in the government were honestly surprised when the program’s revelation caused a public furor. This wasn’t just public posturing. I still remember a private conversation I had with a TIA official who ridiculed my suggestion that the program might turn out to be controversial. This blindness contributed to the program’s counterproductive branding such as the creepy all-seeing-eye logo. Facebook’s error was similar, though of much smaller magnitude.

Of course, privacy is not the only area where organizations misjudge their clients’ preferences. But there does seem to be something about privacy that makes these sorts of errors more common.

What makes privacy different? I’m not entirely certain, but since I owe you at least a strawman answer, let me suggest some possibilities.

(1) Overlawyerization: Organizations see privacy as a legal compliance problem. They’re happy as long as what they’re doing doesn’t break the law; so they do something that is lawful but foolish.

(2) Institutional structure: Privacy is spun off to a special office or officer so the rest of the organization doesn’t have to worry about it; and the privacy office doesn’t have the power to head off mistakes.

(3) Treating privacy as only a PR problem: Rather than asking whether its practices are really acceptable to clients, the organization does what it wants and then tries to sell its actions to clients. The strategy works, until angry clients seize control of the conversation.

(4) Undervaluing emotional factors: The organization sees a potential privacy backlash as “only” an emotional response, which must take a backseat to more important business factors. But clients might be angry for a reason; and in any case they will act on their anger.

(5) Irrational desire for control: Decisionmakers like to feel that they’re in control of client interactions. Sometimes they insist on control even when it would be rational to follow the client’s lead. Where privacy is concerned, they want to decide what clients should want, rather than listening to what clients actually do want.

Perhaps the underlying cause is the complex and subtle nature of privacy. We agree that privacy matters, but we don’t all agree on its contours. It’s hard to offer precise rules for recognizing a privacy problem, but we know one when we see it. Or t least we know it after we’ve seen it.

Workshop: Computing in the Cloud

I’m excited to announce that Princeton’s Center for InfoTech Policy is putting on a workshop on the policy and social implications of “Computing in the Cloud” – the trend where companies, rather than users, store and manage an increasing range of personal data.

Examples include Hotmail and Gmail replacing desktop email, YouTube taking over as a personal video platform, and Flickr competing with desktop photo storage solutions. Facebook, Myspace and other social networks have pioneered new kinds of tools that couldn’t exist on the desktop, and more new models are sure to emerge.

I’m confident that this trend will reshape tech policy, and will change how people relate to technology. But I don’t know what the changes are. By drawing together experts from computer science, industry, government and law, I hope the Center can help those of us at Princeton, and workshop participants from around the country, get a better sense of where things might be headed.

The workshop will be held on the Princeton campus on January 14 and 15, 2008. It will be free and open to the public. We will have a series of panel discussions, interspersed with opportunities for informal exchanges of ideas. We’re still putting together the list of panels and panelists, so we haven’t yet published a schedule. If you’re interested in attending or want to get email updates about the workshop, please email David Robinson (dgr at princeton dot edu).

Here are some of the possible themes for panels we are exploring:

  • Possession and ownership of data: In cloud computing, a provider’s data center holds information that would more traditionally have been stored on the end user’s computer. How does this impact user privacy? To what extent do users “own” this data, and what obligations do the service providers have? What obligations should they have? Does moving the data to the provider’s data center improve security or endanger it?
  • Collaboration and globalization: Cloud computing systems offer new sharing and collaboration features beyond what was possible before. They make shared creation among far-flung users easier, allow or require data to be stored in many different jurisdictions, and give users access to offerings that may be illegal in the users’ home countries. How will local laws, when applied to data centers whose user base is global, affect users practice? Do these services drive forward economic growth — and if so, what effect should that fact have on the policy debate?
  • New roles for new intermediaries: Cloud services often involve new
    intermediaries such as Facebook, MySpace, eBay, and Second Life, standing between people who might have interacted more directly before these services emerged. To what extent are these services “communities”, as their providers claim? How much control do users feel over these communities? How much control do and should users actually have? How does the centralized nature of these intermediaries affect the efficiency and diversity of online experiences? Can the market protect consumers and competition, or is government oversight needed?
  • What’s next: What new services might develop, and how will today’s services evolve? How well will cloud computing be likely to serve users, companies, investors, government, and the public over the longer run? Which social and policy problems will get worse due to cloud computing, and which will get better?

Further adventures in personal credit

In our last installment, I described how one of the mortgage vendors who I was considering for the loan for my new home failed to trigger the credit alerting mechanism (Debix) to which I was signed up. Since then, I’ve learned several interesting facts. First, the way that Debix operates is that they insert a line into your credit reports which says, in effect, “you, the reader of this line, are required to call this 1-800 telephone number, prior to granting credit based on what you see in this report.” That 800-number finds its way to Debix, where a robot answers the phone and asks the human who called it for their name/organization, and the purpose of the request. Then, the Debix robot calls up their customer and asks permission to authorize the request, playing back the recordings made earlier.

The only thing that makes this “mandatory” is a recent law (sorry, I don’t have the citation handy) which specifies how lenders and such are required to act when they see one of these alerts in a credit report. The mechanism, aside from legal requirements, is otherwise used at the discretion of a human loan officer. This leads me to wonder whether or not the mechanism works when there isn’t a human loan officer involved. I may just need to head over to some big box store and purchase myself something with an in-store instant-approval credit card, just to see what happens. (With my new house will inevitably come a number of non-trivial expenses, and oh what great savings I can get with those insta-credit cards!)

So does the mechanism work? Yesterday morning, as I was getting into the car to go to work, my cell phone rang with an 800-number as the caller-ID. “Hello?” It was the Debix robot, asking for my approval. Debix played a recording of an apparently puzzled loan officer who identified herself as being from the bank that, indeed, I’m using for my loan. Well that’s good. Could the loan officer have been lying? Unlikely. An identity thief isn’t really the one who gets to see the 800-number. It’s the loan officer of the bank that the identity thief is trying to defraud who then makes the call. That means our prospective thief would need to guess the proper bank to use that would fool me into giving my okay. Given the number of choices, the odds of the thief nailing it on the first try are pretty low. (Unless our prospective thief is clever enough to have identified a bank that’s too lazy to follow the proper procedure and call the 800-number; more on this below).

A side-effect of my last post was that it got noticed by some people inside Debix and I ended up spending some quality time with one of their people on the telephone.  They were quite interested in my experiences.  They also told me, assuming everything is working right, that there will be some additional authentication hoops that the lender is (legally) mandated to jump through between now and when they actually write out the big check. Our closing date is next week, Friday, so I should have one more post when it’s all over to describe how all of that worked in the end.

Further reading: The New York Times recently had an article (“In ID Theft, Some Victims See an Opportunity“, November 16, 2007) discussing Debix and several other companies competing in the same market. Here’s an interesting quote:

Among its peers, LifeLock has attracted the most attention — much of it negative. In radio and television ads, Todd Davis, chief executive of LifeLock, gives out his Social Security number to demonstrate his faith in the service. As a result, he has been hit with repeated identity theft attacks, including one successful effort this summer in which a check-cashing firm gave out a $500 loan to a Texas fraudster without ever checking Mr. Davis’s credit report.

Sure enough, if you go to LifeLock’s home page, you see Mr. Davis’s social security number, right up front. And, unsurprisingly, he fell victim because, indeed, fraudsters identified a loan organization that didn’t follow the (legally) mandated protocol.

How do we solve the problem? Legally mandated protocols need to become technically mandatory protocols. The sort of credit alerts placed by Debix, LifeLock, and others need to be more than just a line in the consumer’s credit file. Instead, the big-3 credit bureaus need to be (legally) required not to divulge anything beyond the credit-protection vendor’s 800-number without the explicit (technical) permission of the vendor (on behalf of the user). Doing this properly would require the credit bureaus to standardize and implement a suitable Internet-based API with all the right sorts of crypto authentication and so forth – nothing technically difficult about that. Legally, I’d imagine they’d put up more of a fight, since they may not like these startups getting in the way of their business.

The place where the technical difficulty would ramp up is that the instant-credit-offering big-box stores would want to automate their side of the phone robot conversation. That would then require all these little startups to standardize their own APIs, which seems difficult when they’re all still busily inventing their own business models.

(Sidebar: I set up this Debix thing months ago. Then I get a phone call, out of the blue, that asked me to remember my PIN. Momentary panic: what PIN did I use? Same as the four-digit one I use for my bank ATM? Same as the six-digit one I uses for my investment broker? Same as the four-digit one used by my preferred airline’s frequent flyer web site which I can’t seem to change? Anyway, I guessed right. I’d love to know how many people forget.)

Eavesdropping as a Telecom Profit Center

In 1980 AT&T was a powerful institution with a lucrative monopoly on transporting long-distance voice communications, but forbidden by law from permitting the government to eavesdrop without a warrant. Then in 1981 Judge Greene took its voice monopoly away, and in the 1980s and 90s the Internet ate the rest of its lunch. By 1996, Nicholas Negroponte wrote what many others also foresaw: “Shipping bits will be a crummy business. Transporting voice will be even worse. By 2020 … competition will render bandwidth a commodity of the worst kind, with no margins and no real basis for charging anything.

During the 1980s and 90s, AT&T cleverly got out of any business except shipping commodity bits: in 1981 it (was forced to) split off its regional phone companies; in 1996 it (voluntarily) split off its equipment-making arm as Lucent Technologies; in 2000-2001 it sold off its Wireless division to raise cash. Now AT&T long-distance bit-shipping is just a division of the former SBC, renamed AT&T.

What profit centers are left in shipping commodity bits? The United States Government spends 44 billion dollars a year on its spy agencies. It’s very plausible that the NSA is willing to pay $100 million or more for a phone/internet company to install a secret room where the NSA can spy on all the communications that pass through. A lawsuit by the EFF alleges such a room, and its existence was implicitly confirmed by the Director of National Intelligence in an interview with the El Paso Times. We know the NSA spends at least $200 million a year on information-technology outsourcing and some of this goes to phone companies such as Verizon.

Therefore, if it’s true that AT&T has such a secret room, then it may be simply that this is the only way AT&T knows how to make money off of shipping bits: it sells to the government all the information that passes through. Furthermore, economics tells us that in a commodity market, if one vendor is able to lower its price below cost, then other vendors must follow unless they also are able to make up the difference somehow. That is, there will be substantial economic pressure on all the other telecoms to accept the government’s money in exchange for access to everybody’s mail, Google searches, and phone calls.

In the end, it could be that the phone companies that cooperated with the NSA did so not for reasons of patriotism, or because their arms were twisted, but because the NSA came with a checkbook. Taking the NSA’s money may be the only remaining profit center in bit-shipping.

AT&T Explains Guilt by Association

According to government documents studied by The New York Times, the FBI asked several phone companies to analyze phone-call patterns of Americans using a technology called “communities of interest”. Verizon refused, saying that it didn’t have any such technology. AT&T, famously, did not refuse.

What is the “communities of interest” technology? It’s spelled out very clearly in a 2001 research paper from AT&T itself, entitled “Communities of Interest” (by C. Cortes, D. Pregibon, and C. Volinsky). They use high-tech data-mining algorithms to scan through the huge daily logs of every call made on the AT&T network; then they use sophisticated algorithms to analyze the connections between phone numbers: who is talking to whom? The paper literally uses the term “Guilt by Association” to describe what they’re looking for: what phone numbers are in contact with other numbers that are in contact with the bad guys?

When this research was done, back in the last century, the bad guys where people who wanted to rip off AT&T by making fraudulent credit-card calls. (Remember, back in the last century, intercontinental long-distance voice communication actually cost money!) But it’s easy to see how the FBI could use this to chase down anyone who talked to anyone who talked to a terrorist. Or even to a “terrorist.”

Here are a couple of representative diagrams from the paper:

Fig. 4. Guilt by association – what is the shortest path to a fraudulent node?

Fig. 5. A guilt by association plot. Circular nodes correspond to wireless service accounts while rectangular nodes are conventional land line accounts. Shaded nodes have been previously labeled as fraudulent by network security associates.