April 24, 2017

Archives for April 2017

The future of ad blocking

There’s an ongoing arms race between ad blockers and websites — more and more sites either try to sneak their ads through or force users to disable ad blockers. Most previous discussions have assumed that this is a cat-and-mouse game that will escalate indefinitely. But in a new paper, accompanied by proof-of-concept code, we challenge this claim. We believe that due to the architecture of web browsers, there’s an inherent asymmetry that favors users and ad blockers. We have devised and prototyped several ad blocking techniques that work radically differently from current ones. We don’t claim to have created an undefeatable ad blocker, but we identify an evolving combination of technical and legal factors that will determine the “end game” of the arms race.

Our project began last summer when Facebook announced that it had made ads look just like regular posts, and hence impossible to block. Indeed, Adblock Plus and other mainstream ad blockers have been ineffective on Facebook ever since. But Facebook’s human users have to be able to tell ads apart because of laws against misleading advertising. So we built a tool that detects Facebook ads the same way a human would, deliberately ignoring hidden HTML markup that can be obfuscated. (Adblock Plus, on the other hand, is designed to be able to examine only the markup of web pages and not the content.) Our Chrome extension has several thousand users and continues to be effective.

We’ve built on this early success. Laws against misleading advertising apply not just on Facebook, but everywhere on the web. Due to these laws and in response to public-relations pressure, the online ad industry has developed robust self-regulation that standardizes the disclosure of ads across the web. Once again, ad blockers can exploit this, and that’s what our perceptual ad blocker does. [1]

The second prong of an ad blocking strategy is to deal with websites that try to detect (and in turn block) ad blockers. To do this, we introduce the idea of stealth. The only way that a script on a web page can “see” what’s drawn on the screen is to ask the user’s browser to describe it. But ad blocking extensions can control the browser! Not perfectly, but well enough to get the browser to convincingly lie to the web page script about the very existence of the ad blocker. Our proof-of-concept stealthy ad blocker successfully blocked ads and hid its existence on all 50 websites we looked at that are known to deploy anti-adblocking scripts. Finally, we have also investigated ways to detect and block the ad blocking detection scripts themselves. We found that this is feasible but cumbersome; at any rate, it is unnecessary as long as stealthy ad blocking is successful.

The details of all these techniques get extremely messy, and we encourage the interested reader to check out the paper. While some of the details may change, we’re confident of our long-term assessment. That’s because our techniques are all based on sound computer security principles and because we’ve devised a state diagram that describes the possible actions of websites and ad blockers, bringing much-needed clarity to the analysis and helping ensure that there won’t be completely new techniques coming out of left field in the future.

There’s a final wrinkle: the publishing and advertising industries have put forth a number of creative reasons to argue that ad blockers violate the law, and indeed Adblock Plus has been sued several times (without success so far). We carefully analyzed four bodies of law that may support such legal claims, and conclude that the law does not stand in the way of deploying sophisticated ad blocking techniques. [2] That said, we acknowledge that the ethics of ad blocking are far from clear cut. Our research is about what can be done and not what should be done; we look forward to participating in the ethical debate.


[1] To avoid taking sides on the ethics of ad blocking, we have deliberately stopped short of making our proof-of-concept tool fully functional — it is configured to detect ads but not actually block them.

[2] One of the authors is cyberlaw expert Jonathan Mayer.

Dissecting the (Likely) Forthcoming Repeal of the FCC’s Privacy Rulemaking

Last week, the House and Senate both passed a joint resolution that prevents the new privacy rules from the Federal Communications Commission (FCC) from taking effect; the rules were released by the FCC last November, and would have bound Internet Service Providers (ISPs) in the United States to a set of practices concerning the collection and sharing of data about consumers. The rules were widely heralded by consumer advocates, and several researchers in the computer science community, including myself , played a role in helping to shape aspects of the rules. I provided input into the rules that helped preserve the use of ISP traffic data for research and protocol development.

How much should we be concerned? Consumers have cause for concern, but almost certainly not as much as the media would have you believe. The joint resolution is expected to be signed by the President, whereupon it will go into law. Many articles in the news last week announced the joint resolution passed by Congress as a watershed moment, saying effectively that Internet service providers can “now” sell your data to the highest bidder. Yet, the first thing to realize is that Internet service providers were never prevented from doing this, and in some sense, the Congressional repeal simply preserves the status quo, with respect to ISPs and data sharing. That is, the privacy rule that was released last November, never went into effect. That said, there is one thing that consumers might be more concerned about: The resolution also prevents the FCC from making similar rules in the future, which has the effect of removing the threat of regulatory action on privacy. Previously, even though it was legal for ISPs to share your data without your consent, they might not have done so simply for fear of regulatory action from the FCC. If this resolution becomes law, there is no longer such a threat, and we will have to rely on market forces for ISPs to be good stewards of our data.

With these high-order bits in mind, the rest of this post will dissect the events over the past year or so in more detail.

Who regulates privacy? Part of the complication surrounding the debates on privacy is that there are currently two agencies in our government who are primarily responsible for protecting consumer privacy. The Federal Trade Commission (FTC) operates under the FTC Act and regulates consumer protection for businesses that are not “common carriers”; this includes most businesses, with the exception of public utilities, and—recently, with the passage of the Open Internet Order (the so-called “net neutrality” rule) in 2015—ISPs. One of the landmark decisions in the Open Internet Order was to classify ISPs under “Title II” (telecommunications providers), whereas previously they were classified under Title I. This action effectively moved the jurisdiction for regulating ISP privacy from the FTC (where Google, Facebook, and other Internet companies are regulated) to the FCC.

Essentially, there is a firewall of sorts between the two agencies when it comes to privacy rulemaking: The FTC is prohibited by federal law from regulating common carriers, and the FCC has a statutory mandate (under Section 222 of the telecommunications act) to protect customer data that is collected by common carriers.

Are the FCC’s privacy rules “fair”? Part of the debate from the ISPs surrounds whether this separation is fair: ISPs like Comcast and online service providers (so called “edge providers” in Washington) like Google are increasingly competing in the same markets, and regulating them under different rules can in some sense create an uneven playing field. Depending on your viewpoint and orientation, there is some merit to this argument: The FCC’s privacy rules are stronger than the FTC’s rules, as the FCC’s rules govern additional information that cannot be shared without user consent, such as browsing history, application usage history, and geolocation. Companies who are regulated by the FTC (Google, Facebook, etc.) have no such restrictions on sharing your data without your consent. Whether this situation is “fair” depends in some sense on your perspective about whether edge providers like Google and ISPs like Comcast should be subject to the same rules.

  • The ISP viewpoint (and the Republican rationale behind the resolution) of the joint resolution is that for the Googles and Facebooks of the world, your data is not considered sensitive; they can already gather this information about your browsing history and sell it to third-party marketers. The ISPs and Republicans view that if ISPs and edge providers are really in the same market (or should allowed to be), then they shouldn’t be subject to different rules. That sounds good, except there are a couple of hangups. The first is, as mentioned, the FTC cannot regulate ISPs; they are prohibited from doing so by federal law. Unless the ISPs are reclassified again under Title I, they may currently end up in a situation where nobody can legally regulate them, since the FTC is already prevented from doing so, and it is increasingly looking like the FCC will be prevented from doing so, as well. The charitable viewpoint to the situation is that the goal appears to be not to get rid of privacy rules entirely, but rather to shift everything concerning consumer privacy back to the FTC, where ISPs and edge providers are subject to the same rules. But, in the meantime, the situation may be suspended in a strange limbo.
  • The consumer advocate viewpoint is that, in the current market for ISPs in the United States, many consumers do not have a choice of ISP. Therefore, the ISPs are in a position of power that the edge providers do not have. In many senses, that is true: in many parts of the United States, studies from the FCC and elsewhere have shown that consumers have only one choice of broadband ISP. This places the ISP in a position of great power, because we can’t just rely on “market forces” to encourage good behavior towards consumers if consumers can’t vote with their feet. Effectively, in contrast to edge providers such as Google or Facebook, in certain markets in the US, one cannot simply “opt out” of one’s ISP. There are also some arguments that ISPs can see a lot more data than edge providers can; that point is certainly arguable, given the level of instrumentation that a company like Google has on everything from the trackers they place on just about every website on the Internet to their command over our browser, mobile operating system, etc. More likely, we should be equally concerned about both edge providers and ISPs.

The repeal, and the status quo. In essence, the repeal that is likely to come in the coming weeks should cause concern, but it is not quite as simple as “ISPs can now sell your data to the highest bidder”. Keep in mind that ISPs have always legally been able to do so, and they haven’t done so yet. In fact, on Friday, Comcast just committed to not selling your data to third-party marketers, which provides some hope that the market will, in fact, induce behavior that is good for consumers. In some sense, the repeal will do nothing except to preserve the status quo. Ultimately, time will tell. I do expect that increasingly ISPs may look increasingly like advertisers—after all, they have been trying to get into the business of advertising for years. Without the threat of regulatory enforcement that has existed until now, ISPs may be more likely to enter these markets (or at least try to do so). In the coming years, there may not be much we can do about this except hope that the market enforces good behavior. It should be noted that, despite the widespread attention to Virtual Private Networks as a possible defense against ISP data collection over the past week, these offer scant protection against the kinds of data that would or could be collected about you, as I and others have previously explained.

Privacy is a red herring. The real problem is lack of competition. The prospect of relying on the market brings me to a final point. One of the oft-forgotten provisions of the Open Internet Order’s reclassification of the ISPs under Title II is that the FCC can compel the ISPs to “unbundle the local loop”—a technical term for letting competing ISPs share the underlying physical infrastructure. We used to have this situation in the United States (older readers probably remember the days of “mom and pop” DSL providers who leased infrastructure from the telcos), and many countries in Europe still have competitive markets by virtue of this structure. One possible path forward that could give more leverage to market forces would be to unbundle the local loop under Title II. This outcome is widely viewed to be highly unlikely.

Part of the reason this might be unlikely is that if Title II reclassification is walked back and ISPs end up in the Title I regime once again. Oddly, though we are likely to hear much uproar over the “repeal” of the net neutrality rules, one silver lining will be that if and when such a rollback occurs, the ISPs will be bound by some privacy rules. If the current resolution passes, they’ll be bound by none at all.

Finally, it is worth remembering that there are other uses of customer data besides selling it to advertisers. My biggest role in helping shape the FCC’s original privacy rules was to help preserve the use of this data for Internet engineers and researchers who continue to develop new algorithms and protocols to help the Internet perform better, and to keep us safe from attacks ranging from denial of service to phishing. While none of us may be excited at the prospect of having our data shared with advertisers without our consent, we all benefit from other operational uses of this data, and those uses should certainly be preserved.