There’s an ongoing arms race between ad blockers and websites — more and more sites either try to sneak their ads through or force users to disable ad blockers. Most previous discussions have assumed that this is a cat-and-mouse game that will escalate indefinitely. But in a new paper, accompanied by proof-of-concept code, we challenge this claim. We believe that due to the architecture of web browsers, there’s an inherent asymmetry that favors users and ad blockers. We have devised and prototyped several ad blocking techniques that work radically differently from current ones. We don’t claim to have created an undefeatable ad blocker, but we identify an evolving combination of technical and legal factors that will determine the “end game” of the arms race.
Our project began last summer when Facebook announced that it had made ads look just like regular posts, and hence impossible to block. Indeed, Adblock Plus and other mainstream ad blockers have been ineffective on Facebook ever since. But Facebook’s human users have to be able to tell ads apart because of laws against misleading advertising. So we built a tool that detects Facebook ads the same way a human would, deliberately ignoring hidden HTML markup that can be obfuscated. (Adblock Plus, on the other hand, is designed to be able to examine only the markup of web pages and not the content.) Our Chrome extension has several thousand users and continues to be effective.
We’ve built on this early success. Laws against misleading advertising apply not just on Facebook, but everywhere on the web. Due to these laws and in response to public-relations pressure, the online ad industry has developed robust self-regulation that standardizes the disclosure of ads across the web. Once again, ad blockers can exploit this, and that’s what our perceptual ad blocker does. [1]
The second prong of an ad blocking strategy is to deal with websites that try to detect (and in turn block) ad blockers. To do this, we introduce the idea of stealth. The only way that a script on a web page can “see” what’s drawn on the screen is to ask the user’s browser to describe it. But ad blocking extensions can control the browser! Not perfectly, but well enough to get the browser to convincingly lie to the web page script about the very existence of the ad blocker. Our proof-of-concept stealthy ad blocker successfully blocked ads and hid its existence on all 50 websites we looked at that are known to deploy anti-adblocking scripts. Finally, we have also investigated ways to detect and block the ad blocking detection scripts themselves. We found that this is feasible but cumbersome; at any rate, it is unnecessary as long as stealthy ad blocking is successful.
The details of all these techniques get extremely messy, and we encourage the interested reader to check out the paper. While some of the details may change, we’re confident of our long-term assessment. That’s because our techniques are all based on sound computer security principles and because we’ve devised a state diagram that describes the possible actions of websites and ad blockers, bringing much-needed clarity to the analysis and helping ensure that there won’t be completely new techniques coming out of left field in the future.
There’s a final wrinkle: the publishing and advertising industries have put forth a number of creative reasons to argue that ad blockers violate the law, and indeed Adblock Plus has been sued several times (without success so far). We carefully analyzed four bodies of law that may support such legal claims, and conclude that the law does not stand in the way of deploying sophisticated ad blocking techniques. [2] That said, we acknowledge that the ethics of ad blocking are far from clear cut. Our research is about what can be done and not what should be done; we look forward to participating in the ethical debate.
This post was edited to update the link to the paper to the arXiv version (original paper link).
[1] To avoid taking sides on the ethics of ad blocking, we have deliberately stopped short of making our proof-of-concept tool fully functional — it is configured to detect ads but not actually block them.
[2] One of the authors is cyberlaw expert Jonathan Mayer.
@Arvind
I hope a few days after your post that you still will look at the responses too it. Your post included the line “we look forward to participating in the ethical debate.”
I would love to debate some ethics. But, I don’t have a specific handle on the “creative reasons to argue that ad blockers violate the law” — but it seems the courts have already favored the many add blocking techniques in use for TV and Radio advertisements. And with clear cut court statements that include that individuals have every right to determine what they do and do not view or hear on TV/Radio, seems any “debate” regarding online advertisements is a non issue. Users have rights–at least in the USA.
The cases I recall (and I am not a legal expert by any means) revolved around such things as VCR’s used to record shows; and then fast forward over the commercials; DVR’s when things went digital that would simply skip the commercial segment got a lot of attention in the courts; and such. Yet seems to me in each case I have read about over the years the courts have stated that it is well within the rights of an individual to do many things to avoid commercials.
From the “bathroom break” of the 60-80’s; to the VCR and fast-forward; to DVR and skipping; other low-tech methods such as just turning the TV or radio off for a certain amount of time and then back on, or muting it and reading [that was my favorite approach]; or switching channels. Of course providers play the “cat and mouse” game.
There was a time when TV ratings were determine in large part by toilet flushes (each station had its own commercial break times and it was presumed more people would go on bathroom break during the commercials); but what happened when remotes entered the picture and people started lazily changing channels instead of bathroom breaks; when network stations caught on; they began to coordinate commercial breaks at set intervals so all channels would be on commercials at the same time, radio stations do the same thing for the most part.
I noted that the early version of Hulu which was all free paid by advertising; to avoid this type of skipping of advertisements they would cut advertisements randomly in middle of the show (often in middle of a sentence or even a single word); and they would put anywhere from one 15 second commercial to ten 30 second commercials in the break so one wouldn’t know just when to come back.
Despite the cat and mouse aspects of trying to FORCE consumers to view commercials and advertisements; seems to me, the court cases have made it clear; users have every right to decide not to view commercials; and can take any means they want.
Me, I prefer the “ignore” feature of my own brain. It’s not generally hard for a person to recognize advertisements, and my brain just tunes them out. Of course there are times I will click on the random news post which turns out to be nothing more than a plug for some business (fake news for sure–or an advertisement by any other name); but by and large I just don’t click on things that are advertisements.
Still, I can’t find a single reason that blocking advertisements would be “unethical” in any way shape or form.
The only argument I have heard is that the advertiser pays money and expects to get a gain from it when their advertisement is seen. But, that is never a guarantee with any form of advertising. The station (or website in this case) has already been paid by the advertiser, so they have no skin in the game all they must do is serve the advertisement based on the contract, they cannot control how the advertisement is responded to by the public.
The corp. paying for the advertisement, regardless of the type of advertising, must always assume a couple things, 1) that the target audience may not actually even see the advertisement, 2) the target audience may not respond to the advertisement, 3) that it is their own job to determine the effectiveness of advertising and adjust accordingly.
If ad-blockers in web browsers were so popular that no one ever saw an advertisement on a web page; those businesses using web advertising would soon see it is not effective, and would advertise elsewhere. No laws (or ethics) require a revenue stream simply by virtue of posting an advertisement.
The game of cat and mouse with ad-blockers is nothing more than the cat insisting his food source is untouchable (and it is the advertisers who become unethical at that point). Advertisers really ought to look at other means to get their message out if need be.
Still, I suspect that those who use “ad-blocking” software are so in the minority that advertisers are wasting their time (and website operators) targeting ad-blocking software in the first place. In other words; you indicate Facebook has purposefully targeted your software (to protect their revenue stream coming from advertisers); yet your software only boasts “several thousand users” compared to 1.86 billion Facebook users. Their time is misspent chasing the few who dislike advertisements enough to want to block them compared to the billions who don’t use such software (what about 0.00001% of users). Even if one added in all of the ad-blockers it still wouldn’t even begin to reflect enough of a percentage to make a difference; most Facebook users still see most advertisements.
It is like the Tom and Jerry; where the cat (Tom if I remember right) would catch the mouse over and over; but never would eat it; no matter how often Facebook changes their code to keep ad-blockers from working; it still isn’t going to make them any more money, because advertisers already pay expecting some percentage [a very tiny percentage at that] will never see the advertisement anyway, but they have found that advertising to the masses still works regardless of the few who don’t see the advertisements.
My point being, there is not a single “unethical” argument to be made about choosing to block advertisements. It is called living in a free country, the only “unethical” thing going on here is companies that try to force users to view things they don’t want.
Ever wonder what would TV have been like if they were created with a chair to strap the viewer in place so he/she cannot move when advertisements are playing (no bathroom breaks during advertisements); and that the channel dial would lock in place [back when they had “dials”] so no one could change the channel while advertisements were on either, thus FORCING viewers to see advertisements if they wanted to watch the program. I suspect if someone had invented that; it wouldn’t have gained traction; because one being held “hostage” to view an advertisement is the “unethical” behavior; and that is what Facebook and other websites are trying to do.
I say again the only unethical behavior here is that of web developers who in trying to enhance their “revenue stream” by forcing viewers to sit through advertisements. Taking away the free will of an individual is most certainly “unethical;” yet providing the free will to users (which is what ad-blockers do) is the most ethical thing. So, go ahead and complete your project.
Interesting Read! I was also thinking on concepts for side-stepping Ads, including Anti-Adblocker measures. My idea is:
Running the Browser in a sort of “Matryoshka Nesting Doll” approach. It means, allowing all webserver data to flow unimpeded to the browser, including all ads, in a first “public” browser layer. So webservers will not recognise any adblocker activity at all. Only in a second “private” browser layer, of which webservers can’t take notice of, the adblocking tech comes in.
In other words, the first “public” browser-server communication works without hindrances, but does not result in screen display at all. From this “public” data flow, a second “private” browser layer (of data flow) will be taken, ad-filtered and finally displayed on the screen. This kind of “Browser virtualisation approach” would circumvent any Adblocker detection (and its measures) at the roots.
PS: For the moment, I’m happy with “uBlock Origin” in my browsers, it is light weight, resource sparing and has a very active Community. For Facebook, I’m using the great “F.B. Purity”, without of Facebook would be an unenjoyable mess.
Wow, I read “Matryoshka Nesting Doll” and was not sure how you would compare any technological approach to a child’s toy. Yet, while I hadn’t thought to make that comparison; what you described after in essence is exactly how I recently got around the kind of interactive ad that Scott mentions in the first response. I let the data in; but then just hide it; the web server and hence advertiser is none the wiser.
Nice Job. So, by now there is no planning to make it fully functional?
That’s correct.
This is great! Online advertising has been broken for a long time. Complete nonsense really, these poor clients have no idea where their ads are being served, what they are paying for and how much money they are wasting. Stop the insanity.
Have you been in touch with Brave (https://brave.com/) about this work? I would assume so, since the interests are so aligned… They are ideal ones to capitalize on it, and then use an alternate mechanism for funding publisher’s content.
Excellent work.
What I am impressed most it’s the ethical usage of rootkit techniques in defense of the users. Are we seeing a new wave of “force” techniques used for the good? 🙂
Hey Arvind, this is Winder Hughes and I’m the CEO of Relevnt and we are launching what is essentially the only real-time news browser and to pull in clean content via RSS, we use the Embedly scraper/extractor. However, it does not pull in all of the needed content and we are looking for more. Do you all have something more robust or something to customize? If so, please drop me a line:
Thank you
The perceptual ad blocker is not functioning on facebook anymore. I added it today and went onto facebook ad no ads were able to be identified.
Thanks for letting us know! Grant has pushed out a fix; should be working shortly.
In more detail: our extension had an accidental dependence on markup in the way we look for containers that might have ads, and Facebook exploited this to try and break it. This is the second time they’ve been playing cat and mouse with our extension (and we have pretty solid evidence that they’re targeting us specifically). The first time was after our original release last summer, and we were able to quickly fix the issue.
It’s entirely possible that there’s something else we’ve missed, and given that our code is maintained by one undergraduate who’s busy with finals right now, they could certainly give us the run around. But we’re surprised they’re taking this approach. Our paper makes it abundantly clear why the ad blocker has the upper hand.
It would be better for everyone if they instead worked on aligning incentives so that fewer people want to install ad blockers in the first place. They’ve done an admirable job with Facebook ads (and we said so in our original post). Let’s hope they’ll use their clout to try and clean up the ad experience on the rest of the web.
Congrats! I’ve been thinking about something like this for the past year, when I started to download the PDFs of the newspapers/magazines I used to read on the web. To me, PDF (on a big tablet, like the iPad Pro) offline solves the tracking problem: Google, Fakebook et al. cannot learn anymore what I read, what topics I might be interested in, how strong is my interest, my browsing pattern, the daytimes/hours I read, etc etc.
What continued to irritate me is that I need to “clean” my PDFs (of the Economist, say) by hand from the advertisements. (I find advertising fundamentally unethical and harmful, both to society and myself.) It’s a tedious routine (usually once a week, when I go online and download all the stuff).
Perhaps this “perceptual adblocking” technique can be adapted to PDF format so that the ads will be automatically pasted over with a white rectangle.
I will look into this after the doctoral defense.
Thanks for doing this and sharing it freely!! (If I succeed on the PDF front, I shall do the same)
Cheers,
HS
You could use Printliminator to strip out boxes around the content that you don’t want:
https://css-tricks.github.io/The-Printliminator/
Need to turn this addin into an “Adliminator” or something. 😉
Are your ad blockers available for firefox?
Also thanks for the excellent article. I’ve been saying for awhile that advertising is a dead revenue method. It is increasingly only going to reach the least savvy, which will keep them in money for awhile but not indefinitely. What publishers need to do is find new ways to monetize their content.
Paywalls are good, but they prevent search engines from picking up content. Is that good for consumers or bad for consumers? The only people really hurting in this conflict are the publishers. And tough, they’ve been raping our wallets and our time for too many years. It is time for the pendulum to swing back in our favor, and there is nothing in the world that they can do about that. Their desperate attempts to keep the advantage show the contempt they have for their audiences.
Technology is going to allow us to live in an ad-free world, thanks to the brave efforts of people like you. I applaud you.
> tool fully functional – it is configured to detect ads but not actually block them.
The article mentions the ethics and I do not want to be too overly critical since it is
your webpage, but what is the real difference between 50% ad-blocking, 99%
ad-blocking and 100% ad-blocking from a conceptual point of view? Which
ethics apply to one case but not the other? And why not let the users decide
on their own?
I grant you that you gave them more options as most other ad-blockers would
but I do not understand the qualitative argument made between 50% functionality,
99% or 100% here.
The argument is that we wanted to do the minimum that we could to demonstrate the feasibility of our proposed techniques, but no more. Proving the capability to detect ads is necessary for scientific validation of our claims, but actually blocking ads is not necessary.
I think it should not be called ‘50% ad blocking’, but rather ‘100% ad detection’. The argument here is: While ‘blocking’ ads can be subject to legal/ethical questions, ‘detecting’ ads cannot be, by any stretch of imagination.
Excellent work though. Even more awesome is the fact that you have made the source code public.
> Our project began last summer when Facebook announced that it had made ads look just like regular posts, and hence impossible to block. Indeed, Adblock Plus and other mainstream ad blockers have been ineffective on Facebook ever since.
Is this accurate? I use uBlock Origin and see no post-like ads on Facebook. I had to disable UO on Facebook in order to see your extension highlight post-like ads.
In any case fascinating and good work. Thank you!
Thanks. You’re right — uBlock Origin does work on Facebook. Grant points out that this is because Facebook left in a little bit of markup that uBlock Origin is able to examine but ABP isn’t (since it lacks the ‘:has’ pseudoselector).
We conjecture that it would be trivial for FB to change this, but they haven’t bothered to because uBO doesn’t have enough marketshare to be on their radar.
Post-like ads on Facebook? I used Facebook for all of 2 weeks many many years ago. I stopped using it because 99.9% of the so called “posts” were clearly advertisements. It was like watching TV that was non-stop infomercials, except the occasional break to put in a 30 second regular commercial, with even fewer occasional intersperse of news headline teasers–but never any real content.
I don’t know what uBlock does or does not block; but I suspect it doesn’t block most of the advertisements, you probably have become desensitized to most of them. I tried an ad blocker a few years back as well; I noted that it didn’t block most ads; so I suspect most ad blockers really are inefficient.
The tone of this article is funny.
It’s as if ads are bad guys and its some battle against them.
Ads pay for the production and maintenance of content on the web, free to users. If you succeed in your ad blocking project, you will A) kill free content (no more free content), or B) force all websites to be increasingly forceful in charging users in other ways. I suspect: you’s kill small content creators while big ones come up with more powerful gateways to their content.
It would seem the traditional type of web advertising (content and ads mixed together on one page) will eventually completely lose the battle to ad blockers. But there seem to be alternatives already in place. I occasionally see sites that require you to first interact with an ad (like answering a survey or typing in a code) to even get to the content of the site you wish to read. It seems unlikely this could be prevented by ad blockers if done correctly, but maybe I’m wrong on that.
The more annoying endgame is when content and ads blur together (e.g., sponsored blog posts).
Those are excellent points. If publishers are willing to intrude on users’ attention by making them interact with ads, it does seem unlikely that ad blockers can succeed. But that will also drive away many users, and it’s not clear how many publishers would be willing to make that trade off.
Sponsored content / native advertising is again a topic where the law has something to say. These need to be identified clearly as sponsored (and for the most part they are). We’ve found that people aren’t good at noticing these disclosures, but browser extensions can be! Ad blockers could take on the role of prominently alerting readers when a link they’re about to click on is in fact sponsored content.
This type of activity – full page ads that require user interaction before exposing content – are frowned upon by Google to the extent that they will lower search ranking for sites that make use of it. Whether or not that is sufficient to deter the behavior remains to be seen…
I can see Google frowning on that type of behavior; and Google is all powerful in the website realm, developers never want to upset Google.
I would think that if web developers know that Google frowns upon that, it will deter them sufficiently (but until I read your post I had not heard that, so I don’t know how many web developers know Google frowns on that behavior).
Of course such tactics don’t deter me as a user; I get around them (see my post below).
“I occasionally see sites that require you to first interact with an ad”
I ran into a website that did something like that the other day. Was on a “news” website and wanted me to interact in some way in order to read the article. The article was in the background and the text “blurred” so as to make it unreadable, and the ad was hovered over top.
I sure wasn’t going to go clicking through the ad, but wanted to read the article. What did I do?
I clicked on the developer tools in my web browser; found the offending advertisement section; changed its CSS style to “Display:none”; Then I found content section and removed the “class” that was aptly named something like “blur.” Then I happily went ahead and read the article.
I don’t use ad blockers personally, I just ignore advertisements. But, if I were to use an ad blocker, I would want one that simply sets the advertisement’s style to hidden or non-displayed, and hence non-clickable as well; I don’t care if the advertisement downloads (thus making the advertiser think they are still displaying the ad), that is what I would consider “stealth ad blocking.”
“The more annoying endgame is when content and ads blur together (e.g., sponsored blog posts).”
Oh, one more thing, before I stop ranting on this thread.
This technique I find annoying to; but really it is nothing new to web pages. News (even on TV) often had “sponsors” which would be part of the news, usually an entire segment of the news would be dedicated to that company (and you know darn well they paid a pretty premium to get a news segment done).
But, more importantly a little thing I learned in school called “product placement” — every single TV show, every single movie had prominent characters smoking prominent brands before laws outlawed that (only because smoking is very harmful to everyone’s health). The specific one I remember studying was ET. M&M’s was given the chance to place their product (at a pretty hefty price if I recall) but didn’t want to pay the asking rate; Reases Peaces was given the option and they took it; and made out like bandits in huge revenue gains the year ET came out far exceeding the cost for the product placement.
Product placement works; now just as it did then, even if it is now blurred into online content.