Adblocking – History, Impacts, Techniques and Countermeasures.

[KLB]

On Chris' suggestion I'm going to start a thread to discuss ad-blocking as it effects web publishers. Someday I might turn this into a formal article, but for now I'll frame it in the forum of a forum discussion.

I have been researching ad-blocking and ways to counter ad-blocking since late 2003 and first implemented tactics designed to deny access to my website http://EnvironmentalChemistry.com by people were blocking my ads in early 2004. I have participated in countless ad-blocking discussions both in webmaster forums and in ad-blocking type forums. Depending upon who you talk to, I am either the villain or the hero when it comes to this issue.

As the topic of ad-blocking is huge, I'm going to break my discussion of this topic into several parts. This will allow discussion to take place while I write the next section. While I don't plan on giving people exact code to implement on their site, I will provide enough theory and explanation for people to design their own ad-blocking countermeasures by the end of this thread. Also for the purposes of this discussion I am going to totally ignore any and all issues surrounding popup/popunder ads and focus exclusively on on-page advertising (e.g. traditional banner ads).

The reason I won't be giving out exact code is that in order for ad-blocking countermeasures to be effective, each site's code must be unique. If everyone were to use the same code to detect and block ad-blocking users, it would create too large of an incentive to expend considerable resources to defeat that one set of countermeasures. By everyone developing unique code sets that function differently, it creates too many targets for those who would like to circumvent ad-blocking countermeasures to reliably overcome.

Part 1: History

Contrary to what some would like to believe, ad-blocking software and techniques have been around for almost as long as there have been banner ads. In fact I remember a program called JunkBuster from back in the 1995-1996 timeframe. Back then ad-blocking was pretty much used exclusively by those with the skill sets required to deploy those early ad-blocking techniques. What really made ad-blocking easier was when the IAB standardized on specific dimensions for banner ads (e.g. 468x60) thus making it much easier to detect ads.

As with so many forms of advertising from early on, banner advertising become an ever escalating screaming match with advertisers assaulting users with ever more obnoxious ads that flashed, jumped, pulsed, etc. in an effort to get the attention of users. The more obnoxious ads became the more desperate users became to find ways to stop the assault on their eyes. Instead of recognizing how annoying their ads were becoming, some advertisers simply tried find ever more aggressive ways to get around ad-blocking software and force users to pay attention to them.

Eventually this "arms" race lead to the current generation of ad-blocking programs and plug-ins that are highly effective, easy to deploy and easy to maintain. Today if a user really wants to, they can surf the web virtually ad-free using any of dozens of ad-blocking solutions. These solutions include extensions for the Firefox web browser like AdBlock, easy to install software based proxies like AdSubract and security applications like Symantec Norton Internet Security and ZoneAlarm Pro.

Next: part 2 – The impacts of ad blocking on websites.

[Vinnie]

Nice history . Please do better than mook-jon with subsequent parts though

[KLB]

Originally Posted by Vinnie

Nice history . Please do better than mook-jon with subsequent parts though

Oh I'll try. I've only written about this topic umpteen billion times. So its easy enough to write (no research needed). I juat have to form my thoughts coherently.

[Chris]

You know what you have so far is article quality more or less.

[KLB]

To gain insight into just how big of an issue ad-blocking is, I recently crunched all of my server logs for EnvironmentalChemistry.com for the last six months of 2005. I cross referenced the number of daily unique visitors I got against the number of visitors I rerouted to my banner blocking page. I also cross referenced the daily page views reported by my server logs against my Google AdSense stats.

As a point of reference, the stats program I use is AWStats and it has been configured to detect around 400 known robots and automated processes, although I am unable to detect robots that cloak themselves by reporting the UA strings of normal web browsers. Although the browser spoofing will affect the stats I believe the impact is very nominal.

Due to the way I implemented ad-blocking countermeasures on my site, it is only possible to provide a high and low limit as to the percentage of users who are blocking ads not an exact estimate. The low limit was calculated by comparing the total number of visits per month against the number of visits that resulted in the user being redirected to my banner blocking page. Because some ad-blocking methods prevent JavaScripts from running or prevent the JavaScript "onload" event from running correctly, not all users who block ads get redirected to this page. The high limit was calculated by comparing the number of page views that should have displayed a specific Google AdSense ad with the actual number of AdSense ads that were displayed.

Column-1: Month
Column-2: % of Visits Blocked Due to Ad-Blocking
Column-3: % of Page Views Where AdSense Ads Did Not Load

Jun-05 3.54% 12.32%
Jul-05 3.31% 11.97%
Aug-05 3.75% 9.89%
Sep-05 3.39% 9.15%
Oct-05 2.54% 10.94%
Nov-05 3.46% 9.74%
Dec-05 4.40% 10.35%


When I crunched my logs one figure I wanted to see if I could get a grasp on was what percentage of ad-blocking was due to which browser. The real problem with this is that due to the way logs were processed it is not yet possible to breakdown what percentage of users of a specific browser are blocking ads. It was only possible to show the relative percentage of banner blocking for specific browsers. For instance, in June 2005 Firefox accounted for 25.4% of traffic that was blocking ads. More specifically in June 2005 Firefox accounted for 25.4% of requests for the file "/banblock.html" which is only requested when the user gets redirected because they are blocking ads.

If the percentage of total traffic for a specific browser is lower than the percentage of banner blocking traffic for that browser then users of that browser are blocking ads more frequently than users of other browsers. If the percentage of total traffic for a specific browser is higher than the percentage of banner blocking traffic for that browser then users of that browser tend to block ads less frequently than other users. For example, Firefox users consistently block ads more frequently than MSIE users. In part this is because of the Adblock extension that users can download for Firefox. One obvious omission from the following list of browser stats is the Opera web browser. Opera was omitted because it consistently generates less than 0.3% of the total traffic for EnvironmentalChemistry.com making any analysis of it statistically irrelevant.

One very interesting trend that is disclosed by the table below is that the relative percentage of Firefox users blocking ads compared to other users (MSIE users in particular) has been steadily dropping over the past six months even though the total percentage of users using Firefox has climbed modestly over the same period. Why this is happening is anybody's guess.

Column-1: Month
Column-2: Browser
Column-3: % of Total Traffic
Column-4: % of Banner Blocking Traffic

Jun-05 Firefox 7.6% 25.4%
Jun-05 Mozilla 1.5% 1.4%
Jun-05 MSIE 87.4% 68.9%
Jun-05 Netscape 1.3% 1.3%
Jun-05 Safari 1.3% 1.1%

Jul-05 Firefox 8.3% 21.8%
Jul-05 Mozilla 1.6% 1.1%
Jul-05 MSIE 86.5% 73.8%
Jul-05 Netscape 1.4% 1.5%
Jul-05 Safari 1.4% 1.2%

Aug-05 Firefox 8.2% 23.5%
Aug-05 Mozilla 1.4% 1.9%
Aug-05 MSIE 87.3% 71.9%
Aug-05 Netscape 1.2% 1.3%
Aug-05 Safari 1.3% 0.8%

Sep-05 Firefox 8.1% 14.7%
Sep-05 Mozilla 0.9% 0.8%
Sep-05 MSIE 88.0% 81.7%
Sep-05 Netscape 0.9% 1.2%
Sep-05 Safari 1.5% 0.3%

Oct-05 Firefox 7.6% 21.7%
Oct-05 Mozilla 0.7% 3.1%
Oct-05 MSIE 88.4% 72.0%
Oct-05 Netscape 0.9% 1.2%
Oct-05 Safari 1.6% 0.0%

Nov-05 Firefox 7.5% 16.2%
Nov-05 Mozilla 0.7% 1.0%
Nov-05 MSIE 88.2% 79.0%
Nov-05 Netscape 0.9% 1.2%
Nov-05 Safari 2.0% 1.3%

Dec-05 Firefox 8.2% 16.4%
Dec-05 Mozilla 0.9% 0.8%
Dec-05 MSIE 87.0% 79.8%
Dec-05 Netscape 0.8% 0.6%
Dec-05 Safari 2.1% 1.1%


While there is obviously great room for discussion as to what percentage of users are blocking ads and that percentage changes depending upon the demographics of the site in question, it is fairly obvious that ad-blocking is having a real economic impact on websites.

Next: Part 3 – Ad-blocking techniques

[Westech]

This is an excellent read. I'm on the edge of my seat here. This article is sure fire link-bait. If I were you I'd yank this from the forum and get it up in the articles section with my adsense code ASAP! Either way, keep it coming.

Would you be willing to share some information on total visitor numbers to your site? I'm interested in knowing what kind of sample size we're looking at.

[KLB]

I keep some basic stats mostly up to date on my advertising page: http://environmentalchemistry.com/advertise.html

[Masetek]

This is a quality thread and so far a very well written article. Look forward to seeing the next sections. To be honest I never considered ad blocking to be a problem, especially not with such high %'s

[Bleys]

Those are some surpisingly high numbers, Ken. I'll have to sit down and take the time to really read through this and digest it later. Looking forward to the full article.

[KLB]

I figure I'll give my brain a little rest and let people digest and discuss the first two parts some before diving into part three.

[5starAffiliates]

Great articles on a very important topic Ken.

A couple years ago I wrote one if the 1st and most popular articles about Norton ad blocking and how it affects affiliate and webmaster revenue. http://www.5staraffiliateprograms.co...s-revenue.html

Like you I have had lots of discussions on forums about this topic and I mod lots of forums, so the questions keep coming up. I have blogged about many of the individual problems regarding ad blocking, anti-spware that blocks affiliate ads and all type of programs that block or deleted cookies over at my "Adware, Spyware and Cookies" blog.

I would love to have one place - one link to send people to for info. I will probably blog about your articles here next week after you get #3 finished. I get lots of traffic that would be interested in your info here.

Thanks for taking the time to write this!

[Chris]

See exposure... thats why you need to make it a formal article

[ses5909]

Originally Posted by Chris

See exposure... thats why you need to make it a formal article

I agree. Quit your typing in here and save it for an article. That would help the site even more.

[KLB]

Okay I'll follow everyone's suggestion. It might be this weekend or next week before I can tackle this writing. I'm buried in work.

[5starAffiliates]

Hi Ken and Chris,

I just blogged about it and linked here, so let me know when part 3 comes out.

Ad Blocking and You - For Affiliates, Webmasters and Advertisers
http://affiliate-blogs.5staraffiliat...e-advertising/