Alexa, Compete and Quantcast

by Craig Raw

Ever since creating the SearchStatus addon for Firefox, I've spent a good deal of time thinking about measurement online. Marketing on the Web is all about measurement. It is the holy grail that completes the ROI circle and lets you know if your ad, so expensively placed on that media site, is actually doing anything, as opposed to the billboard that gives you the warm fuzzies when you drive past in the morning.

Of course, no-one really knows all the figures. Perhaps one day we will all openly share our logs for objective comparison, but until that day it is necessary to take a representative panel of users and extrapolate to determine some kind of a ranking. A ranking that can, for example, tell you if placing an ad on media site A is better than media site B.

Several enterprising companies attempt to provide this service, and the industry is getting more and more competitive. This post is an attempt to review some of the options out there, with the stats you read below kindly provided by Geoff Mack of Alexa, TJ Mahoney of Compete and Paul Sutter of Quantcast.

Of the online measurement companies, Google must be considered one, and although their PageRank algorithm is more a byproduct of their search focus, their size and scope mean they cannot be ignored. Alexa is more obviously a candidate, since their Alexa rank is a common statistic. Their relative longevity and prominence in the market means Alexa provide the standard to measure against. Compete are generating a fair amount of buzz in the industry, collecting data from a number of different sources. And finally, Quantcast are also coming to the party claiming some impressive numbers.

First, let's take a look at sources: Google uses their extensive network of bots crawling the web to determine page relevance from linking structures and content. The amount of information they have at their disposal is massive, but it's aimed at improving their famously secret search algorithms, not enlightening the world with measurement. One can only dream...

Alexa use a panel of toolbar users, of which SearchStatus users comprise a portion. Although they do not publish user stats on their site, I can tell you a few facts: They are currently on about 20 million users each month, which are qualified and controlled to a panel size of about 2.5 million. While each toolbar has a unique identifier for it's clickstream, Alexa does not have detailed demographics for them all. A nice feature of this method is that it is opt-in only, meaning your activity is not measured without your permission. However, Alexa's stats have been claimed to be biased towards webmaster-related sites, since these are the type of users more likely to have the toolbar installed.

Compete collects data from a variety of sources. Like Alexa, they have toolbar users, which also provide a clickstream without detailed demographics. Data from ISPs is also collected, and there are opt-in panels with demographics - all told, they currently have about 2 million users per month. They do perform normalisation on their data, and claim less bias that the Alexa figures.

Quantcast have two sources of information - They collect "pixel data" from scripts installed on participating websites, claiming a total of about 50 million visitors per month currently, and a further 1.5 million in panel data is provided by relationships with ISPs.  Referrer headers mean the pixel data is useful beyond the actual sites that are tagged, and they claim it has less bias than toolbar-type methods. Normalisation is done on the ISP data according to demographic and other details.

Another important differentiator is that Alexa measures internationally, while Compete and Quantcast currently only take the US domestic market into consideration. Alexa also provide an extensive API to their data, which in my experience can only improve their offering through the greater interaction possible.

But, which measures better? This post from SEOMoz is one attempt to find out - they have taken actual logged stats from certain sites and compared them against many tools in question. My take on it is that the jury is still out on this one - while all sites provide useful information, none should yet be considered as authoritative, particularly as you proceed further down the long tail. I once read an amusing analogy in which Alexa was compared to a candle in a dark cave - not completely illuminating, but certainly an improvement on no candle at all. I say, the more candles the better.

2007/01/31 | permalink | comments (2) | trackbacks (0)

  Bookmark this on ... BlinkBits BlinkList Blogmarks Buddymarks CiteUlike Connotea del.icio.us Digg it FeedMarker feedmelinks Furl Give a Link Gravee igooi Lilisto Linkagogo Linkroll ma.gnolia Maple.nu Netvouz Onlywire RawSugar reddit Scuttle Shadows Simpy Sphinn Spurl tagtooga TalkDigger Wink Yahoo MyWeb