I occasionally get asked this question, and I usually answer that different methodologies may produce different estimates. But I know that doesn’t really get at the heart of the matter. After all, given how many searches are conducted across the U.S. each month, shouldn’t the search share number from the various measurement services be pretty close to each other?

Let’s examine the Google search share figures in November 2008 for the three largest online measurement services, in order: comScore, Nielsen and Hitwise.

comScore and Nielsen are within about half a percentage point of each other, while Hitwise shows Google at a substantially higher 72%. Now, why is it that Hitwise consistently shows a search share about 8 percentage points higher than comScore and Nielsen? A behavior as ubiquitous as Internet search yields extremely robust sample sizes, which means that there should be little variation between the services if each was sufficiently representative of the U.S. Internet audience.

Let’s investigate why that may not be the case…

Both comScore and Nielsen build their Internet panels by recruiting consumers directly and then installing patented software on the panelists’ computers to collect their online behavioral data. comScore is the only service to also include complete AOL proprietary activity. This ensures that all Internet Service Providers (ISPs) are correctly represented in the comScore database. However, Hitwise obtains its clickstream data by purchasing it directly from those ISPs that are willing to sell it to third parties. It’s unclear exactly which ISPs or how many provide their data to Hitwise, since the company does not disclose that information. However, what is publicly known is that, mainly because of privacy concerns, the vast majority of ISPs are, today, unwilling to sell their data to third parties. The most prominent one – and likely only one of the major ISPs – that is willing to sell its data is United Online/NetZero. Interestingly, an analysis of Google’s market share across ISPs (using comScore’s data which represents all ISPs) suggests that the Hitwise data may be mainly reflecting Google’s performance at United Online/NetZero, an ISP composed primarily of dial-up users.

The analysis below shows how Google search share varies across ISPs in the comScore panel:

Here, we can see that Google’s search share peaks among United Online/NetZero users at about 73% – a number that is strikingly similar to what Hitwise reported as Google’s overall U.S. share in November. The other ISPs shown above are all major broadband providers, who do not sell their data to third parties. Google’s market share in each of them is significantly lower than within United Online/NetZero. Note, especially, that Google’s share is lowest among AOL ISP users, an ISP that Hitwise does not measure.

So, is it possible that Google’s high share as reported by Hitwise overstates reality and is simply a result of the fact that Hitwise’s estimates are predominately based on ISP data from United Online/NetZero, the ISP where Google’s market share happens to be highest?

What do you think?