It’s been nearly seven months since comScore first announced the introduction of Media Metrix 360, our new panel-centric Unified Measurement of digital audiences. Our stated premise behind this initiative was to bring the digital media industry a solution which integrates server-side web analytics which do a good job of measuring total page views (if properly filtered for non user-requested traffic and counted correctly as one beacon per page) and panel-based audience measurement which provides insights into the behavior of individual people, as opposed to cookies or machines. The response to this initiative has been overwhelmingly positive, as evidenced by the high level of participation among top publishers – approximately 75% of the top 50 publishers in the U.S. are either fully reportable under this new methodology or in the process of doing so – as well as the reaction we’ve gotten from agencies and other industry stakeholders. This new methodology becomes even more important when considering the evolution of digital media, including the emergence of new channels for media consumption (mobile devices, tablets, e-readers, etc.) and the increasing fragmentation of the content landscape.
In short, our industry requires a digital media measurement infrastructure equipped to handle the realities of the next decade and beyond, and comScore has risen to the challenge. As with any major undertaking, there have been some hurdles and challenges we’ve encountered and overcome along the way. But in the course of these seven months, we have learned an extraordinary amount that we believe is enabling us to quickly vault our industry years into the future.
Through this undertaking, we’ve accumulated a new understanding of the highly complex and fragmented digital media landscape. To say that measuring this environment is complicated would be a severe understatement. But we have rapidly unearthed many intricacies and nuances which lead to a more accurate and harmonious measurement landscape.
One of the earliest – and perhaps most obvious – findings along the way is that we’ve seen ample evidence of the foibles of server-side analytics for measuring the number of unique visitors (i.e. people) who visit a site. Due to many inflationary factors, including cookie deletion and rejection, bot and spider traffic, and site visitation from multiple locations, we’ve found clear and direct evidence that web site servers routinely overstate actual people counts by a factor of two and higher. In particular, the inflationary impact of cookie deletion is consistent with independent research from a variety of other research companies, including Forrester, Belden, Jupiter and Nielsen. The inflation in server data has also now become apparent to the IAB (see their Audience Reach Measurement Guidelines) and academics such as Max Fomitchev, an assistant professor of Computer Science & Engineering at Pennsylvania State University who conducted an exhaustive study and concluded:
“Cookies are about just as inaccurate in estimating unique visitors as unique network addresses. This is the new and unrealized fact in the industry that has a direct impact on Internet advertising as currently reported unique visitor / core audience size numbers tend to overestimate the true audience size by a large factor (7-30, depending on the visitation frequency and the sampling period).”
As we have reached critical mass among websites participating in 360 programs, we have found some astounding statistics. In the month of December alone, we have seen over 1.5 Billion unique cookies just in the US. For certain portals we have observed significantly more cookies than there are Internet users in the U.S. These realities indicate that it is imperative to develop sophisticated methods to remove inflation caused by cookie deletion, rejection, multiple browsers, multiple devices and multiple locations of access such as home and work.
We’ve also discovered instances that illuminate some of the limitations of panel-based measurement. While we have numerous instances where a panel performs extraordinarily well in measuring audiences, there are some types of sites that can be more difficult to measure with a panel, including those visited by heavy Internet users from work computers and certain niche audiences. Huffington Post is one site that falls into both these categories, which has led to a substantial increase in its unique audience counts under the hybrid methodology. On the other hand, we’ve found that other sites show minimal changes under the 360 methodology.
A few people are under the impression that all we do is simply re-publish a Web site’s server data and charge for it. The statistics we publish are based on integrated panel / server methodology and typically show usage levels substantially lower than what one would obtain from raw server data alone. It is important to keep in mind that we are focused on people, which is what is needed for media planning. When an agency plans a campaign, it is usually looking for unduplicated reach across sites to assess the net reach a campaign will deliver. This can only be done by using a panel to track the same user across sites, using a unique and persistent identifier (which, because of cookie deletion and rejection, cannot be a cookie) to calculate the degree of audience overlap and the allocation of impressions between reach and frequency. It’s also important to identify the specific individual using the computer at any point along with their demos. There is no way to do this in an accurate manner without using a panel. Hence, the need for a panel, and the reason why the Media Metrix 360 methodology is based on a “panel-centric” model.
In addition, our position as an unbiased third-party audience measurement standard creates a burden of continuously and closely auditing any server data source that we receive to ensure it fairly represents the relevant audience being reported. Server data are notoriously complex and require significantly greater rigor than simply slapping a beacon on a page and reporting the calls. We need to filter out non-user requested traffic (pop-ups, i-frames, promotional servers, robotic traffic, etc.) and sort traffic according to its point of origin (geography and then home, work, public computer, mobile device, etc.). It is very important that we count everyone applying the same rules and standards. Furthermore, we have to take great pains to make sure that no company is attempting to game the system to try to get higher audience estimates. We also have to diligently apply edit rules along with branding and ownership standards that are strictly applied and rely on the use of our panel for verification. This particular role that we play is akin to the big 4 audit firms auditing a company’s financial statement by verifying the company’s own data, or the Audit Bureau of Circulation, which audits circulation of newspapers and magazines.
While it’s clear that there is widespread consensus that building Media Metrix 360 is the right approach, a small minority have questioned our decision to charge non-clients $5,000 to set up their site’s reporting using the Media Metrix 360 methodology. We believe this is a very reasonable price to charge for six months of reporting back to the client. If at the end of that period, the client does not want to continue subscribing (no-one has yet done so), we will still continue to implement the 360 methodology free of charge as long as the web site continues to maintain our beacons / tags. While we certainly understand that not every site can afford to purchase our data, the simple reality is that we incur substantial incremental costs associated with auditing, processing, editing and storing terabytes of server data and then integrating it with our panel data using our proprietary statistical methodologies. Companies willingly pay just for audit services -- and at a significantly higher rate than we charge. We believe it is only fair that we charge for the work we have to do and the use of the comScore assets we bring to bear on providing the service.
Some have argued that collecting data in this manner should be a cost of doing business and that some other services offer beacon-based measurement approaches for free. However, this argument neglects to account for how those other services monetize their approach – which is by selling publishers’ audiences (using their own cookies) to advertisers (and occasionally to their competitors), sometimes with the owner of the cookies not even being aware that this “Trojan Horse” strategy is happening. comScore is not in the business of selling a client’s cookies and that is part of the reason why our industry views us as the leading unbiased yardstick of Internet behavior. In life and business, there is no such thing as a free lunch, so Web site operators need to carefully consider the true cost of being beaconed by companies who are in the business of selling their cookies or using them for targeted advertising by leveraging information they got from those sites. In fact, we have observed instances where some Web sites were surprised and dismayed to discover that the “free” services they were using were, in fact, off selling their audience cookies on the online ad exchanges without their knowledge. As you can imagine, in such cases the beacons and tags were swiftly removed.
Ultimately, we both understand and expected some growing pains in undertaking such a major initiative as Media Metrix 360. Change, after all, is never easy. But it often reaps significant rewards down the road, and we’re delighted to find that our clients are moving us down this road faster than we ever anticipated. In the end, we believe the industry will be significantly better equipped for the future using our new reporting service and that it represents a more advanced, transparent and accountable measurement platform than has ever existed in the history of media measurement. And that is a very good thing.
Subscribe via RSS
Subscribe via Email
Follow us on Twitter