With the release of our June search data to clients today, I have gotten many media questions focused on the Bing results since its launch in early June. With one full month of metrics available, we can now shed more light on its performance, which can be summarized as follows: It generated significant trial, was successful at increasing Microsoft’s share of search page results, and, to a lesser degree, Microsoft’s share of search sessions and search queries. Here is the comparison of June metrics compared to May:

There was a significant increase of 14.7% in penetration (shown above), driven by new trial which continued to build throughout the month. However, it is important to understand how much of this incremental trial resulted in repeat business. To answer this question, we looked at users who came back for at least another day after their first day of trial during the month of June. We found that 53% of incremental trial users repeated during the month of June. A more important repeat metric is what percent of trial users come back within a 30-day period, which will be available when comScore’s July data are released. Bearing in mind that marketing campaigns are never able to convert all trial users to repeat users, the July data will give us a much clearer understanding of repeat usage.

Despite the strong trial and respectable repeat rate, Microsoft’s share of queries increased by only 4.5% in June. Why? One reason is that while Bing has built a larger base of repeat searchers, it only benefited from their queries after their first trial. And since many of the new trial users came in throughout June, Bing’s share benefited from only a portion of monthly activity from trial users. One would expect the queries from those users to be higher in July, with a full month of usage. Another contributing factor is that, while the share of search sessions increased by 8.9%, repeat users are now conducting fewer search queries per session (down from 2.33 to 2.18). While this adversely affects query share, it may be a reflection of a better search user experience with Bing compared to Live. This would be the case if users were able to get to their ultimate result faster, with fewer queries in a session. We will only know for sure once the dust settles after this initial trial activity.

Bottom line, Bing has had an initial positive impact on Microsoft’s search position. However, one thing is clear: Closing the gap with its competitors won’t happen overnight. Marketing will certainly help generate more trial, but ultimately the product’s biggest opportunity is to capture a higher share of usage among the 41%+ of searchers that are now using Microsoft but only for a portion of their monthly searches. This will ultimately depend on improved user experience, particularly relative to competitors.

Not surprisingly, it all comes back to sustained product performance. Only time will tell if the search experience provided by Bing is compelling enough to increase adoption and change searcher behavior over the long term.

Last month comScore was delighted to receive the M.E.F. Award for Business Intelligence in mobile media, a prestigious award acknowledging the work currently being done by comScore in conjunction with the GSMA and U.K. mobile operators to comprehensively and anonymously measure the mobile world.

Here at comScore we are well aware of the growing importance of measuring online activity across the entire spectrum of digital platforms, and this award highlights the importance of this research as a stepping stone, not only for comScore, but for the digital media industry at large as we pursue the challenge of measuring the entire digital universe.

Paul Goode, SVP of Census Solutions at comScore, and Henry Stevens, GSMA Director of Media & Entertainment, offer their thoughts on winning the prestigious award:

I’ve been in media research for 29 years. I’ve been an executive at Arbitron, and President/CEO at Simmons. So, by now I might know a thing or two about this business. When I joined comScore in 2007, it was largely because I had long considered the company to be the most innovative in the audience measurement space. I wanted to be part of a team doing great, pioneering, innovative work, in an industry too often plagued by inertia and “old school” thinking. In fact, the first thing I did upon joining comScore was to draft a presentation about the methodology, titled “21st Century Audience Measurement.”

Even given my expectations though, when I joined comScore, I experienced quite a bit of culture shock. I always saw media research as a business in which introducing a material change or innovation was a lot like steering an ocean liner. At comScore I learned quickly that for an audience measurement company with a bona fide engineering and technology pedigree, there was no ocean liner. There was only continuous development and innovation.

One of comScore’s innovations that had particularly impressed me in the years before I joined was the company’s recognition, from the beginning, that using a 20th century telephone calling center to recruit an Internet panel was at best untenable, and at worst preposterous. In the marketplace, we talk about using online recruitment, coupled with a smaller, offline randomly-recruited calibration panel to provide the largest possible samples with a minimum of bias. This technique allows us, for example, to include all types of households in our panel - including cell-only (20% of the population), and cell primary (another 14% have landlines but receive all or almost all calls on cell.) These segments of the population are routinely omitted from traditional RDD recruitment; comScore has been including them for years. In addition, the sample sizes that this methodology yields allow us to report audiences for tens of thousands of web entities each month.

Nielsen Online, meanwhile, would turn their nose up at us, holding steadfast to the belief that the old ways were the best ways. They countered Media Metrix with a panel recruited by phone(!), comprised of a sample about one tenth the size of ours and which reported on about one tenth the number of entities.

Needless to say, this won us a lot of business.

Last week Nielsen Online announced, essentially, that they were giving up the ghost. In what they claimed was innovation, they announced that they are moving to the same technique invented by comScore ten years ago: offline recruitment, with a random calibration panel. In the headline of the press release, they claimed that this would now be the largest online reporting panel in the U.S.

I initially saw the press release when Carl Bialik, aka the Wall Street Journal’s Numbers Guy, forwarded it to us asking for a comment. Naturally the first thing I pointed out was that the Media Metrix U.S. in-tab for June was 300,000—fully 30% more panelists than Nielsen was touting. Our Media Metrix panel size has been increasing as a natural outgrowth of our program for constant improvement. We have one million comScore panelists in the U.S., and a subset of these qualify each month for Media Metrix reporting in-tab (the rest are available for other comScore services and analytic work.) Because we have been improving our technology for identifying the demographics of the person using the computer at any point in time (fully two thirds of Internet users in the U.S. are on multi-user machines), many more sample persons have been able to pass our rigorous Media Metrix reporting criteria and qualify for in-tab.

Nielsen seemed to be caught off guard by this development; no doubt they had been devoting a lot of time and money to the initiative to adopt our methodology, and had expected to leapfrog us in panel size. A Nielsen spokesman even backtracked quickly to a statement that “size isn’t everything” - which forces us to ponder the wisdom of touting the benefit of panel size in the headline of the press release in the first place. (By the way: size - and representivity - do matter, if you want to report on more entities with less sampling error.)

Incidentally, our competitor touts the fact of reporting 30,000 entities with their “new” methodology. In the June 2009 U.S. Media Metrix release, comScore reported on 70,000 entities; in our three-month rolling average report, we reported on 95,000 entities.

To be fair, I think Nielsen was legitimately vexed by the announcement of Media Metrix 360 in May. Readers of this blog will know that 360 is comScore’s groundbreaking move to panel-centric hybrid measurement, integrating data from our panel with census-level server data. Advertisers, agencies, and publishers have been widely embracing this approach, and many of the world’s best media brands are already beaconing for us.

Media Metrix 360 brings together the two primary datasets in online metrics - panel measurement, which provides a 360 degree view of the behavior of a sample of persons; and, site-centric server data, which provides a “census” view of activity for a given web entity. The panel data provides person-level insights like demographics, time spent, engagement, and cross-site duplication (which are necessary for buyers and sellers of advertising to disentangle total ad impressions into reach versus frequency.) The server data provides a census of all activity occurring at the website’s servers - every server call.

Of course the server data needs to be filtered to exclude non-human traffic (bots, spiders) and ineligible traffic (e.g., redirects and other non-valid traffic we often see tabulated in publisher server counts, but which we exclude from Media Metrix.) And, in the calculation of unique visitor numbers, server data must also be adjusted for the deleterious impact of cookie deletion. Then it must be parsed by country of origin, so that we only include U.S.-based traffic in the U.S. Media Metrix, and Canada-based traffic in the Canada Media Metrix, and so on. But once properly filtered, server data can enrich and enhance the quality of the overall Media Metrix offering.

Media Metrix 360 hybrid data won’t necessarily match a publisher’s internal server data, for all the reasons cited above. But for the first time, publishers will be able to understand and empirically reconcile the differences between their internal, and comScore’s syndicated, audience data. In addition, the trends in Media Metrix data from month to month will correlate even more closely with trends in publishers’ internal data. Also, the hybrid approach will enable us to report at an even more granular level, and to introduce more frequent and timely data releases. The clients I’ve talked to on both the publisher and the agency side are uniformly excited about these developments,

And in case you’re wondering, the MRC audit process for the 360 component of Media Metrix has already begun. (If this has whetted your appetite for Media Metrix 360, why not click through to comScore Direct and sign up?)

On reflection, I think that Nielsen Online had dedicated so much time and effort to their massive sample catch-up initiative that they did not see Media Metrix 360 coming. Media Metrix 360 is the next generation of online audience measurement; and the 360 announcement pre-empted their news.

I wouldn’t be surprised to see Nielsen begin steering the ocean liner in a new direction. One can only wonder where we’ll be by the time they get here.

You can’t help but read in the news today – ironically -- about the challenges facing the newspaper industry. In the course of the past few months, both the Rocky Mountain News and the Seattle Post-Intelligencer have ceased print news operations, with the Seattle Post-Intelligencer moving exclusively to the digital publishing platform. Both the Chicago Sun Times and the Tribune have filed for bankruptcy. More recently, the New York Times, having been in continued negotiations with its Boston Globe writer’s union (the company’s largest employee union), has reached a tentative agreement on pay cuts that will allow the Boston Globe to at least start their way down the road toward profitability.

In light of such turmoil in the publishing industry, I was interested in investigating the viability of the digital publishing platform as the print medium encounters more and more challenges. Are newspapers losing gross audience, or is the audience simply shifting online? How do these online and print audiences differ? How are newspaper sites performing relative to other online news outlets?

According to comScore Plan Metrix, the overall readership of print newspapers* is down 11 percent in the past year, from 86.4 million in April 2008 to 76.7 million in April 2009, but the total number of visitors to the online newspapers category is up 5 percent during that same period. Clearly, the decline in the number of print readers (-9.7 million people) is far greater than the increase in the number of online newspaper readers (+3.2 million). At the same time, the number of readers of news content online has increased by 8.6 million people. These data indicate that while some print newspaper readers are indeed switching to reading newspapers online, more are switching to reading news online at sites other than newspapers.

* Newspaper Print Readers includes Internet users 18+ who read at least one of the following daily papers in last 7 days: Atlanta Journal-Constitution, Boston Globe, Chicago Tribune, Dallas Morning News, Denver Post, Detroit Free Press, Financial Times, Houston Chronicle, Los Angeles Times, Miami Herald, Milwaukee, Journal, New York Times, Philadelphia Inquirer, San Francisco Chronicle, USA Today, Wall Street Journal, Washington Post.

To gain more insight into readership trends, I did some further analysis by looking at three mutually exclusive segments of newspaper readers:

1. Those who only read print newspapers
2. Those who read both print and online newspaper formats
3. Those who only read the online format

Here we see clearly that readers of print newspapers are abandoning the format. The audience who reads only the print edition declined 12 percent vs. year ago, while those who read the print edition and visited a newspaper site also declined -- by 10 percent. On a slightly more positive note for newspapers, visitation to sites in the online newspapers category grew 26 percent vs. year ago among those who did not read the print edition.

However, if we look at the total combined newspaper reading audience, we find that readership is down 2-percent overall – certainly less alarming than the 12 percent decline among print-only readers but a decline nonetheless, and one that is occurring in the face of an increase of 6% in the number of visitors to sites in the overall news category. So as print newspaper readership declines, we are not seeing a sufficient increase in online newspaper readership to offset the decline. Rather, readers appear to be shifting to non-newspaper sources of online news.

These trends demonstrate the challenge for newspapers to more deeply engage online with the growing number of consumers who do not get any of their news information from the print or online editions of their newspapers. Beyond looking for approaches that will attract these consumers to their own sites, the newspapers must explore alternative ways – including using social media or distributed content as potential distribution models - to reach this audience as the Internet becomes the preferred medium for news consumption. By continuing to evolve their services in a way that aligns with their consumers’ preferences, they may be able to identify alternative ways to offset the revenue lost from their declining print channel.

I’ve been involved in measuring advertising effectiveness for much of my 40 year market research career, most often using behavioral metrics such as sales to gauge advertising’s ROI. During this time, I’ve become a big believer in the importance of reach and frequency (R/F) metrics. To me, they’re crucial in assembling a compelling media plan and then figuring out why some plans worked while others failed. They gain even more importance when putting together a multi-media plan and one is trying to gauge the degree to which adding additional media will represent gains in frequency or reach or both.

So, I’ve long been puzzled as to why the online advertising community hasn’t used R/F more often in its media planning and analysis. Perhaps it’s related to the industry’s long-held obsession with ad impressions and clicks as the metrics of choice in digital media planning. Why worry about R/F when you can use the click as a measure of advertising effectiveness? Well, that idea is fast going out the window as the realization sinks in that with click rates on display ads dropping to levels of 0.1%, the click can’t be viewed as a measure of anything to do with ad effectiveness. Slowly but surely, I believe that the industry is realizing the value of R/F and is beginning to view digital media plans through more of a traditional media lens. And, some of the best minds in the industry are making compelling arguments in that regard. Two recent articles stand out in my mind.

The first is a blog post authored by Young Bean Song, the well-respected media guru from Atlas. In this insightful post, Song argued that “trying to build a brand marketing campaign without traditional target reach and Gross Rating Points (GRP) estimates is like trying to diet without the concept of calories.”

The second was an article authored by Geoff Ramsey, CEO of eMarketer, who admitted, “Until recently, I sided with the anti-GRP arguments and dismissed those campaigning for GRP measurements on the Web as naive, thinking: ‘They just don’t get it, do they?’ But I’ve changed my mind, and now support the adoption of GRPs to integrate digital within the media budgets of big brands.”

Yes, I readily admit that I apparently had some influence on Geoff’s conversion, but I have to tell you that he’s not an easy man to persuade. So, if Geoff is now a believer in R/F, I think it’s a telling moment for the industry.

I do want to mention one aspect about R/F that I think may have led to confusion within the digital community. It’s not that I’m arguing that R/F represent metrics indicating whether or not a plan has worked. That can only be determined by measuring the success of the media plan (and the particular creative that was used) in building brand sales. But, I strongly believe that R/F considerations are vital when deciding how to structure a plan and critical when one is trying to learn from the sales results of a plan why it worked or didn’t. Again, one has to consider the creative that was used, but it seems to me it’s impossible to do any serious retrospective analysis without taking into account how many people were reached and how many times. Simple, but vital.

Ted McConnell, Director of Digital Marketing Innovation at Procter &Gamble, put it well at a recent conference when he said: “Call me old fashioned, but P&G thinks it’s rather important to know what we say, to how many people and how often.”

Couldn’t have said it better myself.

This post was originally published at paidContent.org on July 30, 2009.

It’s high time our industry provide large branding advertisers with metrics that prove that the web offers just as much—if not more—ROI as traditional media. That’s the best way to convince them that it’s OK to shift large portions of their ad spending to the internet.

Consider the following. In a recent blog post, Young-Bean Song from The Atlas Institute (part of Microsoft Advertising) pointed out that if we separate advertising into its two main forms—direct response and branding—and look at the percentage of all measured media that online represents, we see that online direct-response advertising dollars have flowed strongly onto the internet, capturing 30% of all measured direct-response ad dollars) while branding is performing poorly, with only 5% of measured media branding dollars going on online.

Why has the internet failed to attract branding dollars? I lay responsibility squarely at the door of the “click.” Used since the early days of online advertising as an indicator of the effectiveness of an ad, the click originated simply because it could be measured. But not everything that can be measured matters.

In fact, the use of clicks on display ads as a meaningful metric sets the internet up for failure as a branding medium. Doubleclick reports that click rates on display ads today have fallen to approximately 0.1%, an unfortunate reality that has created serious doubts about the value of online advertising in the minds of advertisers that have experimented with the internet as a branding medium. It’s now clear that a publisher would have to be insane to continue using click metrics to try to persuade branding advertisers to turn to the internet.

If the industry can move beyond the click, the future of online branding advertising is bright. By using appropriate metrics, the ability of online display advertising—whether in the form of static display ads, rich media or video—to build brands can be shown to rival or even exceed the effectiveness of traditional media. In a white paper “Whither the Click” (published in the June issue of the Journal of Advertising Research), we summarized the hundreds of studies we’ve conducted using the comScore panel and comparing the behavior of panelists exposed to brand display ads with the behavior of those who did not see the ads. Even in the face of negligible click rates, time and again we observed statistically significant lifts among the ad-exposed consumers in the number of visits to the advertised brand’s web site, the number of trademark search queries, and the sales of the advertised brand, both online and offline.

While these metrics are vital for understanding the true effectiveness of online advertising, reach and frequency (R/F) metrics are also important tools for media planning and analysis. Traditional brand advertisers have been using such metrics for decades, and these metrics should also be central to online media planning and analysis. Let me be clear. I’m not arguing that R/F metrics can indicate whether a particular media plan has worked—that can only be determined by measuring the success of the plan in building brand sales, taking into account the particular creative that was used. But R/F considerations – how many people were reached with ads and how many times—are vital when deciding how to structure a plan and critical when one is trying to understand, based on the sales results, why a plan worked or didn’t.

One problem is that measuring an ad campaign’s reach and frequency on the Internet is not as simple as it is for traditional media because there are so many different locations from where an ad can be delivered on an individual web site. For that reason, R/F needs to be measured at the ad-placement level, not at the site level. To that end, this week comScore today announced an offering with Microsoft Advertising to provide R/F planning and analysis tools at the ad-placement level based on Atlas ad server data and comScore panel data. We believe this is a much more precise approach because it shows the reach of the ad campaign that can actually be achieved, the true potential frequency, and the specific demos of that audience. Campaigns planned at a total site level can overstate reach, understate frequency and may not deliver the desired demographic.

Perhaps Ted McConnell, Director of Digital Marketing Innovation at Procter & Gamble, put it best at a recent conference when he said: “Call me old-fashioned, but P&G thinks it’s rather important to know what we say, to how many people and how often.”

When traditional media thinks about branding advertising, it focuses on creative, reach and frequency. These are time-tested factors. It’s time for online display advertising to go back to the future.