The Measurement Mess | @GeorgeWinslow

Among many other lessons, history has taught us that when currencies collapse, the results are typically catastrophic. In Weimar Germany after World War I, for example, inflation reached such astonishing levels that consumers famously had to cart baskets of paper money to buy a loaf of bread. Waiting on one bread line, a woman turned away from her cache of nearly worthless money long enough to discover a thief had stolen her basket, leaving the cash piled on the street.

Thankfully, the ratings currency that has long been the lifeblood of the TV industry isn't in a similar death spiral. But the turmoil engendered by the exploding popularity of digital media has roiled the research world, calling into question the value of measurement currencies that have been the foundation of commercial TV for more than a half-century.

Here, the problem is twofold: TV uses very different measures of usage than those found in the digital world, making it difficult if not impossible to add up total viewing for a show across TV, online, mobile, social media and other digital platforms.

Worse, the fastest-growing areas of digital media-mobile phones and tablets-are not currently being measured at all by Nielsen, which supplies the ratings currency for the $70 billion TV ad spend.

The result, some researchers complain, is morass of incompatible data that is forcing them to create what several call "Frankenmetrics."

"It's been called Frankenmetrics because we are getting arms and legs and heads from different places that have to be sewn together," says Jack Wakshlag, chief research officer at Turner. "Unfortunately, the result really isn't a whole functioning entity."

"Fixing this is the most important issue facing the industry right now," adds Alan Wurtzel, president of research and media development for NBCUniversal. "Ratings are what we transact and evaluate the business on. If you can't measure it, you can't sell it."

Digital Big Bang

This is particularly important as viewing continues to fragment over multiple devices, explains CBS chief research officer David Poltrack. He notes that in the 2012- 13 TV season, CBS' Elementary had a live TV audience of 9.4 million, which increased by 68% to 15.8 million when DVR, VOD, online streaming and mobile usage was added.

Similar stats can be cited for popular CBS sitcom The Big Bang Theory, which sees its 12.9 million live audience jump to 23.9 million when DVR, VOD and digital media are added over the course of a week.

"The total audience for shows can nearly double when you add in all the other viewing," Poltrack says. "We have to make sure we have a comprehensive measurement system to capture a program's exposure in all of these places."

Accurately comparing all of those audiences will become even more important as operators and programmers expand their TV Everywhere efforts. "In a world where all content is going to be made available on all platforms, it is critical that we have the ability to measure that viewing to monetize it and to better understand consumers," says Matthew Strauss, senior VP of digital and emerging platforms, Comcast Cable, which has been working closely with Nielsen and others to improve ratings measurements.

The vastly different metrics also create an image problem for the TV industry among both consumers and Wall Street investors. During this year's NewFronts, a number of tech writers highlighted a presentation where Google chairman Eric Schmidt seemed to say that YouTube had caught up with TV thanks to the fact that the site now has some 1 billion unique users who consume some 6 billion videos per month.

But those figures reflect the widely used online metrics of unique visitors and video usage "per month for the whole world," not the daily TV ratings Nielsen produces for the U.S. "You have a Tower of Babel with incompatible systems," says Brian Wieser, senior research analyst at Pivotal Research Group. When Wieser translated worldwide TV viewing into figures that could be compared to the Google data, he estimated that monthly worldwide TV viewing would be about 600 million hours, or 100 times the actual consumption of YouTube.

The $10 Billion Question

The confusion surrounding multiplatform measurement also impacts a variety of other players, including ad agencies and media buyers. "We are chomping at the bit over here," says Jordan Bitterman, chief strategy officer of Mindshare North America. "The media and marketing industry is definitely thirsting for crossplatform campaigns and measurement."

Tech giants have an even bigger stake in better measurement. Much of the revenue collected by Google and Facebook in advertising comes from desktop online advertising from PCs, Macs and laptops. But consumers are rapidly switching their usage from those well-measured online platforms to mobile devices and apps, which are at present difficult to measure and monetize.

A December 2012 study by the mobile analytics firm Flurry found that consumers are spending 127 minutes per day using mobile apps, up 35% from 94 minutes per day in 2011. That is more time than is spent on desktop Web usage, which came in at 70 minutes, and it approaches TV usage, which they peg at 168 minutes per day.

But mobile advertising totaled only $3.2 billion in 2012, or only 9% of the $36.6 billion online ad market, according to data from PricewaterhouseCoopers.

Cameron Clayton, president of the digital division at The Weather Co. and global chair of the Mobile Marketing Association, guesstimates that measurement accounts for "20% to 30%" of the problems that have reduced mobile advertising.

"The Nielsens and the comScores of the world have been slow to react to the marketplace and the shift to mobile," says Clayton. "And that creates a real challenge for companies like us that have 100 million mobile users....It's a $5 billion-to-$10 billion problem just in mobile," he adds.

Fixing the problem may not, however, be easy, in part because some companies block access to usage data on their platforms. "There are environments that are hostile to third-party objective measurement," says Josh Chasin, comScore chief research officer.

Netflix, for example, does not allow content holders to put Nielsen tags on programs in order for Nielsen to measure viewing by Netflix subscribers. Likewise, Apple has refused to allow researchers to put software in its iOS operating system, a move that has delayed measurement of iPhones and iPads by several years.

Better cross-platform research will also require advertisers, agencies and major industry execs from the TV, online and mobile world to agree on common terms for defining that usage, which is difficult given the different nature of TV, online and mobile usage. Most TV players would like to see a currency based on gross ratings points (GRP); others believe it will be impressions, preferring to see TV ratings mirror the currency of online measurement. Not surprisingly, the model battle reflects the needs of each side.

"A lot of what is going on is tied to efforts to make online measurement more like TV by using TV metrics like reach and GRP," says Brian Hughes, senior VP of the audience analysis practice lead at Magna Global. "But others would prefer TV to become more like online, where you are really buying audiences and not content."

Frustration and Innovation

Still, progress is being made, most researchers say. They stress that frustration over the multiplatform ratings mess has prompted a heated battle between research companies and a long list of cross-platform ratings initiatives designed to solve the problem.

In June, Telemundo announced an ambitious project to work with Symphony Advanced Media and Vision Critical to measure Hispanic cross-platform media usage of TV, online, mobile and social media.

"The fact that Hispanics are not only adopting mobile and digital media, but adopting it more quickly than the general population made it essential that we get more information on their experiences," says Doug Darfield, Telemundo executive VP of research.

Also last month, ESPN announced the first results of a project to measure five platforms-TV, radio, online, smartphones and tablets-that showed ESPN's content reached 136 million adults, including 85 million men, during February 2013, reports Artie Bulgrin, senior VP of research and analytics for the network.

The numbers were drawn from set-top box data in 4.5 million homes supplied by comScore for TV viewing; radio data from Arbitron Portable People Meter; census data of PC, smartphone and tablet usage from comScore's MediaMetric Multiplatform; and a calibration panel from Arbitron.

Widespread frustration with the progress in fixing the multiplatform measurement confusion is also pushing Nielsen, comScore, Rentrak and others to expand their cross-platform ratings efforts.

"Our biggest single priority is to solve this issue for our broadcast clients," says Megan Clarken, executive VP of global measurement products at Nielsen. As part of that push, Nielsen is currently in pilot testing for tablet and mobile measurement, which it plans to integrate into its TV, DVR, VOD and online measurements systems by fall 2014.

Meanwhile, comScore is planning to expand from its dominant position in the online world to cover all platforms by launching TV measurement services; and Rentrak, which already offers VOD and local TV measurement using set-top box data, continues to ramp up its online and mobile efforts. (See "Big Data Mash-Up," page 13.)

My Own Private Census

All of those efforts are significantly changing the way measurement is approached. One very notable development has been the move in recent years to complement research panels with "census," or big data approaches that draw on millions, if not billions, of data points.

In the long run this could make TV ads more like those in the online industry, where ads are served based on individual characteristics. But it is already improving measurement for TV, VOD and other media.

"Where there is a lot of fractionalization as you see in the multiplatform world, you have to use census data to pull back enormous amounts of information to count all the views," says Bill Livek, CEO of Rentrak, which has built up a successful business tracking VOD and local TV usage based on set-top box data.

Nielsen and comScore are also expanding their use of set-top box data. But at the moment, no single provider has access to all the data collected by the multichannel operators, and there are still issues involved in tabulating the information from different cable systems and set-top boxes. "It is a very important tool, but we are still in the early stages," cautions Strauss of Comcast.

Another major problem is that detailed cross-platform measurement often requires customized proprietary research, notes Colleen Fahey Rush, chief research officer at Viacom Media Networks. She adds that in order to track a campaign for Kraft's BelVita breakfast biscuits and Wheat Thins tied to Jersey Shore on MTV and The Colbert Report on Comedy Central last year, they ended up using Nielsen, Omniture, 3D Accountability and the Insight Express' Ignite panel.

Besides the complexities of measuring such campaigns, Fahey Rush and others also complain that they generally only have access to detailed usage about their own content, but not about their competitors.

"In addition to Frankenmetrics, we are walking into the future with one eye shut," Wakshlag says. "We can see what our own content is doing, but we don't know how big the usage is compared to our competitors."

Still another issue researchers stress is that many of the measurement systems slated to be the basis for cross-platform usage have limitations. For example, Fahey Rush explains that Nielsen's Online Campaign Ratings product gets its demographic data through Facebook, which only had information on people 12 and older, and that comScore's VCE is not reporting usage of people under 18, which creates major problems for tracking numbers on Nickelodeon's content.

"The battle between Google and Facebook is also impacting multiplatform measurement," Wieser adds. He and others point out that Google, which is pushing its own Active GRP measurement solution, does not allow Nielsen to put tags on its OCR product because Nielsen gets its demographic data from its archrival, Facebook.

Over-the-Top Secrecy

Researchers also complain about the Netflix practice of not allowing Nielsen tags to be placed into content on their platform so the usage can be tracked. PricewaterhouseCoopers expects electronic home entertainment revenue to hit $11.5 billion in 2016-more than what is currently spent on DVDs-which will put programmers in a much weakened position during rights negotiations because they have no way to know the value of their content versus other programmers.

"Knowledge is power," Wakshlag says. "They want to make sure they know what is going on within their universe and make it difficult for us to know how powerful the archived content is."

Similar problems occur with Apple and other OTT players. "The problem of getting good data is more common than uncommon," says Fahey Rush. "It is not like we get complete data sets from Apple or some of the other download-to-own players."

Other issues continue to complicate mobile measurements, adds Andy Bovingdon, VP of product marketing at Bango, which provides some innovative solutions for enriching the data so user numbers for Wi-Fi networks and other places can be tracked.

"Data enrichment is crucial if you want to go from ‘big data' to big intelligence, which is more about the meaning of the data," Bovingdon says.

The many black holes in digital measurement also raise doubts about the immediate promise of that so-called "big data," which has emerged as one of the hottest tech topics of the last year.

"There are technical problems in doing it right, and huge privacy concerns," says NBCU's Wurtzel, who sees big data as an important tool but not a panacea. "If it were that simple, we would be already doing it."

The inevitable move to handling more of the big data generated by the digital world will also require some major investments and changes in the way TV companies think about research.

"We are moving in the direction of big data, but we have a dilemma right now," adds Poltrack of CBS. "We need more resources, more personnel to get to the measurement system of the future. But we also need more resources to build up the [current] increasingly obsolescent infrastructure and deal with the fact of what's happening today. There is an upfront market going on right now where $10 billion is being spent."