Measuring Engagement

The CW series One Tree Hill averaged just a 1.3 rating in adults 18-49 for the season to date, to rank 184th among primetime network series.

But a funny thing happens to the teen drama's value if ranked another way: It's in a four-way tie for a lofty sixth place in IAG engagement (see table). Among the other three series tied for sixth in engagement is Fox's Prison Break, which has more than double Hill's audience. Lifetime and Comedy Central were tied in audience rating, in an analysis by NewMediaMetrics, but the audience engagement value was far higher for Comedy (see table).

As marketers develop metrics to measure audience engagement, this and other little-understood new yardsticks are increasingly employed to help steer billions of dollars in advertising buys across all media. Advertising industry executives say that over time, the TV industry will find that its traditional data currency of audience viewership—meaning eyeballs—is no longer the singularly important criterion for ad placement.

Already the new metric promises to wreak havoc on conventional measurement. TV programs and channels with similar eyeball profiles will increasingly attract uneven advertising revenue, as marketers rely on their proprietary formulae of engagement as a tie-breaker in directing the flow of ad dollars.

“Traditionally from a media planning standpoint, everyone started from an advertising reach and frequency as the foundation, and measured them in the exact same way,” says Greg Whiteman, manager of market research for the U.S. Postal Service, which spends an estimated $150 million in advertising annually. “Now what we're doing is measuring changes that advertising makes in consumer perceptions, changes in their perception of our brand and changes in consumer behavior.”

Though definitions often vary, engagement quantifies the degree to which the creative content and media context of any marketing communications results in meaningful communications regarding the brand. A related metric is return-on-investment (ROI), which measures a sales payoff linked to specific marketing activity.

Consumer marketing research executives say that the engagement metric is more popular with advertisers than ROI, which is good news for TV media. That's because TV's sight and sound atmospherics are ideal for creating a favorable aura for consumer engagement. U.S. advertisers are already beginning to deploy these yardsticks, and the impact is beginning to be felt in the marketplace.

Engagement is most often measured by analyzing viewers' answers to various questions about a particular attitude toward media. Consumers rank their attitude toward brands, for example, and determine if they will recommend products to friends. Other tests measure involuntary responses such as brain-wave activity and eye tracking during media exposure.

Sandy Eubank, director of business intelligence for media communications agency OMD, which places an estimated $10.7 billion in ad spending annually, notes that because media ad sellers aren't fully aware of engagement metrics, “The market is not charging a premium, although you don't get a guarantee [that ad buyers get during upfronts]. That's why we find it powerful in the marketplace.”

OMD, which is part of the Omnicom Media Group, employs its own proprietary engagement metric. It is expressed as an index number from 1-100, and is based on analysis of data from Nielsen and Experian Consumer Research, which formerly was known as Simmons National Consumer Study.

“There often is no strong correlation between the [standard] rating of a show and its engagement,” says Huw Griffiths, U.S. director of metrics and brand science at OMD. He says that the biggest impact for engagement comes in TV shows in the middle of the ratings pack.

The introduction of metrics such as engagement and ROI in planning ad buys adds complexity and cost for buyers. For that reason, engagement metrics are employed mainly in big national buys at present, and marketers are developing refinements in conjunction with the Advertising Research Foundation (ARF), including an Engagement Council that is chaired by the Postal Service's Whiteman. Various ARF engagement study groups have included executives from Ford, MasterFoods, Microsoft, Monster, Procter & Gamble, Revlon and Time Warner, and also from all the top advertising agencies.

Their interest is due to several factors. With audience fragmentation, advertisers say they need more objective data on how to guide decisions to allocate spending across multiple media in cross-platform buys, such as TV, Internet and print magazines in a single campaign. Researchers say engagement can be used to evaluate multiple media with the same metric.

Moreover, ad agencies purposefully draw attention to their “special brew,” their own unique media buying techniques to land and hold clients. Likewise, as tenures of chief marketing officers (CMOs) at advertisers have become increasingly short, new arrivals put their stamp on company strategy by introducing state-of-the-art metrics. CMOs impress top corporate brass with presentations that apply a numeric precision to consumer initiatives that are akin to financial accounting.

Alan Wurtzel, NBC Universal president of research and media development, predicts that “everyone will have their own 'secret sauce.' We understand there's lot of customization.”

Media companies are presenting advertisers with their own versions of engagement and cross-platform advertising metrics to bag ad spend, often using catchy names to brand their media platforms. NBC will soon present advertisers with its Total Audience Measure, which it dubs TAMi. Marketing executives have heard MTV talk up what it calls “return on innovation,” which is a twist on the industry acronym using the word “investment.” Discovery, The Weather Channel, Turner Broadcasting and CBS are other TV networks active in early engagement research initiatives.

Audience data giant Nielsen—which agreed to buy IAG Research for $225 million in early April for its consumer engagement expertise in TV shows, commercials and product placements—intends to be the dominant player for such data. Nielsen Co. Executive VP Susan Whiting says that it's still early, and data needs to be better integrated so the media industry can easily connect various strands. “We're not there yet,” Whiting observes. “I see over the next two years a step to presenting information in combination. For example, how exposure on TV translates to activity on the Internet, or how information online translates to sales in stores. Right now, we are presenting the pieces individually. As a first step, we are testing the TV-Internet combination now.”

Whiting says Nielsen's ultimate engagement goal is to put media audience count, brand buzz, ad recall and sales in a single seamless data package.

Engagement measurement is now mostly confined to national media, but some advertisers get excited when the metric surfaces occasionally at the TV station level. Hearst-Argyle Television VP/Sales Kathleen Keefe received a flood of media buyer calls when her station group released a custom research report in March showing high viewer engagement with local newscasts.

“Senior agency executives on the planning side and some advertisers suddenly are sending me e-mails asking when they could meet with me,” she recalls. “I was a bit surprised by the response because it's not always easy to reach some of these executives.” Researcher Frank N. Magid Associates conducted Hearst-Argyle's Local Television Advertising Effectiveness Study, which showed that local TV station news surpassed cable and broadcast network news in terms of viewer trust, engagement and impression of advertisers.

Still, some may argue that advertiser interest in engagement is a passing fad because it is an amorphous metric that is not easily defined. Furthermore, critics say its mash-up of advertising, programming and consumer response is overly complex and perhaps too subjective; viewing data is gleaned from digital TV set-top boxes, portable devices to measure consumer media consumption out-of-home, and the TV viewing habits of DVR users.

Others disagree that engagement will replace the current ratings metric, largely because different classes of advertisers have different goals. For example, retailing and consumer prices for automobiles, fast food, financial services, toothpaste, shoes and pharmaceuticals have little in common.

Says John Lowell, VP and director of captivation at Starcom USA, which buys an estimated $7.8 billion in U.S. advertising annually, “There is no one size that fits all in engagement metrics.”

There's even disagreement whether the ad research industry will eventually develop a standard data set for calculating engagement that is accepted by the entire media industry. A white paper in 2006 said the engagement measure is “a critical advertising paradigm to replace GRPs [gross rating points] in the 21st century.”

“The thing that everyone agrees on is that we need audience metrics that go beyond simply counting eyeballs or readers, although there's not much everyone agrees about after that,” says Robert Barocci, president of the ARF. “It's clear that the days of buying TV just on GRPs is already past for advertisers like Toyota and others. So the big question is, what else do you measure? That thing is what we call engagement.”