Category: Measurement innovation
Client/Entering Company: Golin
Campaign title: Understanding communications as a predictor of business change and operationalizing the Barcelona Principles
Company Name: Golin

Objective/Brief

As a profession, public relations has notoriously fallen short when it comes to measuring the impact of what we do. It is a shortcoming that AMEC members know all too well and that AMEC and the Barcelona Principles exist to correct. There are all sorts of reasons why this has been the case, but if public relations is going to claim its rightful seat at the table, we need to figure this out.

The recently quoted comments of one PR professional may represent the fundamental issue for all of us: “The problem is that the profession, by and large, has not adopted a single definition for what we actually do . . . . This has created inconsistency in how we measure results.”

At Golin, we’ve taken some steps to solve the problem. And, as with all problems that have been tough to solve, we went back to some basic first principles and asked some basic questions.
Like “What do clients pay us to do?” The real answer isn’t “get media” or “do events” or even “build award winning programs.”

It’s “create change” … specifically it’s to change the attitudes and behavior of the people our clients care most about. That is the definition of our fundamental mission that works for all clients — whether they are a large consumer brand, or the advocates of a social cause or a company deep in the B2B supply chain.

Clients hire us to create change.

In order to establish that we are doing what we’ve been hired to do, we have to demonstrate the change — or “delta” — between people exposed to our clients’ communications and those who are not exposed.

The result of this return to first principles is a new suite of fast, affordable primary research products from Golin — called the DeltaSet — that measure communications exposure as a predictor of attitude and behavioral change.

Strategy

To build the DeltaSet, we mined decades of engagement and experience with Golin clients to understand the gaps that have existed between the types of measurement clients have asked for and the types agencies (including ours) have historically delivered.

We identified two principal reasons for this gap:

• A lack of standardization in overall methodology, leading to a “reinventing the wheel” approach to every new client request;
• A bias toward allowing outputs measurement (media hits, impressions, social media touchpoints etc.) to stand without proof as a proxy for changes in stakeholder education, attitudes or behavior.

We knew we had to go deeper.

Our insight was a simple one: If our goal is change, then ideas and messages are our products. And, like all efforts to understand the impact of products on the lives of people, to know the impact of what you’re doing, you have to understand the difference between people exposed to those products and people who have not been exposed.

It’s a question of understanding the relationship between stimulus and response.

For most companies, brands and programs, direct observation can be difficult, time consuming and expensive. But, in the era of online research and surveys, asking can be increasingly fast, affordable and reliable.

The DeltaSet is about asking people two critical things: how do you recall having been exposed to communications from our client and how does our client’s business, products and category impact your attitudes and behavior?

Execution/Implementation

The answers to these two critical questions, repeated over hundreds or thousands of respondents who are our clients’ most important audiences, permit a statistical analysis of how patterns of exposure (from question 1) correlate to patterns of attitude and behavior change (from question 2).
Our strategy for achieving this has involved:

• Creating an approach to survey design and analysis that simplifies the collection of the business endpoints that matter;
• Establishing innovative relationships with a range of online survey panel providers that give us the best chance of matching clients with the respondents who are most relevant to them;
• Building on the specialization afforded by our agency’s g4 agency model — specifically, the development of our Golin Strategist Community of analysts and planners — to train and create a critical mass of specialists with the ability to conduct DeltaSet research.

It is the fundamental purpose of the Golin Strategist Community to generate the actionable insights that support planning, creative and media strategy. To deliver on that mission, we have organized the DeltaSet into four standardized research products designed to provide the specific outputs that are most useful to providing insights that support communications strategy:

• A communications mix model – DeltaMix – explores how audiences exposed to various channels (social media, mainstream newspapers, TV, owned media, etc.) have changed their behavior vs. those that have not been exposed;
• A message and program and concept testing framework – DeltaIf – estimates the likelihood that an audience will change its attitudes or behavior if it knows or learns important new information about a brand, product or organization;
• A positioning “white space” identification tool – DeltaSpace – identifies an audience’s unmet needs to determine the brand or reputation equities that will resonate best;
• A return on investment modeling approach – DeltaNet – provides a window into communications ROI by quantifying and measuring the amount of change (reputation increase, purchase intent, etc.) per dollar of investment in the communications being evaluated.
• A message and program and concept testing framework – DeltaIf – estimates the likelihood that an audience will change its attitudes or behavior if it knows or learns important new information about a brand, product or organization;
• A positioning “white space” identification tool – DeltaSpace – identifies an audience’s unmet needs to determine the brand or reputation equities that will resonate best;
• A return on investment modeling approach – DeltaNet – provides a window into communications ROI by quantifying and measuring the amount of change (reputation increase, purchase intent, etc.) per dollar of investment in the communications being evaluated.

Conducting a DeltaSet inquiry involves one or more of the following steps:

• Asking survey respondents to quantify the business endpoints our clients identify are important;
• Asking about and segmenting survey respondents on the basis of their exposure to communications;
• Conducting statistical analysis of how patterns of exposure to stimulus correlate to patterns of attitudes;
• Introducing new information, program concepts or creative to respondents within a survey to understand the potential power of this information to change attitudes or behavior;
• Asking respondents to evaluate the brand or reputational attributes that are (a) the most/least important to them and (b) the most/least experienced by them in a category;
• Estimating the extent of recalled exposure (to a campaign, say) and quantifying the amount of attitude or behavior change among people exposed vs. not exposed.

Conclusions

Historically, communications measurement has focused on the question, “How much stimulus did we create?” or, more simply, an exercise in measuring what we do. By itself, the answer to this question provides no concrete evidence that the stimulus we create will, or can, correlate to the kinds of response we seek, even if it reaches the right audiences.

Among the many objectives we have laid out for the DeltaSet, none has been more important than the need to establish a framework for operationalizing and commercializing several of the Barcelona Principles, namely:

• Principle 1: A platform for goal-setting and re-connecting measurement with communications strategy adaptation and improvement;
• Principle 2: Measuring the effect of communications on actual outcomes, rather than simply counting outputs; understanding the correlation between communications exposure; and stakeholder attitude/behavior change;
• Principle 3: The reconnection of this attitude/behavior change metrics to the business results that matter to clients;
• Principle 6: Establishing a platform (through surveys of reported exposure and engagement) for understanding the differences between traditional and social media exposure on stakeholder attitudes/behavior;
• Principle 7: Establishing a transparent, reproducible framework for repeating DeltaSet analysis over time, and across multiple clients, industries, stakeholder groups and issue categories.

To date, Golin has successfully applied the DeltaSet model to communications assignments of all kinds – from major retailers seeking to understand how media exposure correlates to monthly spend, to consumer brands building public education campaigns, to financial services marketing, corporate reputation building programs, issues management and many others.

We are confident that DeltaSet is an important step forward in the journey toward understanding and reporting on the business impact of public relations.

Contacts: Greg Sendi
Email: gsendi@golin.com
Telephone: 312 729 4275