Campaign title: Country reputational perception study

Client/Entering Company: CARMA

Company Name: Anonymous Government entity

Summary

One-off study to analyse the reputational perception of a Middle Eastern country in opinion-forming media and user- generated coverage from seven regional markets and key international media. Perceptions and awareness analysed for reputational themes and initiatives in social and online media, containing deep insight into differences in opinion regarding the country’s policy and initiatives between the markets. The report was delivered in Arabic and English with a 13-day turnaround.

Objective/Brief

The client, a government department of a Middle Eastern country, wanted to discover how the country’s reputation fared in opinion-forming online sites and social media across seven Arabic-speaking target markets in the region, plus key international news sites, in relation to six key themes of reputational interest:

  1. Culture
  2. Education
  3. Humanitarian
  4. Political/Security/Safety
  5. Renewable Energy/Scientific Endeavour
  6. Tolerance

The client had conducted an awareness campaign highlighting numerous government initiatives and events intended to improve the lives of its citizens, resident expatriates and neighbouring countries. These were widely publicised through a number of public relations activities including press releases and conferences, interviews, advertising (press, online and OOH) and social media campaigns.
The question for this client was not how effective a particular communications campaign was in the immediate aftermath of that campaign with its Tier 1 target (home) audience, but rather how much of an impact these positive activities have on the country’s reputation in other markets, against the backdrop of other events.

As a general rule, the country’s own traditional media, and to some extent users on social media, tend to be highly supportive of government initiatives. The more challenging communications goal for the client was to have resonance for their positive messages outside of their home market which would translate into favourable public opinion for the country.

The main objectives of the study were to identify and compare within and across target (non-home) markets:

  • awareness of/interest in the key themes
  • attitudes towards the country
  • key reputation drivers of the country by market, media outlet, topic and sentiment
  • top sites, active users and influencers

The client needed the report – in Arabic and English – to be delivered within 13 working days.

Strategy

Establishing the media universe
The study was to include articles and posts discussing the themes, in Arabic, across seven markets in the GCC, Levant and North Africa, plus English language articles in key international online news sites. Due to the limited time and budget, coupled with the necessity of comparing fairly across
markets, a systematic sampling methodology had to be devised.

With the requirement to measure resonance of themes, and their effect on reputation over an extended period of time, albeit in a one-off snapshot, it was decided to review coverage published/ posted in the past 12 months, but with a cap of 15,000 posts and 1,000 online articles undergoing full analysis.

To ensure that only opinion-forming media were included in the study, the top 15 Arabic news websites were selected from the target markets, plus 20 of the top international English-language news sites as listed on 4imn.com. The social media study included conversations relating to the reputational themes on Twitter and Facebook, restricted to users with 1k or more followers and only posts with engagement.

Since the study required an assessment of attitudes towards the country’s policy and activities, the selection criteria was set to include only content expressing strong opinions.

Planning the content analysis
The time constraints necessitated the deployment of a large internal team, and the importance of the client allowed for no mistakes.

To that end, the code frame, metrics, and structure of the analysis report had to be mapped out even before the start of the data collection phase, in order to ensure that all facets of the project were documented and understood by the whole team.

The high-level reputational themes identified by the client needed to be broken down further into specific initiatives, events and topics for three reasons:

  • strengthening the accuracy of item selection
  • augmenting the granularity of the content analysis by assessing sentiment for each
  • enriching the insight provided in the narrative

It was crucial to storyboard the content of the report right at the start of the project, to ensure that the results were presented in such a way so as to guide the client through the contents, from top level overview to initiative-level detail, comparing social and traditional, market by market.

A multi-stage process was planned to ensure complete transparency regarding the project amongst all stakeholders, both internal and external:

  • Internal preparation – desk research was undertaken to discover the underlying proof points for each of the reputational themes; analysis and design teams worked on the report template, ensuring adequate space for both clear graphics and a large amount of narrative, ensuring clarity and visual appeal in both reporting languages
  • External approval – a workshop was held with the client to discuss, and achieve buy-in, on all aspects of the project
  • Internal briefing – on top of a written briefing document in English and Arabic, a critical path map was distributed across the multiple teams, and verbal briefings given to ensure cross-team understanding

Execution/Implementation

Content analysis
Content analysis of each item was a multi-level process:

  • For online media
    • Our native language research coding team read each captured article to establish whether it contained a clear opinion
    • Articles which did were scrutinised for mentions of any events, initiatives and/or topics listed under the broad reputational themes
    • These were then assessed for sentiment – Positive (support/praise for the specific element) or Negative (criticism/dismissal )
    • Purely neutral and factual comments were not captured as they added no value to this particular study, with its focus on public opinion
  • Our social media team
    • First used our unique Natural Language Processing (NLP) technologies as well as data mining technologies to isolate articles potentially containing opinions on the themes
    • Then adopted the same methodology as for traditional media of identifying and analysing sub-themes for sentiment

Charting and segmentation
The analysis teams then took the raw data and created volume and sentiment charts per initiative/ per theme by market.
The report was segmented by market, then Social and Traditional, each further segmented by theme, then specific initiatives, with a top line overview per market showing:

  • percentage split between Facebook, Twitter and Online
  • Pos/Neg tonality split for Social vs Online overall
  • By theme – Pos/Neg tonality split for Social vs Online

Qualitative insights
The real value was then added, as native-speaking analysts with in-depth knowledge of the country, the nature of the media of each market under analysis, and a clear understanding of what questions the client wanted answering, performed a deep dive on the data.

The subsequent narrative accompanying every chart provided the client with detailed insight into not only how each market’s media perceived the country’s efforts to “improve the lives of its citizens, resident expats, and neighbouring countries”, but also what the conversational drivers and potential future touch points were.

Insight was given regarding the awareness and perception of each single initiative in each market, whether ignored, praised or scorned, with example quotes from individual users, journalists and third parties used to illustrate key points. A second layer of insight looked at each theme in turn as a whole (including, but not limited to, named initiatives).

This widened the difference between markets to a certain extent, as some were not aware of specific initiatives, but were vocal on the broader themes. The analysts were able to detect and describe the underlying motifs driving sentiment for each theme because of the granularity of the code frame.

Client stakeholders
For our direct client stakeholders, this was the level of detail and insight they required.
Other stakeholders in the client’s organisation required a more top line overview, so we provided a one-slide overview awareness of Key Themes of Reputational Interest, broken out by social and online, by market. Others wanted to understand who were the top influencers and most active users/online news sites, which we were able to provide along with thumbnails of representative posts/articles.

For distribution to members at the very highest level of the organisation, and who did not require the details, our Middle East political analyst reviewed the data, in parallel with the market analysis teams working on the detail, and provided a succinct, yet insightful Executive Summary, covering perceptions of the country at that given moment.

This summary included recommendations on how to shape communications efforts in the future in terms of ‘friend’ and ‘foe’ markets, which favourable themes should continue to be pushed, and which unfavourable themes to tackle sensitively, highlighting differences between social and traditional media.

Effectiveness of Assignment

The nature of the client dictated that the analysis and insight of the issues had to be conducted and presented with the utmost sensitivity. CARMA’s regional awareness, coupled with our strict adherence to transparency, consistency and validity under AMEC’s global principles and guidelines, made us the ideal partner for the project. This, together with our on-the-ground native language and human analytical capabilities, meant that we were able to deliver a data and insight-rich, 168-slide report in two languages simultaneously – with market-specific, culturally-sensitive analysis of public opinion-forming media perception – within the 13-day deadline.

Highlights

A country’s reputation based on opinion-forming online sites and social media across seven Arabic-speaking target markets within the region, plus key international news sites, in relation to six key themes of reputational interest.

CARMA – Supporting Materials