This article aims to answer the question “why measure and evaluate communication campaigns?” and then discusses key approaches of measuring communication campaigns with examples. It may be irrelevant to the theme of this blog, however, it is one of my precious academic thoughts. Therefor, I thought I should share with you guys. Hope it would help, even just a bit.
Why measure and evaluate communications campaigns?
Evaluation focuses effort demonstrates effectiveness, efficiency, encourages good management and accountability. (Tench and Yeomans, 2006). It helps to keep the campaign in the right track and assesses the impact of communication campaigns. Usually when the campaign is complete it will need to be evaluated to discover whether it has met its overall objectives. However, it does not just happen at the end of the campaign. During the process, communication campaigns also need to be measured and evaluated. Evaluation is an ongoing process and should be considered at the objective setting stage: All the planning approaches emphasise the importance of ongoing monitoring (Tench and Yeomans, 2006).
Bissland’s (1990:25) definition illustrates that “evaluation is the systematic assessment of a program and its results. It is a means for practitioners to offer accountability to clients – and to themselves.” (Tench and Yeomans, 2006). It is perfectly reasonable that the clients want to evaluate their investment in the communications campaigns. Actually, not only clients care about evaluating the campaigns. “Audiences include numerous parties that have an interest in the evaluation – the organisation, the public relations practitioners involved, target publics and the evaluators themselves.” (Tench and Yeomans, 2006). The organisations need to know how much benefit they get from the communications campaigns; the campaign practitioners need to know the results of their work; how target publics of the campaigns have influenced and been influenced by the campaigns; evaluators, such as media monitoring company and certain government departments may also be interested in the impact that the campaigns gave to the publics.
The PRCA, PR Leaders’ Panel has found that 84% of PR Agency Heads believe that evaluation is very important to the credibility of PR. (Prca.org.uk, 2009) Evaluation can be used as creditably for the client to assure the plans made will be efficient and worth while for them. Leggetter. B (2009), Executive Director of AMEC, and Group Chairman of Bite Communications said “In a recession, evaluation is already playing a more important part in PR campaigns as clients demand proof that the program is working.” Accurate evaluation and measurement of certain strategies and tactics of the campaign can help audiences make improvement, in order to achieve their goals.
Evaluation is a concept among public relations practitioners. Even some countries have set up their own version of evaluation guide or toolkits; there is still no worldwide agreement on a single standard. However, there are some key approaches that will make communications campaigns evaluation easier.
- Set Objectives
A campaign needs to be evaluated to discover whether it has met its overall objectives. Sometimes the objective can be easy to measure, such as achieving certain sales or views. In this case when it has achieved the certain number, then clearly it was successful. Sometimes the situation can be more complicated. For example, if the plan is to increase a brand reputation, that will take a long time, and require different strategies and plenty of effort. Therefore, a SMART objective is essential.
SMART stands for specific, measurable, achievable, realistic and time bound (Tench and Yeomans, 2006). A clear and measurable objective is relatively easy to judge whether it has been achieved, or how much of it has been achieved. For example,
- Increase editorial coverage of services by 20 per cent over 18 months.
- Double job applications from local school leavers within 2 years.
- Inform ten targeted investors of reasons for management buyout before the AGM and obtain their support.
- The 3 ‘O’s
Three Os are outputs, outcomes, and outgrowths (Green, 2015). Outputs can be production of material; media coverage, events held or web traffic etc. Anything that has been produced for the purpose of achieving the campaign objectives can be output. Outcomes are the results of outputs. They can be awareness, opinions, attitudes or behavior of audiences. Individuals may change their behavior, or government may change the policy. However, they might be opposite from the objectives of the campaign or not helpful to the objectives. Therefore, the most important part is the outgrowths. This is the part that shows the positive impact of the campaign, and how much benefits the campaign brings.
The definition of insight is “The ability to discern the true nature of a situation”. Insight can help practitioners to do better research and know well about the campaign. As many tools available, insight data can be found easily. For example, Google Analytics, a website analysis tool that help the user to know more about audience, it shows how many views, when the views happen, where are those from and some other information. Google Trends, provide information about every hot topic, trend, and a better picture about what is going on and what are people talking about. YouGov using online data to produce a brand cameos to provide more knowledge about the brand users. Software like these can help campaign practitioners get to know about their audience in detail, and understand the situation based on the fact in real time.
However, tools can only provide answers to the question that are asked by people. Keeping every question specific, focus on solving one problem. Tools cannot answer a general question such as “how to improve my reputation?” but they can provide data to answer the questions such as “what are we known for?”, “When can we reach more audience?” and ”What topic attracts more people?” etc.
According to Green (2015), the traditional marketing model was to invest 80% of the team’s effort before launching, with 20% post-launch to optimise the campaign. But in an effort to be more nimble, marketers such as Adidas have turned the traditional model on its head. This allows Adidas produce great content in a very short time.
Adidas spends 20% of their time upfront to inform big-picture creative and media decisions, reserving 80% to test and iterate in the moment.
The 20:80 model allowed Adidas to take advantage of the world’s best focus group: its real audience in real time. It didn’t invest all of its resources in one World Cup ad produced months in advance. Instead, it took in audience feedback after posting each video and pivoted content accordingly. Its process of testing and iterating from one episode to the next helped Adidas increase its views 7X from the first episode of its video “The Dugout” to the last
- Three types of problem
Measurements are not easy. The best way of assessing the value of PR should be through a combination of measures: volume of positive coverage, impact on sales leads, impact on website hits, impact on the way people search for your company on Google. (Aspectuspr.com, 2015) And some regular researches on attitudes, understanding and behavior changes within target audience are also necessary. So there could be some problems when measuring the campaigns.
There are three types of problems that could be met when measuring campaigns. The first type of problem is the easiest of all to overcome. It is visible and solvable. Solving it is like taking an underground to another place within zone 1 in London. For example, measure sales after a marketing campaign, or measure views after a media campaign.
The second type of problem is harder than the first one. It is visible, but looks really complicated. Measuring this type of problem is like going to another place within zone 6 in London by underground. It may not find a direct way to measure; several transfers are needed. But it is solvable; everything that needs to solve the problem is extra work. Such as measuring audiences’ awareness changes after the campaign, or measuring if the reputation increased.
The last type of problem is not so easy to find. It may be invisible, even unsolvable. Measuring it is like going to another unknown place by underground in zone 10, or maybe not in zone 10. It is really complex.
- Create frameworks
“Frameworks are like heroes. Whenever there is a doubt, just go for them; do as what they do.” (Green, 2015). There are many PR agencies all over the world who have set up their own measurement models. Such as ‘7 Barcelona Principles’, ‘AMEC Valid Metrics model’, ‘Business measurement balanced scorecard’, ‘CIPR Toolkit’ and so on. It is important to choose the framework, which suits the campaign the most or create a specific framework for the certain campaign.
Old school evaluation uses advertising value equivalent (AVE) to measure a communication campaign. However, one of the ‘7 Barcelona Principles’ is that AVE cannot evaluate public relations effectively. AVE cannot measure positive or negative. Reputation takes years to build but can be destroyed in seconds. The UK jewellery company Ratners once lost £500 million share price because of a negative report about their founder. And ultimately it led to the company going bust (Bagnall, 2015). AVE cannot measure all of this.
Advertising works in a different way in online media than it does in the printed press. Accordingly attributing any kind of meaningful AVE to an online piece of mainstream content is impossible (Bagnall, 2015). For example, it’s not possible to assign a value to someone’s tweets, blog content, Facebook and Linkedin updates.
- Iteration measurement
Iteration is a single development cycle, it may also be defined as the elapsed time between iteration planning sessions. It means a process of constant checking, evaluation and re-evaluation (Green, 2015). It is a new way of thinking and doing PR. In communications campaigns, especially digital communications, throughout its duration, practitioners will be regularly checking to see if the programme is on track (Tench and Yeomans, 2006).
In Obama campaign fund-raising Splash page had 2 elements:
- Turquoise photo of Obama
- Bright red ‘Sign Up’ button
Several buttons were tested, such as ‘Learn More’, ‘Join Us Now’, and ‘Sign Up Now’. Among these, ‘Learn More’ increased 18.6% sign-ups. For pictures and videos, using video alone made it 30.3% worse, using Obama family photo then made it increasing 13.1%. The most amazing finding is that if using video and photo together, then it increase 40% sign-ups and $75m donation.
Another example, in communications campaigns, media coverage will be evaluating monthly to see if the selected media are using the material supplied (feature articles or press releases) and to judge how they are using it, then improve it. Everything is a prototype in iteration measurement. It keeps the good methods and abandons the ones, which do not work well. This gives more chance to improve the campaign, and before it reaches the end, there is always a way to make it better.
- Key Performance Indicator
A Key Performance Indicator (KPI) is a measurable value that demonstrates how effectively a company is achieving key business objectives. Organisations use KPIs at multiple levels to evaluate their success at reaching targets (Tench and Yeomans, 2006). In an age of Big Data, it is essential to establish specific KPIs for a certain campaign, and focus on the critical ones, which are more important than the others. Such as Return on Investment (ROI), Key message appearances, Product reviews, Media stratification etc.
“Programme evaluation is the systematic measurement of the outcomes of a project, programme or campaign, based on the extent to which stated objectives were achieved. As part of the strategic planning process, establishing appropriate and practical evaluation wraps up all the previous plans, ideas and recommendations.” (Smith, R.D. 2009)
PR focuses on creating an ideal image or reputation to further develop for the clients. Without evaluations, there is no proof of improvement or utilization of all tools. The key approaches of evaluations are: SMART objectives, which are an essential start for all campaigns. They give clear directions and unique goals to the campaigns. The 3 O’s measures the work of a practitioner as well as the results of their work. Insight is the best way to research and get to know the publics, especially with the help of those tools. Then there are 3 types of problem to be discovered. The frameworks help solve these problems; even some of them are too difficult to locate. Iteration measurement keeps the method of campaign updated to the best all the time. Key Performance Indicator makes sure those most important elements have been taken care of. These key approaches bring a safe routine to the campaigns, to keep them moving towards the right directions.
Aspectuspr.com, (2015). Measurement and evaluation in PR – Aspectus PR. [online] Available at: http://www.aspectuspr.com/intelligence/measurement-and-evaluation-in-pr/ [Accessed 19 Mar. 2015].
Bagnall, R. (2015). AVEs don’t measure PR. Here’s why. | | Measurement MattersMeasurement Matters. [online] Gorkana.com. Available at: http://www.gorkana.com/measurement-matters/measurement-matters/pr-measurement-media-evaluation/aves-dont-measure-pr-heres-why/ [Accessed 19 Mar. 2015].
Green, A. (2015). Campaign Analytics 1.
Green, A. (2015). Campaign Analytics 2.
Green, A. (2015). Campaign Analytics 3.
Green, A. (2015). Campaign Analytics 4.
Green, A. (2015). Campaign Analytics 5.
Gregory, A. (2010) Planning and Managing a Public Relations Campaign, London: IPR/Kogan Page
Prca.org.uk, (2009). 84% of PR Leaders’ say evaluation makes PR credible. [online] Available at: http://www.prca.org.uk/?pid=403&sid=8 [Accessed 17 Mar. 2015].
Rose, C. (2010) How to Win Campaigns, London: Routeledge
Schulzstephanie, (2014). Evaluation and It’s Importance: Especially in PR. [online] Available at: https://schulzstephanie.wordpress.com/2014/12/02/evaluation-and-its-importance-especially-in-pr/ [Accessed 19 Mar. 2015].
Smith, R. D. (2013) Strategic Planning for Public Relations, London: Routeledge
Tench, R. and Yeomans, L. (2006). Exploring public relations. Harlow, England: FT Prentice Hall.
Watson, T & Noble P (2014) Evaluating Public Relations (CIPR), London: IPR Kogan Page