How to measure the impact of your media campaign

Written by Brian Ligomeka, Centre for Solutions Journalism, Malawi

Tell us more about what your organisation does

In Malawi, abortion is restricted, and CSJ and other organisations are advocating for the law reform. The CSJ works with the media by producing content – articles, videos and stories – for television, radio, newspapers, magazines and online platforms. The media amplifies the messages we want to get out.

We also offer journalists specialised training in reporting SRHR and LGBTI issues so they can positively cover them. Besides, we work with community groups by civic-educating them on SRHR issues and training members of the community in advocacy. Finally, we also collaborate with the government by advocating for SRHR law reforms.

How do you measure the impact the Center has on people’s knowledge and attitudes to SRHR?

To measure the impact of the media content we are producing is having on our target audiences by among others changing attitudes, we conduct pre and post-project surveys. We identify a sample of our intended target audience and administer a questionnaire, which measures their views and knowledge of SRHR issues. Then we invite the same people to watch our SRHR television programmes or ask them to read a column we run in the newspapers, or online.

About three to six months later, depending on what timeframe we have decided on, we go back to them and get them to answer the same questions as before. In this way, we assess if our media content has improved their knowledge or changed their attitudes on SRHR issues.

Sometimes people change their attitudes. In our pre-project implementation survey, one participant, for example, indicated that abortion should remain restricted. Six months later, after watching our SRHR television programme, the same participant changed his position when answering the same questionnaire. He observed that “government should legalise access to safe abortions to save lives of women.”

What have you achieved?

Out of 60 religious and traditional leaders, we offered training on the intersection of religion and sexual and reproductive health and rights, 42 of them are supportive of abortion law reform. Some of them have written articles in the newspapers while some have expressed their support on national television. To have a religious leader who was opposing abortion law reform change his views and start supporting law reform is an achievement made through our key stakeholders’ engagement.

On the impact of our media content, we once randomly selected 20 students. After we had done an initial assessment (through a questionnaire) of their attitudes, we made available to them 32 positive articles on the Sexual and Reproductive Health and Rights. The objective of the survey was to assess if professionally written articles on abortion could improve the students’ knowledge of, and attitude towards, safe abortion. We also wanted to find out if positive media articles can lead to support for abortion law reform.

When we did the second survey, we found that 16 students had positively changed their attitude. We held a focus group discussion with them. During the discussion, some of the students who changed their positions from being anti-choice to pro-choice argued that it was important for abortion to be legalised in Malawi to reduce maternal deaths contributed by unsafe abortions.

How do you measure your impact with the media itself?

We also have methodologies that we use to check the success, strengths and shortfalls of our media training.

One of the methods we use is that of doing a desk analysis of the stories that some media practitioners have covered before and after we offer them training. Over either three month or six-month period, we classify the quality of their stories as positive, negative or incomplete. In this way, we can track their ‘coverage journey’ of abortion and other SRHR issues. Apart from assessing quality,  story tracking helps to quantify articles produced.

Most of the time, after training, they begin to publish and broadcast positive stories. To us, that is an indicator that the training had a positive impact. However, if we see that the trained journalists continue to publish negative or incomplete stories, we know that something was wrong with our training. We review the training content, presentations made, and post-training evaluation forms to identify weak areas that require improvements.

Can you tell us more about how you categorise the journalists’ stories?

We categorise the stories into three groups:

• Positive
• Negative
• Incomplete

A positive story is when it:

• Includes positive voices or those of abortion survivors or even pro-choice activists
• Is able to challenge stereotypes
• Has a variety of sources, one of them being the abortion survivors or pro-choice activists, others even being the doctors
• Is fair in its approach to the issue
• Doesn’t ridicule or moralise issues or people
• Shows the journalist has done research, has history and there is good context.

We brand a story a negative when it:

• Lacks the voices of those who have had an abortion or even those of us who are doing activism
• Perpetuates stereotypes
• Is very biased in a negative way, for example, when only one group has been approached for their comments and the other group hasn’t been contacted to give their point of view
• Is full of jargon or even language which doesn’t add any value to the story
• Includes discriminatory language, for example, calling pro-choice activists ‘baby killers’
• Lacks research, history and context.

Incomplete stories include those that:

• Quote correct data but lack human voices, again, for pro-choice activists and even abortion survivors to put the data in correct context.
• Fail to address the stereotypes
• Has only uniform sources. For example, a reporter quotes four or five religious’ leaders, all of whom condemn abortion but doesn’t quote any gynaecologists or even advocates for provision of abortion
• Fail to address key issues and quote pro-choice activists out of context
• Publish stories on SRHR and abortion where all sources are anonymous.

What have you learned from this experience?

We’ve learnt about the impact of our work and how to adjust it to achieve maximum impact.

We have, for example, learned that to reach your target audience, you need to use the media product that they typically access. For instance, to reach those who can change the law, the cabinet ministers and MPs, it is better to use newspapers and national television. Most politicians read newspapers in the morning and watch television programmes in the evening. The youth gets most information from peers and social media.

The best effective way to disseminate to rural people in Malawi, the radio is the most effective tool. It is crucial to package the media content in local languages. Most of the content we produced was initially in English. The feedback we received was that to reach rural communities; we needed to deliver content in local languages as well. We adjusted accordingly.

During focus group discussions, we learnt that in a conservative society like Malawi when dealing with the issue of abortion, we should use a phased approach. We looked at SRHR issues as a continuum with some key issues being teenage and unplanned pregnancies, contraceptives scarcity and failure, unsafe and safe abortions, post-abortion care and the need for law reform on abortion. The approach enabled many stakeholders we engaged, including traditional and religious leaders, to understand the linkages between the different SRHR issues.

What are your tips for someone facing the same or similar issues?

1. When conducting advocacy through the media identify the target audience you want to reach.
2. Assess the knowledge or the attitude of the target audience through a simple survey, for example, a questionnaire administered to a sample chosen from the target audience.
3. Choose the media channel such as television, radio, newspaper or social media that is best for delivering the key messages to your target audience.
4. After a period of advocacy, assess the knowledge or attitude levels of your target audience to see if the advocacy intervention has an impact.
5. Depending on the feedback received, review or upscale the intervention. If the intervention is failing to make any difference, consider discontinuing it and explore other strategies instead.

Anything else?

We also ask audiences generally to give us feedback on the media content we produce. That feedback, whether negative, neutral, or positive, assist us to measure the success of the projects that we are conducting with the media to influence attitudes. It also helps us to increase our knowledge of audiences as we advocate for law reform.

Extra information

Do you have examples of the Before and After questionnaires you use?

Yes, here they are:

Example of a completed Before – Media advocacy questionnaire

Example of a completed After – Media advocacy questionnaire

Do you have any stories that would help or inspire another organisation or group facing the same problems?

Here are a couple of examples of positive stories on SRHR/abortion in the media:

Unsafe abortions haunt women in Malawi

The author of the above story is among the journalists who attended our training on how to report on abortion issues held in Malawi capital Lilongwe in 2018.

What about negative stories?

Here is an example:

There-is-no-justification-for-abortion-catholic-church

And incomplete stories?

Here is an example:

woman-arrested-for-abortion

One of the questions being: Is the story about abortion or baby dumping?

Did you use any external resources to help you solve this issue that you would recommend to other organisations?

For anyone involved in abortion media advocacy, I recommend the book titled “How to Talk About Abortion: A Rights Messaging Guide”. It can be accessed on https://www.ippf.org/sites/default/files/2018-08/ippf_abortion_messaging_guide_web_0.pdf

Another resource is Public Health Media Advocacy Action Guide: Elements Of A Media Advocacy Campaign. It can be accessed on: https://preventepidemics.org/wpcontent/uploads/2019/06/Elements_of_a_media_advocacy_campaign.pdf      

Brian Ligomeka, Centre for Solutions Journalism, Malawi

Brian Ligomeka is a communications expert and sexual and reproductive health and rights activist currently serving as a Programmes Advisor for the Centre for Solutions Journalism (CSJ).

Brian has led in the implementation of projects advocating for decriminalization of abortion and the promotion of sexual minority rights in Malawi. Brian has addressed hostile audiences on several occasions when discussing ‘taboo’ issues.

Centre for Solutions Journalism website