Planning
Build evaluation into your engagement plans from the beginning.
Evaluation should be tailored to your project – depending on your particular aims, objectives and what you feel is important to measure.
What to measure
Think about what to measure first – measure what is important, rather than what is easy to measure.
The following may help you think about what’s important to your project.
Aims
- Have you found out what you wanted to know?
- Were your objectives for working with people and communities clear enough?
- Were you clear with participants what could and couldn’t be influenced?
Reach and promotion
- Did you work with people with lived experience?
- Did you reach people from communities experiencing health inequalities?
- How did you promote the opportunities for people to get involved? For example, a newsletter or via community groups?
- If you used a particular way to promote the opportunity, such as a newsletter or social media post, are you able to measure how many people were recruited or spoken to as a result?
Audience
- Did you reach the people you were aiming to?
- Did you collect demographic monitoring data for the participants?
- Were there any groups you needed to reach, but did not hear from?
- Were there any groups that were initially missed in your engagement planning?
- What would help identify and reach the right people if you did this again?
Methods
- Were the engagement methods such as promoting through your networks or attending a community group appropriate at reaching your target groups?
- Was your approach flexible enough to suit those you were engaging with?
- Would you choose different engagement methods to reach the same groups if you did this again?
Supporting participation
- How did you ensure people were listened to?
- Was information made available in suitable formats such as an easy read version?
- Was support provided to enable participants to engage, for example interpreters?
Timing
- Did you give people enough time to participate and contribute to your activities? For example, did people have enough time to respond to your advert? Did they have enough time to contribute their views?
- Did you keep to your original timescale?
Cost
- Did you budget for your engagement activity?
- Did you use NHSE guidance for remuneration?
- What other costs were there?
- Was the process proportionate / value for money?
Impact and feedback
- How has feedback/insight been considered by decision-makers?
- Will there be changes to policy or services following engagement?
- Have you provided feedback in a way that is meaningful to the community? For example, have you let them know how their feedback will be used?
- Are you involving people in the on-going monitoring of any changes made as a result of the project/programme such as task and finish groups?
Learning
- What worked well?
- What would you do differently in the future?
- How will you share this learning with colleagues?
How to measure
Once you’ve decided what you want to measure, it’s important to think about how you might go about measuring those things. Here are some examples that might help you:
Qualitative methods
- Desktop research – for example, reviewing all documentation produced by the programme
- Project team reflection sessions
- Verbal or written feedback from participants on the difference made by the exercise
- Decision makers having a discussion, that is minuted, about the feedback at key meetings
- Quotes from participants – anonoymising responses.
Quantitative methods
- Setting targets for the numbers of responses you want to receive/ events you want to hold (both overall and from specific communities)
- Surveys before and after engagement work to understand if attitudes have changed towards the project
- Demographic analysis of participants
- Social media or website or distribution statistics around promotional activities
More about how to develop good questions
Evaluation
Once you are clear about what you are evaluating and how, you can begin your evaluation.
Sometimes it can be helpful to start this process while you are still actively engaging. This can help you make adjustments to your approach – for example, through reviewing demographic data, you might notice you are not reaching people from specific communities and can try different techniques to encourage participation.
Evaluation should lead to action – use your evaluation to enhance and improve how you work and share your findings with partners, so they also get the benefit. Consider how you will share what you have learned more widely and how it will influence future stages of your project.
Involve local people
Involve local people with lived experience and the communities you have worked with in the project to plan how you will evaluate your engagement work – and ask those who worked with you to feedback once the project is complete.
Evaluation is a team effort.