Evaluate and assess impact
Evaluation can help you move beyond describing what you do (delivery and outputs) towards evidencing what difference it makes (the impact or effectiveness). It can help you
- determine if and how a programme or policy works
- weigh up the success of a programme or policy against the resources used
- inform decisions about whether to continue, stop or change a programme
Performance management and evaluation are complementary but different. Evaluation and assessing impact should align with and enhance the information collected to report on performance.
- Performance management - the collection and reporting information that demonstrates delivery of our priorities, objectives and targets.
- Evaluation - when we want to know if something is being effective, what’s its value?
It is important to consider which approach is most appropriate for the questions you are trying to answer. If there is already good evidence that something works, improvement science can be used to explore whether the known effective intervention is transferable into a different setting.
If you are planning to use evaluation, it is important that you do it early and that it is proportionate to your needs and resources.
Steps to Evaluation
To help you think through the process of evaluation here are seven steps to consider.
- Understanding the intervention and the problem it is intended to solve
- Defining your evaluation questions - what is it you want to know?
- Reviewing what evidence or data are currently available or would be needed to answer this question: how practical or feasible is it likely to be to collect this data?
- Designing your evaluation
- Collecting the evidence or data
- Analysing the evidence or data
- Reporting on the findings and drawing conclusions or making recommendations.
You can find out more about step 3, reviewing evidence or data, within our guidance on using the right indicators.
Choose the approach to take
There are many approaches to evaluating a programme or policy so it can be difficult to decide which one to choose. The Better Evaluation Rainbow Framework (external website) can help you work out the best evaluation approach for your needs.
Use an outcomes approach
We promote an outcomes approach to evaluate the impact of policies and programmes on population health and wellbeing. As the names suggest, this approach is focused on outcomes and considers the changes that occur as a result of the interventions, activities and outputs, namely what difference do they make? The tools that we commonly use to support an outcomes approach to evaluating and assessing impact include
- logic models
- outcomes chains (sometimes called results chains).
These tools are collectively known as ‘outcomes frameworks’ and can be used to identify shared outcomes with relevant partners. They are also useful for planning policies or interventions. These outcomes frameworks should be informed by the best available evidence.
A logic model provides a road map for the implementation of the programme or intervention and a framework for monitoring progress and for evaluation. Logic modelling is a useful process in partnerships where there are shared outcomes.
We have worked with partners to create a series of topic-specific frameworks which help link activities with long term outcomes. You can learn more about how to use these tools to support outcomes planning and evaluation of our Outcomes Framework website.
Evaluation Support
We work with a range of partners to develop outcome focused approaches to planning, managing and evaluating actions to improve health and reduce inequalities. These include
- planning of evidence-informed public health interventions in a way that will reduce health inequalities
- using monitoring and evaluation to help us understand the impact of these actions and their contribution to National and Local outcomes
- a process of evaluability assessment that helps prioritise where best to invest in evaluation resources
- action to build evaluation and learning into improvement, funding and policy making processes.
We can provide various methods and tools to support partners in an outcomes focused approach to monitoring progress and assessing impact.
The Community Planning in Scotland website provides details of organisations and websites offering support and guidance on evaluation for Community Planning Partnerships and on evaluation and external review for Health and Social Care Partnerships (external websites).