How did we create the logic models?
This section explains the criteria and information that have been used by NHS Health Scotland to construct the logic models in the outcomes frameworks on this website. It also briefly outlines two other issues (health inequalities and monitoring/evaluation) that are relevant when creating and using logic models.
In developing the logic models, NHS Health Scotland has drawn on its ten formally adopted decision-making principles: doing good, not doing harm, fairness, sustainability, respect, empowerment, social responsibility, participation, openness and accountability.
We see logic models as a useful tool in developing accountability for health improvement outcomes.
What activities and outcomes are included in the logic models and why?
1) What?: What are our health priorities? What are the key determinants or risk factors where action is required to address these priorities? Evidence of association and evidence of causation are normally the key evidence types here.
2) How?: How should we intervene to tackle these health priorities and the associated risk factors? Evidence of effectiveness is the key type in this instance.
Ideally, each link in a logic model should be supported by a recognised evidence base demonstrating that an activity is likely to be effective in health improvement terms. It should ‘do good’ in a sustained way and it should be unlikely to cause harm.
In practice, there are a number of reasons why the types and sources of evidence used in the logic models vary, e.g., the diversity in types of evidence available and the wide range of issues/topics covered. Also, the logic models have evolved using different processes and methods, and over differing time periods.
The evidence base used in this work is mainly from highly processed sources such as formal guidance, e.g., NICE, SIGN. For each logic model we have defined the sources of evidence used and commented on the volume and quality of evidence available.
We don’t always have a strong evidence base for the possible actions and links to include in logic models for health improvement. Using highly processed evidence may narrow the evidence base further because this kind of evidence is often based on particular types of study of particular types of intervention i.e. controlled studies of discrete, non-complex interventions.
As such, it may exclude important parts of the evidence base relating to complex public health interventions, which are often multi-faceted, context-dependent and delivered in multiple settings. The risk is that the conclusions drawn from outcomes frameworks reflect the narrow scope of the highly processed evidence base, rather than what may work to improve health.
Moreover, evidence from evaluations of individual health improvement interventions delivered in isolation can only partially inform logic models representing complex ‘packages’ of interventions that may interact with each other. These points reinforce the place of plausible theory, as well as available evidence, in making judgements about the combinations of potential health improvement interventions or actions to include in a logic model, and about the links that can reasonably be made between actions and outcomes.
This is reflected in the explanations given for the links identified in each model. In addition to commenting on the volume of evidence from defined sources, the outcomes frameworks explain the plausible theory used in making judgements about what to include in the logic models. In this context, we have defined plausible theory as judgements that are considered reasonable on the basis of formal scientific theory, or reasoned argument or apparently trustworthy and credible sources (including recognised experts in the field) outwith the defined evidence sources.
It is important logic models incorporate key strands of National Policy/Action Plans as often these are the major drivers of existing and planned action. Therefore links between various components in the logic model have also been included if underpinned by a current Scottish Government Policy or Action Plan. “Policy” sections in the explanations for each logic model outline the relevant Policy/Action Plan where appropriate.
Our knowledge base on how to intervene effectively to reduce health inequalities is currently not extensive. However, in the outcomes frameworks we highlight if the link in question is considered to be particularly important for reducing inequalities and make a statement on how the link could/should contribute.
Monitoring and Evaluation
The logic modelling process can also help identify and prioritise key elements or links in the model that should either be monitored or evaluated. For example, if strong evidence of effectiveness already exists for a particular link then basic monitoring may be all that is required. However, if there is a lack of effectiveness evidence for a particular link (and it is consequently based mainly on plausible theory) this might be an area where further evaluation research is required. We are using logic models to inform the development of monitoring and evaluation strategies.