Understanding why an intervention worked or failed: the H-WORK Evaluation Toolkit (HET)

by USFD team

What is the HET?

The research team at the University of Sheffield has developed the H-WORK Evaluation Toolkit (HET) to assess the implementation process of multi-level interventions. This enables us to understand why an intervention succeeded or failed and helps us to understand the key factors that influence outcomes.

How is the HET structured?

The HET include different measures to assess the process of implementing interventions in different phases: 1) Pre-intervention; 2) Monitoring intervention and participants’ feedback; 3) Integration monitoring process; 4) Post-intervention.

  • Phase 1 – Pre-intervention: Before implementing interventions, it is essential to evaluate some contextual factors that may influence intervention effectiveness, for example, employees’ readiness for change and the levels of commitment of senior and line managers.
  • Phase 2 – Monitoring intervention and participants’ feedback: It is essential to assess the participants’ experiences and levels of engagement in intervention activities. Gathering feedback on the ongoing interventions allows for changes as you go along. This can be assessed with the Intervention Appraisal Questionnaire.
  • Phase 3 – Integration monitoring process: Evaluating whether participants adopt the learnings into their daily job activities is important. If participants have kept how they feel, think, and behave at work, chances are that a given intervention was useless in positively impacting their mental health. In these cases, action plans can be implemented to allow participants to apply learnings to their daily work. This can be assessed with the Intervention Integration Questionnaire.
  • Phase 4 – Post-intervention: It is also essential to assess the extent to which the intervention has achieved its intended outcomes should be evaluated. Therefore, considering whether any changes to employee mental health and well-being were produced after interventions were implemented is crucial to understand their effectiveness.

What kind of results can the HET provide?

In summary, the HET can provide information about the context in which the interventions are implemented, participants’ appraisals of intervention activities and design, some fundamental mechanisms that explain participants are implementing what they learn into their daily work, and changes in their mental health and wellbeing after interventions were implemented.

How is the HET related to other toolkits (HAT and HIT)?

Given the participatory approach adopted by the H-WORK project, the HET is strongly related to other project toolkits. For example, before interventions are implemented, conducting a need analysis (HAT) to prioritise actions and select and implement suitable interventions according to those needs (HIT) is necessary.