Workplace Health and Safety Strategy:
Outcome Monitoring Framework - Document for Consultation

Appendix A – Creating the Outcome Monitoring Framework

Outcome Hierarchy

The Strategy contains a "Framework for Action" which outlines a simplified intervention logic, showing how actions are intended to lead to the achievement of objectives and outcomes that support the Strategy vision. As an adjunct to the framework for action, the Workplace Health and Safety policy team also developed the 'Roadmap'. The Roadmap model shows that achievement of some of the objectives/outcomes in the Strategy is contingent on the achievement of others. That is, developing safe workplaces consists of a series of phases - the building blocks have to be in place, before performance improves and better outcomes are achieved.

The Framework for Action and the Roadmap were used as the basis for developing an outcome hierarchy. The Framework for Action identifies three key outcome areas, and these are used as a way of organising the hierarchy vertically. The Roadmap indicates that some outcomes are dependent on others, and this concept is used to organise the hierarchy horizontally. For example, building blocks (immediate outcomes) are at the base of the hierarchy.

Commonly, the term 'health' is used in relation to workplace illness and 'safety' in relation to workplace injury. However, in this hierarchy, the concepts of 'health' and 'safety' are conceptualised in a different way, to align with the Strategy vision statement - "Healthy people in safe and productive workplaces". That is, 'health' is considered to be an attribute of the person (being free from illness or injury caused in or by their workplace). On the other hand, 'safety' is considered to be an attribute of the workplace (where systems and practices are in place to eliminate or minimise injury or illness to workers).

Identifying Indicators

For each outcome in the hierarchy, further work was done to identify:

the Strategy activities that should contribute to the outcome (what needs to be done to achieve this outcome?)
a theory of change (why or how do we expect the activities to contribute to the outcomes)
one or more indicators (what would we see if this outcome was achieved?)
one or more measures (how can we monitor changes in this indicator?).

The process for identifying indicators and measures involved developing indicator statements based upon the outcomes and contributing activities, by answering the question "what would we expect to see if this outcome was achieved". Availability of appropriate data against which to measure the indicator was not one of the criteria considered during indicator selection. This was to avoid the risk of initially selecting indicators based on ease of reporting, rather than choosing those that best reflect performance against the outcomes. However, indicator development was also an iterative process, so at times the existence of a measure led to the development of additional indicators.

Identifying Measures

Once indicators were identified, an extensive literature and data review was conducted to identify what, if any, measures against these indicators were currently available. Where measures where available, the source was noted. Where measures were not available, this was noted along with a suggestion about what would be required in order to collect the necessary information.

All of the above information is presented in a matrix format, with separate tables for the three major outcome areas.

View Government Leadership Matrix - see Appendix B

View Industry Leadership and Community Engagement Matrix - see Appendix C

View Preventive Workplace Cultures Matrix - see Appendix D

Selecting Indicators for the Outcome Monitoring Framework

The outcome framework currently contains 37 expected outcomes for the WHSS. Each outcome has one or more associated indicators, and each indicator has one or more actual or potential measures. In order to maximise efficiency and minimise cost, a subset of the indicators/measures needs to be chosen for ongoing monitoring.

This paper presents a proposed monitoring framework with two phases: Phase One consists of indicators where measurement information is currently available, or can be collected/reported at minimal cost. Phase Two consists of indicators that are necessary to fully assess if the WHSS has been successful, but that would require significant extra cost to collect/report.

Selection criteria for the indicators and measures in the proposed monitoring framework included:

Validity Often it is possible to measure indicators in a variety of different ways. The differences in measures may depend on who collects the data, where it is collected from or how the indicator is defined. Some measures are more valid than others - because the definition used maps more closely to the indicator, or because the data collection method is more reliable, accurate or thorough. In cases where there are a number of different measures available, the measure(s) with greatest validity are suggested for the monitoring framework. Other measures which have been considered and discarded are noted.

Alignment As noted in the WHHS, where possible indicators have been aligned with the NZIPS or other relevant government work programmes

Availability For Phase One of the monitoring programme, measures which are currently available (and are likely to be repeated in future) have been selected. Using only these measures, progress in workplace safety could be monitored at minimal expense. However, the extent to which the WHSS has contributed to these outcomes would be difficult to assess. Also, the lack of any current measures of workplace health mean that this aspect of the Strategy cannot be monitored.

Phase Two of the monitoring programme suggests additional measures that will provide a fuller picture of outcomes (including health outcomes), and may provide a better sense of the extent to which WHSS has contributed to these outcomes. However, collection of this information is likely to involve significant extra expenditure.

Contents | << Previous page | Next page >>