Application of risk methodology to drinking water testing laboratory and system inspection results
Learn how the risk rating methodology applied to drinking water inspections supports the oversight of drinking water safety.
Overview
We have a comprehensive inspection program for licensed and out-of-province laboratories that test Ontario drinking water and municipal residential drinking water systems. We assess whether these laboratories and drinking water systems are meeting the requirements under the Safe Drinking Water Act, 2002 and associated regulations. It is the responsibility of laboratory and drinking water system owner to ensure that they comply with all applicable legal requirements.
This page describes the risk rating methodology we use to evaluate the results of these inspections. The goal of the application of this methodology is to encourage the on-going improvement of compliance rates at laboratories and drinking water systems and to establish a way to measure this progress.
We review the inspection protocols on a periodic basis so that they align with legislative and technological changes and emerging issues that may affect drinking water safety. This helps to ensure we continue to protect human health under the Safe Drinking Water Act, 2002.
Inspection protocols
Our inspection protocols for drinking water testing laboratories (laboratory) and municipal residential drinking water systems (systems) contain multiple inspection modules. Each module contains multiple questions that address a wide range of regulatory issues. Ministry inspectors use the guidance for these questions when conducting inspections to assess everything from administrative procedures, water treatment processes for systems and adverse water quality reporting requirements for laboratories.
Additionally, the inspection protocol can contain non-regulatory questions, such as information gathering questions. Non-regulatory questions are not risk rated. It is possible for a laboratory or system owner to have a recommended action as part of this module with no change to their risk rating associated with the inspection.
Based on the number of areas where a laboratory or system is deemed to be non-compliant during the inspection, and the significance of these areas to health consequences, we calculate a risk-based rating for each inspection of a laboratory and system. Each regulatory inspection question has an individual risk rating.
It is important to note that an inspection rating of less than 100% does not necessarily mean that the laboratory is not analyzing drinking water properly or not reporting data as required, or that the system owner is not operating the system properly or that the drinking water is unsafe. It provides laboratories and system owners with insight they can use to better inform decisions and improve compliance. To that end, we work with the laboratory or system owners so that they undertake corrective actions to achieve full compliance.
The inspection rating reflects the results for each inspection period. At least two inspection cycles are carried out per year for laboratories. Since the two cycles have different inspection questions, the inspection ratings from one cycle cannot be directly compared to the other. By contrast, the inspection ratings reflect the results of the specific system for the reporting year and can be compared year over year. The rating record encourages continuous improvement and allows laboratories and system owners to identify specific areas requiring attention.
Determining the risk rating
The risk management approach used for inspection aligns with the Government of Ontario’s risk management framework. Risk management is a systematic approach to identifying potential hazards (harms); understanding the likelihood and consequences of the hazards; and taking steps to mitigate the risks if necessary.
The risk management framework provides a formula to be used in the determination of risk:
Risk = Likelihood (L) × Consequence (C)
The question risk rating (L×C) for each regulatory inspection question is derived from an evaluation of the likelihood of the underlying harm occurring should the answer to a legislative question indicate non-compliance. Then, assuming the underlying harm does occur, the most probable consequence to health (C) is determined. The likelihood and consequence values can be found in Table 1 and Table 2.
| Likelihood of underlying harm occurring | Likelihood value |
|---|---|
| Almost certain | L = 5 |
| Very likely | L = 4 |
| Likely | L = 3 |
| Unlikely | L = 2 |
| Rare | L = 1 |
If there is no chance of harm occurring to health, then a likelihood of zero (L=0) is used.
| Most probable health consequence | Consequence value |
|---|---|
| Extreme | C = 5 |
| High | C = 4 |
| Medium | C = 3 |
| Low | C = 2 |
| Negligible | C = 1 |
The question risk rating quantifies the risk of non-compliance of each question relative to the others. Questions with higher values are those that have a more significant impact on drinking water safety and a higher likelihood of harm occurring when they indicate non-compliance. The highest possible value would be 25 (5×5) and the lowest would be 1 (1×1). If there is no chance of harm occurring, then the question risk rating would be 0.
The following presents a sample laboratory question showing the risk rating determination process:
Inspection Question ID: LABDW1079000
Inspection Question Text: Does the laboratory have a suitable, authorized adverse/exceedance reporting policy/procedure?
Risk = Likelihood × Consequence
Likelihood of Underlying Harm Occurring: L=2 Unlikely
Consequence: C=2 Low Health
Question Risk Rating: 2×2 = 4
The following presents a sample municipal residential drinking water system question showing the risk rating determination process:
Inspection Question ID: DWMR1007000
Inspection Question Text: Is the owner maintaining the production well(s) in a manner sufficient to prevent entry into the well of surface water and other foreign materials?
Risk = Likelihood × Consequence
Likelihood of Underlying Harm Occurring: L=3 Likely
Consequence: C=3 Medium Health
Question Risk Rating: 3×3 = 9
Reporting results
After each inspection, we calculate an overall inspection risk rating. Inspectors answer regulatory compliance questions and input their responses as “yes,” “no” or “not applicable”. A “no” response indicates non-compliance.
The maximum number of regulatory questions asked by an inspector varies by the inspection. For drinking water testing laboratories, it is impacted by the type of inspection (for example annual 1, annual 2 and more) and the scope of the work performed by the laboratory (for example types of parameters analyzed). For municipal residential drinking water systems, it is impacted by the category of system (large versus small municipal residential), type of inspection (focused versus detailed), and source type (for example groundwater or surface water).
To calculate the inspection rating, we add up all non-compliant (no) answers associated with a health risk and divide by the total risk ratings of all questions with a health risk asked (maximum question rating) for each inspection. The resulting inspection risk rating (as a percentage) is subtracted from 100% to arrive at the final inspection rating.
A summary of findings for each inspection is generated in the form of an inspection rating. The findings are grouped into the different modules of the inspection protocol (for example sample handling and testing methods, treatment processes and monitoring etc.), which would provide the laboratory or system owner with information on the areas where Improvement is needed. Only the modules that are included in the inspection are shown on the inspection rating.
Laboratories and system owners receive a rating with each inspection report. The rating is confidential to each laboratory and is not publicly reported. However, aggregate results for laboratories and systems are part of the Chief Drinking Water Inspector annual report. Individual inspection risk rating results for systems are also posted in the Ontario Data Catalogue.
For further information, please visit Ontario’s drinking water page or contact llcp@ontario.ca for laboratories, and drinking.water@ontario.ca for systems.