Application of the risk methodology used for measuring municipal drinking water system inspection results
This document describes the risk rating methodology which has been applied to the findings of the Ministry’s municipal residential drinking water system inspection results since fiscal year 2008-09.
On this page Skip this page navigation
The Ministry of the Environment (MOE) has a rigorous and comprehensive inspection program for municipal residential drinking water systems (MRDWS). Its objective is to determine the compliance of MRDWS with requirements under the Safe Drinking Water Act and associated regulations. It is the responsibility of the municipal residential drinking water system owner to ensure their drinking water systems are in compliance with all applicable legal requirements.
This document describes the risk rating methodology, which has been applied to the findings of the Ministry’s MRDWS inspection results since fiscal year 2008-09. The primary goals of this assessment are to encourage ongoing improvement of these systems and to establish a way to measure this progress.
MOE reviews the risk rating methodology every three years to account for legislative and societal changes that affect acceptable risk levels. As a result of the most recent review, the methodology has been modified to present an improved metric for the evaluation of the risk/safety of MRDWS operations.
The Ministry’s Municipal Residential Drinking Water Inspection Protocol contains up to 14 inspection modules and consists of approximately 120 regulatory questions. Those protocol questions are also linked to definitive guidance that ministry inspectors use when conducting MRDWS inspections. The questions address a wide range of regulatory issues, from administrative procedures to drinking water quality monitoring. Additionally, the inspection protocol contains a number of non-regulatory questions.
A team of drinking water specialists in the ministry have assessed each of the inspection protocol regulatory questions to determine the risk (not complying with the regulation) to the delivery of safe drinking water. This assessment was based on established provincial risk assessment principles, with each question receiving a risk rating referred to as the Question Risk Rating. Based on the number of areas where a system is deemed to be non-compliant dur ing the inspection, and the significance of these areas to administrative, environmental, and health consequences, a risk-based inspection rating is calculated by the ministry for each drinking water system.
It is important to be aware that an inspection rating that is less than 100 per cent does not mean that the drinking water from the system is unsafe. It shows areas where a system’s operation can improve. To that end, the ministry works with owners and operators of systems to make sure they know what they need to do to achieve full compliance.
The inspection rating reflects the inspection results of the specific drinking water system for the reporting year. Since the methodology is applied consistently over a period of years, it serves as a comparative measure both provincially and in relation to the individual system. Both the drinking water system and the public are able to track the performance over time, which encourages continuous improvement and allows systems to identify specific areas requiring attention.
The ministry’s annual inspection program is an important aspect of our drinking water safety net. The ministry and its partners share a common commitment to excellence and we continue to work toward the goal of 100 per cent regulatory compliance.
Determining Potential to Compromise the Delivery of Safe Water
The risk management approach used for MRDWS is aligned with the Government of Ontario’s Risk Management Framework. Risk management is a systematic approach to identifying potential hazards; understanding the likelihood and consequences of the hazards; and taking steps to reduce their risk if necessary and as appropriate.
The Risk Management Framework provides a formula to be used in the determination of risk:
Risk = Likelihood (of the consequence) × Consequence
Every regulatory question in the inspection protocol possesses a likelihood value (L) for an assigned consequence value (C) as described in Table 1 and Table 2.
|Likelihood of Consequence Occurring||Likelihood Value|
|0% - 0.99% (Possible but Highly Unlikely)||L = 0|
|1 - 10% (Unlikely)||L = 1|
|11 - 49% (Possible)||L = 2|
|50 - 89% (Likely)||L = 3|
|90 - 100% (Almost Certain)||L = 4|
|Medium Administrative Consequence||C = 1|
|Major Administrative Consequence||C = 2|
|Minor Environmental Consequence||C = 3|
|Minor Health Consequence||C = 4|
|Medium Environmental Consequence||C = 5|
|Major Environmental Consequence||C = 6|
|Medium Health Consequence||C = 7|
|Major Health Consequence||C = 8|
The consequence values (0 through 8) are selected to align with other risk-based programs and projects currently under development or in use within the ministry as outlined in Table 2.
The Question Risk Rating for each regulatory inspection question is derived from an evaluation of every identified consequence and its corresponding likelihood of occurrence:
- All levels of consequence are evaluated for their potential to occur
- Greatest of all the combinations is selected.
The Question Risk Rating quantifies the risk of non-compliance of each question relative to the others. Questions with higher values are those with a potentially more significant impact on drinking water safety and a higher likelihood of occurrence. The highest possible value would be 32 (4×8) and the lowest would be 0 (0×1).
Table 3 presents a sample question showing the risk rating determination process.
|Medium Administrative Consequence||Major Administrative Consequence||Minor Environmental Consequence||Minor Health Consequence||Medium Environmental Consequence||Major Environmental Consequence||Medium Health Consequence||Major Health Consequence|
|L=4 (Almost Certain)||L=1 (Unlikely||L=2 (Possible)||L=3 (Likely)||L=3 (Likely)||L=1 (Unlikely||L=3 (Likely)||L=2 (Possible)|
Application of the Methodology to Inspection Results
Based on the results of a MRDWS inspection, an overall inspection risk rating is calculated. During an inspection, inspectors answer the questions that relate to regulatory compliance and input their responses as "yes", "no" or "not applicable" into the Ministry’s Laboratory and Waterworks Inspection System (LWIS) database. A "no" response indicates non-compliance. The maximum number of regulatory questions asked by an inspector varies by: system (i.e., distribution, stand-alone), type of inspection (i.e., focused, detailed), and source type (i.e., groundwater, surface water).
The risk ratings of all non-compliant answers are summed and divided by the sum of the risk ratings of all questions asked (maximum question rating). The resulting inspection risk rating (as a percentage) is subtracted from 100 per cent to arrive at the final inspection rating.
Application of the Methodology for Public Reporting
The individual MRDWS Total Inspection Ratings are published with the ministry’s Chief Drinking Water Inspector’s Annual Report.
Figure 1 presents the distribution of MRDWS ratings for a sample of annual inspections. Individual drinking water systems can compare against all the other inspected facilities over a period of inspection years.
Figure 1: Year Over Year Distribution of MRDWS Ratings
|Year A||Year B||Year C||Year D||Year E|
|>50% to 80%||18||13||5||7||7|
|>80% to 85%||12||16||8||12||9|
|>85% to 90%||33||26||22||18||17|
|>90% to 95%||137||117||92||77||75|
|>95% to 100%||233||281||349||344||406|
Reporting Results to MRDWS Owners/Operators
A summary of inspection findings for each system is generated in the form of an Inspection Rating Record (IRR). The findings are grouped into the 14 possible modules of the inspection protocol, which would provide the system owner/operator with information on the areas where they need to improve. The 14 modules are:
- Permit to Take Water
- Capacity Assessment
- Treatment Processes
- Process Wastewater
- Distribution System
- Operations Manuals
- Contingency and Emergency Planning
- Consumer Relations
- Certification and Training
- Water Quality Monitoring
- Reporting, Notification and Corrective Actions
- Other Inspection Findings
For further information, please visit Drinking Water Ontario.