Open Data Publication

Any data that is not determined to be ‘exempt’ for reasons described above, must be published in adherence to the principles outlined in section 4.0 of the Open Data Directive (ODD).

Open Government has developed a process (described below) for ministries to assess risks and publish their data. Provincial agencies should consider the ministry requirements as an example of best practice and use them as a guide to develop their own internal publication process. As an additional final step in publication, provincial agencies should notify their responsible ministry when they have published data.

An overview of the process and roles/responsibilities is found in Appendix 2.

Step 1: Identify, Describe and Prioritize Data

Once a dataset has been identified it should be described and added to the data inventory with an ‘Under review’ status. In rare cases, the government cannot admit some data exists – this data should not be listed as part of the inventory.

The OG team is available to review draft dataset description forms before they are sent for translation to provide advice on the plain language and style to promote consistency with the broader catalogue.

Using their inventory as a guide, organizations are expected to focus their efforts and prioritize high-value datasets for release sooner.

There are several factors that can help you decide what data should be a priority for release:

• High Value Data:

  Data that is frequently requested by the public, or collected or produced in support of government priorities, mandated by act or statute, also released by other jurisdictions, or identified as high value by reputable international organizations, such as the Open Government Partnership or the Open Knowledge Foundation. See Appendix D of the Open Data Directive.

• Public engagement and feedback:

  Data is considered of high interest or value if it is the subject of a high number of public website searches, related webpage usage (e.g., similar datasets or info), relevant to public engagements, FOI requests or email/correspondence requests. Work with your web team, correspondence unit, call centres, program areas or FOIP coordinators to help determine what data the public would like opened.

• Transparency & accountability:

  Data that increase transparency and accountability, specifically data used to create legislation, regulation, policy, program and service evaluation, or data related to asset management, procurement contracts and audited financial information (e.g., Public Accounts) is considered high-value.

• Policy evaluation:

  Releasing data that is useful for internal and external stakeholders’ evaluation of the delivery of policy, programs or services can be considered high-value. However, usefulness requires timely and frequent release to support effective evaluation.

• Government Priority:

  Data that aligns with new or important initiatives are considered high-value.

• Program efficiency:

  Data that can help improve the delivery of programs and services are considered high-value. To determine if data are of high-value to others, examine the inter-agency or inter-ministerial data requests (e.g., Early Development Instrument).

• Cost/return on investment:

  Ministries and provincial agencies may want to prioritize data that will cost less to release, such as data that is already published or collected in a machine-readable format and will take less time and fewer resources to prepare as a result.

  Alternatively, data that has a large return-on-investment, such as data that can reduce FOI requests, create more efficient programs through research, or have positive economic or social impacts, should also be prioritized for release.

Given the prioritization criteria above, organizations must develop a plan, including timelines, for the publication of all datasets which are not deemed ‘exempt’ according to the Open Data Directive.

Additionally, the Simpler, Faster, Better Services Act empowers the Chief Digital and Data Officer to identify and compel the release of high value data.

Step 2: Assess Data

To determine whether data is eligible to be Open, the data must be assessed prior to publication for:

• legal, security, confidentiality, privacy and/or commercial-sensitivity issues
• issues management

Ministries are responsible for leading the assessment of their respective datasets. For corporate-wide or financial data, OG will work with the data custodians to assess the data.

Prior to publishing datasets that contain de-identified data, they should be assessed in the context of existing available data to manage the risk of identification through linking with existing data in the catalogue, or other similar datasets that are publicly available

A detailed list of teams that can be involved in assessments and how the data should be assessed can be found on the Open Government intranet site. The following is an overview of roles that could be involved in the risk assessment process:

Open Government Lead

The ministry Open Government lead should:

• identify their ministry’s assessment team,
• convene the assessment team each time datasets need to be assessed, and determine timelines,
• document the results.

Program/Policy Lead

The ministry program or policy lead (e.g., the data custodian or coordinator) should consider and document the dataset’s content, any data quality issues, and update the release schedule.

Considerations when assessing data:

• whether the data can be interpreted by other people
• whether the data can be used to add value
• the limitations of the dataset
• similar data released in other jurisdictions
• the demand for and importance of a dataset as it may determine a dataset’s economic or social impact potential.

Legal Branch

Legal representatives can provide advice on the data’s legal issues.

Privacy Team

FOIP coordinators can assess whether the datasets contain personal and/or confidential information. They can also provide background on whether the dataset was included in an FOI request.

If the data contains personal information, they are required to make recommendations on whether it should be redacted, de-identified or exempt from publication.

The Information and Privacy Commissioner for Ontario has published De-identification Guidelines for Structured Data. The guidelines introduce institutions to the basic concepts and techniques of de-identification, outline the key issues to consider when de-identifying personal information in the form of structured data and they provide a step-by-step process that institutions can follow when removing personal information from datasets.

IT Cluster

Ministry IT clusters or ministry analytics branches can assess the data quality and accessibility and suggest how to extract the dataset in a machine-readable format.

Communications Branch

Communications can help identify potential issues and opportunities associated with the release of the datasets. They can advise when issues management plans are needed and help develop appropriate plans.

French Language Services Coordinator

Raw data is posted to the catalogue in the language in which it was created, but FLS coordinators can be consulted to ensure that language requirements under the French Language Services Act are met and will provide advice for identified datasets on French translation needs.

Step 3: Prepare Data and Metadata

The following steps should be taken to prepare data for publication:

1. When recommended by Communications, write an issues management plan.
2. Convert data into an Open Format with help from IT cluster as needed. Ensure that specific technical requirements are met and that the data is in a machine-readable format.
3. Update the data description and metadata as needed. Review mandatory requirements using the checklist.
4. Check in with OG for advice on metadata updates. It is good practice to share a draft version of the data if possible as well, when seeking guidance on Open Formats.
5. Translate the metadata updates, the data dictionary, and other material for the Data Catalogue. For greater clarity:
   • Data files are published in the catalogue in the language in which they were collected. They are exempt from translation requirements.
   • The dataset description, metadata, and accompanying data dictionaries or technical documentation must be translated. These will be separate documents that accompany the datasets. Any separate documents to accompany a dataset must also comply with AODA requirements.
   • The ministry or affected program area is responsible for obtaining and covering the costs of translation.
6. Un-formatted data files that are structured for database consumption meet accessibility requirements. Formatted tables must pass Excel accessibility checks (e.g., no merged cells, alternative text, properly named columns and sheets, etc.)
7. If deem ‘Restricted’ the rationale needs to be assessed for clarity and translated as part of the metadata.

Step 4: Review Assessment

Before determining data as Open or Restricted the Program Director responsible must review the recommendations of the assessment team. This review will focus on the following key elements:

• The data access level will be confirmed as “Open” or “Restricted”
• If “Restricted” the rationale not to release will be reviewed for clarity
• Any preparations of files to protect identity or mitigate other risks are reviewed and confirmed as appropriate
• If required, an associated issues management plan is reviewed by the Assistant Deputy Minister or delegate to confirm messaging and mitigation strategy is appropriate

Each dataset is unique and will require an assessment by multiple perspectives to ensure potential risks are mitigated. The director or delegate may ask that additional teams be engaged to review the assessment team’s recommendations based on the nature of the dataset.

Step 5: Submit Data and Metadata for Publishing

Once the recommendation of the assessment team is reviewed, the submission package is ready to be sent to the OGO.

The submission package to OGO must include:

• Either the data file, a link to the data file, or documentation required to access a database directly or the rationale not to release
• Updated data description form
• Documentation that the assessment was reviewed by Program Director
• If required, an issues management plan reviewed by the Assistant Deputy Minister’s office

Before the dataset is updated to ‘Open’ or ‘Restricted’ the OGO will:

1. Assess the data and verify its compliance with the directive and the guidebook.
2. Share any associated issues management plan with strategic communications team to ensure the mitigation plan aligns with broader issues management strategies.

The OGO will work with the appropriate teams to publish the data and update the metadata in the Data Catalogue on Ontario.ca.

Once published, the OGO will send the relevant links to the ministry.

After publication, ministries should work with their Communications Branches and Cabinet Office to:

• promote through social media and traditional media in both English and French (e.g., links in News Releases).
• let their ministry and Ontario Public Service (OPS) colleagues know that they have published a new Open Dataset
• engage with partners