Introduction

A. About this Committee Report

On 25 March 2021, the Office of the Auditor General of Canada (OAG) tabled an audit report in the House of Commons entitled “Follow-up Audit on Rail Safety – Transport Canada,” which was referred to the House of Commons Standing Committee on Public Accounts (the Committee) for consideration.^[1] On 6 May 2021, the Committee held a meeting on this report, with the following individuals in attendance:

• OAG – Karen Hogan, Auditor General of Canada; Dawn Campbell, Principal; and Isabelle Marsolais, Director.
• Transport Canada – Michael Keenan, Deputy Minister; Aaron McCrorie, Associate Assistant Deputy Minister, Safety and Security; and Michael DeJong, Director General, Rail Safety.^[2]

B. Background

The OAG’s 2021 audit followed up on “selected recommendations from [its] 2013 Fall Report of the Auditor General of Canada, Chapter 7—Oversight of Rail Safety—Transport Canada. In that audit, [the OAG] concluded that Transport Canada needed to address significant weaknesses in its oversight of safety management systems implemented by federal railway companies to manage safety risks on a day-to-day basis. [The OAG] recommended that the department

• improve its risk-based planning of oversight of railway companies, including collecting relevant risk and safety information from railways
• conduct timely follow-up on deficiencies affecting the safety of railway companies’ operations, including assessing the effectiveness of railway companies’ safety management systems.”^[3]

C. Audit Objective

The OAG’s audit “focused on whether Transport Canada implemented selected recommendations from [its] 2013 audit regarding the department’s oversight of the safe transportation of people and goods on federally regulated tracks. In particular, [the OAG] examined the risk-based planning process for rail safety oversight and whether the department confirmed that railway companies corrected deficiencies affecting the safety of their operations.”^[4]

The audit covered the period from 1 September 2018 to 31 August 2020, to which the audit conclusion applies. However, to gain a more complete understanding of the subject matter of the audit, the OAG also examined certain matters that preceded the start date of this period.^[5]

D. Roles and Responsibilities

Transport Canada – Under the Railway Safety Act, the department “is responsible for overseeing safety requirements for federally regulated railway companies. The department’s oversight role includes monitoring compliance and enforcement activities.”^[6]

Railways – Railway companies “are responsible for the safety of their own operations pursuant to the Railway Safety Act and the accompanying Railway Safety Management System Regulations adopted in 2001. These were repealed and replaced in 2015. Each railway company is required to develop and follow its own safety management system in its day-to-day operations. These systems generally include a safety policy, safety targets, a risk-assessment process, and processes for continuous improvement. Safety management systems complement and do not eliminate or replace other regulatory requirements. They are intended to enable railway companies to manage risks before Transport Canada intervenes and before major safety issues emerge.”^[7]

Findings and Recommendations

A. Risk-based Planning for Oversight

According to the OAG, “Transport Canada could not demonstrate that its oversight planning activities improved railway companies’ regulatory compliance.”^[8]

1. Improved Risk-based Planning Process

The OAG noted that, since its 2013 audit, “Transport Canada made progress in improving its risk-based approach to identifying and overseeing safety issues. In the 2016–17 fiscal year, 40% of inspections were risk-based, and this proportion increased to 72% in the 2019–20 fiscal year.”^[9] Michael Keenan, Deputy Minister, Transport Canada, described the risks the department focuses on as follows:

The number one indicator is the risk to the health and safety of Canadians, not the economic cost of the accident. That’s very much secondary, if not hardly even considered. It’s the risk to the health and safety of Canadians, the risk of disrupting communities and the risk to the environment. Those are the factors that we consider.^[10]

The OAG also found that “each regional office followed different planning practices to prioritize their oversight activities. Although Transport Canada identified risks for rail safety at both the national and regional levels, the department’s risk assessment was inconsistent across regions and was not always adequately documented, even though the department provided guidance to regional offices on how to identify areas of highest risk. The department was aware of this situation and acknowledged that it needed to work with the regional offices to assess risk consistently.”^[11]

2. Overall Effectiveness of Oversight Activities not Measured

The OAG found that “Transport Canada did not measure the overall effectiveness of its rail safety oversight. It had not identified whether rail safety has improved as a result of its inspections and audits of safety management systems.”^[12] Karen Hogan, Auditor General of Canada, explained that, when “time, effort and energy are invested, it is very reasonable to expect that the investment will pay off. This final step is extremely important in determining whether the efforts are in the right place, adequately targeted at risk, and effective.”^[13]

Furthermore, “Transport Canada senior management received only partial information on the results of oversight activities to make informed risk-based planning decisions. Although the department’s senior management received year-end reports, the following information to support decision making was not included:

• sufficient data on railways’ compliance with regulations
• results of departmental audits of railways’ safety management systems
• inspection results—departmental data focused instead on the number of inspections performed.”^[14]

Recommendation

The OAG therefore made the following recommendation:

Transport Canada should determine the extent to which its inspections and audits have improved the railway companies’ compliance with regulations that mitigate key safety risks. The department should integrate these results into its risk-based planning decisions, including determining which and how many inspections to perform.^[15]

Karen Hogan encouraged Transport Canada “to consider what other programs and jurisdictions are doing on this front, both in Canada and in other countries. The Canada Energy Regulator, for example, has established indicators that measure components of effectiveness. In the United Kingdom, the Office of Rail and Road has developed tools to assess railway companies’ ability to manage health and safety risks. The resulting information is used to make year-over-year progress comparisons. Furthermore, in the United States, the Office of Transit Safety and Oversight has committed to monitoring the effectiveness of state safety agencies.”^[16] Michael DeJong, of Transport Canada, said:

[W]e’ve actually done follow-up with our U.S. counterparts as well as with our U.K. and European Union counterparts to look into potential indicators for how we can improve our measurements of the oversight regime, including by potentially leveraging some best practices from the U.S. as well as our Canadian Energy Regulator in terms of potential performance indicators.^[17]

Michael Keenan added the following:

There are many players and there are many factors in the safety outcome. The work the Auditor General has asked us to do is to start to isolate those factors and to identify in some analytical and, if possible, quantitative manner how each specific factor of each element of the SMS [safety management system] and the other elements of the safety system contribute to improving performance to have lower accidents with higher volumes.

We are in the process of doing that. The fact that it is ongoing does not undermine the fact that the operators are accountable to a much more exacting standard in their safety management systems, and have been since 2015, and they’re being rigorously audited on it. When they’re coming up short, they are subject to enforcement measures from Transport Canada.^[18]

According to its action plan, Transport Canada will address this recommendation by developing better performance indicators “that can be used on a regular basis to monitor performance.”^[19] The expected completion date for this stage is September 2021. The department will also implement regulatory compliance rates for railway companies (November 2021), which will be monitored on a regular basis as of January 2022. The department will then proceed with “the program evaluation that will determine the extent to which oversight activities are improving rail safety.”^[20] That stage is expected to be completed in December 2022. From there, the department “will provide a detailed action plan to the evaluation recommendations outlining activities necessary to better measure program effectiveness on an ongoing basis” by February 2023.^[21]

Therefore, the Committee recommends:

Recommendation 1 – on the effectiveness of oversight activities

That Transport Canada provide the House of Commons Standing Committee on Public Accounts with (1) a report on the development of indicators that can be used to assess performance, by 31 October 2021; (2) a report on railway companies’ compliance with regulations, by 28 February 2022; and (3) a report on the evaluation of the rail safety oversight program and the action plan to better measure program effectiveness on an ongoing basis, by 28 February 2023.

3. Inadequate Information to Plan for Inspections

The OAG found that Transport Canada began to implement its 2013 recommendation “to collect relevant risk and safety performance information from federal railways.”^[22] However, the department “did not provide sufficiently detailed guidance to the railway companies on how to report their safety data. As a result, the information was sometimes late, incomplete, or of varying quality.”^[23]

Recommendation

The OAG therefore made the following recommendation:

Transport Canada should improve guidance to railway companies so that safety data submissions are complete, reliable, and timely.^[24]

According to its action plan, the department “will develop an information request template (September 2021) and issue guidelines to railway companies and stakeholders to facilitate data analysis and streamline reporting. These measures will be completed by October 2021.”^[25]

Therefore, the Committee recommends:

Recommendation 2 – on the information available to plan for inspections

That, by 31 October 2021, Transport Canada provide the House of Commons Standing Committee on Public Accounts with a report on the changes made to guidelines issued to railway companies regarding the safety data they must submit to the department.

4. Audit Results of Safety Management Systems not Considered

The OAG found that Transport Canada had made progress in conducting more audits of railway companies’ safety management systems since its 2013 audit. According to the department, it audited all the railway companies in operation at least once between 1 April 2016 and 31 March 2020.^[26] Michael Keenan reported that the number of safety management system audits being done went from 4 a year to 25 a year.^[27]

However, the OAG noted that “Transport Canada’s oversight planning process did not consider findings from departmental audits of the railways’ safety management systems.”^[28] According to the OAG, if “safety management system audit findings had been included in Transport Canada’s planning oversight process, the department’s annual inspection focus may have shifted. In [the OAG’s] view, it is important that the department integrate the results of its audit activities into the risk-based planning for rail safety oversight, so that it considers all available information.”^[29]

Recommendation

Hence, the OAG made the following recommendation:

Transport Canada should integrate the findings from audits of safety management systems when planning its oversight activities and adjust its selection of planned inspections to address areas of highest risk.^[30]

In response to that recommendation, the department plans to review “the existing risk‑based planning procedure and update to include participation of Safety Management System Staff in functional risk-based planning meetings.”^[31] Furthermore, guidelines will be established for “Safety Management System Staff with respect to the information they will need to bring to the functional meetings for planning purposes.”^[32] The expected completion date for these steps is May 2021.

Therefore, the Committee recommends:

Recommendation 3 – on taking into account the audit results

That, by 31 August 2022, Transport Canada provide the House of Commons Standing Committee on Public Accounts with a report outlining the changes to its planning procedures for oversight activities to ensure that they integrate the findings from audits of safety management systems.

B. Conducting Audits and Inspections

According to the OAG, “Transport Canada made progress in assessing and following up on railway companies’ corrective actions but did not measure the effectiveness of their safety management systems.”^[33]

1. Good Progress on Assessing Railway Companies’ Corrective Actions

The OAG reported that, “for most of the inspections and audits [it] examined, Transport Canada assessed whether railway companies took corrective actions when non-compliances were identified. In [the OAG’s] view, Transport Canada’s process to validate the railway companies’ plans to address problems was reasonable.”^[34]

2. No Standard Time Frame for Assessing Whether Railway Companies Completed Corrective Actions

According to the OAG, “the department had no standard time frame for how long it should take to assess whether railway companies completed corrective actions.”^[35] Karen Hogan explained that, while Transport Canada is “doing follow-up and making sure corrective actions are taken, [it is] not really setting a timeline. It is left to the inspector to go back at some point in time. More regular, focused review or follow-up would keep the pressure on railway companies to increase and enhance safety.”^[36]

In fact, the “department’s standards for on-site audits and inspections focused primarily on how quickly the railway companies needed to provide a plan to correct the issues—and not on confirming that railway companies made timely corrections to the deficiencies affecting safety.”^[37]

Recommendation

The OAG thus made the following recommendation:

Transport Canada should set standards for the time it should take to assess whether railway companies corrected deficiencies identified in inspections and audits of safety management systems.^[38]

In response to that recommendation, the department stated that it “will establish a standard by December 2021 on how long it should take to assess whether railway companies corrected deficiencies resulting from inspections and audits of safety management systems.”^[39]

Therefore, the Committee recommends:

Recommendation 4 – on the lack of standard time frames

That, by 31 December 2021, Transport Canada provide the House of Commons Standing Committee on Public Accounts with a report outlining its new standards on time frames to assess whether railway companies have corrected the deficiencies identified in inspections and audits of safety management systems.

3. Improved Information Management for Inspections but not for Safety Management System Audits

According to the OAG, “Transport Canada updated its information management system to better manage inspections. Refinements included tracking railway companies’ corrective actions, inspection results, and departmental communication with railway companies. The system also generated reports for management.”^[40]

However, “the department did not integrate information and results from its safety management system audits into its information database, nor did it have another way to track this data. Furthermore, Transport Canada did not adhere consistently to its documentation standards, which made it difficult to determine whether the department uniformly assessed the railway companies’ safety management systems.”^[41]

Recommendation

Thus, the OAG made the following recommendation:

Transport Canada should improve data management for its safety management system audits by

•   adhering to its documentation standards
•   integrating the results of its safety management system audits into a data management system.^[42]

Transport Canada stated that, in April 2021, it had completed a review of “a sample of audit files from across headquarters and the regions for compliance with documentation standards.”^[43] Michael DeJong summarized the results of that review as follows:

Essentially, the sample showed that we are collecting a wealth of information from railway companies, including information on operating characteristics, traffic patterns, tonnage and previous incidents. As the deputy minister mentioned, however, to use all of this data to drive our risk-based approach, we need it to be formatted in a way that allows Transport Canada to collate and analyze it quickly so that it can help support our risk-based planning.^[44]

A new data management system would also “be developed to facilitate the tracking of audit findings and identification of compliance trends. This system will be in place by December 2021.”^[45]

Therefore, the Committee recommends:

Recommendation 5 – on information management for safety management system audits

That, by 31 December 2021, Transport Canada provide the House of Commons Standing Committee on Public Accounts with a report on the implementation of its new information management plan for data from safety management system audits.

4. Effectiveness of Safety Management Systems not Assessed

Karen Hogan explained that the railway companies’ safety management systems were “formal frameworks to proactively integrate safety into day-to-day railway operations. In-depth, systematic assessments of these systems are called audits. They are meant to verify whether the systems meet regulatory requirements and integrate safety into daily railway operations. Over the past 14 years, several reports have recommended that Transport Canada undertake such assessments.”^[46]

The OAG found that “the department’s audits continued to focus on whether safety management systems complied with regulatory requirements and not whether these systems were effective and actually improved rail safety. Determining whether the supporting processes are effective is an important next step for the department in assessing the overall effectiveness of railway companies’ safety management systems.”^[47]

Recommendation

The OAG therefore made the following recommendation:

Transport Canada should make it an immediate priority to regularly assess the effectiveness of railway companies’ safety management system processes.^[48]

Transport Canada committed to putting “a system in place that assesses the effectiveness of a railway company’s safety management system’s processes”^[49] by December 2021. It added that consultations “will be launched in April 2022 with the rail industry and with the public. Consultations will commence to identify potential amendments to the Safety Management System Regulation.”^[50] To explain why consultations will not start until April 2022, Michael DeJong said:

Much of our regulatory development work will be informed by the findings of our effectiveness audits on [safety management systems], which will be launched in September 2021. We’ll be able to use those results to help drive our regulatory action in this space.^[51]

Therefore, the Committee recommends:

Recommendation 6 – on assessing the effectiveness of safety management systems

That Transport Canada provide the House of Commons Standing Committee on Public Accounts with: (1) a report on the effectiveness of rail companies’ safety management system processes, by 31 December 2021; and (2) a report on its consultations on the Railway Safety Management System Regulations, 2015, by 31 October 2022.

Conclusion

The Committee concludes that Transport Canada had not fully implemented all of the selected recommendations from the OAG’s 2013 audit report on rail safety regarding the department’s oversight of the safe transportation of people and goods.

The department made important improvements to both its safety oversight planning and inspection follow-up, but it did not assess whether its oversight activities improved rail safety. Also, the department did not measure the effectiveness of the railway companies’ safety management systems or integrate the results of those audits into its rail safety oversight planning.

Given the critical nature of rail safety, and given the tragic accidents that occurred in 2013 in Lac-Mégantic, Quebec, and in 2019 in Field, British Columbia, the Committee has made six recommendations to ensure that Transport Canada improves its processes in order to ensure the safety of people and the environment.

Summary of Recommended Measures and Timelines

Table 1—Summary of Recommendations and Timelines

Recommendation	Recommended Measure	Timeline
Recommendation 1	Transport Canada should provide the House of Commons Standing Committee on Public Accounts with (1) a report on the development of indicators that can be used to assess performance; (2) a report on railway companies’ compliance with regulations; and (3) a report on the evaluation of the rail safety oversight program and the action plan to better measure program effectiveness on an ongoing basis.	(1) 31 October 2021 (2) 28 February 2022 (3) 28 February 2023
Recommendation 2	Transport Canada should provide the Committee with a report on the changes made to guidelines issued to railway companies regarding the safety data they must submit to the department.	31 October 2021
Recommendation 3	Transport Canada should provide the Committee with a report outlining the changes to its planning procedures for oversight activities to ensure that they integrate the findings from audits of safety management systems.	31 August 2022
Recommendation 4	Transport Canada should provide the Committee with a report outlining its new standards on time frames to assess whether railway companies have corrected the deficiencies identified in inspections and audits of safety management systems.	31 December 2021
Recommendation 5	Transport Canada should provide the Committee with a report on the implementation of its new information management plan for data from safety management system audits.	31 December 2021
Recommendation 6	Transport Canada should provide the Committee with: (1) a report on the effectiveness of rail companies’ safety management system processes; and (2) a report on its consultations on the Railway Safety Management System Regulations, 2015.	(1) 31 December 2021 (2) 31 October 2022

---