Introduction

The Office of the Auditor General of Canada (OAG) outlined that in 2017, the unemployment rate for Indigenous Canadians was just over 11%, compared with just over 6% for non-Indigenous Canadians. In fact, “many Indigenous people face barriers to sustained employment, such as living in isolated communities and having low levels of education,” and “need training and support to build the skills they need to find and keep jobs.”[1]

To address such issues, Employment and Social Development Canada (ESDC or the Department) “funds programs to help Canadians develop skills for employment and improve their workforce mobility. The Department prioritizes skill development for groups that are underrepresented in the workforce, such as Indigenous people.”[2]

To do so, in 2010, “the Department launched its two current programs: the Aboriginal Skills and Employment Training Strategy [the Strategy] and the Skills and Partnership Fund [the Fund].”[3]

In the spring of 2018, the OAG released a performance audit to determine whether ESDC “managed the … Strategy and the … Fund to increase the number of Indigenous people getting jobs and staying employed. Specifically, the [OAG] analyzed the Department’s actions to implement, monitor, report on, and improve the programs.”[4]

The Strategy and the Fund share a similar objective, which is to increase “Indigenous participation in the Canadian labour market, ensuring that First Nations, Métis, and Inuit people are engaged in sustainable, meaningful employment.” They were both initially created for fiscal years 2010-2011 to 2014-2015. Contribution agreements under the two programs are for a maximum of five years. According to the OAG, the Strategy should be ending in 2017‑2018 and the Fund in 2020‑2021,[5] but the ESDC 2017‑2018 Departmental Plan forecasts expenditures for the Strategy until 2019‑2020, when the next evaluation is set to take place.[6]

The main difference between the two programs lies with the type and amount of funding: the Strategy “provides funding for the delivery of training and employment support to Indigenous clients” and received funding of $2.4 billion between 2010‑2011 and 2017‑2018, while the Fund is “project driven in an effort to take advantage of emerging economic development opportunities in sectors such as natural resources,” and received funding of approximately $300 million over that same period[7] (forecast spending of $50 million per year over the next three fiscal years).[8]

In the context of these programs, “the Department does not provide services directly to clients. Instead, it enters into contribution agreements with Indigenous organizations”[9] (agreement holders). Furthermore, under the Strategy, “agreement holders can deliver services directly to clients or enter into sub-agreements with other Indigenous organizations to deliver services on their behalf. Under the Fund, agreement holders may put in place sub-projects or agreements with third parties to help deliver training and employment support to Indigenous clients.”[10]

Still according to the OAG, in addition “to occupational skills training, services eligible under the programs include client assessments, employment counselling, essential skills development, and wage subsidies to encourage employers to hire clients.” Moreover, “the Department approves funding to the agreement holders and provides guidance, information, and training to support them in delivering services to clients. The Department is responsible for monitoring the agreement holders’ use of program funds, for measuring and reporting program results, and for adjusting the programs as necessary.”[11]

On 29 October 2018, the House of Commons Standing Committee on Public Accounts (the Committee) held a hearing on this audit. The following witnesses were in attendance: from the OAG were Michael Ferguson, Auditor General of Canada (AG), and Glenn Wheeler, Principal; from ESDC were Graham Flack, Deputy Minister, Leslie MacLean, Senior Associate Deputy Minister and Chief Operating Officer for Service Canada, and Rachel Wernick, Senior Assistant Deputy Minister, Skills and Employment Branch.[12]

Findings and Recommendations

Program Results

Performance Measurement Strategies

According to the OAG, the Department “developed a performance measurement strategy for each of the programs to measure and report on results. However, [the OAG] found that the Department did not fully implement these strategies.”[13]

Performance Indicators and Targets

The OAG also found that the “Department’s performance measurement strategies for the programs contained many performance indicators: 30 for the Strategy and 10 for the Fund. The Department considered 3 indicators to be particularly important for both programs and required all agreement holders to measure them. These 3 key indicators were

• the number of clients served;
• the number of clients who found employment after receiving services, and
• the number of clients who returned to school after receiving services.”[14]

Furthermore, “the Department had targets for the number of clients who found employment after receiving services. … For example, under the Strategy, the Department’s target was for 14,000 to 16,500 clients to find employment each year. However, [the OAG] found that the Department did not establish targets for nearly half of the remaining performance indicators under the Strategy, including its third key indicator (the number of clients who returned to school). However, the Department did establish targets for the remaining indicators under the Fund.”[15]

Another of the OAG’s findings was that the “Department based its employment targets on the results of the Indigenous labour market programs it had in place before 2010.” However, “it did not revisit the targets to ensure that they were still relevant, given changes in the programs and other factors, such as changing economic conditions, between 2010 and 2017.”[16] In particular, “the Department had not clearly defined its employment indicator. It viewed all employment outcomes as the same, regardless of their nature or duration.”[17]

Reporting Program Results

Given “the significant limitations in the Department’s implementation of the performance measurement strategies for the programs, the Department could not meaningfully report on their performance. Although the Department publicly reported employment results for most years for the Strategy and for three years for the Fund, the numbers it reported did not indicate whether clients had found sustainable employment.”[18]

The OAG therefore made the following recommendation with regard to program results:

Employment and Social Development Canada should, in collaboration with Indigenous organizations, measure and report on whether the Aboriginal Skills and Employment Training Strategy and the Skills and Partnership Fund increase the number of Indigenous people who have sustainable and meaningful employment. It should do so by fully implementing its performance measurement strategies for the programs, including clearly defining performance indicators and targets.[19]

In its Detailed Action Plan,[20] ESDC stated that a “new performance measurement strategy with strengthened outcomes, indicators, and clearly defined targets will be developed with Indigenous organizations for the new Indigenous Skills and Employment Training (ISET) Program and implemented in April 2019.” Also, “an enhanced with clearly defined performance indicators and targets [will be] implemented”[21] by June 2019.

Furthermore, Deputy Minister Graham Flack explained that ESDC was “on track to have that new co-developed performance measurement framework in place when the new program launches in April of 2019.”[22] The new ISET program will replace the Strategy, and Mr. Flack added that ESDC would ensure that “the new ISET, like its predecessor ASETS, will also be complementary with the [Skills and Partnership Fund].”[23] As well, the performance measurement strategy will focus on measuring the program objectives to “reduce the skills gap between indigenous and non-indigenous people by 50% and the employment gap by 25%. [The strategy] includes strengthened indicators and enhanced reporting on post-program results.”[24]

Lastly, the Deputy Minister assured the Committee that ESDC is “not going to start spending the new money until we have the framework [strategy] in place and that framework will be in place April 1.”[25]

Consequently, the Committee recommends:

Recommendation 1—On program results

That by 30 June 2019, Employment and Social Development Canada present the House of Commons Standing Committee on Public Accounts with a detailed report on its Performance Measurement Strategy for the new Indigenous Skills and Employment Training Program and the Skills and Partnership Fund, including a list of the chosen performance indicators and their targets; this list should include metrics on employment advancement.

Use of Data to Improve the Programs

Data Completeness and Accuracy

According to the OAG, “the Department did not make sure that all agreement holders provided data on the results their clients had achieved after receiving services. Some agreement holders consistently failed to report their results. Since 2011, four of the agreement holders had reported less than 50% of their results to the Department. Moreover, [their] analysis of the data indicated that the Department did not know whether 22% of all clients who had received services under the programs since 2010 actually found a job or went back to school.”[26] Moreover, “it did not confirm the accuracy of most of the data the agreement holders provided about whether clients found jobs.”[27]

Data Analysis

The OAG also found that “the Department did not analyze program data to identify trends, problems, or good practices that could guide agreement holders to improve their services and results. This represents a lost opportunity for the Department.”[28]

The OAG therefore “analyzed the data to determine whether it contained information that would be useful in managing the program … [and] identified several issues that … merit further analysis by the Department.”

For example, given the high cost of the wage subsidy, “repeated use by some clients raises questions about whether clients who received this service continued to be employed after the wage subsidy ended, or were more likely to find future employment elsewhere compared with those who did not receive the subsidy. Without knowing this, [ESDC] cannot be sure whether the programs are helping clients find sustainable employment or only subsidizing employers without benefiting clients over the longer term.”[29]

Regarding the lack of data, Glenn Wheeler, Principal, OAG, stated that there were two issues: the first pertained to the “data that the department did collect. There wasn’t a lot of work done to assess how reliable, accurate and complete that data was.”[30] The second concerned the fact that “there also exists a lot of other departmental data that could be used to as a proxy for success. A good example of that would be the wage subsidy.”[31]

Program Evaluation

According to the OAG, “because of the lack of data, the Department used some results from its previous Indigenous training programs to evaluate the results of the current programs. This means that the results of the 2015 evaluation were not a good measure of the current programs’ success in helping clients find sustainable employment.”[32]

Thus, the OAG made the following recommendation regarding the use of data to improve programs:

Employment and Social Development Canada should ensure that it has the information it needs to improve the Aboriginal Skills and Employment Training Strategy and the Skills and Partnership Fund, where necessary, by working in collaboration with agreement holders to identify, collect, confirm, and analyze program data.[33]

It is Detailed Action Plan, ESDC stated that it would

• “[a]lign data collection templates & systems with the enhanced Performance Measurement Strategy for the new ISET Program” by March 2019;
• “[c]ommence collection of performance data”[34] by April 2019;
• “[w]ork collaboratively with partners to begin annual performance data analysis and identify potential programming and service delivery improvements”[35] by September 2020; and
• “[analyze] long-term outcomes, key successes and lessons learned from the ASETS and SPF to inform and adjust as needed on-going implementation of the new ISET program”[36] by September 2020.

Consequently, the Committee recommends:

Recommendation 2—On the use of data to improve the programs

That Employment and Social Development Canada present the House of Commons Standing Committee on Public Accounts with: 1) a report on the selected data for the programs’ evaluations by 30 June 2019; 2) a progress report on the analysis of program outcomes and possible improvements to program and service delivery by 31 December 2019; and 3) a final report on this analysis by 30 September 2020.

Program Funding

Funding Formula for the Strategy

According to the OAG, when “the Department introduced the Strategy in 2010, it allocated funds to agreement holders in accordance with the 1999 funding formula that it had used for the program that preceded the Strategy.”[37]

The OAG found, however, that “the Department tried to update its funding formula in 2014, but could not reach consensus with agreement holders on what changes to make. Any changes to the formula would have increased funding for some agreement holders and decreased funding for others. This means that the Department allocated funding to agreement holders as recently as the 2017‑18 fiscal year on the basis of data that was more than 20 years old.”[38]

The OAG therefore made the following recommendation:

Employment and Social Development Canada should allocate funding under the Aboriginal Skills and Employment Training Strategy in accordance with current needs and the capacities of individual agreement holders to achieve results. This should include

• updating the funding formula;
• using up-to-date population and socio-economic data; and
• considering past performance of individual agreement holders.[39]

According to ESDC’s Detailed Action Plan, “Budget 2018 announced new, incremental funding to non-Indigenous peoples, based on updated socio-economic and demographic data. As part of the co-development of the new Indigenous Skills and Employment Training Program, the Department will engage with Indigenous partners to determine the approach to allocating new funding, to be implemented by April 2019.”[40]

The Committee is satisfied with this plan; however, it wants to ensure that the allocation will not focus solely on new funding from the 2018 federal budget, but rather for all funding allocated to the new ISET Program.

Consequently, the Committee recommends:

Recommendation 3—On the funding formula

That, by 31 May 2019, Employment and Social Development Canada present the House of Commons Standing Committee on Public Accounts with a report on the formula used for allocating all funding for the new Indigenous Skills and Employment Training Program among the various communities and regions of Canada.

Duplication with Other Programs

According to the OAG, “the Department identified a risk that services offered under the Strategy and the Fund could overlap with each other, as well as with services under programs offered by other federal, provincial, and territorial organizations that support training for Indigenous people. This could lead to a duplication of services, confusion among agreement holders, and inefficient use of the Department’s resources.”[41] The OAG also found “that the Department had not formally assessed the extent to which the Strategy and the Fund duplicated services.”[42]

The OAG therefore made the following recommendation:

Employment and Social Development Canada should conduct the analysis necessary to determine whether its Indigenous labour market programs overlap with each other. The Department should also complete work to determine whether its labour market programs overlap with those offered by other federal, provincial, and territorial organizations. The Department should also take the necessary steps to address any identified areas of overlap.[43]

In its Detailed Action Plan, ESDC commits to completing “an overview of existing federal, provincial/territorial supports for Indigenous labour market integration and job creation” by December 2018 and analyzing it to “identify opportunities for potential improvements”[44] by April 2019.

The Committee is satisfied with the actions planned by the Department, but questions remain about duplication between federal and provincial/territorial programs.

Consequently, the Committee recommends:

Recommendation 4—On the duplication between the various Indigenous labour market programs

That Employment and Social Development Canada present the House of Commons Standing Committee on Public Accounts with: 1) an overview of all instances of duplication in and between federal and provincial and territorial Indigenous labour market programs by 31 January 2019; and 2) a report including a plan addressing how to eliminate this duplication by 31 May 2019.

Program Management

Guidance to Agreement Holders

The OAG “found that the Department supported agreement holders by providing formal guidance on the programs and templates to help them meet their reporting requirements. [It] also found that Department officials worked with agreement holders to help them develop business and operational plans and to help them ensure that funding proposals contained all the required information.”[45]

Furthermore, “the Department provided support to agreement holders by holding information sessions on topics such as revised reporting requirements and best practices in program implementation. However, the Department offered these sessions intermittently.”[46]

Labour Market Information

According to the OAG, the “Department has known since 2010 that it did not have sufficient labour market information about Indigenous communities. In particular, the Department identified a lack of up-to-date, on-reserve labour market information to support program design, service delivery, and decision making.”[47] ESDC has taken some action to address this problem: new data will be available through a pilot project, but not before 2022.[48]

The OAG therefore made the following recommendation:

Employment and Social Development Canada, in collaboration with Indigenous organizations, should identify and provide labour market information that individual agreement holders need to align their services with demand in the labour market for their particular regions.[49]

According to its Detailed Action Plan, ESDC committed to “further improve available labour market information through an extended survey pilot, beginning in April 2018, to test processes and tools to improve the level of detail and frequency of labour market information for on-reserve communities. The annual results of the survey and skills inventory will be used to inform program design and better tailor client interventions to labour market demand.” As well, ESDC plans to proceed with pilot “continuation in Fall 2018 with up to 15 additional new ISET Program organizations (serving 60 First Nations).” Additionally, the “Department is working to link jobs available on the Job Bank directly with Indigenous organizations. This work will be implemented by April 2019.”[50]

Consequently, the Committee recommends

Recommendation 5—On labour market information provided to agreement holders

That Employment and Social Development Canada present the House of Commons Standing Committee on Public Accounts with: 1) a progress report on the preliminary results of linking the Job Bank with Indigenous organizations by 30 June 2019; 2) a report listing the organizations involved in the labour market information pilot project as of 15 June 2019 and the date on which these organizations will join the pilot project by 30 June 2019; and 3) a report on the results of the survey and skills inventory by 30 April 2020 and 30 April of the three subsequent years.

Administrative Burden

The OAG noted that “the Department did not always make use of the information that it required agreement holders to provide. … In 2011, the Department committed to using the information in these annual reports to assess what was working well in the partnerships that agreement holders formed to deliver services. However, the Department never did this.”[51]

According to the OAG, during “consultations for the renewal of the Strategy, various Indigenous organizations expressed a desire to reduce the administrative burden imposed by the program. … [The] Department reduced the detail required from agreement holders in the plans they submitted to obtain funding, eliminated the requirement for them to submit annual reports, and eliminated the Department’s mid‑year monitoring activity.”[52]

The OAG therefore made the following recommendation:

Employment and Social Development Canada, in collaboration with Indigenous organizations, should assess how the changes it has made to reporting requirements have affected the agreement holders’ administrative burden and the Department’s ability to manage the programs. It should also make any required adjustments.[53]

According to its Detailed Action Plan, ESDC, “has already taken concrete steps in consultation with Indigenous organizations to reduce administrative burden. The Department will continue this collaborative effort and will assess the impact of previous changes in order to further improve the delivery of the programs.”[54] As well, “new financial and administrative reporting requirements for inclusion in new ISET agreements”[55] will be added by January 2019.

Hence, the Committee recommends

Recommendation 6—On administrative burden

That, by 30 April 2019, Employment and Social Development Canada present the House of Commons Standing Committee on Public Accounts with a report on the concrete measures taken to reduce administrative burden in the new Indigenous Skills and Employment Training Program agreements.

Oversight of Agreement Holders

Monitoring of Agreement Holders

The Department’s program officers conduct three types of monitoring: financial, activity and results. According to the OAG, “the Department did not always complete the three monitoring activities within the required timelines.”[56] (Exhibit 6.3 of the OAG report provides more details on this subject.)

The OAG “found that the Department did not use the information it gathered through monitoring agreement holders to explore ways to improve program delivery or to identify systemic issues requiring attention and best practices to share with Department officials or agreement holders.”[57]

The OAG therefore made the following recommendation:

Employment and Social Development Canada should complete all required monitoring of agreement holders within the required timelines and use the resulting information to adjust the contribution agreements and overall programs as necessary.[58]

According to its Detailed Action Plan, ESDC will update “monitoring strategies as part of the risk management framework model renewal”[59] by April 2019. Additionally, the Department plans to develop “an approach for annual review and complete the first review of the new ISET Program monitoring results to identify potential programming and service delivery improvements”[60] by September 2021.

Although the Committee considers these measures to be adequate to address the OAG’s recommendation, it questions the time required by the Department to complete the first review (i.e., September 2021).

Consequently, the Committee recommends:

Recommendation 7—On the monitoring of agreement holders

That Employment and Social Development Canada present the House of Commons Standing Committee on Public Accounts with: 1) its update on the monitoring strategies as part of the risk management framework model renewal by 30 June 2019; 2) a report explaining the length of time to complete the first report on the new Indigenous Skills and Employment Training Program monitoring results by 30 June 2019; and 3) if these explanations are deemed satisfactory, the first report on the new Indigenous Skills and Employment Training Program monitoring results by 30 September 2021. Otherwise, the Committee reserves the right to request this report at an earlier date.

Oversight of Agreement Holders

According to the OAG, many of the “agreement holders used sub-agreements with other organizations to deliver services under the Aboriginal Skills and Employment Training Strategy. Some offered services in remote locations where the agreement holders did not have a presence. As with the agreement holders, sub-agreement holders were required to be Indigenous organizations.”[61]

Specifically, the OAG found the following:

• “The Department had no mechanism by which to learn whether agreement holders had entered into new sub-agreements, whether agreement holders had effectively monitored their sub-agreement holders, or whether program officers had fulfilled all of their related responsibilities, such as visiting any new sub-agreement holders.
• The Department provided a checklist to program officers to help them ensure that sub-agreements included the necessary clauses. Programs officers were to use this checklist to ensure that sub-agreements were consistent with program requirements. However, [the OAG] noted that the Department had not established a way to find out whether this checklist was consistently used or whether agreement holders addressed any deficiencies noted by program officers.
• The contribution agreements gave the Department the authority to audit the sub-agreement holders [and] that the Department had not conducted any audits of sub-agreement holders in the Western and Territories Region. Furthermore, Department officials indicated that the Department did not track the monitoring or auditing of sub-agreements.”[62]

The OAG therefore made the following recommendation:

Employment and Social Development Canada should more fully assess the risks related to sub-agreement holders, identify a strategy to address any identified risks, and implement the required changes.[63]

According to ESDC’s Detailed Action Plan, analysis of the “elements of risk and mitigation strategies associated with the management of sub-agreements, including capacity building”[64] was completed in September 2018 and will be implemented in April 2019.

Consequently, the Committee recommends:

Recommendation 8—On the risks related to sub-agreement holders

That, by 31 March 2019, Employment and Social Development Canada present the House of Commons Standing Committee on Public Accounts with a report on the elements of risk and mitigation strategies associated with the management of sub-agreements.

Program Officer Training

According to the OAG, there were “gaps in the Department’s training for program officers who delivered the programs. For example, while there was mandatory training on grants and contributions as of September 2016, the Department did not address some of the specific requirements related to the Strategy. The Department made training available to program officers on how to manage relationships with agreement holders and monitor sub-agreements, but this training was limited, and it was not mandatory.”[65]

The OAG did not make a formal recommendation on this subject, but did note that addressing gaps in training “could help program officers monitor agreement holders more effectively.”[66]

Thus, the Committee recommends

Recommendation 9—On program officer training

That, by 30 June 2019, Employment and Social Development Canada present the House of Commons Standing Committee on Public Accounts with a report on the mandatory and optional training delivered to Indigenous training program officers.

Conclusion

The Committee concludes that Employment and Social Development Canada’s management of the Aboriginal Skills and Employment Training Strategy and the Skills and Partnership Fund was not sufficient to demonstrate that these programs increased the number of Indigenous people getting jobs and staying employed.

To rectify this situation, the Committee has made nine recommendations for the Department to ensure that its Indigenous training programs achieve their objectives. In his Message from the Auditor General, in the spring of 2018, the AG opined that the lack of progress on the Aboriginal file is an “incomprehensible failure.”[67] In response, the Committee wants all Canadian to know that it takes this opinion very seriously; hopefully, the recommendations laid out in this report are steps in the right direction.

Summary of Recommended Actions and Associated Deadlines

Table 1—Summary of Recommendations and Deadlines

Recommendation	Recommended action	Deadline
Recommendation 1	Employment and Social Development Canada (ESDC) should provide the Committee a detailed report on its Performance Measurement Strategy for the new Indigenous Skills and Employment Training Program and the Skills and Partnership Fund, including a list of the chosen performance indicators and their targets; this list should include metrics on employment advancement.	30 June 2019
Recommendation 2	ESDC should provide the Committee: 1) a report on the selected data for the programs’ evaluations; 2) a progress report on the analysis of program outcomes and possible improvements to program and service delivery; and 3) a final report on this analysis.	1) 30 June 2019; 2) 31 December 2019; 3) 30 September 2020
Recommendation 3	ESDC should provide the Committee a report on the formula used for allocating all funding for the new Indigenous Skills and Employment Training Program among the various communities and regions of Canada.	31 May 2019
Recommendation 4	ESDC should provide the Committee: 1) an overview addressing all instances of duplication in and between federal and provincial/territorial Indigenous labour market programs; and 2) a report including a plan detailing how to eliminate this duplication	1) 31 January 2019; 2) 31 May 2019.
Recommendation 5	ESDC should provide the Committee: 1) a progress report on the preliminary results of linking the Job Bank with Indigenous organizations; 2) a report listing the organizations involved in the labour market information pilot project as of 15 June 2019 and the date on which these organizations will join the pilot project; and 3) a report on the results of the survey and skills inventory	1) 30 June 2019; 2) 30 June 2019; 3) 30 April 2020, 2021, 2022 and 2023
Recommendation 6	ESDC should provide the Committee a report on the concrete measures taken to reduce administrative burden in the new Indigenous Skills and Employment Training Program agreements.	30 April 2019
Recommendation 7	ESDC should provide the Committee: 1) its update on the monitoring strategies as part of the risk management framework model renewal; 2) a report explaining the length of time to complete the first report on the new Indigenous Skills and Employment Training Program monitoring results; and 3) the first report on the new Indigenous Skills and Employment Training Program monitoring results.	1) 30 June 2019; 2) 30 June 2019; 3) 30 September 2021 or earlier if deemed necessary.
Recommendation 8	ESDC should provide the Committee a report on the elements of risk and mitigation strategies associated with the management of sub-agreements.	31 March 2019
Recommendation 9	ESDC should provide the Committee a report on the mandatory and optional training delivered to Indigenous training program officers.	30 June 2019