A. Employment and Monitoring Assessment Report

The Canada Employment Insurance Commission’s EIMAR provides an overview of EBSMs and their results based on selected performance indicators. The key indicators are the number of active claimants served; the number of EI clients who return to employment; total clients served; and the amount of unpaid Part 1 EI benefits resulting from a client’s return to work. Figure 5 illustrates the change in the first three indicators from 2002–2003 to 2012–2013. Figure 6 shows the amount of unpaid EI benefits as a result of a return to employment.[36]

Figure 5 – Selected LMDA Performance Indicators, 2002–2003 to 2012–2013

Source: Canada Employment Insurance Commission, 2012/13 Employment Insurance Monitoring and Assessment Report, Chapter 3, Section 6, Key Performance Indicators, 2014.

Figure 6 – Unpaid EI Benefits ($ Million), 2002–2003 to 2012–2013

Source: Canada Employment Insurance Commission, 2012/13 Employment Insurance Monitoring and Assessment Report, Chapter 3, Section 6, Key Performance Indicators, 2014.

An annual analysis of net impacts and outcomes of EBSMs published in the 2012/13 EIMAR revealed the following results, among others:

1. SD participation increased employment earnings of active claimants. It led to the largest net employment earnings increases among all EBSMs. The incidence of employment of active claimants also increased. It is notable that these net impacts on earnings and employment continuously grew over the five years that followed the end of the claimants’ participation in an SD program. “SD also reduced the use of EI for active claimants.”[37]

2. TWS active claimants increased their earnings over the five years that followed the completion of their intervention. They were also more likely to be employed after benefiting from a TWS work experience. “Their use of EI increased modestly after participation, but mainly in the fourth and fifth years after the program end.”[38]

3. SE participants saw a decrease in employment earnings and incidence of employment after taking part in this measure. The size of the impact diminished over time. However, the analysis provides only a partial picture of SE’s effectiveness. The evaluation does not capture the influence of other factors such as success in business on SE participants’ accomplishments.[39]

4. JCP active claimants increased their employment earnings, and the incidence and duration of employment during the five years that followed their participation. As for the need for EI benefits, this measure decreased both in terms of the amount of EI collected and the number of weeks in receipt of benefits by individuals who participated in a JCP intervention. However, these last results were only statistically significant for the first two years following participation.[40]

5. EAS active claimants[41] saw a decrease in their employment earnings in the first two years following their participation but their earnings started to slowly increase in the third year. Overall, the impact of EAS on earnings was limited. Active claimants had incremental increases in their incidence of employment in year two to five following their participation in EAS. “The use of EI decreased in all years after participation.… Overall, the increases in incidence of employment and the decreases in EI use indicate that active claimants found employment following their participation in EAS-only.”[42] Their employment was also likely to be maintained over the five years post-intervention.

B. Framework for a renewed accountability and evaluation policy for Labour Market Development Agreements

1. Annual provincial and territorial reports on delivery of Employment Benefits and Support Measures

Under LMDAs, provincial and territorial governments report annually on the implementation of their plans to deliver EBSM-similar programming, activities and results. They are also required to develop an evaluation framework jointly with the federal government and subsequently carry out evaluations of their programs to determine their impacts and outcomes. The Hon. Jason Kenney, Minister of ESDC, confirmed to the Committee that there has not been a full evaluation done of all the LMDAs.[43] Even though there is some accountability already built into the LMDAs, many witnesses told the Committee that there are reporting limitations and gaps. Some stated that data collection is neither consistent nor reliable.

Many witnesses criticized annual reports that only provide information on spending, activities and results at a broad level. At this point in time, witnesses stated that it is impossible to make interprovincial/territorial comparisons and that no national picture of labour market programming emerges from annual provincial/territorial reports on LMDAs.

[I]t is nearly impossible to know the true impacts of the nearly $2 billion in annual LMDA training expenditures. The data available for the amount of money invested through LMDA is currently and has always been very vague.[44]

Mathew Wilson
Canadian Manufacturers and Exporters

I would argue that the [EI] monitoring and assessment report is probably the best place for information … because it’s the only place you can try to get consistent data. When I’ve gone back and tried to look at the annual reports that the various provinces are submitting to Ottawa, it’s frankly been hard to tell what the outcomes are.[45]

Tyler Meredith
Institute for Research on Public Policy

At least one business association has worked hard to garner information about LMDA-funded programs for its members by visiting ESDC and provincial and territorial websites, writing to ministers involved, and consulting the monitoring and assessment report. Despite doing this, they have found it virtually impossible to get a concrete picture of what programs are offered using LMDA money that may apply in their industry, let alone how well that money is being spent.[46]

Judith Andrew
Canada Employment Insurance Commission

Witnesses who suggested improving the evaluation process indicated that there is a need for better and more specific data to be collected. Without the proper data, a thorough program evaluation cannot be completed. Some stated that this issue needs to be resolved in the next generation of LMDAs. A few witnesses advocated for the creation of a pan-Canadian research institute that would be dedicated to collecting and analyzing data related to labour market programming and policies in the renewed LMDAs as well as labour market information. This is discussed in the next chapter on improving labour market information. On the other hand, the Canadian Federation of Independent Business asked for a “public accounting of how the LMDA funds are used.”[47] Michael Mendelson, Senior Scholar, Caledon Institute of Social Policy, summarized the concerns expressed by many witnesses:

Good evaluation requires third-party objective review where it’s possible, randomized controls, and rigorous statistical and economic analysis. We need to know what is working well and what is not, what makes financial sense and what does not, not as a way of attaching blame but as a way of improving our programs.

The federal government should be requiring high standards for evaluation and should be aggressively promoting best practices.[48]

Michael Mendelson
Caledon Institute of Social Policy

2. Labour Market Partners’ Forums

Barbara Byers, secretary-treasurer, Canadian Labour Congress, suggested that accountability could be strengthened in three areas. First, by establishing a “Labour Market Partners Forum with representation from key stakeholders including government, labour, employers, education and community organizations;” second, by expanding performance indicators “to measure the number of participants receiving a certificate or credential, the duration of employment and earnings for participants returning to the labour market;” and third, by requiring that annual LMDA reports by provinces and territories be submitted to provincial and territorial legislatures and include “reporting requirements which would allow for comparison of outcomes across jurisdictions.”[49]

The recommendation to create labour market partners’ forums was supported by many witnesses; some of whom argued that the current lack of collaboration and coordination between and among federal and provincial/territorial partners and other stakeholders could be corrected by the establishment of these forums. Others expressed a need for a federal labour market partners’ forum to play a very broad role, from identifying successful training and gaps in labour market programming to the establishment of spending priorities. Several witnesses provided testimony on this topic:

My second point relates to the limited extent of collaboration and coordination between and among federal and provincial partners. For various reasons, since the introduction of the LMDAs in the 1990s, transfers and policy-making have proceeded on a bilateral basis between the federal government and each province or territory. While this is not unique to the area of labour market policy, it has in many respects inhibited responsiveness to labour market concerns at the national level.[50]

Tyler Meredith
Institute for Research on Public Policy

A permanent federal labour market partners’ forum should be established to contribute to the successful training initiatives and to identify other labour market measures that are needed as part of a new Canadian job strategy.

…

Provinces or territories without a labour market forum should be required to establish a forum with one of its responsibilities being an annual review and advice on how the LMDA and LMA spending priorities are made.[51]

Cammie Peirce
Unifor

In discussion with labour stakeholders, it is clear that consultation is key. To that end, the labour side feels very strongly that labour market partners forums be established in all jurisdictions, with representation from government, labour, employers, education training providers, and community organizations.[52]

Mary-Lou Donnelly
Canada Employment Insurance Commission

As an alternative, some argued for a greater role to be given instead to the Forum of Labour Market Ministers which currently engages “in intergovernmental collaboration on issues relating to Canadian labour markets, Canadian employers, and Canadian workers.”[53] Donna Wood, Adjunct Assistant Professor, University of Victoria, appearing as an individual, suggested that the federal, provincial and territorial governments work together to reform and expand the Forum of Labour Market Ministers with a mandate to act as a multilateral, pan-Canadian intergovernmental forum responsible for consulting on and determining all aspects of employment training and policy in Canada. She believes that the Forum of Labour Market Ministers Secretariat should be made permanent and that it should build formal linkages with businesses, unions, communities, experts, Aboriginal organizations and other intergovernmental forums (e.g., the Council of Ministers of Education).[54]

Witnesses would like to see the renewal process of LMDAs include an overhaul of the monitoring and governance mechanisms, which would go beyond bilateral policy-making to actually enable provincial and territorial labour market partners’ forums and an active pan-Canadian forum for planning, priority setting, and intergovernmental collaboration and innovation. These mechanisms would be permanent and meet on a regular basis.

RECOMMENDATION 5

The Committee recommends that the federal government with provincial and territorial governments consider expanding the mandate of the Forum of Labour Market Ministers given the needs of the evolving labour market.

Michael Mendelson, Senior Scholar, Caledon Institute of Social Policy, stated that “Ottawa is the steward of these funds [EI funds] and has the right to insist on more accountability on behalf of EI contributors.… [G]overnments should work together to develop uniform reporting standards so a national picture can be obtained.”[55]Many witnesses agreed with this recommendation, for instance:

In the recent debate about the Canada Job Grant, there was a lot of discussion on who is responsible for training. Let me answer. We all are, and the solutions are local, provincial, and national. This is why we strongly support a federal government that sets standards and requires clear and tangible deliverables for the funding it gives.[56]

Serge Buy
National Association of Career Colleges

While we recognize that labour market needs differ significantly from region to region, we believe that there needs to be national standards and some type of mechanisms in place that would result in better sharing of information among jurisdictions with regard to priorities, plans, and result.[57]

Joyce Reynolds
Restaurants Canada

Many witnesses asked for the development of accountability standards that would guide the production of provincial and territorial annual reports on LMDAs. Based on the testimony, these reports should be more transparent, outline performance measurement and outcomes, and provide for comprehensive public accountability. The standards should allow for comparable high quality annual reporting on the training programs and other activities funded by LMDAs.

RECOMMENDATION 6

The Committee recommends that the federal government in collaboration with provinces and territories develop a coherent set of guiding principles and accountability standards to be included in the new generation of Labour Market Development Agreements.

Other witnesses told the Committee that there should be enough flexibility in reporting on outcomes to allow for the collection of data coming from various sources. Best practices and data collected by service delivery organizations, business and labour should be channelled up to the provincial and territorial departments that are signatories of LMDAs to enhance the sharing of information and annual reporting. In addition, witnesses mentioned to the Committee that the role of the federal government should be to set a national vision and broad operational parameters to guide reporting on outcomes.

We are unanimous in our belief that the provinces and territories are better equipped to be responsive to regional and local labour market issues and that the role of the federal government is best suited to set the vision and the broad operational parameters. At present there is no Canada-wide framework on goals, objectives and measures.… Each training agreement has different accountability provisions, making it almost impossible to paint a pan-Canadian picture to better inform the policy realm. Under these agreements there are no formal ways for business, labour, or the CBT [community-based employment and training] agencies to provide consistent and meaningful data to the system it is meant to serve.[58]

Chris Atchinson
Canadian Coalition of Community-Based Employability Training

Make information on all LMDA-funded programs, provincial and federal, and the results of these programs easily accessible for review and sharing by employers, allowing for as much flexibility as possible in labour market development agreements in order to accommodate the regional and sector-specific needs and opportunities.[59]

Bard Golightly
Canadian Home Builders’ Association

Many witnesses raised the Quebec model as an example of active participation by employers, workers and service delivery organizations in the overarching labour market development strategy of the province; a best practice in terms of partnerships. Alain Noël, Professor, Department of Political Science, Université de Montréal, testifying as an individual, described the Quebec model:

Quebec has had its own way of using the agreements and it has proved fruitful. In 1997, following agreements signed with the federal government, the Government of Quebec created Emploi-Québec, a complex structure that brings together all of the partners, employers, unions, the education sector, community organizations, regions, as well as committees that focus on particular needs, such as those of youth, disabled persons and those who are being released from jail.

Emploi-Québec managed the funds it obtained through the Labour Market Development Agreements. It also managed additional funds from the Quebec government to allow persons who were not entitled to employment insurance benefits to have access to training through labour market integration programs.…

The very systematic studies carried out by Emploi-Québec demonstrated that the program had very beneficial effects. Labour market integration programs worked for people who, for instance, were receiving employment insurance benefits. The difference was even greater for people who were receiving social assistance. They were the ones who benefited the most from these programs.[60]

Alain Noël
As an Individual

Many witnesses praised the Quebec model and suggested that as a best practice it could guide the development of labour market partners’ forums. The role of employers in the Quebec model and other best practices are explained in Chapter 6.

The Quebec model does work well. Emploi-Québec sits down at the table with people from business, education, from ministries and from the community sector. We all sit down to work together in order to respond to the challenges of the labour market and craft an action plan.[61]

Monique Sauvé
Réseau des carrefours jeunesse-emploi du Québec

Another element is that different provinces have different approaches to partnerships. Quebec, of course, is the leader on that one, in how their labour market partners’ council works.[62]

Donna Wood
As an Individual

The Hon. Jason Kenney, Minister of ESDC, further raised the possibility of providing a “pay-for-performance bonus” to a provincial/territorial government or for a specific program that is effective in delivering savings to the EI fund by helping EI claimants quickly return to work. However, the Minister acknowledged that at the present time the performance measures are insufficient and do not provide the kind of data needed to determine which provinces/territories or programs should be offered a performance bonus for saving dollars to the EI fund.[63]

Jean-Denis Fréchette, Parliamentary Budget Officer, Library of Parliament, told the Committee that, as a result of the lack of good data, reliable, and more granular labour market information, it is very difficult to evaluate labour market training programs and determine their influence on relieving labour market pressures associated with labour shortages and skills mismatches in certain regions and/or occupations. The current data can only allow for an examination of the impact on labour supply.[64] The next chapter will discuss in more detail the concerns related to labour market information and make recommendations to improve the latter.