INTRODUCTION

According to the Office of the Auditor General of Canada (OAG or the Office), the Canada Revenue Agency (CRA or the Agency) “processes more than 30 million tax returns annually. It also administers and collects business payroll payments and goods and services taxes. The Agency’s call centres are meant to give individuals and businesses timely, accurate information about their taxes, credits, and benefits.”^[1]

Moreover, the “Agency’s online services and telephone call centres are the primary ways for the public to obtain tax information. The Agency’s call centres help callers with specific questions. They are also the main source of information for those who do not have Internet access, those who are uncomfortable using computers, and those who cannot find answers on the Agency’s website.”^[2]

In the fall of 2017, the OAG released a performance audit that focused on determining “whether [CRA’s] call centres provided Canadian taxpayers with timely access to accurate information.”^[3] Regarding methodology, the OAG

• examined calls received on the call centres’ lines for addressing questions from individuals and businesses, and questions about benefits;^[4]
• defined an answered call as either one that was answered by an agent or one that was answered by the Agency’s automated self-service system and lasted at least a minute (to evaluate whether taxpayers had timely access to information through the call centres);^[5]
• examined whether callers reached an agent on the first attempt;^[6] and
• examined whether the responses that agents gave reflected relevant tax legislation correctly (to evaluate whether agents gave callers accurate information).^[7]

It should be noted that for this audit, the Office “did not examine the accuracy of information given by the automated self-service system” or “how easy it was to find information on the Agency’s website or the quality of that information.”^[8]

On 30 November 2017, the House of Commons Standing Committee on Public Accounts (the Committee) held a hearing on this audit.^[9] From the OAG were Michael Ferguson, Auditor General of Canada, and Martin Dompierre, Principal.^[10] From the CRA were Bob Hamilton, Commissioner of Revenue and Chief Executive Officer (the Commissioner); Frank Vermaeten, Assistant Commissioner, Assessment, Benefit, and Service Branch; and Gillian Pranke, Director General, Call Centre Services Directorate.^[11]

FINDINGS AND RECOMMENDATIONS

A. Access and Timeliness

According to the OAG, between “March 2016 and March 2017, individuals and businesses made more than 53.5 million telephone calls to the Canada Revenue Agency about taxes or benefits. During the period covered by the audit, the Agency’s Call Centres Services Directorate managed the nine call centres. The Directorate had a number of telephone lines, including those for addressing questions from individuals, from businesses, and about benefits. Depending on their questions, callers could respond to recorded voice prompts and select from a variety of options to hear general information, and in some cases, to get account-specific information.”^[12]

Additionally, CRA “had a traffic team in each of its nine call centres. The teams were tasked with ensuring that wait times to speak with an agent did not exceed two minutes. One of the ways the traffic team accomplished this was by blocking calls or directing them to the automated self-service system when the wait time approached two minutes. Blocked calls were any calls that did not reach either an agent or the automated self-service system. Callers who did not succeed in reaching an agent needed to redial the Agency if they still wished to speak with one.”^[13]

The OAG notes that CRA’s Taxpayer Bill of Rights “describes 16 taxpayer rights and the treatment to which taxpayers are entitled when they deal with the Agency. These include the right to complete, accurate, clear, and timely information.”^[14]

Overall, the OAG found that “between March 2016 and March 2017, the Agency answered only 36 percent of the calls it received—32 percent were answered by an agent, and 4 percent went to the automated self-service system and lasted at least a minute” and that on average, “callers made three or four call attempts per week. However, even after several attempts, callers did not always reach either an agent or the automated self-service system.”^[15]

Furthermore, the OAG reported that the “remaining 64 percent of calls were unanswered calls: 54 percent were blocked (that is, received a busy signal or a message to go to the website or call back later), and 10 percent were directed to the automated self-service system, but the caller hung up in less than a minute. Compared with similar tax administrations in other countries, the Agency blocked more calls.”^[16]

Finally, specifically regarding blocked calls, the OAG contends that CRA did that so “it could meet its service standard for agent wait times. Depending on the telephone line, the service standard stipulates that 75 to 80 percent of the time, callers should wait no more than two minutes to speak with an agent. Because the Agency blocked calls, [the OAG] found that on average, in the 2016–17 fiscal year, callers made three or four call attempts per week. Even so, their calls were not always answered.”^[17]

Therefore, the OAG recommended that CRA should

• “review how it manages its incoming calls to improve access by considering practices such as increasing their maximum wait times to speak with an agent; and
• consider giving callers information on call wait times to access an agent so they can decide if they prefer to wait, use self-service options such as the website, or call back later.”^[18]

The CRA agreed with this recommendation stated the following in its Detailed Action Plan:

• Based on the findings of recent pilot projects, the Agency has increased queue wait times to 5 minutes, which allows “more callers to have access to agent queues;”
• In the summer of 2017, the Agency signed “a commitment to transition its call centres to a new telephony platform as part of the Government of Canada’s Contact Centre Transformation Initiative;”
• Between 2017 and 2018, the “CRA will consult Canadians on wait time preferences and to identify opportunities to further expand self-service options in its overflow system” (which has self-service options); and
• “[Once] the Hosted Contact Centre Services (HCCS) solution is implemented (spring of 2018 businesses, summer of 2018 for individuals) the CRA plans to transform its traffic management strategy; and, it will “allow for the introduction of a national queue which will result in a more effective distribution of calls across the CRA’s national network.”^[19]

When questioned about this matter, the Commissioner explained how a test case conducted by the Agency found that when callers were allowed to wait longer than the current standard of two minutes (thereby allowing more to get through), it had positive results. Furthermore, he offered the following explanation regarding how a new technology platform will improve matters:

[Once] we get the new technology and are able to tell people upfront what their wait times will be, we'll be operating the business in a slightly different way. I think that may give rise to a different kind of service standard.^[20]

Given how deeply this situation can affect Canadian taxpayers, the Committee was shocked and disappointed to learn of the huge challenge many of them face in merely being able to connect to an agent. Therefore, notwithstanding the Agency’s plans to improve matters with new telephony technology, the Committee recommends

Recommendation 1 - Regarding improving access and addressing wait times

That, within 120 days of the tabling of this report, the Canada Revenue Agency provide the House of Commons Standing Committee on Public Accounts with a report detailing what progress has been made with regard to 1) reviewing how it manages its incoming calls to improve access; and 2) giving callers information on call wait times to access an agent so they can decide if they prefer to wait, use self-service options such as the website, or call back later.

B. Accuracy

Table 1, below, shows the difference between the results of the Agency’s internal quality assurance and testing protocols versus those done by external organizations (namely, the OAG and the Canadian Federation of Independent Business).

Table 1 – Comparison of Testing Results by Source

Source	Telephone Lines Tested	Percentage of Inaccurate Response
Canada Revenue Agency National Quality and Accuracy Learning Program	All	2016: 6% 2015: 7% 2014: 7%
Audit, Evaluation, and Risk Branch	Business and individual	2016: 20%
	Business	2013: 17%*
Canadian Federation of Independent Business	Business	2017: 31% 2013: 24%
Office of the Auditor General of Canada	Business, individual, and benefits	2017 overall: 29% Business and goods and services tax (GST): 31% Individual: 36% Canada child benefit: 21%

Source:  Office of the Auditor General of Canada, Call Centres—Canada Revenue Agency, Report 2 of the 2017 Fall Reports of the Auditor General of Canada, Exhibit 2.4.*The OAG noted that CRA’s “Study of the Accuracy of Telephone Responses to Non-Account Specific Business Enquiries, dated May 2014, omitted one of the nine questions because it was considered a significant outlier. Including all nine questions in the report would have resulted in a 25 percent inaccuracy rate.”

Notably, the OAG’s test results found that CRA’s call centre agents gave wrong information to callers almost 30% of the time, similar to results found by other assessments over the past five years.^[21] Also, the OAG noted that “the Agency’s National Quality and Accuracy Learning Program did not test the accuracy of agents’ responses effectively or independently. Therefore, the results of its tests were unreliable.”^[22]

Lastly, the Office also “found that the Agency knew the individual training needs of its call centre agents. However, agent training records were not centralized, which made it difficult for headquarters to follow up on whether the training had been provided.”^[23] Also, “the Agency was inconsistent in assessing agents’ training results. Agents were tested after the initial training, but not always after subsequent training.”^[24]

Thus, the OAG recommended that CRA “should ensure that its quality assurance practices generate more effective results in order to improve accuracy, identify opportunities for continuous improvement, and identify and monitor training needs.”^[25]

In response to this recommendation, in its Detailed Action Plan, the Agency agreed and stated the following:

• Implementation of a new approach to training began in the fall of 2017 and is being rolled out across all call centres;
• “In October 2017, improvements were made to the suite of agent desktop tools allowing agents to access information more quickly;”
• “New call recording and listening tools available under HCCS will strengthen CRA’s ability to monitor quality and accuracy;”
• “The new skills-based routing functionality available with HCCS will allow for improved accuracy as call topics can be directed to specific agent groups rather than all calls flowing through front line “general” agents;” and
• A new quality and accuracy evaluation tool that “will increase the consistency of listener evaluations by reducing ambiguity and complexity” will be adopted by the summer of 2018.^[26]

When questioned on why it took the Agency so long to properly address the problem of CRA agents providing accurate information, given that it had been aware of it for some time, the Commissioner’s response was as follows:

As for our training system, every agent gets six weeks of training when they go through. This is something we look at on a continuous basis. We check whether we're seeing any issues and whether we need to make adjustments. We have recently made significant changes that we're rolling out. As [Assistant Commissioner Frank Vermaeten] mentioned, we have a new gating process for how agents get into being on the lines, as well as a new nesting function whereby they work together to improve. We are making improvements right now before the technology comes. We didn't decide after the Auditor General's report to invest in new technology—that decision was taken ... and it will arrive next year.^[27]

Again, the Committee cannot stress enough the paramount importance of the Agency’s ability to provide accurate information to its callers; thus, it recommends

Recommendation 2 – Regarding improving the accuracy of responses

That, within 120 days of the tabling of this report, the Canada Revenue Agency provide the House of Commons Standing Committee on Public Accounts with a report detailing what progress has been made with regard to ensuring that its quality assurance practices generate more effective results in order to improve accuracy, identify opportunities for continuous improvement, and identify and monitor training needs.

C. Reporting Results

According to the OAG, when the Agency measured caller access, it “disregarded how many calls a person needed to make before they got through to the Agency’s call centre services. For example, the Agency considered that a caller had 100 percent access to call centre services even if the caller had to try four times, experiencing three blocked calls before getting through. Other countries, such as the United Kingdom, Australia, and New Zealand, reported performance measures that either reported or accounted for all calls, including blocked calls. By contrast, the Agency did not report on or account for blocked calls.”^[28]

Additionally, the OAG reported that between “the 2012–13 to 2016–17 fiscal years, the Agency’s service standard was to have its agents answer calls within two minutes between 75 and 80 percent of the time” and found that “this applied only to calls that were routed to agents, not to blocked calls or calls redirected to the automated self‑service system (with no option to speak with an agent). The Agency achieved its service standard by blocking or redirecting calls to the automated self-service system with no option to speak with an agent.”^[29] This may have given callers the impression they could speak with an agent, but in reality, over 60% of calls were blocked or automatically sent to the automated system, which did not give the callers the choice to speak with an agent.^[30]

Consequently, the OAG recommended that CRA “should assess and improve its performance indicators and its reporting on the accessibility, accuracy, and timeliness of its call centre services.”^[31]

The Agency agreed with this recommendation, and stated the following in its Detailed Action Plan:

• In annual reporting documents, the CRA “now provides data on the number of calls answered, abandoned in the queue, served in the [automated overflow] system] or presented with a busy signal;”
• “Adjustments to service standards will be assessed for 2019-20 based on consultations and experience with the new technology;”
• “Beginning in April 2018, the CRA will report internally on caller satisfaction. This data will be incorporated into the Agency-wide feedback model;” and
• Once implemented, CRA will begin to use the new quality and accuracy evaluation tool available under the Hosted Contact Centre Services solution to measure accuracy levels.^[32]

When the Committee questioned Agency officials about why this information was not accurately reported to Parliament and Canadians—and whether or not the Agency “misled” them—the Committee did not receive a clear response; instead, the Commissioner acknowledged that the Agency must do better and that it has begun to implement some of the measures outlined above.^[33] As a Committee mandated to ensure the accountability of the Government of Canada, it must ensure that federal organizations report their results accurately to Canadians; as such, the Committee recommends

Recommendation 3 – Regarding improving the accuracy of reporting

That, within 120 days of the tabling of this report, the Canada Revenue Agency provide the House of Commons Standing Committee on Public Accounts with a report detailing what progress has been made with regard to assessing and improving its performance indicators and its reporting on the accessibility, accuracy, and timeliness of its call centre services.

ADDITIONAL COMMENTARY

During the hearing, the Committee was disappointed with the Agency’s responses to its questions, especially those that could be reasonably categorized as “basic.” Overall, although the Committee acknowledges the Agency’s plans to implement new telephony technology and improved training for agents, it still has concerns. As such, the Committee will invite the CRA back for another hearing to provide updates regarding improvements to caller wait times and access, accuracy of responses provided, and more accurate public reporting of its results. As part of this amplified follow-up process, the Committee recommends

Recommendation 4 – Regarding improving wait times and response accuracy

That, within 120 days of the tabling of this report, the Canada Revenue Agency provide the House of Commons Standing Committee on Public Accounts with the results of a study that uses a sampling period of 10 business days during the normal hours of operation, using a random sample of call centre agents from across the country, that measures 1) the percentage of calls that connect with an agent, that connect with the automated overflow system, or that receive a busy signal; and 2) the percentage of accurate responses provided.

CONCLUSION

The Committee concludes that the Canada Revenue Agency did a poor job of providing taxpayers with reasonable access to call centre agents and providing accurate information to those callers. And, alarmingly, although “the Agency reported that it met its targets for both access and timeliness, its performance measures were incomplete and its call centres’ results were overstated.”^[34] Consequently, the Committee has made four recommendations to help ensure the Agency will work diligently towards improving these matters.

SUMMARY OF RECOMMENDED ACTIONS AND ASSOCIATED DEADLINES

Table 1 – Summary of Recommended Actions and Associated Deadlines

Recommendation	Recommended Action	Deadline
Recommendation 1	The Canada Revenue Agency should provide the Committee with a report detailing what progress has been made with regard to 1) reviewing how it manages its incoming calls to improve access; and 2) giving callers information on call wait times to access an agent so they can decide if they prefer to wait, use self-service options such as the website, or call back later.	120 days after the tabling of this report
Recommendation 2	The Agency should provide the Committee with a report detailing what progress has been made with regard to ensuring that its quality assurance practices generate more effective results in order to improve accuracy, identify opportunities for continuous improvement, and identify and monitor training needs.	120 days after the tabling of this report
Recommendation 3	The Agency should provide the Committee with a report detailing what progress has been made with regard to assessing and improving its performance indicators and its reporting on the accessibility, accuracy, and timeliness of its call centre services.	120 days after the tabling of this report
Recommendation 4	The Agency should provide the Committee with the results of a study that uses a sampling period of 10 business days during the normal hours of operation, using a random sample of call centre agents from across the country, that measures 1) the percentage of calls that connect with an agent, that connect with the automated overflow system, or that receive a busy signal; and 2) the percentage of accurate responses provided.	120 days after the tabling of this report