PACP Committee Meeting

     Good morning, everyone. This is meeting number 12 of the Standing Committee on Public Accounts.

    I'll just remind committee members, and those in the audience today, that we are being televised. If you could please mute your phones or other communication devices, that would be appreciated.

    As well, committee members, at the close of this meeting we will take some time to go in camera for committee business. Please bear that in mind. It may be that a couple of rounds of questioning would be cut short, but there should be ample time for us to question our Auditor General today.

    Today we are considering the main estimates for 2016-17: vote 1 under the Office of the Auditor General; and the report on plans and priorities for 2016-17 of the Office of the Auditor General of Canada.

    We have witnesses today from the Office of the Auditor General to help us with this study. We have our Auditor General, Michael Ferguson; Sylvain Ricard, assistant auditor general; Ronald Bergin, principal; and Susan Seally, principal.

    I would invite Mr. Ferguson, the Auditor General of Canada, to begin, please.

    Mr. Chair, we're pleased to have this opportunity to discuss our 2014-15 performance report and our 2016-17 report on plans and priorities. With me today is Sylvain Ricard, assistant auditor general of corporate services and chief financial officer; Susan Seally, principal of human resources; and Ron Bergin, principal of strategic planning.

    As the legislative auditor of the federal government and the three northern territories, we support Parliament and territorial legislatures by providing independent and objective assurance, advice, and information about government financial statements and the management of government programs. The commissioner of the environment and sustainable development carries out our mandate related to the environment and sustainable development.

     We conduct all of our audits in accordance with Canadian auditing standards. We subject our audits, and our system of quality control, to internal practice reviews and to periodic external reviews, to provide assurance that you can rely on the quality of our work.

    In addition to carrying out our audit work, we are engaged in the advancement of legislative audit methodology, accounting and auditing standards, and best practices. We also work internationally, supporting projects funded by Global Affairs Canada, sharing knowledge, building professional capacities, and promoting better-managed and accountable international institutions.

    As reflected in our financial statements, our net cost of operations was $92.4 million in the 2014-15 fiscal year, of which $77.7 million was provided through the main estimates. We had a budget of 565 full-time equivalent employees, and employed the equivalent of 547 full-time employees.

    With these resources, we delivered 87 financial audits, three special examinations, 25 performance audits, and two case reports under the Public Servants Disclosure Protection Act. This represents all planned audit work except for one performance audit, on sustainable development of commercial fisheries, which was cancelled.

    Our 2014-15 performance report identifies several indicators of the impact of our work, along with measures of our operational performance, which are attached to this statement as appendix A.

     In monitoring the impact of our work, specifically the extent to which federal government organizations addressed the issues and concerns raised in our audits, our targets were met for special examinations and performance audits. Our target was not met for financial audits, though the affected audits represent less than 4% of our total work.

    In the 2014-15 performance report, we note that parliamentary committees reviewed 32% of our performance audits. This finding represents an increase from 24% in the 2013-14 fiscal year, although it is below our target of 65%. We participated in 21 parliamentary committee hearings and briefings on our audit work.

     Among senior managers of organizations subject to performance audits, 59% either agreed or strongly agreed that our audit reports were understandable and fair and added value. This result was below our target of 80% and below recent results, which ranged from 74% to 79%.

    Our analysis shows that most of the remaining respondents were neutral, neither agreeing nor disagreeing that our reports added value, and they identified no particular opportunities for improvement.

    Our measures of organizational performance remained generally positive. In particular, our practice reviews, which serve as a key quality control in our audit methodology, found that our audit reports were appropriate and supported by proper evidence.

    Turning now to our 2016-17 report on plans and priorities.

    Our strategic plan identifies a number of client, operational, and people management objectives that we use to manage the office and direct our continuous improvement efforts. In the 2016-17 fiscal year, we will focus most of our improvement efforts on the following three priorities.

    First, we will improve the governance and management of the office through the implementation of new senior management roles and responsibilities, and improvements to our risk management procedures. These actions should lead to more streamlined decision-making at the most appropriate level in the organization.

    Second, to develop and maintain a skilled, engaged, and bilingual workforce, we will monitor employee participation in our required professional development program, develop and deliver empowerment workshops, and implement second-language learning plans.

    We are committed to providing our staff with the training and development opportunities they need, and to supporting them in meeting the language requirements of their positions.

    Third, to meet our objective of ensuring that we continue to select audits that are likely to have significant impact and value, we will review our audit mandates and the allocation of our audit resources to focus on audits of greatest importance.

    We are also reviewing the performance indicators that we use to manage the office and to report to you on our performance.

    We expect to complete this work in the coming months.

    Tables containing our current performance measures are attached to the statement as appendix B.

    For the 2016-17 fiscal year, we are requesting parliamentary appropriations of $78.5 million in our main estimates.

    Our planned number of full-time equivalent employees is 570. With these resources, we expect to complete more than 85 financial audits, 24 performance audits, and six special examinations.

    In conclusion, my staff and I look forward, in the coming year, to continuing to provide you with products of high quality that add value.

    We thank you for your ongoing support of our work. We would be pleased to answer your questions.

    Thank you.

    Thank you very much for your testimony.

    We'll move into the first round of questions.

    For seven minutes, go ahead, please, Mr. Lefebvre.

    Thank you, Mr. Chair.

    I'd like to thank the witnesses for being here with us today.

    Mr. Ferguson, in your report, you state that 32% of the reports are reviewed by a parliamentary committee and that your target is 65%. That's a fairly significant gap.

    Could you provide some explanations in that regard? Is the 65% target achievable? What accounts for the discrepancy?

    I can only say that those are our numbers. We try to establish how many hearings would be needed in order to discuss our reports with parliamentary committees. Naturally, it's up to the committees to determine whether they consider it important to hear from us on certain audits that we've tabled.

    We welcome the opportunity to discuss our reports and consultations, but, again, it's up to each of the committees to decide whether the subjects are of interest to them.

    Before how many committees have you appeared? Which ones review the reports that you submit?

    It depends on each case, but if you look at the past, we've appeared before a variety of committees.

    There is also our mandate from the commissioner of the environment and sustainable development. Naturally, those reports are submitted to the Standing Committee on the Environment and Sustainable Development

    In addition, we sometimes appear before other committees—notably on defence and indigenous affairs—and before certain committees of the Senate.

    Are you able to measure the extent to which your audit products result in tangible changes to the workings of government? After you've done an audit, can you measure the changes made?

    Finding the right way to assess the outcomes of each of our audits is an ongoing challenge for us. We certainly do follow-up audits, from time to time, on topics that were the subject of past audits. That's one possible approach. However, it's not really a good way to achieve helpful measurements of outcome, given all the time that elapses before we carry out a new audit on the same subject.

    However, it's the only way we've found to determine whether our audits have a particular impact. Departments certainly look favourably on all the recommendations we make in our various audits, and they sometimes prepare action plans to resolve the problems we've identified. But the only way to ensure that the recommendations and improvements are implemented is to do a follow-up.

    Thank you.

    I will give the floor to my colleague Ms. Mendès.

    Thank you very much.

    I'd like to thank all the witnesses for being with us again this morning.

    Mr. Ferguson, I have a question about the targets that you say were not achieved.

    Based on the financial reports, it looks like you did not achieve all your objectives for management audits and financial audits. Is there a reason that you weren't able to achieve those targets? Is it because the departments themselves had trouble answering your questions in a timely manner? Are there other explanations for this situation?

    There are two audits, in particular—financial and pension plan audits—for which the information often is not prepared within the time needed for the audit to be carried out in accordance with the audit schedule. The entities involved are the reserve force pension plan and the Canadian Forces pension plan.

    We sometimes face the same type of challenge with a few crown corporation audits in the three northern territories. As you know, we also perform the auditor general function for those territories. That's another situation where it's often difficult to get information in a timely manner.

    It is possible to give us a list of all the departments that did not respond, that remained, shall we say, neutral, or that did not say yes or no to your recommendations? Would it be possible to have that list, further to your performance audit and management audit reports?

    I see two aspects to your question.

    The departments generally agree with all our recommendations and prepare action plans to resolve the issues we found.

    In my opening statement, I referred to the results of a survey of departments and organizations. They stated that our recommendations added value and that the audits were an important exercise. Some of the responses in those cases were neutral. These were simply evaluations given by the departments audited by our office—their assessment of the value of those audits.

    Thank you.

     Thank you very much. We'll move to our second round.

     Mr. Godin, you have seven minutes.

    Thank you, Mr. Chair.

    I thank the Auditor General and his colleagues for being here with us.

    You help us fulfill our function. What we need to determine today is whether you use the right methods to give us information. I think you do.

    In your 2016-17 report on plans and priorities, you specify three objectives: "ensure effective, efficient, and accountable governance and management of the Office; develop and maintain a skilled, engaged, and bilingual workforce; and ensure the selection and continuance of audit products likely to have significant impact and value."

    Come next year, how will we be able to assess whether you've attained your objectives? Which measures will you put in place to enable us to determine whether you've achieved them at a rate of 25%, 50%, 100%, or 150%? Which are the measures for assessing the attainment of those three objectives?

    The performance measures for the 2016-17 fiscal year are set out in appendix B to my opening statement. They are the different indicators that we must put in place to measure our progress. It's possible to see the progress we made on those three priorities. For example, it's our objective to give our employees all the training necessary. To that end, we can identify all the available courses and the participants in each of those courses.

    In addition, we have second-language training and development plans covering both official languages. Therefore, we are actually able to see what progress we make in that area as well.

    Appendix B sets out all the indicators that we will put in place.

    Currently, how does the workforce of the Office of the Auditor General compare with that of other departments with respect to bilingualism?

    I can't speak to the situation within other organizations, but I can say that, in view of the requirements, we need to have certain bilingual positions, including supervisory positions. It's important that those positions be held by people who can communicate in both official languages. Our priority, as expressed in our official languages plan, is to ensure that all our supervisors have up-to-date proficiency levels and the skills to communicate in both languages. But I cannot make any comparisons with other departments. Mr. Ricard might be able to address that.

    We are subject to the same standards. All our executive positions are designated bilingual imperative, which means a CBC proficiency profile. This is the case with our management promotion process as well. The same standards apply to us.

    Can you assert that the relevant requirements are completely met at the Office of the Auditor General?

    Yes, it's clear that all the plans, and the training and development programs, are established with a view to training bilingual staff that meets the requirements.

    Thank you.

    Mr. Ferguson, at paragraph 17 of your statement, you say that you're asking to have 570 full-time equivalent employees so you can carry out 85 financial audits, 24 performance audits, and 6 special examinations. In 2014-15, you had 547 employees. Does this mean you're operating with reduced staffing levels? Are you overextended? Do you need more employees to give us similar results and audits on the same scale?

    Are we overextended? I suppose we are, somewhat, and our situation in that regard is similar to everyone else's. Our staff has certainly been reduced as a result of the workforce reduction initiated some years ago. We have reduced the scope of some of our work, and, in recent years, we've reached the limit of our capacities. We're evaluating options in this area.

    But you have nonetheless been able to provide us satisfactory reports and audits in compliance with professional standards. In light of that, I'm wondering why you require that many additional staff at this time.

    For the short term, things are going well, and we can survive and provide high-quality work. Longer term, given the nature of our activities, the situation is different. If the objective is to invest in knowledge and everything that entails, the short-term outlook is not sufficient. In the course of the most recent year, we never entertained the prospect of reducing the quality of our work.

    Our briefing note states that "the OAG accomplished the following activities during 2014-2015".

    How do you choose which audits to carry out?

    There's a way to assess the risks that exist in the various programs. We have a process to identify the various programs within the departments and to identify the risks. Once we have that information, we can carry out an audit. As I said earlier, it's important to bear in mind that half our work consists in financial audits regarding the way a handful of crown corporations prepare financial information. These reports are not part of our performance audits. They're different. For the performance audits, the process is to identify the programs, identify the existing risks within those programs, and select the programs of highest importance.

    Thank you.

     Thank you very much. We'll move now to Mr. Christopherson for seven minutes.

    Very good. Thank you, Mr. Chair.

    It's good to see you all again.

    First up, I just want to comment on paragraph 9 of your opening statements, and underscore this issue of the committee and our performance. I just want to draw attention to the abysmal record in the last few years, in the last Parliament, and how already the numbers are starting to jump. Just doing quick math in terms of what we'll be looking at in the future and the chapters we've picked, I think we're going to come not only close to that 65% we stand a chance of exceeding it, which is just wonderful.

    I know that the Auditor General agrees that the more public airing we can give his findings, the more effect they will have and the more we are doing our job. I just wanted to mention that, because it's a good thing.

    Then, I want to move to page 12 of the opening statements. Mr. Ferguson, under point 2 of the strategic objectives, if you go to the far right-hand column, discussion of targets not met, and go down to the last point in that column, it says:

    There's a footnote, which says:

    What's going on there?

     In general, the information related to those two sets of financial statements wasn't ready for us at the time we did the audit. We've had, and we've expressed, problems with the reserve force pension plan and with being able to do the audit of that in the past.

    These have been long-standing issues we've been wrestling with. I believe it was on the reserve force pension plan that we stopped doing the audit for a period of time. Then we came back in to do another audit.

     Maybe Mr. Bergin could give us the details.

     As the Auditor General has said, we've had issues with two audits, the Canadian Forces pension plan and the reserve force pension plan.

    In the case of the reserve force pension plan, we were not able to issue an opinion on their 2014 financial statements. We are going in, and we are doing limited procedures to get as much assurance as we can—

    Sorry to interrupt, Mr. Chair. Is that the one that was recently implemented, the one we did a hearing on, the reserve pension plan?

    The reserve force pension plan has been around for quite a while. We used to try to do audits on it. We weren't able to express an opinion, so we stopped doing it for a while. Then we came back in, and in 2014 we still weren't able to provide an opinion.

    This is the financial statements of the pension plan for the reserve forces.

    I thought I recalled a report from your office that they were dragging their heels in implementing it.

    We did a performance audit report also on the reserve force pension plan. That was before I took this position, so I don't have the details of it right now. Certainly this has been an issue that we've raised before.

    I'll bet you if we were to look— Anyway, sorry. I apologize.

    Go ahead, sir.

    We're in the process of trying to go back in to do a limited audit, so that we can give some opinion on it, but we have not been able to do that yet. We're still trying.

    It says “statutory deadline”. Help me understand. I think I know, but I'm going to ask you. What's a statutory deadline?

    Often, under the legislation, it says the organization is supposed to make its audited financial statements public by a certain date. That's what we're measuring, whether they get their financial statements prepared and out by that date required by legislation.

    How can they not? How can they just decide to do it or not do it? It's statutory or it isn't.

    I guess in terms of the direct question, that's a question for National Defence to answer.

    Certainly there's a requirement, that's why we measure it, and that's why we highlight it in our report. It's to let Parliament know when there are situations where organizations are not meeting their statutory deadlines for issuing those audited statements.

    Is there no penalty whatsoever for missing a statutory deadline? You just have to acknowledge it and move on?

    I don't have all of the statutes in front of me, but I cannot recall any statute that has a penalty associated with not meeting that deadline.

    Then the best we have is public pressure and public shaming.

    Is there anything we can do? This is an ongoing problem. You've identified it before. It's here again before us. There's nothing to indicate it's going to be any different a year from now. Is there anything we can do as a committee to assist in getting them to meet their legal obligations?

    That's a good question.

    One of the concerns that I have, and that we're trying to sort out is.... This committee, in particular, doesn't know very much about the work we do on our financial statement audit. When we present reports to this committee, it's about our performance audits. Now we've been including the special examinations, as well, but there isn't anything we bring to the committee about our financial statement audit.

    We are trying to put together a report to bring to this committee about our financial audit practice that would help bring these types of issues forward.

    I can take back and think about your comment on this particular issue. In the longer term, we're trying to find a better way to bring this information forward, so there's a regular way you can help us with these types of issues.

    Excellent. Thank you.

    I think I have moments left.

    Just quickly—and it's been touched on by others—on the senior managers' feedback, the 80% to the 59%, I noted your extra notes on page 15. It says “we are analyzing the responses in detail”.

    If you touched on that while I was busy looking at other documents, I apologize, but what's your sense, at this point, of why there is such a dramatic departure?

    Very quickly, we're way out of time.

     I guess we really don't have a reason for the departure. What we're doing with it, though, is identifying that we need to spend more time on our relationship with those organizations so that we can go in and get a better understanding of how they are viewing the performance audits that we're doing.

    Thank you.

    We will now move to Mr. Arya, please.

    Thank you, Mr. Chair.

    I would like to speak on the bilingualism you mentioned in paragraph 14. Let me be very clear: I do support bilingualism. However, for millions of immigrant families, English or French is neither their first nor their second language. They come here, they study hard, they work hard, and they become proficient in one of the official languages. For them, entering the public service is a major, major problem.

     Two weeks back when I went door-knocking, I met an immigrant family where the dad and his daughter were professionals. She spoke to me at length of her experience. She interned in one of the federal departments here, and she was not recruited because her manager told her that she was not bilingual. Now she works with one of the four major audit firms. Not only did we lose a good professional, but it leaves a very bad taste in the mouth of the family.

     I understand that some of your recruitment ads mention something like “bilingualism is an asset”. Maybe the managers are using that to recruit only bilingual candidates. I think unilingual candidates should be recruited and then trained in the second language. With our aim to improve bilingualism, if you start recruiting only among the groups of young professionals who are already bilingual, it creates a major problem.

    I can speak to it, and I can ask Ms. Seally to provide some more details, but I can speak to what our approach is. Our approach is simply to make sure that we are respecting the laws and the policies of the Government of Canada as they relate to bilingualism and bilingualism requirements. That means that people in supervisory positions need to have achieved a certain level of bilingualism as identified through testing through the federal government.

    I understand, sir. I'm not talking of the supervisory level, or the middle management, or the senior management level. I'm talking of young professionals.

    I understand. In terms of other positions, we don't have requirements for most of those positions. Of course, if you're talking support positions, a number of those maybe have to be bilingual, but in terms of audit professionals starting out, we don't require people to be bilingual.

     We do have a capacity within our office to train people on their second language. That's why we've put together these language plans. Perhaps I'll ask Ms. Seally to provide more detail about the hiring process.

     It's exactly the case of what Mr. Ferguson was saying. We hired 55 people last year, and 26 of them were at the entry level. At the entry level, we request that our candidates are proficient in either English or French. In fact, we do not require bilingualism at the entry level of the audit trainee program.

    Yes, but as I said, I know personally a lot of families here. I'm not talking about a middle-aged immigrant like me who may or may not have the capacity to be proficient in both official languages. I'm talking about youngsters who go to universities here. They work hard, but they can't seem to get entry. I'm very happy to note that at least you are recruiting unilingual professionals and then training them to be bilingual. That's good. Thanks.

    Ms. Zahid, you have approximately three minutes.

    Thank you, Chair.

    I would like to ask you how you determine which departments you are going to audit. How do you keep a balance between auditing new departments or revisiting the departments for which you have done audits before?

     In looking across all of our mandate and everything we are supposed to audit, first of all, we have a mandate to audit financial statements of the federal government, the government of the three territories, and crown corporations. Those financial statement audits we do every year on all of those organizations.

    We have a mandate to do special examinations of the systems and practices that exist within federal crown corporations, and we have to do a special examination of each crown corporation at least once every 10 years—at least, of all those listed in the Financial Administration Act, which is most of them. We have to do that special examination once every 10 years.

    There are certain things, then, that we have to do, that we are legislated to do.

    In terms of our performance audits, we look across the federal government. We've divided the federal government, I suppose, into a number of envelopes, whether international affairs, the environment, aboriginal affairs—those types of things—and then we will do what we call a strategic audit plan to try to identify the risks in those areas. It's not so much about looking really at a particular department as about looking at an area and doing a strategic audit plan that looks at all of the departments within that area, looking at the types of programs they offer and the risks associated with them. Then, the ones in which we feel there is the most risk that something needs to be improved will be the ones we select for a performance audit.

    Do you go back to the departments that you have audited before? What is the process of making sure that the recommendations you made when you audited a particular department are being implemented?

    We do follow-up audits. The process for the departments is that when we make the recommendations, they will agree with our recommendations and they will prepare an action plan. From time to time we will do a follow-up audit, but remembering the time frames involved, by the time we finish an audit we probably have to give the department about three years or so to implement those recommendations.

    Then we would need to come back to do an audit, and the work we do in that world is to make sure that we still have an audit level of assurance. That means that we go in and do another full audit, which will include at least some follow-up of the recommendations we made previously. To complete that second audit from the time we choose it until the time we report it could easily take 18 months.

    Counting from the point in time that we present the first audit report until we can present a follow-up audit report, we may be five years out from when we made the original recommendations.

    It's certainly why the role of this committee is particularly important. The follow-up audits are important, but they can't be the only way of making sure that departments are implementing the recommendations. The fact that departments have to come here and have to tell you what they're going to do, or maybe you bring them back in after a year or a year and a half and they explain to you the progress they've made—all of those types of things—are I think a fundamental aspect of making sure that departments are in fact implementing the recommendations.

    Thank you very much. We're well over.

    We'll now move over to the official opposition one more time and go to Mr. Doherty, please.

    Thank you to our guests.

    Mr. Ferguson, you've been the Auditor General since 2011. Is that correct?

    That's correct.

    My questions are going to touch more on organizational behaviour and performance, similar to the line of questioning we had earlier. I'll be the first to say that it looks as though you have been taking some steps to streamline the process and make your organization more effective; however, your key indicators in your performance reports or your performance audits are almost failing, I would say, in terms of the measures you are putting forward. You're far below your targets.

    Could you say that perhaps the process you use in determining the audits may be flawed and may need to be revisited to make the work more relevant and perhaps mitigate the apathetic responses you're getting from some of the organizations you are reporting on?

     In terms of the particular indicator, which is that the people we have done a performance audit on believe that the audit has added value, when we do fall below our target on that, I think it is very much an indication that we need to spend more time understanding, with the organizations that we've done the audit of, why they felt the audit didn't provide value, remembering that this is always a difficult measure because people don't like to be audited. Our audits in particular are not just audits; they are also publicly available and there's public discussion about them. That can make some people uncomfortable.

    It's not always a surprise that some people didn't like being audited and could therefore say that they didn't feel as though the audit process added value. For us, it's a matter of understanding with them, having the relationship with them to understand why they felt perhaps that value wasn't there.

    That leads me to my next point. Going back to strictly business and customer service, ultimately you have customers. If we were to look at just the customer side of it, you are falling below in the customer service side of it in terms of the relevance or the importance that your customers are placing on your reports. I think that a better understanding of what your customers are looking for is very important, as is spending the time organizationally to be able to develop that. It is also important to develop that skill set internally, to be able to go in and actually extrapolate that information and build that relationship with your customers.

    Is there a penalty for non-compliance by your customers? You provide a report, you audit an organization or department and, in your own words, you follow up from time to time. Having one to three years gives any organization the time to choose whether they will follow up or not or act on your recommendations. What is the penalty for non-compliance?

    The only thing we do when we do an audit is to identify where we think things need to be improved. We make recommendations. The department comes up with an action plan to deal with the issues we've identified. I guess their penalty is that if things are not working the way they should, they have a very uncomfortable couple of hours in front of this committee and maybe come back again.

    There's nothing in the legislation that requires people to implement recommendations of the Auditor General, nor in fact should there be, in my opinion, because these are recommendations we're making to the department and the government, and it's up to the government to decide what they're going to do with those recommendations.

    You're here before us asking for money. What are you going to do differently so that we get different results?

    In terms of the performance audits, remember that there are more clients than just the people we're auditing. There is also this committee and the public accounts committee. To the extent that the public accounts committee feels that our audits are adding value, that's our primary measure. We do from time to time sit down. Mr. Bergin will go around and talk to individual parliamentarians to get the sense of whether they feel those audits are adding value. We are constantly working towards making sure that we are selecting the right audits; things that are going to be of interest, things we believe there are risks with. We are also always working on ways to better communicate those audits and trying to make sure that the written communication we have is clear and concise. We're constantly trying to improve both how we select the audits and how we report the audits.

    Thank you very much.

    We'll move to Ms. Shanahan for five minutes.

    Again, thank you to the Auditor General and to your team for being here.

    This session gives us a wonderful opportunity to understand more about the work you do, because indeed, in the various committees I'm involved in, I have the pleasure of seeing you and your work quite often.

    I want to clarify something. Are the departments your customers, or are your customers Parliament and the people of Canada? Do the departments actually pay you for your services? This is why we're here this morning, to look at your estimates for your budget.

     Our budget is provided as a budget to the Office of the Auditor General, and within that we make the decision about what to audit.

    Certainly, what we consider to be our primary client is Parliament. Also, we would consider the crown corporations and the territorial legislatures, of course.

    In terms of the performance audits, again, we see that more as making sure that we have relationships. The term “client” in that sense would be used very loosely. It is about making sure that we have the right relationships. They are certainly important stakeholders in the work that we are doing, but they are not asking us—and therefore paying—to have a performance audit done. We are choosing to do those performance audits. Once we have chosen something, the performance audit is essentially going to be done, whether the department wants it done or not.

    Thank you very much for clarifying that. I think that is really the object of this committee here, to examine your audits and take those conclusions and recommendations in order to better serve the Canadian public. That is really the whole purpose, although I agree that departments...wise managers would be very well advised to take those recommendations in their work going forward.

    I would like to take this on a completely different tack. Although I will be interested in the financial reporting side, as you bring forward that report in the future, a completely different tack would be the environmental auditor. I had the pleasure of hearing her report while sitting in on the environment committee. Is there not a place, a time, or an appropriate occasion for the environmental auditor to report to this committee?

    Certainly, I have talked about that in the past. One of our indicators is the number of our reports that have committee hearings. In the past, we struggled to have hearings on any of the audits that were being produced under the mandate of the commissioner of the environment and sustainable development.

    I think the environment committee has started to have some hearings, but I would certainly encourage this committee to look at the audits and the work that have been done under the banner of the commissioner of the environment and sustainable development. Again, just so everybody is aware, the commissioner is within our office. All of the work that is done by the commissioner of the environment and sustainable development, either audits or studies, is done according to our methodology. What is produced by the commissioner is audits, just like any of the audits that I have presented.

    The commissioner will be presenting a report, a series of three audits, I think, in a couple of weeks, at the end of May. I would certainly encourage this committee to look at those and decide whether the committee is interested in having a hearing on any of those reports.

    Just another quick question.... Again, I am trying not to cross committee eyes too much, but it is important for there to be consistency when we are looking at estimates and when we are looking at end results. Do you have any opinion on cash versus accrual accounting methods?

    I am going to have an opportunity to appear before the government operations committee this afternoon. Certainly, in terms of financial statements, accrual accounting is how financial statements have to be prepared. There is no alternative to accrual accounting for the government's financial statements or the financial statements for crown corporations.

    Certainly, in those sets of financial statements, there is always a statement of cash flows. That is as important a statement as the income statement, the balance sheet, or any other statement. That provides the information about cash.

    I think the question about cash versus accrual is very much more a question of how the estimates are prepared, and what Parliament is voting on. My preference would be to keep everything consistent—the budget is on an accrual basis; the financial statements are on an accrual basis—and therefore to put the estimates on an accrual basis. However, you have to remember that it would mean that different information would be presented than is presented right now. I could talk about that for a long time, but I think I'd better cut it off there.

     We'll save that.

    Thank you. If you wouldn't have cut it off, I would have cut you off.

    Thank you very much.

    We'll go back and go to Mr. Doherty, please.

    Mr. Ferguson, since 2011 how has the organization changed since you've been in place?

    I can tell you how I think it has changed, but all I know is what the organization has been like since I've been there. Maybe Monsieur Ricard, or Ms. Seally, could provide you with some more information.

    There are a few things we have done in particular. One is we have reduced the number of senior assistant auditors general level. At the top level of the organization, we have reduced that. I believe when I started in the organization the executive table was about 16 people. We have a couple of pending retirements, and that will get us to nine. We've significantly reduced that.

    We also went through the strategic operating review process. In that process we were able to identify some financial audits we were doing that we felt didn't need to be done, or didn't need to be done by us, so we made that reduction.

    We identified that we were not meeting our requirements on official languages at all of our supervisory levels. We put in place a strategy to deal with that.

    Again, because we've reduced the number of assistant auditors general, it meant we had to better define the roles and responsibilities of the next two levels in the organization, what we refer to as the principals and the directors. We've had to clearly define what their responsibilities are, who's responsible for delivering audits, and who's responsible for making sure of the quality of audits. Those would be a number of the main things we have done.

    The other things we continue to struggle with are that we do promote performance measurement, and we do promote reporting on performance. We constantly struggle with trying to get a good performance measure for understanding the value that is coming from a performance audit, because right now we have that unsatisfactory measurement of just being able to survey people and ask, “Do you think it added value, or do you not?” We also have the follow-up audits, but that might be five years down the road.

    We are trying to find additional ways to measure the value we bring from performance audits.

    I appreciate your comments, and in no way are my questions meant to diminish any of your impact on that organization, but only to get some more information.

    My final question is, how can we as Parliament help make you more effective, make your team more effective, and help the process?

    I think that's interesting. I think there are probably a few things. Some of them might be legislative. There are some requirements in legislation that we have to do. Whenever there are legislative requirements, that indicates to us that Parliament's interested at that point in time, but it also can tie our hands.

    For example, in the case of the commissioner of the environment and sustainable development, there's a requirement that we look at the sustainable development strategies of the government once every year, and we do an audit of it.

    If we had more flexibility about exactly what types of audits we were going to do, when we were going to do them, that would mean that we wouldn't have our hands tied per se.

    Similarly, as I said, we are required to do a special examination of crown corporations at least once every 10 years. Of course, if we think there's a particular crown corporation that we feel it's risky how they're running their operations, we can go in more often than once every 10 years, but there may be crown corporations that are well run. We can look at indicators that say they are well run, and maybe they don't need us in there once every 10 years. Maybe they do, and maybe they don't. Some things that would provide us with a bit more flexibility would be a starting point.

    Thank you, Mr. Doherty.

    We will now move over to Mrs. Mendès.

    Thank you very much, Mr. Chair.

    I'd like to refer to your report of the plans and priorities for 2016-17. You mention, or you alert us to, the possibility that labour negotiations may have an impact on future performance audits, if I read it correctly. Is this something we should be aware of?

     Mr. Chair, that's just sort of the normal scan of our environment and looking at what possible risks are out there, and of course, whenever you're in a situation of contracts having to be negotiated and put in place, if there should happen to ever be any disruption, that could have an impact on us—not that we are expecting one, but it is a risk we have to be aware of and we have to plan for.

    So it's just one of the elements that you take into account.

    Right.

    Another question I have is about your periodic external reviews of your own work. Who would be the one who does them?

    I guess there are a few things. First of all, we are audited by the Institutes of Chartered Professional Accountants in all of the jurisdictions we operate in terms of the work that we do on our financial statement audits. We operate in Vancouver, Edmonton, Montreal, Halifax, and Ottawa. We have offices in all of those areas, so any of those Institutes of Chartered Professional Accountants could come in at any time and audit the work that we've done through those offices and evaluate whether our audit files are appropriate.

    Once every 10 years there's also a peer review, and this happens at an international level. Auditor general offices around the world have put in place a mechanism to do peer reviews. It happens once every 10 years, so for us that's essentially once a mandate for an Auditor General. The last one that was done on our office was done, I believe, in 2009-10, about that time period. It was led by the Australian National Audit Office and would also have had participation from some other national audit offices.

    We are right now in the process of planning for the next one, which will probably happen in maybe 2018-19, something like that, when we sort of put our name forward and say we're going to want this done. We also actually participate in some of these peer reviews of others. So we will put auditors on peer reviews of other national audit organizations as well.

    Thank you.

    Mr. Harvey, we'll go to you.

     I just have one quick question. Earlier you mentioned about the two audits for the reserve force pension plan and the Canadian Forces pension plan that were cancelled, right? Or they weren't conducted?

    No. They were issued after the statutory date. In terms of the reserve force pension, we weren't able to do that one. I'll ask Mr. Bergin to give you the details.

    We did the Canadian Forces pension plan about a year late, but we have done it, and we have just issued the most recent one, so it is proceeding. It's the reserve forces pension plan where we have the issue, and right now we are trying to do a smaller audit there. We're in the process of trying to do that, but it's not done yet.

    What are the inhibiting factors that are leading you to not be able to conduct that audit?

    The fundamental problem in terms of the reserve force pension is that the quality of the underlying data they have just hasn't lent itself to being audited. It hasn't been up to date. They have been doing a lot of work to try to make sure that the data is clean and appropriate, but that's been the underlying issue.

    We talked about accountability earlier and these statutory deadlines. It's the same idea. You talk about a statutory deadline and how there aren't really any enforcement tools to go along with that. I would compare that to this. There aren't really any tools in place to make departments accountable for providing the information.

    Again, I think part of what's important about this conversation is that very often parliamentarians don't know that we issue audit opinions on sets of financial statements. In terms of something like the reserve force pension, our audit opinion might be a denial of opinion or something, but it would not be a regular opinion such as, “This set of financial statements has been prepared in accordance with generally accepted accounting principles.” Because so many federal organizations get a standard audit opinion on their financial statements, they are able to prepare financial statements that can be audited and are fairly presented, I think it's often taken for granted, but every now and then there are some that either are not able to meet their authority, statutory deadline for example, or don't have the information that's needed to be able to produce a set of financial statements that can be audited.

    I think that those are very much things that parliamentarians should be alert to. This is sort of right now the one way that we have to let people know about that. If we produced a summary report about our work, that might put a little bit more light on it. Again, I think what it very much comes down to is that all we can do is provide the information and say, “This is what the situation is.” Now, there's nothing else that we can do in those instances.

    Thank you very much.

    We'll move to Mr. Christopherson, please.

    Thank you, Mr. Chair.

    I'll just pick up on the last point. This reserve pension still troubles me.

    Through you, Mr. Chair, I would ask if perhaps the analyst would consider doing a check back. I seem to recall dealing with this. The issue at the time was that they were dragging their heels on setting up this plan. It was quite scathing in terms of the things they were supposed to do and hadn't done.

    It's pensions, and it's always worrisome when anything to do with pensions is a problem. Number one, it's big money, as a rule. Secondly, we're talking, at the end of the day, about people's ability to survive when they're retired. It's kind of a big issue these days.

    Perhaps the analyst would be kind enough to check that. I could be totally wrong—I don't trust my memory well—but I seem to recall that.

    Second, it needs to be said, and I don't think it has—if it has, it can be said again—that it's an excellent, excellent report, actually. If you look at it, “target met” is in here far more than anything else, and certainly more than most. Of course, you always put yourself in an unusual position when you come here, because you spend all year pointing out where everybody else is not running things quite the way they should be. Then you're on the line once a year in terms of your operation, and you run the risk of being a hypocrite. But in fact it's quite the opposite. I think you're showing stellar leadership. We know there have been surveys about how your department is one of the best, if not the best, to work in, and that's consistent.

    So credit where credit's due: it's an excellent report in terms of the areas you've met. We appreciate that. We know you're serious about getting things up to speed in a couple of the areas that aren't quite where we'd all like them to be.

    I also want to give a quick shout-out to my friend Daryl Kramp. I'm sure he's not watching, but if he's out there...or if anybody knows Daryl, let him know that accrual finances and auditing are right back again. That was his thing for over 10 years.

    I want to tell colleagues that I've been on this committee, as you know, forever and a day now, and it still is a challenge to understand accrual accounting. At some point, we do need to have a bit of a briefing on it, because I think we still have the issue of purchasing not being in accrual, although that may now be resolved.

    At any rate, I just wanted to give a shout-out to Daryl, who invested a lot of time and effort, and who cared about this committee in a way that I think reflects the gold standard of what this committee is about. Daryl, wherever you are, man, accrual is still with us.

    That said, I do have a serious question. Under your performance measures and results for 2014-15, you state, “Build and maintain relationships with parliamentarians and key stakeholders.” That's a big statement. Then, under your indicators, you state, “Development of a relationship management plan and related objectives.”

    I see that it's in development, but I'm curious as hell as to what a structured “relationship maintenance plan” looks like, especially when you're dealing with types like us. I'd just like some thoughts on how that work is going, and where it's going.

    Again, I think that's relevant to the conversation we had before with regard to the indication of how the people we are auditing are perceiving the value of our audits. In a relationship management plan, essentially at a very high level, what we will do is first identify who the key people are that we need to have the relationships with, and who on our side of the fence should be maintaining that relationship. It can be to understand the issues and to understand issues around audit. It can be various stakeholders. It's not just the people we audit. It could be representatives of the department; the crown corporation; chairs of audit committees, particularly on crown corporations; audit committees of departments, because departments have internal audit committees; and parliamentarians.

    So it's identifying all of those different people we need to have relationships with. Then it's making sure that periodically we are making the contacts we need to make with them to understand how they perceive our work and our value. We're trying to put some structure around that, and it's primarily around who we should be making the contact with, then who should be making the contact, and what types of things we need to make sure we are discussing in those conversations.

    Thank you.

    I'm not certain if there's anyone more on the government side, but we do have Monsieur Godin for cinq minutes.

    Thank you, Mr. Chair.

    Mr. Ferguson, you said that the departments are audited and that you submit your report after that. When your report raises several questions, department officials must spend a couple of difficult hours before our committee—the time it takes to answer our questions. After that, it's on to the next audit, and the next report.

    Through your experience and your many dealings with other legislatures, have you discovered any methodology that could equip us with better tools? You check into things, you share your findings with us, and the department officials come and testify before us. What disappoints me is that, after that, we don't seem to have the teeth to hold the people in authority to account for their actions, with the appropriate consequences.

    The topic of the reserve force pension plan came up a bit earlier. There is still no resolution in that file. Things continue as before, and it's not your fault. Perhaps you don't have the necessary tools to bring pressure to bear in cases like that and impose requirements.

    Could you recommend some specific measures that could be used to correct such situations? We could then see whether Parliament could adopt measures that would equip us better. It seems to me that there's a step missing after the work we do.

    Based on the experience you've acquired, could you suggest measures that we could put in place so we can be even more effective?

    A few indicators could potentially be used. During an audit, we often carry out various analyses. The analysis reveals some indicators—some numbers that point to the problems.

    We are seeking a way to use the indicators. We need to find a way to keep the indicators up to date following an audit. In our audit preparation process, we identify the data needed to carry out the analysis and establish a percentage for a specific program-related aspect. In the future, perhaps the same information can be used for the same analysis. It would not be another audit; rather, it would simply be an update of the indicators identified during the exercise. These indicators might enable us to see whether the improvements requested have been put in place and whether the results are conclusive.

    That could work with organizations that comply with your recommendations. But what about cases like the reserve force pension plan? Do you not have the legal power to require people in positions of authority to comply with your recommendations?

    What should be done with that type of organization, which is not delivering the goods, is not cooperating with you, and is not meeting the deadlines?

    The issue with the reserve force pension plan was not a lack of collaboration. Rather, it was about problems related to the data in the system. We have already carried out an audit concerning that issue. We might find a way to revisit the analyses done in the past.

    I do not have a precise answer for you on this subject, but I might be able to identify a few steps that can be taken to resolve the issue with the reserve force pension plan.

     Mr. Ferguson, what my colleague was trying to get at, using the pension reserve as an example, was that for those other organizations that are not compliant or that are challenging for your organization, what are the things we can do here to make it easier so that they are compliant?

    Again, I think it's the same answer. We do have good co-operation from the departments and organizations we audit, so I don't want to leave any impression that we're saying people don't collaborate with us, because they do. Oftentimes, we will find that as soon as we go in and do an audit, the departments are trying to correct the problem before the audit gets out, because when the audit gets out, they want to be able to say they've already dealt with it. But from time to time, there are issues we raise that take a long time, and because we can't go in and do a follow-up on every audit, or because by the time we do a follow-up audit it takes a while, some of the audits and the results we brought forward can be forgotten.

    Today, the focus has been around this question of the reserve force pension. We can use that as an example. Yes, we can go back to identify if in fact there was a performance audit and if there was a piece of work that was brought to this committee. Then the committee can decide whether there's something they can do about that, given the fact that we still can't provide an audit opinion—there was a piece of work done a while back. Is that something the committee could think about doing?

    To me, the role of the committee is very much about helping to keep the departments to their commitments, right? I think it's changing now, but I think unfortunately the world had moved into what I'll call a “one and done” situation with these types of audits. A department would have to come here, they would talk to the audit for a couple of hours, and then they would know that they would never have to come back and talk about it again, unless we were going to do a follow-up audit.

     I think it will keep the pressure on departments if this committee is keeping more of an eye on specific issues and specific items that have been brought forward and saying, “Okay, you're coming in, you're going to have a hearing, and then you're going to come back in a year or a year and a half, even if there isn't another audit, to tell us about what you've done.”

    Thank you.

    We'll go to Mr. Harvey, please.

    I have another one quick question and then I'm done.

    We used the reserve force pension as an example, and you've just referenced that again, but is that a common issue among departments? Is it a situation you can find yourself in on more than one occasion where a department is unable to provide the information you need in order to perform your audit, due to their data management or their lack thereof?

    I think we talked about that in the context of the performance audits I released last week, but in the context of financial audits, it's actually very unusual to find ourselves in a situation where the underlying information is not sufficient for us to be able to provide an opinion on a set of financial statements. On that side of the work we do in the financial audit side, the data is usually managed in such a way that the quality is there for us to be able to express an opinion on a set of financial statements. It is not usually a problem that we have there.

    In terms of the performance audits, as we talked about last week, certainly there we see a lot more problems with the way the administrative data is captured. We talked about the problem with addresses at Citizenship Canada and the problem with the information about the preparedness of reserve force soldiers at National Defence. On the administrative side, which is what we tend to look at in our performance audits, there are a significant number of problems with data. On the financial audit side, it's much rarer to have those types of problems, but the one instance where we have it is the reserve force pension.

    That said, we're talking specifically right at this moment, I guess.... I mentioned the reserve force pension. For me, I'm way more concerned about the inability of that department to have accurate financial data for the reserve force portion than I am about their ability to give us the data about who is going to be affected by the drug plan. It's not that I'm not concerned about that, because I am, but I think it's a real big problem if that department can't provide the data in order for you to do a proper audit on that pension plan.

     That is certainly a significant problem, a problem, again, that we identified a number of years ago, and I believe Mr. Christopherson is correct, although it was before my time so I'm just going by a vague memory of what I've been told. I believe that we did prepare a report on the issues around the reserve force pension and we stopped doing audits on them for a period of time because there just wasn't any sense. Then we've gone back in to try to see whether the information is auditable yet, and it still isn't. I think you're right to be concerned about that particular situation.

    And in closing, from my point of view, what does that say about us as government? We're saying the data is not auditable so we're going to leave it for a while and then we're going to come back, and if it's still not auditable, then we'll worry about it later.

    That doesn't make any sense to me, especially when you're talking about fiscal accounting on an ongoing basis. To me, this is a very grave problem so I just wanted to highlight that.

    Thank you, Mr. Harvey.

    Mr. Doherty, you had a final comment?

    Absolutely. I echo my colleague from across the way. In today's time it is absolutely troubling that an organization does not follow the general accounting practices and cannot provide this data, and the fact that our Auditor General can't get this information from them is troubling for me. I would expect something to be done rather than waiting another six or seven years for us to follow up on it.

    Mr. Ferguson, what is the percentage of audits that you do, that your organization does do follow-ups on?

    In terms of the performance audit, I don't think I can give an exact percentage. What we do in our planning is two ways of potential follow-up. One is to consider whether we want to do a specific follow-up on exactly what we did sometime in the past. So we will just make a decision. We'll look at audits we've done in the past and we'll ask our auditors to identify any they feel are in need of follow-up. Also though, the other way that a follow-up might happen is sometimes we will go in and we will look at a program, and the program may be similar to a program we've audited in the past, and we may be able to follow up on some of the recommendations we made in the past because of that overlap.

    It wouldn't be called a specific follow-up audit but it may follow up on some recommendations. In the last series of audits that we released, we did a specific follow-up audit on the Governor in Council appointments. We changed slightly the way that we did it. We identified that there were still vacancies on administrative tribunals that make decisions for people, that there were still backlogs in some of the decisions those tribunals were making, so we identified the same types of issues. That was a follow-up.

    In the audit on a prescription drug program for veterans, we followed up on one aspect. We had done an audit in the past on prescription drug programs across multiple departments. This time we just went in and looked at Veterans Affairs and we talked about what they were doing the last time we audited in terms of monitoring the information they have on prescription drug usage by veterans, and we provided a bit of follow-up this time about the progress they had made on that.

    So in terms of the percentage, I can't give you a definite percentage. It's simply a decision that we make in each planning cycle.

    Can you tell me why you don't follow up on all of the audits?

    If we were to follow up on all of the audits, what would end up happening is at some point all of our work would just become follow-up, right? So if we do 10 audits this year, in three years' time if we then have to follow up on all 10 of the audits, then it doesn't give us much room to audit new programs. That's why if we can find some ways to provide some indicators, to look for some indicators of whether departments have been making progress in the meantime, then we would be able to reduce that work and maybe provide the committee with a little bit more information on whether it looks like departments are on the right track, without necessarily having to come in and do another audit.

    Perfect.

     Thank you. I would also say that I think all of our committee have hit on part of our responsibility here. Yes, we can ask the question of the Auditor General, “Why aren't you following up on more? Why aren't you...?” The question really comes back to this committee, that we have the opportunity to hold these folks accountable as well. I think even today we've maybe been given a little prodding towards the reserve pension plan may be coming. We should at least be sitting down and asking how we can hold them a little more accountable, so that the books can be open, so that there's transparency, so that Canadians know, and as Mr. Christopherson said, pensioners will have a certain degree of confidence in what's going on.

    Maybe at the next subcommittee, the steering committee, we will discuss this.

    Is there any way of knowing which audits are ongoing or under way right now? I know that all of a sudden we get your reports and we know what's coming down, but is it a secret? Is there a way to know what you're reporting on now?

    Certainly, Mr. Chair, in the 2016-17 report on plans and priorities, at the back, in section III, we provide all of the planned audits for 2016-17. That includes the fall 2016 reports that we are planning to bring to you.

    First, there's the spring report of the commissioner of the environment and sustainable development that I mentioned. That doesn't actually come through this committee. That will be on safety of consumer products, federal support for sustainable communities, and federal support to mitigate the effects of severe weather.

    In fall 2016, reports of the Auditor General of Canada are there. Then the fall report of the 2016 report of the commissioner of the environment and sustainable development is there, as well.

    Our website also has the spring 2017 reports listed now, which are there. In this report, it goes up to the end of fall 2016, and on our website, the audits that we have planned for spring 2017 are there.

    However, you will conduct an audit should, all of a sudden, there be a massive issue in the media or in Parliament, where there is a high question as to practices within a certain department. You also have the ability to perform an audit on those that may come up that are outside the planned audits that you have.

    Our audit process is a very structured way of doing audits. Yes, from time to time, we can make the adjustments. We try to do our planning out so that our auditors know what they're going to be working on, and they can start to prepare, and so the departments know that we're coming in, too.

    Every now and then, we do have to adjust for something. I think probably the most visible example of that in the most recent past was when we agreed to do the audit of senators' expenses. That, of course, meant that we had to make a significant adjustment to our plans and what we were going to do.

    Certainly, we prefer to have a structured plan. We want to audit things of importance. We try not to just react to what the media story of the day is. By the time we get an audit organized, and we execute that audit and report, we're probably two years out from when something became the issue of the day.

    We need to understand whether that issue will still be important throughout that cycle. We have to take all of those things into consideration.

    Thank you very much. I think that is pretty well all the questions.

    Shall vote 1 under the Auditor General, less the amount voted in interim supply, carry?

    (Vote 1 agreed to)

    The Chair: Under vote 1 of the Office of the Auditor General, shall the chair report this to the House?

    Some hon. members: Agreed.

     We thank you very much again for coming. You appear very regularly before this committee, and we appreciate it. We appreciate the answers we had in your testimony today.

    We're going to suspend, and then we're coming back to do committee business in camera.

    [Proceedings continue in camera]