Internal Audit Report - Audit of the Management Control Framework of the Canada Strategic Infrastructure Fund and of the Border Infrastructure Fund

Final Audit Report

PDF Version (Size: 149.12 KB) (270 Kb)

Help on accessing alternative formats, such as PDF, PPT and ZIP files, can be obtained in the alternate format help section.

Table of Contents

Appendix A: List of Recommendations and Management Response
Appendix B: Audit Opinion and Statement of Assurance (Information on the types of Audit Opinion and Statement of Assurance)

Planning phase: November 2006 to March 2007

Execution phase (fieldwork): March to May 2007

Report Date: November 15, 2007 (CSIF) – November 23, 2007 (BIF)

Management Response: December 3, 2007

Tabled and Approved by DAC: December 13, 2007

Executive Summary

Audit Objective

The audit objective was to assess the adequacy of the Management Control Framework (MCF) in place at Infrastructure Canada (INFC) for the administration of the Canada Strategic Infrastructure Fund (CSIF) and Border Infrastructure Fund (BIF) contribution programs, and to determine whether the MCF is working as intended.

The assessment of the adequacy and effectiveness of the MCF served to determine whether the four audit objectives detailed below were met. These four objectives are divided into ten audit areas, which were assessed in terms of the adequacy and the effectiveness of the MCF. In some cases, the audit areas relate to more than one audit objective. The following presents the four audit objectives and their relationship with the audit areas:

1. The CSIF and BIF contribution programs adhere to applicable legislation, policies and procedures:

  • Program Design (1);
  • Funding Instrument (2);
  • Compliance with Treasury Board Transfer Payment Policy (TPP) and the Financial Administration Act (FAA) (3); and
  • Monitoring of Projects (Contribution Agreement) (4).

2. The objectives of the CSIF and BIF Programs are met:

  • Monitoring of Projects (4); and
  • Output Indicators (5). (Covered under objective # 4 below)

3. The CSIF and BIF Programs are appropriately planned, implemented and monitored:

  • Strategic and Operational Planning (6);
  • Governance (including Third Party Delivery) (7);
  • Recruitment and Retention of Personnel (8);
  • Monitoring of Projects (4); and
  • Risk Management (9).

4. Timely and useful information is available and used for decision-making:

  • Information for Decision Making (10); and
  • Output Indicators (project results) (5).

Below is our assessment by audit objective with respect to the ten areas of the audit.

Objective 1

The audit did not reveal any significant issues regarding the first objective. The Programs were designed through proper mechanisms and have been reviewed and approved by appropriate authorities. The funding instrument is appropriate and in line with the Treasury Board (TB) Transfer Payment Policy (TPP) for contributions to CSIF and BIF projects. Considering the strategic implications of the Programs, we found the current project selection process to be in line with the Programs' design that was established. We found that overall the terms and conditions contained in the project level Contribution Agreements ( Contribution Agreement s) respected those imposed by Treasury Board at the Program level and that although the Contribution Agreement s varied in structure and content, they respected the TPP. The INFC claim and payment process complies with the provisions of the TPP and the FAA, however most payments are now made directly by the Federal Delivery Partners (FDPs).

Objectives 2, 3 and 4

We found significant weaknesses impacting the management control framework of INFC for objectives 2, 3 and 4 with respect to the monitoring, information for decision making, recruitment and retention of personnel and risk management.

An important aspect of an effective management control framework includes the organizations' ability to provide strategic direction to its staff, to effectively monitor its activities and risk, in addition to the collection of key information that will assist with decision making, risk mitigation strategies and reporting on both the projects' and the Programs' progress. Despite the fact that INFC personnel attend the Agreement Steering Committee meetings one of the major findings relates to our inability to fully assess whether effective monitoring is being exercised due to the lack of documentation related to risk management and the monitoring of activities. Furthermore, this lack of documentation significantly reduces INFC's ability to generate appropriate information for decision making. These weaknesses have been amplified by the issue with respect to the recruitment and retention of personnel in the Program Operations' Branch and the lack of strategic planning processes in place at INFC. These factors combined create risk with respect to the management of the Programs, thus require management attention as detailed in this report.

Audit Opinion

In our opinion, INFC has moderate issues requiring management focus. More specifically, there are significant control weaknesses in some areas; however, it is our assumption that the risk exposure is not serious because the Federal Delivery Partners (FDPs) are mature government organizations, which in general, have adequate control frameworks in place. Caution should be used in interpreting this opinion as we have not tested or in any way validated our assumption of the adequacy of the FDP's control framework. Therefore we believe that the findings raised in this report require management attention since important weaknesses can be present even in sophisticated control environments and INFC has the responsibility to ensure an adequate control framework in the context of the third party delivery model. (Appendix B provides information on the types of audit opinion that can be formulated, as per the Office of the Comptroller General's Statement of Internal Audit Standards – Preliminary Draft).

Statement of Assurance/Reliance

In our opinion, based on our professional judgment as auditors, sufficient and appropriate audit procedures have been conducted, in accordance with the Treasury Board (TB) Policy on Internal Audit and the evidence gathered supports the accuracy of the conclusions contained in this report. Criteria were developed to provide a framework in assessing the extent to which the MCF of CSIF and BIF was meeting the audit objectives. The criteria were based on the Draft TB Audit Guide for Grants, Contributions and Other Transfers and were reviewed for appropriateness and approved by the Head of Internal Audit prior to executing the audit plan. Consequently, the evidence gathered provides senior management with reasonable assurance of the accuracy of the conclusions drawn from this audit.

Key Recommendations

In order to help INFC improve its MCF with respect to monitoring, information for decision making, recruitment and retention of personnel, risk management, and other key areas, we are presenting the following key recommendations with respect to our major findings. All the recommendations made as a result of this audit, are presented throughout this report, immediately following each relevant section, and are also listed in Appendix A.

Strategic and Operational Planning

  • INFC should implement a strategic planning process and develop an integrated operational plan. The integrated operational plan would be for a period of at least one year and should be revised annually. Furthermore, it should include subject matters related to Finance, Human Resources and risk mitigating strategies to address CSIF and BIF concerns.

Governance

  • INFC should document what is expected from each Branch, Directorate and Division to fulfill its mandate with respect to CSIF and BIF and its other programs. This information should be communicated throughout the organization to ensure that there is a common understanding.
  • INFC should formalize a process that would ensure that the Control Framework of the more significant FDPs is being reviewed periodically. This will help ensure that appropriate monitoring and reporting mechanisms exist at the FDP level and allow INFC to develop its own monitoring procedures based on the assurance provided by this information.

Recruitment and Retention of Personnel

  • Additional procedures should be initiated urgently to ensure that INFC has the required capacity to deliver on its responsibilities, especially in POB as the resources are critically low.
  • The problem of staff retention in INFC should be examined and an action plan developed to address issues identified.

Monitoring of Projects

  • An explicit project monitoring accountability framework should be drafted and communicated to the relevant INFC staff. This framework should detail what areas should be monitored, what monitoring duties should be performed and what key documentation should be retained in the files to support this monitoring process.

Information for Decision Making

  • A decision should be made with respect to the usage (or not) of the Shared Information Management System for Infrastructure (SIMSI) as the key reporting system of relevant project/program information for the management of CSIF and BIF.
  • The development of the executive report designed by the ADM of Program Operations Branch (POB) should be completed, implemented, and revised, if needed, when a decision is reached with respect to the use of SIMSI. This report should be developed to facilitate the dissemination of information to be shared with the Executive Committee.

Output Indicators (project results)

  • INFC should develop a reporting system that would allow the reporting of information with respect to its output indicators established.

Risk Management

  • Responsibility at the Branch, Directorate, Division or individual levels should be assigned for risks areas identified in the Risk Profiles.
  • INFC management should incorporate, into their integrated planning process, the specific actions to be undertaken to address the risks identified.
  • INFC should develop and implement a project risk management guideline with formal risk assessment procedures to be initiated on each CSIF and BIF project before the Contribution Agreement (Contribution Agreement) is signed. This procedure should detail the level of assessment to be performed, who would be responsible to execute it, when it should be executed and the documentation that would result from this process.
  • Detailed mitigating strategies should be developed and incorporated into the Contribution Agreement or guidelines.

Head of Internal Audit's Signature

Yolande Andrews

Yolande Andrews - Head of Internal Audit

Audit Team Members

Pierre Samson, M.Sc., FCGA, Samson & Associates, Team Leader
Dan Rosborough, Samson & Associates, Senior Auditor
David Cleary, CA, Samson & Associates, Senior Auditor
Mathieu Farley, CA, CPA, Samson & Associates, Senior Auditor

1.0 Background

Canada Strategic Infrastructure Fund (CSIF)

The Canada Strategic Infrastructure Fund (CSIF) is a cost-shared contribution program for strategic infrastructure projects. This program received funding in the 2001 and 2003 federal budgets, for a total of $4 billion fund. It is expected that the approved funding will provide federal support to approximately 50 projects across Canada.

Investments are directed to projects of major national and regional significance, and are to be made in areas that are vital to sustaining economic growth and supporting an enhanced quality of life for Canadians. CSIF is delivered through negotiated agreements with provincial, territorial or local governments, or private partners; these are considered the recipients. Contribution agreements are tailored based on the project requirements.

The CSIF Act outlines the categories of investments in projects that involve fixed capital assets that are used or operated for the benefit of the public. The categories eligible under the CSIF are:

  • Highway and Rail Infrastructure;
  • Local Transportation Infrastructure;
  • Tourism or Urban Development Infrastructure;
  • Northern Infrastructure;
  • Water or Sewage Infrastructure; and
  • other categories approved by regulation, e.g. Advanced Telecommunications and High Speed Broadband.

The overall planned results INFC expects to achieve through CSIF are to invest in projects which:

  • facilitate the movement of goods and people on Canada's National Highway System for the purposes of increasing the productivity, economic efficiency, and safety of Canada's surface transportation system;
  • facilitate the safe and efficient movement of goods and people, ease congestion, or reduce greenhouse gases and airborne pollutants;
  • ensure that tourism continues to contribute to the economic well-being of Canadians and to serve as a bridge between Canada and the world;
  • ensure that drinking water is safe, clean, and reliable at drinking water facilities, and ensure sustainable treatment of wastewater; and
  • expand broadband networks in Canada.

CSIF is delivered in partnership with the federal departments and three federal regional economic agencies. The federal departments include Industry Canada for Ontario and broadband projects, Indian and Northern Affairs (INAC) for northern and First Nations projects, Canada Mortgage and Housing Corporation for a project in Nunavut, and Transport Canada for all transportation-related projects. The three federal regional economic agencies are Atlantic Canada Opportunities Agency (ACOA), Canada Economic Development - Quebec (CED-Q), and Western Economic Diversification (WED).

Roles and responsibilities of partners are outlined in Memoranda of Understanding (MOU) between Infrastructure Canada (INFC) and the implementing agencies or departments. As at the time of this report, all Federal Delivery Partners (FDP) had signed the MOU with the exception of the FDP in Quebec.

CSIF was designed using the legislative and policy architecture of the federal government. Due to their size, and the fact that the projects are often of major national or regional importance, TB Submissions are prepared for each individual project.

Border Infrastructure Fund (BIF)

The Border Infrastructure Fund (BIF), which was announced in the 2001 budget, is a $600 million cost-shared contribution program. It complements some of the Government of Canada's other infrastructure programs, such as the CSIF, and the Strategic Highway Infrastructure Program, a Transport Canada program.

As part of "Canada's commitment to address land border pressures, such as traffic congestion, and to continue to facilitate the large volume of trade across the Canada - United States border," BIF contributions are directed at, or are on routes leading to Canada's border crossings, with a particular focus on the six largest boarder crossings:

  • Windsor, Ontario;
  • Sarnia, Ontario;
  • Fort Erie, Ontario;
  • Niagara Falls, Ontario;
  • Douglas, British Columbia; and,
  • Lacolle, Quebec.

BIF also directs some funding towards smaller and regionally important border crossings throughout Canada. Once completed, projects supported under BIF will help to alleviate traffic congestion, increase system capacity and further the the Smart Border Declaration.

The overall planned results that INFC expects to achieve through BIF are to invest in projects that contribute to safe and efficient border crossings. Expected outcomes are to alleviate border congestion and increase border-crossing capacity, increase security and safety at border crossings, thus leading to cross border trade efficiencies.

BIF is delivered in partnership with Transport Canada.

BIF was designed using the legislative and policy architecture of the federal government. Due to their size and the fact that the projects are often of major national or regional importance, TB Submissions are prepared for each individual project.

2.0 Audit Objectives, Scope and Approach

2.1 Audit Objectives

The broad audit objective was to assess the adequacy of the Management Control Framework (MCF) in place at Infrastructure Canada for the administration of the CSIF and BIF contribution programs, and to determine whether the MCF is working as intended.

The assessment of the adequacy and effectiveness of the MCF served to determine whether the four audit objectives detailed below are being met. These four objectives are divided in ten audit areas, which were assessed in terms of the adequacy and the effectiveness of the MCF. In some cases, audit areas relate to more than one audit objective. Following are the four audit objectives and their relationship with the audit areas:

5. The CSIF and BIF contribution programs adhere to applicable legislation, policies and procedures:

  • Program Design (1);
  • Funding Instrument (2);
  • Compliance with Treasury Board Transfer Payment Policy (TPP) and the Financial Administration Act (FAA) (3); and
  • Monitoring of Projects (Contribution Agreement) (4).

6. The objectives of the CSIF and BIF Programs are met:

  • Monitoring of Projects (4); and
  • Output Indicators (5).

7. The CSIF and BIF Programs are appropriately planned, implemented and monitored:

  • Strategic and Operational Planning (6);
  • Governance (including Third Party Delivery) (7);
  • Recruitment and Retention of Personnel (8);
  • Monitoring of Projects (4);
  • Risk Management (9).

8. Timely and useful information is available and used for decision-making:

  • Information for Decision Making (10); and
  • Output Indicators (5).

2.2 Audit Scope

The scope of the audit included both the program and project life cycles for CSIF and BIF, specifically as they relate to INFC responsibilities with regard to: program design, implementation and evaluation as well as project eligibility, selection and approval, environmental assessment, project agreement negotiation, and project monitoring and completion.

We examined the Management Control Framework put in place for the CSIF and BIF contribution programs from the inception of the program up to December 31, 2006. This involved reviewing the adequacy and effectiveness of the internal controls which include those policies, procedures, processes, structures, systems, and practices that management has at its disposal to ensure sound management of, control over, and accountability for its contribution programs. The audit scope also included the controls relating to the proper management of third party delivery arrangements (implementing departments/agencies for CSIF and BIF), and the overall governance of the Programs under the structure and processes that existed at December 31, 2006. As such, project and Program information and documentation was reviewed from the inception of each Program to December 31, 2006. As well, joint responsibilities of INFC and third parties were included in the scope - project agreement negotiation, preparation of Treasury Board submissions, and project monitoring.

Third parties' responsibilities were not included in the scope of this internal audit, except:

  • when these responsibilities relate to reporting and communicating with INFC, as provided for in the third party delivery arrangement, so as to enable INFC to discharge adequately its accountabilities; and/or
  • when these responsibilities relate to joint responsibilities of INFC and third parties.

In that context, unless they relate to the responsibilities described above, the following third party responsibilities were not part of the scope of this audit:

  • overseeing implementation of mitigation measures identified in the environmental assessment,
  • assessment of the eligibility and reasonability of project costs,
  • provision of information pertaining to cash flow and budget,
  • approval and payment of claims,
  • conduct of project audits and evaluations,
  • ensuring adherence to information management requirements, including the use of the Shared Information Management System for Infrastructure (SIMSI), and provision of communication support.

The planning phase was conducted from November 2006 to March 2007. The execution phase occurred from March to May 2007, followed by the drafting of the report.

2.3 Audit Approach

2.3.1 Audit Survey

The audit team carried out an audit survey to gain an understanding of the CSIF and BIF Programs. The audit survey phase consisted of the conduct and completion of the following components:

  • Program and project documentation review;
  • Policy & governance review;
  • Interviews;
  • Production of program delivery process flowcharts; and
  • Risk assessment.

2.3.2 Execution Phase

In the execution phase a sample of 16 project files for CSIF and 5 files for BIF were tested in the context of the MCF as well as to assess the adequacy of the contribution agreements entered into with the recipients.

In addition, follow up interviews and documentation review were undertaken to further assess the areas identified as requiring further audit work during the survey phase of the audit. The cumulative results of our audit survey and execution phase are detailed in the following section on Findings, Recommendations and Management Response.

3.0 Findings and Recommendations

As indicated previously, we assessed the adequacy of the MCF with respect to the ten audit areas of the audit. The following observations have been made.

3.1 Program Design

CSIF and BIF were designed using the legislative and policy architecture of the federal government. As a result, we acknowledge that the Programs were designed through proper mechanisms and have been reviewed and approved by appropriate authorities.

The audit addressed whether the terms and conditions at both the Program and project level have adequate authority through TB and the contribution agreement (Contribution Agreement) process.

As part of the audit, we also reviewed the project selection process. Project selection is a collaborative effort with each Province and Territory through an exercise where INFC will decide to fund Provincial and Territorial priorities as long as they meet the CSIF and BIF objectives and that INFC have the funds available.

Considering the strategic implications of the Programs, we found the current project selection process to be in line with the Programs' design established as part of the Memoranda to Cabinet.

Findings for 3.1

Although the Agreement Steering Committee (ASC) is the key monitoring mechanism and the Contribution Agreement is an important guiding document that outlines the agreements terms and conditions with respect to the projects, it was observed that the monitoring process only commences when the ASC is formed, which is after the signing of the Contribution Agreement . In many cases, we were informed that this does not occur on a timely basis. This is the result of the design of the programs because many infrastructure projects have commenced or incurred significant progress after the Ministerial announcement is made. In some cases, the Contribution Agreement was only signed three years after the Ministerial announcement. This inherent lag in the monitoring process was noted but no recommendation is made because the current design of the programs has caused this. The design of future infrastructure programs should take this into consideration in order that effective monitoring mechanisms are in place when the project commences.

Recommendation for 3.1

There is no recommendation.

3.2 Funding Instrument

The risks at this level should be mitigated by the processes involved in securing approved Terms and Conditions and spending authority through Treasury Board for the Programs. The auditor considered the risk that the funding instrument (contribution) chosen for the programs might not be the most appropriate in light of program implementation.

Findings for 3.2

The audit found that the funding instrument is appropriate and in line with the Treasury Board (TB) Transfer Payment Policy (TPP) for contributions to CSIF and BIF projects. The budgetary allocation for the Programs, are approved under the Canada Strategic Infrastructure Fund Act (2002). There are approved Terms and Conditions for each project. Contribution agreement and spending authority obtained through TB are required for all projects. We were satisfied that contributions are the appropriate mechanism, given the current design and intent of the Programs.

Recommendation for 3.2

There is no recommendation.

3.3 Compliance with the Treasury Board Transfer Payment Policy (TPP) and the Financial Administration Act (FAA)

3.3.1 Compliance with Legislative and Policy Requirements

As part of the audit, we assessed whether efficient and effective accounting and other procedures are in place and implemented at INFC to ensure that payments meet legislative (Section 33 and 34 of the FAA) and policy requirements, and hence ensure that funding is used for the purposes agreed.

For CSIF, INFC has entered into MOUs with some FDPs for the delivery of the Program in a number of Provinces. For these projects, the funds are advanced to the FDP several times in the year: at interim supply, at approval of main estimates, and at approval of the various supplementary estimates, based on cash flow needs and Section 33 and 34 are exercised by the FDP. INFC only retained the Section 34 over specific recipient project claims in selected cases where there is no MOU with the FDP. Another particularity is that INFC is using the financial system of Industry Canada. As a result, Industry Canada exercises Section 33 on behalf of INFC.

For BIF, INFC has entered into a MOU with Transport Canada for the delivery of the Program in a number of Provinces. For these projects, the funds are advanced to Transport Canada several times in the year: at interim supply, at approval of main estimates, and at approval of the various supplementary estimates, based on cash flow needs and Sections 33 and 34 are exercised by Transport Canada. Since the BIF project claims are handled directly through the claim and payment processes of Transport Canada, no further work was performed concerning payments as this was beyond the scope of the engagement.

Findings for 3.3

Certain controls have been established with respect to the proper handling of public funds. The Programs have a number of financial controls in place including:

  • financial signing authorities are established for the various levels of management;
  • INFC or FDPs are required to certify claims submitted by recipients under FAA Section 34; and
  • contribution agreements contain clauses outlining obligations of the recipient as well as a clause giving INFC the right to audit.

One of our audit concerns relates to the lack of assurance provided to INFC by the FDP concerning the effectiveness of their control environment, since Section 33 and 34 authorities have been delegated to these delivery partners.

For the files that were handled directly by INFC because there was no MOU, we tested the adequacy of INFC claims and payment procedures through a sample of project claims submitted through INFC for settlement. We also assessed compliance with the Terms and Conditions of the Contribution Agreement and relevant sections of the FAA. We found that for all INFC claims examined, TPP and FAA requirements were fully met. In cases where INFC is responsible for claims settlement, we found a duplication of confirmation of contract performance and price (Section 34). The FDP Officer prepares the project claims documentation and certifies the claim under FAA Section 34. The claim is then forwarded to the INFC Project Officer who conducts a second level review and recommends FAA Section 34 certification to the ADM POB. Once the ADM POB approves the claim, it is then sent to Industry Canada for release under FAA Section 33 as INFC does not have Section 33 authority (the payment authority phase of the accounts payable process).

In summary, we conclude that the claims and payment process comply with the provisions of the Transfer Payment Policy and the FAA. However, most payments are now paid directly by the FDPs and there is limited assurance obtained regarding their compliance with Sections 33 and 34. This is further detailed in Section 3.5.2 (Governance: Third party delivery) of this report with a related recommendation.

Recommendation for 3.3

There is no recommendation.

3.3.2 Financial Planning

Cash management ensures that sufficient funds are available in current and future fiscal years to meet program and recipient requirements. It also helps to ensure that there are no significant lapses of funds at year-end.

INFC's programs can be subject to significant cash flow fluctuations due to their nature. As a result, special rules are in place, which provide that lapses can be re-profiled to future years with the approval of the Department of Finance. However, large unanticipated lapses can create issues for the Department of Finance in managing the fiscal framework, and therefore the department is expected to manage so that lapsed funding is minimized.

Schedule B in the Contribution Agreement details the cash flow for the project funding and is the basis for committing the funds. Changes to Schedule B cash flows don't have to be signed by the Minister; rather they can be signed by the Federal Agreement Steering Committee (ASC) Co-chair, usually INFC's federal delivery partner.

INFC coordinates requests for cash management adjustments with all federal delivery partners, and approves changes based on availability of funds at the program and vote level. Once the cash management change is approved by INFC, an amended Contribution Agreement Schedule B can be signed between the recipient and the Agreement Steering Committee Co-chair.

Findings for 3.3.2

The audit team found that Schedule B's are not always amended based on an approved project cash flow forecast. We also found that several projects Schedule B of Contribution Agreement 's had not been amended to re-profile funds that were not expended in previous financial years. We were advised by INFC Finance that FDP cash forecasts are not submitted on a timely basis.

We expected that Finance would have copies of all the most recent Contribution Agreement Schedule B's in order to commit the funds and make the appropriate payments to the FDPs.

Although the amendment process of Contribution Agreement Schedule B is presented in the ASC Handbook, not all parties presently follow the Handbooks requirements regarding schedule B updates. Therefore, the process is not consistently understood or applied. There is no procedure in place that sets out the timelines and expectations of INFC Finance with respect to cash forecasting and amendments to Contribution Agreement Schedule B. Also, amended Contribution Agreement Schedule B's are not copied to INFC, who therefore do not have an accurate record of commitments. In summary, the cause of this condition is the lack of clear guidance and procedures with respect to the modification of the cash flows, including the Contribution Agreement Schedule B's.

If Finance does not have the most recent Contribution Agreement Schedule B, it can result in unnecessary delays in the payment to recipients at the end of one year or, result in claims being submitted in excess of Contribution Agreement Schedule B's cash flow – Section 32 non-compliance since the FAA requires a proper control of commitments.

Recommendations for 3.3.2

1. Cash management calendar and procedures need to be developed and communicated by INFC to all federal delivery partners.

2. The Contribution Agreement and Schedule B template wording needs to be improved to confirm that the fiscal year breakdown is a ceiling, not an estimate and that any adjustment required to Contribution Agreement Schedule B for Canada's contribution be approved and requested to INFC by the ASC.

3.4 Strategic and Operational Planning

In order to properly address priorities and risk management and in order to use resources diligently for the implementation of CSIF and BIF, INFC needs to have an integrated planning process.

Findings for 3.4

INFC does not have a strategic or integrated operational plan in place. Planning is not formally integrated within the organization. Although there was one initiative to implement an operational plan for the Program and Operations Branch in 2005, it has not been followed through and it did not involve all branches of the organization.

This situation can be attributed to the fact that INFC is a relatively new department and has not yet put in place all of the management control framework components which would exist in a mature organization. Staffing issues have also been identified as one reason for not having developed such a planning process. Also many program changes have occurred at INFC since its establishment.

The lack of an integrated planning process reduces INFC's ability to make progress in areas requiring improvement for CSIF and BIF, including human resource capacity, appropriateness of IT systems, funding, etc. A planning process is more important in a new organization as processes and procedures are not fully defined yet.

Recommendations for 3.4

3. INFC should implement a strategic planning process and develop an integrated operational plan. The integrated operational plan would be for a period of at least one year and should be revised annually. Furthermore, it should include subject matters related to Finance, Human Resources and risk mitigating strategies to address CSIF and BIF concerns.

4. Regular reporting on the progress towards the integrated operational plan should be made to the Executive Committee to ensure there is appropriate accountability for the management of CSIF and BIF.

5. INFC should ensure that they have an adequate and qualified resource responsible for putting in place and managing the planning cycle.

3.5 Governance

The principles of the governance structure related to the CSIF and BIF Programs need to be understood in relationship with the delivery model that INFC has chosen for these programs. As in previous and similar versions of infrastructure programs, the use of third party delivery partners (the same as the ones currently being used) has been selected. INFC has entered into a Memorandum of Understanding (MOU) to implement CSIF and BIF, with the various Federal Delivery Partners (FDPs), except for Quebec.

The oversight of the contribution agreements (Contribution Agreement) being implemented by the FDPs is done through participation on an oversight committee called the Agreement Steering Committee (ASC). The ASC is composed of a Provincial co-chair, a Federal co-chair, INFC representatives, FDP representatives as well as recipient representatives.

Two Branches within INFC are primarily involved with the implementation of CSIF and BIF; the Policy and Communications Branch (PCB) and Program Operations Branch (POB). These Branches are in communication with the FDP and the recipient at different times during the project's life cycle. PCB is involved at the early stage in reviewing and discussing potential projects before the Ministerial announcement, while POB is responsible for the detailed negotiation of the contribution agreement and its management, including monitoring.

The Risk-Based Audit Framework (RBAF) and the Results Based Management Accountability Framework (RMAF) provide a general understanding of how INFC, the FDP, the ASC and the Recipient exercise their respective roles. Further details are provided in the MOU with the various FDPs and in the contribution agreements.

3.5.1 INFC Internal Organizational Structure

A clear understanding of responsibilities within an organization is critical to effectively achieve common objectives and fulfill the mandate for which the organization was created, including CSIF and BIF.

Findings for 3.5.1

We noted a lack of common understanding regarding the internal organizational structure at INFC which has implications with respect to the management of CSIF and BIF. Although some employees have a sense of what their role and responsibilities are, it is not necessarily the same understanding for everyone and we have not found a description of the roles and responsibilities of the various Branches, Directorates and Divisions in relationship to INFC's mandate for managing CSIF and BIF or other programs.

In particular, there are different views regarding the role that the Human Resources Division should exercise, the adequacy and use of SIMSI as a reporting system, the role of POB before the Ministerial announcement is made, and the extent of documentation that should be provided by PCB to POB following the Ministerial project announcement.

This situation can partially be attributed to the fact that INFC is a relatively new department and has not yet put in place all the management control framework components which would exist in a more mature organization. Furthermore, INFC has been through major program changes under its responsibility in addition to experiencing turnover of its top executives. These changes have required a lot of flexibility within the organization and have created changes to employee's roles and responsibilities since its establishment in 2002.

Unclear understanding of everyone's roles and responsibilities leads to ineffective management and frustration. Currently, there are several examples of differences with respect to the understanding of each others roles and responsibilities within INFC. This lack of common understanding has already reduced INFC's ability to effectively perform its responsibilities. For example, consequences of this include the fact that SIMSI is not being used as a reporting mechanism for CSIF and BIF projects and no alternative system has been put in place.

Recommendation for 3.5.1

6. INFC should document what is expected from each Branch, Directorate and Division to fulfill its mandate with respect to CSIF and BIF and its other programs. This information should be communicated throughout the organization to ensure that there is a common understanding.

3.5.2 Third Party Delivery

To deliver the CSIF and BIF Programs, INFC uses third party delivery agents called Federal Delivery Partners (FDP). These partners are established organizations working with multiple levels of government (Federal/Provincial/Municipal) and other private sector organizations.

The principles that guide the organization of the Programs were established using the legislative and policy architecture of the federal government. The RMAFs for CSIF and BIF also provide some guidance with respect to the Governance framework but they do not detail the specific responsibilities. Instead, MOUs have been prepared and signed with all FDPs currently involved with the implementation of CSIF and BIF projects, except for Quebec. These MOUs include descriptions of the roles and responsibilities for each of the two parties involved. Based on interviews with personnel from the Atlantic Canada Opportunities Agency, Western Economic Diversification Canada and Industry Canada, they are largely satisfied with the MOU. However, Transport Canada is not satisfied and is currently working with INFC to revise the MOU to better detail the respective roles and responsibilities.

3.5.3 Internal Audits Performed at the FDP Level

As a result of many responsibilities being delegated to the FDP, key internal controls and reporting mechanisms at the FDP level can directly impact on the MCF at INFC. We have noted the following facts related to current internal audits being performed at the FDP level.

It was noted that both Transport Canada and Industry Canada have initiated internal audits as part of their own internal review procedures, related in part to their management responsibilities with respect to CSIF (and BIF, in the case of TC). Internal Audit at INFC is monitoring these audits as they relate to programs that are administered by INFC and for which INFC is ultimately accountable. Since the absence of proper controls and reporting mechanisms at the FDP level can affect INFC, the Organization should continue to monitor the results of these audits and ensure that appropriate actions are taken, as required.

Findings for 3.5.3

There is no formal process in place at INFC to ensure that the FDPs have adequate controls in place to discharge their responsibilities under the MOU. INFC should require assurance that the FDP has adequate controls in place. The FDPs may already have this type of information, such as recent internal audits, which could provide some assurance as to the state of their control framework to INFC management. Our interviews revealed that the responsibility to request and analyze such information on a proactive basis was not assigned at INFC.

Such a procedure will increase the effectiveness of the MCF at INFC's level. If such a process is initiated, weaknesses at the FDP level can be identified and corrected on a timelier basis, which in turn enhances the overall control framework for CSIF and BIF.

Recommendation for 3.5.3

7. INFC should formalize a process that would ensure that the Control Framework of the more significant FDPs is being reviewed periodically. This will help ensure that appropriate monitoring and reporting mechanisms exist at the FDP level and allow INFC to construct its own monitoring procedures based on the assurance provided by this information.

3.5.4 Agreement Steering Committee

The ASC is the key project monitoring body that has been established for CSIF and BIF projects. It is through its participation on the ASC that INFC is exercising most of its oversight with respect to CSIF and BIF funding and project results. According to the Risk Based Audit Frameworks (RBAF), the ASC is expected to manage, implement and govern the Contribution Agreement .

In order to ensure consistent management, implementation and governance of the Contribution Agreement , the ASC requires guidance with respect to its roles and responsibilities.

Findings for 3.5.4

We found that INFC has prepared an ASC Handbook to provide detailed guidance to the ASC, but at the time of our audit, the Handbook was still draft and had not been distributed to the ASC members as formal guidance. As a result, there is no formal guidance for the ASC to ensure that they discharge their responsibilities in an appropriate and consistent manner.

This situation can be attributed to the fact that no individual at INFC has been assigned responsibility for this document. Furthermore, the need for guidance has not been considered urgent since similar committee's have been used as a monitoring mechanism for other programs and the individuals involved have similar experience.

However, as the ASC meetings are the primary monitoring mechanism for INFC with respect to CSIF and BIF projects, a lack of clear guidance could result in inconsistent and/or inappropriate management and implementation of the Contribution Agreement .

Furthermore, the draft ASC Handbook does not contain certain provisions that would ensure common and appropriate execution of the ASC meetings. As such, the draft ASC Handbook does not detail the ASC objectives and provide guidance on the administration of the ASC: frequency of meetings, required attendance (quorum), reporting requirements, etc.

Recommendations for 3.5.4

8. The draft ASC Handbook should be finalized and communicated to all members of the ASC.

9. The Handbook should detail the ASC objectives and provide guidance on the administration of the ASC: frequency of meetings, required attendance (quorum), reporting requirements, etc.

10. An office of primary responsibility should be established for the ASC Handbook.

3.5.5 Policies and Guidelines

INFC has issued and distributed numerous guidelines to provide appropriate guidance to the recipients and FDPs involved with the implementation of CSIF and BIF projects. These guidelines provide details on key aspects of project implementation and monitoring for the following areas:

  • Reporting - Audit and Evaluation guideline (draft);
  • Project Closure Process guideline (draft);
  • Environmental Assessment guideline (draft);
  • Claims and Payment guideline (draft);
  • Information Management guideline (related to SIMSI) (draft); and
  • Communication Protocol guideline (draft).
Findings for 3.5.5

The CSIF and BIF guidelines were never officially issued and remain in draft format. As a result, although they are being communicated, some FDPs expressed concern that they were not final and in some cases were too general. In addition, the guidelines have not been regularly updated to reflect experience gained with respect to the delivery of the CSIF and BIF Programs.

The primary reason why these guidelines are still in a draft format is that there is no office of primary responsibility established for these guidelines, and no employee has taken ownership to ensure that they are being finalized, approved and regularly updated.

This situation has resulted in inconsistent use of the guidelines by the FDPs and recipients. As no formal sign-off procedures exist for guidelines, we found that INFC employees with coincidental interest are not always advised as to their related responsibilities within the guideline.

Recommendations for 3.5.5

11. For each guideline, an office of primary responsibility should be established and sign-off procedures be put in place to take into account stakeholders with coincidental interest in the respective guideline. These guidelines should be referenced in the MOU and/or the Contribution Agreement as governing instruments.

3.6 Recruitment and Retention of Personnel

Proper management of human resource needs is critical to ensure that INFC has the capacity and the expertise to discharge its responsibilities with respect to CSIF and BIF. Human resource planning should be discussed regularly and appropriate actions initiated on a timely basis to address the Organizations needs.

Findings for 3.6

INFC has had high turnover of personnel over the past year. Some Branches more than others.

POB has an important role with respect to CSIF and BIF implementation and monitoring. It is within this branch that the Contribution Agreement 's are prepared, risks are managed and monitoring of all projects is performed. Since the beginning of 2006, the full time equivalents (FTE) positions have significantly decreased. FTE levels have been as low as 65% (33 staff compared with 53 positions in December 2006). Although there have been hiring efforts over that period, the current level of FTEs has not improved in the last twelve months because many employees continue to leave. This poses a significant risk to the ability of INFC; including POB where the situation is more critical, to appropriately discharge its roles and responsibilities with respect to CSIF and BIF.

A cause of this condition is the lack of retention of staff as shown by the particularly high number of staff who left POB in 2006, including top executives, which in turn caused a bottle neck in the processing of an unusually high number of staffing actions. In this context, the resources dedicated to staffing initiatives could not address the extraordinary additional demands to fill the numerous vacancies. The uncertainty surrounding INFC because it lacks A Base funding has also been raised as an unfavorable condition for attracting and retaining personnel.

Furthermore, the lack of a common understanding concerning the roles and responsibilities of the Human Resources Division with respect to Recruitment and Retention of Personnel (hiring activities especially) is also a cause of this ongoing problem.

After almost 12 months of shortages of personnel (to December 31, 2006), the situation has not been resolved and INFC still has critical personnel shortages. More specifically in POB, this impacts the appropriateness of the monitoring activities. Staffing actions will not solely correct this situation, if POB continues over time to experience an unusual level of staff turnover. Retention of staff will need to be improved as well.

Considering all this and notwithstanding our recommendations below, we recognize that INFC personnel have been working hard and have demonstrated a high level of commitment in making sure the key activities were executed even when they were short staffed. This can be demonstrated by the number of Treasury Board Submissions prepared since the beginning of the Program (46 in total at the time of our audit).

Recommendations for 3.6

12. Additional procedures should be initiated urgently to ensure that INFC has the required capacity to deliver on its responsibilities, especially in POB as the resources are critically low.

13. The problem of staff retention in INFC should be examined and an action plan be developed to address issues identified.

14. The roles and responsibilities of the Human Resources Division and the other Branches and Directorates should be clearly defined and communicated with respect to the Recruitment and Retention of Personnel.

15. As part of the establishment of an integrated planning process, INFC should include a Human Resource plan, detailing the specific competitions required to address the current and future needs of the Programs.

3.7 Monitoring

To monitor the performance of the projects, Agreement Steering Committees (ASC) have been put in place with the expected intention that they will monitor the implementation of the Contribution Agreement . Since an INFC representative is a member of the ASC for each project, it was expected that it is through their attendance that most of the project performance monitoring would be executed. An important aspect of monitoring is to ensure that accountabilities are well defined. Accountability is the duty to report on the fulfillment of one's responsibility. As such, a key enabler of accountability is an appropriate organization structure that clarifies authorities, responsibilities, and the reporting responsibilities. The ASC Handbook outlines the roles and responsibilities of the different parties involved with the implementation and project monitoring.

3.7.1 Monitoring of Projects

At INFC, the performance of the recipient should be adequately monitored, (also considering the FDPs monitoring responsibilities), the actual results of the project should be measured, and appropriate reliable information should be gathered with respect to the projects performance. This should be done with formal directions to ensure there is adequate and consistent monitoring of projects. To assess the adequacy of the framework outlining the responsibilities for INFC project monitoring by Project Officers in the Intergovernmental Operations Directorate-POB, we reviewed a sample of 16 project files for CSIF and 5 project files for BIF and had discussions with INFC staff.

Findings for 3.7.1

The project monitoring procedures are mostly executed through the participation on the Agreement Steering Committee (ASC), but there is little documented evidence to support the monitoring activities in the project files. There are few notes in the files regarding the participation to the ASC (including minutes of meetings) or notes regarding the analysis that was performed pertaining to key monitoring documents such as audit and progress reports. Furthermore, there is inconsistent gathering of information and there were no documents pertaining to key decisions made, in the files reviewed, with respect to the projects maintained in the project files.

Although expectations for monitoring have been verbally communicated, there are no formal established guidelines to direct Project Officers with respect to their project monitoring related duties.

Based on the information maintained in the files reviewed, it was not clear what actual monitoring procedures have occurred at INFC. Based on interviews and on the review of the information in the files, we were unable to fully assess whether effective monitoring is being exercised. However, we were made aware from the interviews that monitoring activities such as attendance to the ASC meetings is occurring.

This lack of formal written guidance for monitoring has the following potential impacts:

  • document collection is not consistent across all projects;
  • project monitoring activities may not be consistent or effectively preventing INFC from taking remedial actions for any issues that would otherwise be identified; and
  • consolidation of key information may not be an efficient process should management require specific project information to be consolidated for management reporting.
Recommendation for 3.7.1

16. An explicit project monitoring accountability framework should be drafted and communicated with the relevant INFC staff. This framework should detail what areas should be monitored, what monitoring duties should be performed and what key documentation should be retained in the files to support this monitoring process.

3.7.2 Project Filing System

An adequate filing structure with respect to the monitoring of projects can enhance operational efficiency and support an effective monitoring process. INFC (Intergovernmental Operations Directorate-Program Operations Branch) maintains files for each CSIF and BIF project. Documentation is stored in these files which act as a repository of project monitoring related information for INFC. We examined a sample of sixteen files for CSIF and five files for BIF to determine if they support an effective monitoring process.

Findings for 3.7.2

The project filing system was found to be inconsistent and lacks uniform structure. Each project has many files associated with it. These are indexed, but the indexing is not always consistent. In some cases the same index lettering exists on multiple folder tabs for the same project, therefore one would need to look at the folder cover for more details. However, the details on the folder cover do not always reflect what is in the file folder. There is no specific sort order in the file structure. There is no file checklist to ensure that the project officers gather the minimum expected documentation to support the monitoring process. For example, many of the files reviewed did not contain audit reports, progress reports or a complete set of the ASC meeting minutes.

The review of the sample of project files demonstrated that guidance is required to ensure that a structured and consistent approach is implemented.

Recommendation for 3.7.2

17. A filing system for CSIF and BIF projects should be documented and implemented to allow for efficient and uniform retrieval of documents, especially numerical indexing of files that would allow easy retrieval of documents. A file checklist should be implemented detailing the minimum expectations with respect to file content. The report control sheet template, which was drafted by an employee in POB, but never implemented, should be discussed, revised and implemented as soon as possible. This checklist will help ensure that the minimum required project monitoring documentation is filed appropriately.

3.7.3 Monitoring of Financial and Compliance Audits

The audit function is an important part of the monitoring process. Transfer Payment Policy states that Departments…

are responsible for determining whether recipients have complied with the terms and conditions applicable to the contributions. This responsibility includes the audit of recipients when deemed necessary.

Departments must develop a risk-based audit framework for the audit of contributions including:

  • determining which recipients are to be audited;
  • selecting appropriate auditors or indicating the acceptability of auditors when retained by the recipient;
  • determining whether the scope, frequency and scheduling of audits meet program requirements;
  • coordinating audits with others involved in the audit of the same recipients; and
  • determining follow-up action required on audit findings.

The responsibility to ensure that compliance and financial audits are undertaken has been delegated to the FDPs through the MOUs.

To ensure effective oversight at the program level, INFC should delegate accountability for project audits to an individual who could, in turn, summarize and report to senior management the status, results and follow up concerning the audits performed.

To assess the adequacy of the monitoring process, we reviewed a sample of files, discussed with INFC staff any additional oversight mechanisms within INFC and reviewed the link between the FDPs and INFC regarding the audits performed for CSIF and BIF projects.

Findings for 3.7.3

Although we were advised by the FDPs that audits are being performed at the project level and that payments are generally released only after the financial audit reports are received, we found that INFC did not have any employee with designated responsibilities to summarize which audits are planned, their progress, or whether they are completed or the results/actions taken as a result of the audits undertaken.

Very few audit reports were found in the files and those that were examined varied in type and scope.

We also found that the audit selection process was not conducted using a risk-based approach, rather all projects are subject to a financial and compliance audit every year according to the Contribution Agreement . However, this is not what the Transfer Payment Policy (TPP) suggests and could result in too many audits being conducted for low risk projects.

In addition, we found that the Draft Audit Guideline was not clear and has created confusion with some FDPs, the ASCs and with the external auditors. For example, the scope of the audits is not always clearly defined and as previously noted in section 3.5.5, the Guideline remains in draft form.

We also noted that Transport Canada was in the process of creating their own audit guidelines to further define the frequency and scope of the audits for CSIF and BIF related projects.

Some delegation of responsibility has occurred to one employee from the Corporate Services Branch for providing functional advice on recipient audits, but there was no written guidance or enough time for her to adequately perform such a function.

The responsibility to monitor all of the audit reports, summarize the results of previous audits and analyze for appropriate follow up action at the time of the audit, pertain to the Program Operations Branch. POB has a resource planned to act in this capacity within that Branch at INFC.

A proposal outlining the main roles of the FDPs, Project Management Committee, Program Operations (INFC) and Internal Audit (IA) in terms of their respective involvement in RBAFs, audit guidelines, audit plans, project audits, corrective actions, monitoring, and periodic review of the MCF for the CSIF and BIF Programs was presented to the Departmental Audit Committee (DAC) at its meeting on June 15, 2007. The proposal confirms that the primary accountability for administering programs, including the recipient audit component of the control framework, lies with Program Operations.

Recommendations for 3.7.3

As a result of the above findings, we recommend the following:

18. INFC should require that the FDPs provide a periodic report summarizing which audits are due to be performed, which audits have been performed, and their results, along with management actions concerning any issues identified.

19. The Audit Guidelines should be revised to add clarity concerning scope, reporting requirements and other weaknesses that have been identified. This could be done in consultation with Transport Canada, since they have already initiated this process.

20. To be consistent with the RBAF, the Audit Guideline should allow for a risk-based audit methodology to determine the extent of audits required.

21. INFC should ensure that the center of expertise for the recipient audit monitoring function in POB is adequately staffed and that the responsibilities for this position are clearly defined and communicated.

3.7.4 Contribution Agreements (Contribution Agreement)

The Contribution Agreement should provide an adequate basis for the parties to exercise their respective roles and responsibilities to ensure effective financial and operational monitoring by the department. Signed agreements have been drafted from templates which have been improved over time by Intergovernmental Operations as well as the Issues Management group.

Findings for 3.7.4

Our examination confirmed that without exception, the basic requirements as mandated by TPP have been respected. Legal services have been involved with this process and the templates have been reviewed by Treasury Board Secretariat (TBS). The MOUs were drafted with the assistance of legal services.

Recommendation for 3.7.4

There is no recommendation.

3.8 Information for Decision Making

3.8.1 Information for Decision Making

The ADMs and the Executive Committee need appropriate information in order to make necessary decisions on which they are held accountable. As such, we have assessed the adequacy of the process in place to ensure that appropriate and sufficient information is available and communicated to management to ensure that an effective decision making process is in place.

Findings for 3.8.1

At this time, although the ADM of POB feels she obtains the required information to manage CSIF and BIF, the information is communicated through informal debriefings and is not formally documented. Project Officers attend the ASC meetings and communicate with the FDPs and the recipients on an on-going basis. Project Officers meet with the ADM of POB regularly to debrief her on the key issues and decisions made.

The absence of a documented reporting mechanism has been acknowledged by POB and as a result an executive report detailing for each project, the status, the risks and the key issues, is being designed to address this weakness and satisfy the ADM's management requirements.

As discussed seperately below, according to the RMAFs, SIMSI was supposed to be used as a complete reporting system and include the required information for management. However, there is currently not enough information in SIMSI to provide management a summary report as described in the preceding paragraph.

The potential impact of the current reliance through informal reporting of verbal communcations is that some key information may not be reported to management and there is no documented corporate memory of any issues discussed. As part of our file testing, we were not able to assess whether any key information had not been collected or reported since the CSIF and BIF project monitoring files at INFC did not provide complete or consistent monitoring documentation as previously described in this report.

Recommendations for 3.8.1

We recommend that:

22. A decision be made with respect to the usage (or not) of SIMSI as the key reporting system of relevant project/program information for the management of CSIF and BIF.

23. The development of the aforementioned executive report be completed, put in place, and revised, if needed, when a decision is reached with respect to the use of SIMSI. This report should be developed to facilitate the dissemination of information to be shared with the Executive Committee.

3.8.2 Reporting System - SIMSI

The Shared Information Management System Infrastructure (SIMSI) is the project reporting system developed by INFC for its various programs. We examined the CSIF and BIF modules of SIMSI as part of the Management Control Framework as it supports the implementing agencies and INFC discharge their responsibilities with respect to administering and reporting on CSIF and BIF.

At the time CSIF and BIF were put in place, INFC established a SIMSI Operations Committee to address the program and project data requirements. A Joint Application Development approach involving POB, Finance and IT of the CSIF and BIF modules was utilized. Amendments to data in the CSIF and BIF modules agreed to at Steering Committee meetings must be updated by SIMSI Program Operations Analysts, including all financial information.

As described in the RBAFs and RMAFs, SIMSI is to be used for on-line reporting for program and project information on such aspects as project funding and claims, communications events and the performance of CSIF and BIF. From this perspective the audit examined the development of the SIMSI CSIF and BIF modules to provide assurance that the CSIF and BIF stakeholders' user requirements are adequately addressed and that SIMSI is effective for the purposes of the RBAFs and RMAFs.

Findings for 3.8.2

We found that SIMSI is not being used for reporting on CSIF and BIF project finances or performance as it was intended. Federal Delivery Partners, the major intended users of the system, do not find that SIMSI is appropriate for their needs and reported that they have not been involved in the development of the SIMSI CSIF and BIF modules.

By examining the CSIF and BIF modules' data we found that the financial component of the CSIF and BIF modules reflects the internal INFC allocations used by INFC in cash managing the project and that Contribution Agreement Schedule B data is not current.

The SIMSI Operational Committee meets monthly and part of its mandate is to address data integrity issues. However, we found that validation of data procedures has not been established. Consequently, there are no formalized sign-off procedures in place to change data within the system nor are there formalized sign-off procedures attesting to the accuracy of the data.

Recommendations for 3.8.2

We recommend the following:

24. A SIMSI CSIF and BIF Project Charter should be developed to operationalize the governance structure approved by the Executive Committee. The Project Charter would define the respective roles and accountabilities of the INFC stakeholders and FDPs in the context of the evolving functionality of the SIMSI CSIF and BIF modules.

25. A SIMSI CSIF and BIF Service Level Agreement (SLA) should be prepared that defines Offices of Primary Interest (OPI) responsible for SIMSI CSIF and BIF data. The SLA would specifically address data integrity issues by providing guidance on such matters as the accuracy and timeliness of data in the OPI area of responsibility, data validation/sign-off procedures and service levels.

3.9 Output Indicators (project results)

Output indicators have been established to define the results of a project. These Output indicators have been established for each project through the due-diligence process performed and included in each Treasury Board Submission.

Findings for 3.9

Through our discussions with POB, we found that INFC does not gather program and project information with respect to the output indicators identified in the Treasury Board Submissions.

We have been told that this is the case because the output indicators established for the individual projects are often immeasurable or unrealistic. As a result, there is no reporting concerning the achievement of expected results with respect to the output indicators for each project.

A lack of reporting concerning the output indicators established for the projects is not enabling INFC to acknowledge its performance in implementing the CSIF and BIF projects and fulfil its accountability.

Recommendations for 3.9

26. INFC should develop a reporting system that would allow the reporting of information with respect to its output indicators established.

27. If the output indicators identified for the projects are considered inappropriate, INFC should ensure that any new Treasury Board Submission has measurable and realistic output indicators.

3.10 Risk Management

As part of our audit, risk management has been assessed from a program and from a project perspective. The Issues Management Directorate (IMD), part of POB, is responsible for risk management at the CSIF and BIF program level and project level. There is also a Departmental risk management perspective for which the Corporate Services Branch has initiated actions, namely the Corporate Risk Profile; however, we focused on the program and project level risk management, directly affecting the management of CSIF and BIF.

3.10.1 Program Risk Management

Managing program risk is important to proactively address potential issues affecting program delivery. The Program Risk Management was initiated starting with the preparation of the RBAF in 2004. Since then, Issues Management Directorate (IMD) has prepared and updated Risk Profiles for each key Program risk area of CSIF and BIF. These Risk Profiles include an identification of the risk area, an assessment of the risk and mitigating strategies to be implemented in order to manage the related risk.

Findings for 3.10.1

The Risk Profiles have been reviewed at INFC several times since the inception of the documents in 2004, with the last review completed in December 2005. The strategies to mitigate the risks identified as part of the risk assessment have not been incorporated into a work plan and most of the identified risks have not yet been addressed at the time of our audit.

We acknowledge that IMD has put in place a risk management framework (documented in its "Update on Risk Management" presented to the Departmental Audit and Evaluation Committee in June 2006), but the actions identified under mitigating strategies in the Risk Profile to address the risks have not yet been acted upon.

This situation can be attributed primarily to the fact that the accountability for risk areas has not been identified. As a result, there is no integration of the risk assessment initiatives into the management of the Programs.

Consequently, risks to CSIF and BIF identified as low, medium and high have not been managed and the Programs continue to be exposed to the same risks as they originally were three years ago. We noted that many risks identified had been assessed as "high risks."

Recommendations for 3.10.1

28. Responsibility at the Branch, Directorate, Division or Individual level should be assigned for risks areas identified in the Risk Profiles.

29. INFC management should incorporate in their integrated planning process the specific actions to be undertaken to address the risks identified.

3.10.2 Project Risk Management

As part of a sound project management approach, risk management should be documented and monitored to mitigate any risk jeopardizing project implementation.

With respect to large scale CSIF and BIF projects, many circumstances/risks can result in significant project modifications that may result in cost overruns or scope reductions. These situations can be the result of:

  • environmental assessments;
  • delays to the start of construction that have an impact on initial budget estimates, and on the political will; and
  • engineering issues.

Currently, the Policy and Priorities Directorate (PPD) is involved in assessing and reviewing these risks before a project is announced. The Issues Management Directorate (IMD) has been involved in assessing and managing these risks after the projects have been announced.

Findings for 3.10.2

Environmental risks are always addressed formally before a Contribution Agreement is signed. Although other significant project risks are addressed informally before the Contribution Agreement is signed, there are no formal project risk management guidelines or systematic risk assessment performed before a Contribution Agreement is signed for a project. Furthermore, project risk assessment and management initiatives, other than environmental risks, are not formally documented. These initiatives would be undertaken before the Contribution Agreement is signed, by IMD as part of its risk management responsibilities.

Starting with CSIF II, an interim due diligence was performed by PPD before the Ministerial Announcement, but it did not involve a thorough risk assessment and was not documented.

Currently, an interim due diligence for BIF is performed by PPD before the Ministerial Announcement, but it does not involve a thorough risk assessment and was not documented.

Two reasons have been identified to explain the current condition. The first reason is that there are no project risk management guidelines in place to formalize project risk management, with the exception of environmental risks. The second reason is that the staffing levels are insufficient to prepare and formalize documented procedures and guidelines for project risk management. Specifically, there is only one position responsible for this area and it has been vacant since December 2006.

The lack of project risk management guidelines, systematic risk assessment and management procedures prior to entering into a Contribution Agreement , could result in significant risk being left unidentified and/or no mitigating strategies being implemented as part of the Contribution Agreement . Significant unidentified project risk could have implications on a project's results, costs or successful implementation.

Recommendations for 3.10.2

30. INFC should develop and implement a project risk management guideline with formal risk assessment procedures to be initiated on each CSIF and BIF project before the Contribution Agreement is signed. This procedure should detail the level of assessment to be performed, who would be responsible to execute it, when it should be executed and the documentation that would result from this process.

31. Detailed mitigating strategies should be developed and incorporated into the Contribution Agreement or guiding documents.

Appendix A

Audit of the Management Control Framework of CSIF and BIF List of Recommendations and Management Response

Recommendations resulting from findings (in the order they appear in the audit report)

The responses below, approved by Senior Management, reflect the actions taken or about to be taken to address the recommendations presented in the audit report.

Compliance with Treasury Board Transfer Payment Policy and the Financial Administration Act.

Recommendations Responsibility Response and Actions to be taken (including timeline)
1. Cash management calendar and procedures need to be developed and communicated by INFC to all federal delivery partners. ADM Corporate Services Branch The calendar and the procedures were developed over the summer by CS and POB and were shared with TC in September 2007 and were sent to delivery partners in November, 2007. Implementation will follow discussions with the federal delivery partners.
2. The Contribution Agreement and Schedule B template wording needs to be improved to confirm that the fiscal year breakdown is a ceiling, not an estimate and that any adjustment required to Contribution Agreement Schedule B for Canada's contribution be approved and requested to INFC by the ASC. ADM Corporate Services Branch This approach will be adopted for the MIC component of the BCF (under development) and implemented for existing CSIF and BIF projects in the 08-09 fiscal year, through discussion at the next ASC meeting for each project.
Strategic and Operational Planning
3. INFC should implement a strategic planning process and develop an integrated operational plan. The integrated operational plan would be for a period of at least one year and should be revised annually. Furthermore, it should include subject matters related to Finance, Human Resources and risk mitigating strategies to address CSIF and BIF concerns. INFC Executive Committee The Executive Committee will determine before March 31, 2008 the nature and extent of strategic and integrated operational planning that is appropriate given the mandate, size and structure of the Department. This will occur after the reorganization has been agreed to and will be in place in 2008-9.
4. Regular reporting on the progress towards the integrated operational plan should be made to the Executive Committee to ensure there is appropriate accountability for the management of CSIF and BIF. INFC Executive Committee The Executive Committee will determine (in conjunction with the preceding recommendation) the nature, extent and timing of reporting against the integrated operational planning that it decides to institute for 2008-9
5. INFC should ensure that they have an adequate and qualified resource responsible for putting in place and managing the planning cycle. ADM Corporate Services Branch Resourcing of the planning function will be determined in conjunction with the two previous recommendations, in the first quarter of 2008.
Governance
6. INFC should document what is expected from each Branch, Directorate and Division to fulfill its mandate with respect to CSIF and BIF and its other programs. This information should be communicated throughout the organization to ensure that there is a common understanding. INFC Executive Committee The roles and responsibilities of each branch, division and directorate will be documented once the reorganization of INFC is complete (spring 2008) and communicated to all staff.
7. INFC should formalize a process that would ensure that the Control Framework of the more significant FDPs is being reviewed periodically. This will help ensure that appropriate monitoring and reporting mechanisms exist at the FDP level and allow INFC to construct its own monitoring procedures based on the assurance provided by this information. ADM Program Operations Branch The DG, Issues Management will formalize and implement the periodic review of FDP activity for existing programs and for the BCF by March 31, 2008.
8. The draft ASC Handbook should be finalized and communicated to all members of the ASC. ADM Program Operations Branch The ASC Handbook for CSIF and BIF projects will be finalized by June, 2008 and communicated to the ASC members of each project.
9. The Handbook should detail the ASC objectives and provide guidance on the administration of the ASC: frequency of meetings, required attendance (quorum), reporting requirements, etc. ADM Program Operations Branch The Handbook will state the objectives of the ASC and give guidance on its administration.
10. An office of primary responsibility should be established for the ASC Handbook. ADM Program Operations Branch The DG, Issues Management in POB has the primary responsibility for the ASC Handbook
11. For each guideline, an office of primary responsibility should be established and sign-off procedures be put in place to take into account stakeholders with coincidental interest in the respective guideline. These guidelines should be referenced in the MOU and/or the Contribution Agreement as governing instruments. ADM Program Operations Branch The ADM, POB, in conjunction with ADM, CS will agree on the primary and secondary responsibilities for each guideline. It is not intended to alter existing MOUs or Contribution Agreement s, but any new Contribution Agreement s under CSIF and BIF will do so.
Recruitment and Retention of Personnel
12. Additional procedures should be initiated urgently to ensure that INFC has the required capacity to deliver on its responsibilities, especially in POB as the resources are critically low. ADM Program Operations Branch Since December, 2006, INFC has proactively introduced innovative practices to enhance staffing activities, in particular the use of collective staffing actions for ES positions. When these did not produce entirely satisfactory results, the guidelines for the use of non-advertised and bilingual imperative staffing were reviewed and modified to permit greater flexibility. The impact of these changes will be assessed in January, 2008
13. The problem of staff retention in INFC should be examined and an action plan developed to address issues identified. INFC Executive Committee

Director HR

The issue of retention is one of the priority areas identified in the Strategic HR plan; specific strategies will be elaborated at the Corporate and Branch levels in the first quarter of 2008. Significant work has been done on improving the orientation of new employees, including the preparation of a booklet to introduce them to the department and discussion of how best to structure orientation sessions. Specific actions will be determined in the context of the department's strategic HR plan.
14. The roles and responsibilities of the Human Resources Division and the other Branches and Directorates should be clearly defined and communicated with respect to the Recruitment and Retention of Personnel. INFC Executive Committee This will be dealt with in conjunction with the action on the recommendation to document all roles and responsibilities as well as in the elaboration of the Strategic HR plan (first quarter of 2008)
15. As part of the establishment of an integrated planning process, INFC should include a Human Resource plan, detailing the specific competitions required to address the current and future needs of the Programs. ADM Corporate Services Branch A Strategic HR Plan was developed and presented to the HR Committee and Executive Committee in early November and submitted to the Deputy on Nov 15 for his views. The Deputy has approved the Strategic HR Plan in early December.
Monitoring
16. An explicit project monitoring accountability framework should be drafted and communicated to the relevant INFC staff. This framework should detail what areas should be monitored, what monitoring duties should be performed and what key documentation should be retained in the files to support this monitoring process. ADM Program Operations Branch The DG, IM will document the project monitoring accountability framework for CSIF and BIF (and for MIC/BCF), with the concurrence of her colleagues in POB and the relevant parties in CS by June 2008.
17. A filing system for CSIF and BIF projects should be documented and implemented to allow for efficient and uniform retrieval of documents, especially numerical indexing of files that would allow easy retrieval of documents. A file checklist should be implemented detailing the minimum expectations with respect to file content. The report control sheet template, which was drafted by an employee in POB, but never implemented, should be discussed, revised and implemented as soon as possible. This checklist will help ensure that the minimum required project monitoring documentation is filed appropriately. ADM Program Operations Branch The DG, Intergovernmental Operations will implement a standardized filing system, including a checklist, for major projects (in CSIF, BIF and MIC/BCF) by March 31, 2008, with the assistance of CS, Information Management staff. The DG, Issues Management will develop the model project file and provide guidance for its implementation in POB.
18. INFC should require that the FDPs provide a periodic report summarizing which audits are due to be performed, which audits have been performed, and their results, along with management actions concerning any issues identified. ADM Program Operations Branch The ADM has already contacted the FDPs to determine the audit activity under ICP in the summer of 2007 and will do the same for the other infrastructure programs by June 30, 2008. The process will be documented in the ASC Handbook which, as mentioned in an earlier response, will be finalized by June 30, 2008.
19. The Audit Guidelines should be revised to add clarity concerning scope, reporting requirements and other weaknesses that have been identified. This could be done in consultation with Transport Canada, since they have already initiated this process. ADM Program Operations Branch The DG, Issues Management, in conjunction with the CFO, will clarify the audit guidelines for major projects in CSIF, BIF and MIC/BCF by March 31, 2008. All delivery partners will be consulted.
20. To be consistent with the RBAF, the Audit Guideline should allow for a risk-based audit methodology to determine the extent of audits required. ADM Program Operations Branch The audit guidelines mentioned in the previous recommendation will be based on a risk based assessment and will in turn require audit plans of the delivery partners to be based on a risk based assessment of each project.
21. INFC should ensure that the center of expertise for the recipient audit monitoring function in POB is adequately staffed and that the responsibilities for this position are clearly defined and communicated. ADM Program Operations Branch This position was staffed in August, 2007 and additional support (2 staff) was added in November. The responsibilities were mapped and presented to DAC in June 2007.
Information for Decision Making
22. A decision should be made with respect to the usage (or not) of Shared Information Management System for Infrastructure (SIMSI) as the key reporting system of relevant project/program information for the management of CSIF and BIF. ADM Program Operations Branch SIMSI will be the key reporting system of relevant project/program tombstone information for the management of all infrastructure programs.
23. The development of the executive report designed by the ADM of Program Operations Branch (POB) should be completed, put in place, and revised, if needed, when a decision is reached with respect to the use of SIMSI. This report should be developed to facilitate the dissemination of information to be shared with the Executive Committee. ADM Program Operations Branch In order to obtain the information necessary to respond with accurate, timely information on project status and issues, POB will work with CS/IMIT to determine what information resides/should reside in SIMSI and what additional information must be gathered by other means from the delivery partners by March 31, 2008.
24. A SIMSI CSIF and BIF Project Charter should be developed to operationalize the governance structure approved by the Executive Committee. The Project Charter would define the respective roles and accountabilities of the INFC stakeholders and FDPs in the context of the evolving functionality of the SIMSI CSIF and BIF modules. ADM Corporate Services Branch The roles and responsibilities related to SIMSI will be documented in the department wide documentation that will be prepared as the reorganization is completed in the first quarter of 2008. As further development of the CSIF and BIF component of SIMSI will be limited because of the advent of the BCF/MIC, no project charter will be developed for CSIF and BIF. However, the SIMSI Steering Committee, which includes representatives from the department and from FDPs oversees the residual development work.
25. A SIMSI CSIF and BIF Service Level Agreement (SLA) should be prepared that defines Offices of Primary Interest (OPI) responsible for SIMSI CSIF and BIF data. The SLA would specifically address data integrity issues by providing guidance on such matters as the accuracy and timeliness of data in the OPI area of responsibility, data validation/sign-off procedures and service levels. ADM Corporate Services Branch The SIMSI Steering Committee is already addressing the issue of data quality in a series of collaborative initiatives covering the various infrastructure programs. The OPI is the DG, Intergovernmental Operations, who works closely with the other DGs in POB and the directors in IMIT. The appropriate documentation will be prepared, but until the nature and extent of the problems is explored, it is not possible to determine a timeframe for completion of this documentation.
Output Indicators (Project results)
26. INFC should develop a reporting system that would allow the reporting of information with respect to its output indicators established. ADM Program Operations Branch and ADM Policy and Communications Branch The ADM, Policy and Communications presented a proposed approach for redefining INFC's strategic objective and improving performance reporting to EXEC on November 21, 2007. The work is targeted for completion by June, 2008. Any adjustments to the IT system would be made following the completion of this work and with due regard to cost/benefit. Until the extent of changes is determined, it is not possible to estimate the timing for these system changes.
27. If the output indicators identified for the projects are considered inappropriate, INFC should ensure that any new Treasury Board Submission has measurable and realistic output indicators. ADM Program Operations Branch and ADM Policy and Communications Branch INFC has taken a new approach in the RMAF/RBAF (ARAF) for the BCF and PT Base, which reflects the search for measurable and realistic output indicators. As outlined in the previous response, a department-wide initiative was launched on Nov. 21 to review our approach to performance measurement in light of the current mandate and programs, as well as current TB requirements.
Risk Management
28. Responsibility at the Branch, Directorate, Division or individual level should be assigned for risks areas identified in the Risk Profiles. ADM Program Operations Branch The DG, Issues Management will ensure that responsibility for risks identified in the risk profile of the branch, the program and the projects is identified by June, 2008. A draft project level risk assessment template has already been developed for use by POB analysts.
29. INFC management should incorporate, into their integrated planning process, the specific actions to be undertaken to address the risks identified. ADM Program Operations Branch This is the responsibility of all senior management, not just the ADM, POB. Senior management has already accepted the Corporate Risk Profile in early 2007 and work is proceeding on lower-level risk profiles as time permits.
30. INFC should develop and implement a project risk management guideline with formal risk assessment procedures to be initiated on each CSIF and BIF project before the Contribution Agreement is signed. This procedure should detail the level of assessment to be performed, who would be responsible to execute it, when it should be executed and the documentation that would result from this process. ADM Program Operations Branch and ADM Policy and Communications Branch The ADMs, P&C and POB will design the due diligence process for the MIC/BCF to ensure that risk is assessed and documented for each project before the Contribution Agreement is signed. The due diligence and risk assessment has already been done on the majority of CSIF and BIF projects, and is done in conjunction with the Treasury Board submission for those that are still in the pipeline. The new process will be implemented in January, 2008 both for BCF and the residual CSIF and BIF projects.
31. Detailed mitigating strategies should be developed and incorporated into the Contribution Agreement or guiding documents. ADM Program Operations Branch Contribution Agreement s already include (at least implicitly) risk mitigation strategies. What will be addressed as outlined in the previous recommendation is the strengthening of the risk assessment process and an improved documentation of both the process and the results.

Appendix B

Audit of the Management Control Framework of CSIF and BIF

Audit Opinion and Statement of Assurance (Information on the types of Audit Opinion and Statement of Assurance – as per the Office of the Comptroller General's Statement of Internal Audit Standards – Preliminary Draft)

Types of audit opinion that can be formulated

The auditor's opinion is based on logical specific deductions about the audit objectives based on the findings against criteria and noteworthy accomplishments in the report. Standardized language is needed in order to ensure consistency in the formulation of an opinion across different audit areas and different time periods. In addition, it allows the deputy head and other readers to better understand the opinion being expressed.

In my opinion, the [audit entity examined]:
is well controlled (at least one of the first two criteria needs to met and the third criterion needs to be met)

  • well managed, no material weaknesses noted; or
  • well managed, but minor improvements are needed; and
  • effective and sustainable.

Or: It has moderate issues requiring management focus (at least one of the following two criteria need to be met)

  • significant control weaknesses, but exposure is not serious; or
  • multiple risk areas, but exposure is not serious.

Or: It requires significant improvements (at least one of the following four criteria need to be met)

  • financial adjustments material to line item or area; or
  • control deficiencies represent serious exposure; or
  • major deficiencies in overall control structure; or
  • financial adjustments material to the company.

Statement of Assurance

The opinion is supported by a clear indication to users of the level of assurance being provided. Users need to be informed of the auditor's judgment about the confidence that may be placed in the opinion above. This section informs the reader that the auditor has conducted the engagement in accordance with the Treasury Board Policy on Internal Audit (and IIA Professional Internal Auditing Standards - provided that the results of the quality assurance program indicate that the internal audit function is in compliance with the Standards). This section also explains that the auditor has examined sufficient, relevant evidence and obtained sufficient information and explanations to provide a reasonable level of assurance on the reported opinion or conclusions. (Not all assurance engagements are designed to provide, or result in providing, a reasonable level of assurance. When this is the case, the auditor's statement of assurance shall identify the circumstances (e.g., limited procedures, inconclusive evidence, etc.) that affect the level of assurance and shall indicate the auditor's caution on reliance on these opinions or conclusions).

Date modified: