Internal Audit Report - Audit of the Management Control Framework for the Public Transit Fund
Final Report–June 2009
Table of Contents
- 1. Executive Summary
- 2. Background
- 3. Audit Objective
- 4. Audit Scope
- 5. Audit Approach
- 6. Audit Findings
- 7 Audit Opinion
- 8 Statement of Assurance
1. Executive Summary
The Public Transit Fund PTF Transfer Payment Program was established in 2005 to contribute to the Government of Canada's environmental objectives of cleaner air and reduced greenhouse gas (GHG) emissions through targeted support for public transit infrastructure in Canadian communities.
In May 2005, the Government of Canada approved an amount up to $800 million to be allocated to public transit in the provinces and territories on a population-based formula. During the fiscal year 2005-06, Infrastructure Canada allocated $400 million to the provinces and territories. The remaining $400 million was redirected to the Public Transit Capital Trust Fund, managed by the Department of Finance Canada.
The audit recognizes that the PTF Transfer Payment Program was designed to build upon the program design created for the Gas Tax Fund (GTF) Transfer Payment Program. By utilizing the GTF architecture for the PTF, the Government of Canada simplified the negotiation process with the provinces and territories and minimized the need for an additional delivery mechanism and reporting structures. With that being said, the audit found that any term or one time allocation of funds program should be regarded and managed as a separate independent program in all its aspects.
The audit found that the governance and accountability frameworks that were established for the PTF program were implemented through agreements that were negotiated, written and signed by the parties. The PTF Agreements were developed in a manner that respected the three levels of government (Federal-Provincial-Municipal), and recognized the need to build strong partnerships that balanced the flexibility in delivering the program while establishing the conditions for governance and accountability. More specifically, the PTF Agreements entail a tripartite of roles and responsibilities where INFC designed and monitors the program, the provinces and territories (or municipal associations) implement and administer the program based on their jurisdictional requirements by receiving and redistributing the funds and managing the reporting on the fulfillment of their obligations and commitments, and the municipalities make the decisions on which eligible projects to invest in based on local priorities, manage the implementation of the eligible projects and report on results.
The audit determined that INFC manages the PTF program effectively by overseeing three key areas:
- nurturing the relationship with the provinces and territories;
- reviewing the Annual Expenditure Reports and the Audit Reports (AER/Audit Report); and,
- defining and reporting on the results achieved through the use of Public Transit Funds.
A summary of our observations and of the opportunities for improvements related to each of these topics follows:
- In order to nurture its relationship with the provinces and territories INFC has created a distinct and dedicated organizational unit that is staffed with a strong team of experienced employees who have evolved in line with the development and implementation of the GTF transfer payment program and the addition of the PTF.
- INFC has made the review of the AER/Audit Report the focal point of its monitoring and reporting activities and has developed a number of tools and processes to perform and support its reviews. The implementation of these processes, however, should be strengthened to ensure that any PTF problems are identified, addressed, and resolved in a timely manner, and to provide feedback to recipients.
Infrastructure Canada's Shared Information Management System for Infrastructure (SIMSI) is the key reporting system of relevant project and program tombstone information for the management of all its infrastructure programs. The recipients of the PTF program, however, were given the opportunity to use their own systems and only two provinces have adopted SIMSI. As a result, the information provided to INFC arrives in various formats and the program area must perform a manual process of national consolidation that is cumbersome, time-consuming, vulnerable and sensitive to error.
Overall, the monitoring and reporting system adopted by INFC needs to be better documented and enhanced to ensure that program and senior management are provided with timely, reliable, and complete information to support their decision making and to communicate the results of PTF Agreements.
- The recipients of the PTF program must provide an account of the results achieved by incorporating performance information in their annual AER/Audit Reports and by submitting an Outcomes Report by September 30, 2009, and periodically thereafter. INFC, in turn, relies on this information to prepare a number of national reports such as the Departmental Performance Report, the Report on Plans and Priorities, or other special reports.
The audit found that many jurisdictions were uncertain about the approach that they should follow to select the performance indicators to report on their outcomes. INFC has undertaken a significant amount of work to resolve this problem but more needs to be done.
Our overall opinion on the adequacy and effectiveness of the management control framework and the appropriateness of the management practices that have been put in place for the Public Transit Fund Transfer Payment Program is that they are satisfactory to ensure compliance, program effectiveness and financial integrity.
Original signed by
Linda Saunders
Chief Audit Executive
Office of Infrastructure Canada
2. Background
Established in 2005, the Public Transit Fund (PTF) Transfer Payment Program is designed to contribute to the Government of Canada's environmental objectives of cleaner air and reduced greenhouse gas (GHG) emissions through targeted support for public transit infrastructure in Canadian communities.
In May 2005, the Government of Canada approved an amount up to $800M to be allocated to public transit in the provinces and territories on a population-based formula. During the fiscal year 2005-06, Infrastructure Canada allocated $400M to the provinces and territories for the PTF Transfer Payment Program. The remaining $400M were redirected to the Public Transit Capital Trust Fund, managed by the Department of Finance Canada.
The PTF Transfer Payment Program was designed to build upon the program design that was created for the Gas Tax Fund (GTF) Transfer Payment Program. By utilizing the terms and conditions of the GTF as a base for the agreements for the PTF, the Government of Canada minimized the need for an additional delivery mechanism and reporting structures, as well as simplified and facilitated the negotiation process with the provinces and territories.
Like the GTF Transfer Payment Program, the PTF is categorized as an 'Other Transfer Payment Program' as it does not fit either the typical contribution program model or the grant model. Unlike typical contribution programs, the funding under this program flowed to the recipients before the expenditures for which it was intended were incurred. Unlike typical grant programs, the PTF is based on a formula that requires annual appropriations that had been determined for the two year period, and imposes a significant number of conditions on the recipients until the end of the agreement. Finally, the transfer payments were made to a combination (that varied by province/territory) of provinces, territories, and at the request of provincial/territorial municipal entities, transit agencies or municipalities.
The PTF Agreements signed by the parties entail a tripartite of roles and responsibilities where INFC designed and monitors the program, the provinces and territories (or municipal associations) implement and administer the program based on their jurisdictional requirements by receiving and redistributing the funds and managing the reporting on the fulfillment of their obligations and commitments, and the municipalities make the decisions on which eligible projects to invest in based on local priorities, manage the implementation of the eligible projects and report on results.
As of May 5, 2009, the PTF provided $400M of funding to all provinces and territories. In turn the provinces and territories, through their respective municipalities, have invested $230.04M as follows:
The Gas Tax Fund provided $1,950M of funding to all provinces and territories (excluding First Nations) and the provinces and territories, through their respective municipalities have invested $1,793M as follows:
Amounts in $000's
| Signatory | Public Transit Fund Jurisdictional Allocation (2005 - 2006) |
Cumulative $ Received from Canada (2006-2007) |
2007-2008 $ Invested |
Cumulative$ Invested and/or Committed (2006-2008) |
|---|---|---|---|---|
| British Columbia | 52,543 | 52,543 | 6,251 | 43,058 |
| Alberta | 40,091 | 40,091 | 16,352 | 36,355 |
| Saskatchewan | 12,463 | 12,463 | 0 | 0 |
| Manitoba | 14,653 | 14,653 | 1,535 | 2,374 |
| Ontario | 155,169 | 155,169 | 11,141 | 133,896 |
| Quebec | 94,443 | 94,443 | 2,711 | 2,711 |
| New Brunswick | 9,408 | 9,408 | 0 | 1,246 |
| Nova Scotia | 11,732 | 11,732 | 0 | 8,288 |
| Prince Edward Island | 1,727 | 1,727 | 0 | 1,727 |
| Newfoundland & Labrador | 6,473 | 6,473 | 0 | 0 |
| Yukon | 391 | 391 | 0 | 391 |
| Northwest Territories | 536 | 536 | 0 | 0 |
| Nunavut | 371 | 371 | 0 | 0 |
| TOTAL1 | 400,000 | 400,000 | 37,991 | 230,044 |
- [1]Please note that the sum of the values presented in each column may vary in comparison with the total presented due to rounding.
- In June 2008, the Infrastructure Canada (INFC) Departmental Audit Committee (DAC) approved a management control framework audit of the Public Transit Fund Transfer Payment Program.
The following sections identify the audit objective and scope that have been defined for this audit.
3. Audit Objective
The objective of the audit was to provide assurance that an effective management control framework (policies, procedures, systems and controls), and appropriate management practices are in place, to ensure compliance, program effectiveness and financial integrity.
4. Audit Scope
The scope of the audit included an assessment of the effectiveness of the management control framework and appropriateness of management practices for the PTF Transfer Payment Program, as well as an assessment of a sample of key financial and non-financial documents associated with the PTF Transfer Payment Program.
5. Audit Approach
The audit was conducted in accordance with the Government of Canada internal auditing standards.
The auditors used the technique of directed sampling to determine the sample size and to select the items to be reviewed and persons to be interviewed. This approach is the most appropriate given that many of the audit tests and procedures fall outside a statistical context e.g. review of meeting agendas, records of discussions, and interviews with key stakeholders. A directed sample is a non-statistical sampling technique and the findings cannot be extrapolated to the whole population.
Given that the transfer payment program design for PTF was created as a mirror image of the GTF and both programs are managed by INFC's Program Operations branch, it has been determined that these two management control framework audits would be performed concurrently.
As a result, the examination phases of both audits were performed concurrently and began at the end of January 2009 to conclude at the end of April 2009. The examinations employed various techniques including interviews, review of financial and non-financial documentation, sampling of files/records and analytical reviews. The audit criteria were based upon applicable policies, procedures, and legislation as well as Treasury Board Secretariat's Core Management Controls.
As part of the preliminary planning for these management control framework audits, a risk assessment of the Public Transit Fund Transfer Payment Program was conducted against the Management Accountability Framework, developed by the Treasury Board of Canada Secretariat (TBS).
The results of the risk assessment indicated that the components of the framework listed below merited further examination for the Public Transit Fund Transfer Payment Program and were included in the scope of this audit.
Governance & Strategic Direction – The audit examined the governance processes in place to support the Public Transit Fund (PTF) Transfer Payment Program. These governance processes included the program design and risk management processes that support corporate priorities (including the allocation of resources), the monitoring and reporting of program results, and the overall organizational and accountability structures.
People – The audit examined Infrastructure Canada's internal capacity to support the PTF Transfer Payment Program to determine if the research and analytical capacity is developed and sustained to support monitoring, reporting, and program advice to senior management.
Stewardship – The audit examined Infrastructure Canada's federal partnership agreements with the provinces, territories, and municipal (or transit) associations to ensure that the agreements were established in compliance with the terms and conditions specified in the Treasury Board Submission related to the Public Transit Fund Transfer Payment Program. The audit also determined if Infrastructure Canada's control regime (financial, human and informational resources) is integrated, effective, and communicated to all staff.
Accountability – The audit examined Infrastructure Canada's federal partnership agreements with the provinces, territories, and municipal (or transit) associations to ensure that roles, responsibilities, and accountabilities for results for the PTF Transfer Payment Program, are clearly assigned and consistent with resources, and delegations are appropriate to capabilities, communicated and understood.
Policy and Programs – The audit reviewed the policies and procedures in place for the PTF Transfer Payment Program to ensure that they are appropriate and adequate for ensuring compliance, program effectiveness and financial integrity. The audit also determined whether policies and procedures have been communicated and understood.
Results and Performance - The audit reviewed the process in place to support monitoring and reporting of relevant information on results of the Public Transit Fund Transfer Payment Program and determined the extent to which it is used to support program decisions.
Reporting
Following the examination phase the audit team prepared a number of Finding Sheets together with Recommendations and provided them to senior management of the Programs Operations branch for validation and comments.
Once the Finding Sheets were completed and validated, the auditors gathered all relevant documentation and prepared a draft report for the ADM, Program Operations. The ADM, in turn, provided feedback, comments, and formulated the management action plan in response to the audit recommendations.
This report, including the management action plan, was presented and approved by the Departmental Audit Committee on June 11, 2009.
6. Audit Findings
For purposes of this report, the residual risk rankings associated with the audit findings use a low, moderate, high three-point scale and are subjectively judged based on our knowledge of the PTF management control framework gathered during the audit. The subjective criteria are:
High – Threats/Opportunities have very significant impact on INFC's objectives, are imminently likely and no, or uncertain mitigation measures, are in place.
Moderate – Threats/Opportunities have significant impact on INFC's objectives, have a longer-term likelihood and reliable mitigation measures are planned or are being established.
Low – Threats/Opportunities do not have a significant residual risk to INFC's objectives.
6.1 Governance and Strategic Direction
The PTF was designed to build upon the Gas Tax Fund (GTF) agreements by providing funding targeted to public transit infrastructure. As a result, the governance processes of the GTF, the accountability framework defined in the terms and conditions of the GTF agreements, and the organization structure involved with the delivery of the GTF were all used to create and deliver the PTF. This approach allowed INFC to minimize the need for an additional delivery mechanism and reporting structure and to benefit from the experience and dedication of the team of employees currently responsible for the GTF.
The audit reviewed a number of PTF Agreements signed with the provinces and territories and found that they were negotiated and written in a manner that respects the three levels of government (Federal-Provincial-Municipal). The Agreements also respect the need to balance the flexibility in delivering the program while establishing the conditions for accountability.
The audit also finds that the key Governance processes currently in place to support the PTF, i.e. the Program Design, the Oversight Committee, the National Workshops, the Issues Management Committee and Program Management Committee are appropriately conceived.
On the other hand, the implementation and the integration of the PTF into the GTF have produced a number of unintended consequences. For example, the audit observed that the Oversight Committee meetings are structured around the GTF and no specific agenda items are devoted to the PTF. The National workshops are geared towards the eligible project categories and commitments of GTF whereas the PTF items are incorporated as required through the GTF eligible project category 'public transit'. Our interviews with the participants confirmed that they view the PTF as a 'top-up' of the eligible project category 'public transit' of the GTF rather than an independent program.
Our discussions with INFC management confirmed that the risks associated with the PTF are lower than the GTF and the PTF was not on the GTF agenda because there were no issues to discuss.
In conclusion, the experience with the PTF should be used as a lesson learned: any term or one time allocation of funds program should be regarded and managed as a separate independent program in all aspects. This means that they should be included as explicit items on Oversight Committee agendas and internally managed as specific programs rather than a component of another program. Additionally, INFC should ensure that complete and accurate records are maintained.
The PTF is a sunsetting program and all the distributed funds must be spent by March 31, 2010. At the time of audit work, the information presented indicated that a number of provinces and territories had not spent the funds that they had received and in some cases had never reported on the use of those funds. The audit identified this as an area of risk in that all of the Public Transit funds may not be spent when the program comes to an end.
Management recognizes the existence of a potential risk; however, the risk level needs to be qualified given that the PTF program parameters and systems are established in the provincial and territorial jurisdictions. In addition, the project activities within the provincial and territorial jurisdictions are progressing and expect to meet the spending commitment deadline of March 31, 2010.
| Recommendations | Management Action Plan |
|---|---|
| 6.1.1 No action is required. | 6.1.1 N/A |
| Manager Responsible: | Assistant Deputy Minister, Program Operations |
6.2 Stewardship
Financial Controls:
The audit found that the financial authorities, policies and procedures related to the accounting of expenditures are established and communicated, the payments made to initial recipients comply with the terms and conditions specified in the TB Submission and the initial recipient's Agreements, and the payment processing requirements were completed in a timely manner.
Monitoring and Reporting Controls:
In order to ensure that provinces and territories are responsible and accountable for the monies that they have been allocated and received under PTF transfer payment program, as well as the PTF investment decisions that have been made within their jurisdiction, Infrastructure Canada requires that an Annual Expenditure Report and Audit Report (AER/Audit Report) be submitted annually by each province and territory.
The audit established that the AER/Audit Report is the key monitoring and reporting instrument that INFC receives and reviews on an ongoing basis to ensure compliance to program requirements and the achievement of program results. INFC has made these reports the focal point of its monitoring and reporting activities and has developed a number of tools and processes to perform and support its analyses.
The auditors selected nine initial recipient AER/Audit Reports for detailed review and found a number of weaknesses and inconsistencies: the analysis of the financial statements is generally limited to verifying the accuracy of the numbers without a thorough analysis of items such as the investment rate of return used to calculate the interest earned on unspent funds, and administrative costs; the list of projects is accepted without further examination of key information such as their status of implementation or completion; the control matrix used to summarize the analysis of the AER/Auditor Report by the program officer is not employed consistently and a number of omissions were observed. In some cases there is no evidence on file to indicate that the AER/Audit Reports had been analyzed and accepted by the program, and in one case the initial recipient has not yet provided an AER/Audit report to account for its PTF expenditures.
The audit concluded that the implementation of the AER/Audit Report monitoring and reporting process is adequate but it needs to be strengthened to ensure that any PTF problems are identified, addressed, and resolved in a timely manner, and to provide feedback to recipients regarding opportunities for improvement.
Finally, the tracking mechanism adopted by INFC to monitor the status of the PTF investment decisions made by each provincial and territorial jurisdiction and to provide a national consolidation to senior management is a critical element in the overall monitoring and reporting process.
Our review of these documents exposes important gaps in the information provided and there was no indication that the problems identified were being communicated to senior management and properly addressed. Specifically, there was no indication at the time of the review to suggest that the unspent PTF dollars highlighted in the national consolidation of PTF AER/Audit Report information had been communicated to senior management.
Consequently, there is a risk that all PTF funds will not be spent when the program comes to an end, some projects that have been funded may, in fact be ineligible, and INFC may not be able to account for the use of all the PTF funds.
Information Management
Infrastructure Canada's Shared Information Management System for Infrastructure (SIMSI) is the key reporting system of relevant project and program tombstone information for the management of all its infrastructure programs. The audit found, however, that the SIMSI system was not designed to easily support the reporting and information management requirements of a non-application based infrastructure transfer payment program like the Public Transit Fund.
As part of the PTF program design, the audit found that the provinces and territories were encouraged to use their existing reporting mechanisms to capture the required financial and results information from their municipalities. As a result, only two provinces have adopted SIMSI while all the other provincial and territorial jurisdictions have decided to use their own application or other tools such as Excel spreadsheets or Word documents to prepare and submit their AER/Audit Report information.
By allowing the PTF recipients to utilize their existing reporting mechanisms, INFC has increased its difficulty in securing timely, complete and reliable information. The program area must collect the provincial and territorial AER/Audit Reports in various formats and consolidate all of the information into a national summary. The resulting data is stored in an Excel spreadsheet format and is manually entered and stored into the enterprise data warehouse of SIMSI that was originally created for PTF data. The audit finds this reporting system to be cumbersome, time-consuming, vulnerable and sensitive to error.
In order to resolve this problem INFC has issued new guidance and the situation has improved but additional clarification is required.
The audit concludes that the reporting system adopted by INFC needs to be enhanced to ensure that program management and senior management are provided with timely, reliable, and complete information that supports decision making and ensure that the results of PTF Agreements are communicated appropriately.
| Recommendations | Management Action Plan |
|---|---|
6.2.1 Given that the PTF is a sunsetting program and all the distributed funds must be spent by March 31, 2010, INFC should monitor and document for those provincial and territorial jurisdictions that have unspent PTF funds, the progress that is being made on completing project activities to ensure that the spending commitment is met. If the monitoring determines that there is a risk that some of the PTF funds will not be spent by the deadline, INFC should seek the guidance of their Senior Financial Management Advisor and take the actions needed to implement corrective measures. |
6.2.1
|
| Manager Responsible: | Assistant Deputy Minister, Program Operations |
| Due Date: | Various – Latest Management Action is December 7, 2009 |
6.3 Results and Performance
INFC is required to track the performance of the PTF recipients to demonstrate accountability for spending and to confirm that the anticipated program outcomes have been met.
In order to obtain the required performance information from the PTF recipients, the Agreements include a provision where each province and territory will compile and submit to INFC, as part of their annual AER/Audit Report, data associated with the performance and achievements regarding the use of PTF funds. The Agreements also include a provision that each province and territory will produce by no later than September 30, 2009, an Outcomes Report for the Gas Tax Funds (GTF) that will include the contribution of the PTF to the ultimate objectives of clean air and lower greenhouse gas emission.
The projects eligible for the PTF are a mirror image of the projects eligible under the Transit category of the GTF Transfer Payment Program and both programs share a common structure. Given the commonality of the performance indicators the auditors determined that the performance measurement problems experienced by the GTF could be applied to the PTF.
The outcome indicators for the GTF transfer payment program, which are linked to the program's objectives, were identified in the Agreements. The audit found however, that the approach to select the performance indicators that was provided to the recipients at the inception of the program was not well defined and confused many jurisdictions. This was also noted in the March 18, 2008 report, Implementation Evaluation of the GTF, where INFC confirmed that many jurisdictions were still confused about how and what was supposed to be measured. In our estimation, this confusion also applies to the PTF transfer payment program.
In an effort to resolve this issue, the audit recognized that INFC, in consultation with the provincial and territorial counterparts, has undertaken an enormous amount of work through its National Workshops and sub-committees to develop a Performance Measurement Framework for GTF and PTF that contains a set of performance indicators and a methodology to capture the performance data. The performance indicators included in the framework comprise a category "Transit Indicators" that will be used to capture performance data for both the GTF and the PTF transfer payment programs.
In spite of this hard work however, there are still recipients that need to continue the performance measurement work within their respective jurisdictions to ensure full acceptance (i.e. oversight committee and municipalities) and appropriate implementation.
If the performance measurement framework defined by INFC in consultation with the provinces and territories is not accepted and implemented by all jurisdictions, there is a risk that the performance data required to produce the Outcome Report may not be collected and analysed to the extent necessary to meet the September 30, 2009 deadline and INFC's requirements. Furthermore, there is a risk that INFC may not have all the information necessary to clearly demonstrate at a national level, the achievement and performance of the PTF funds that have been expended, when it comes time to prepare the department's Departmental Performance Report, and other reports.
| Recommendations | Management Action Plan |
|---|---|
6.3.1 INFC should document that all provincial and territorial jurisdictions have adopted and implemented a Performance Measurement Framework that reflects the concepts identified in the PTF segment of the INFC's GTF and PTF Performance Measurement Framework. |
6.3.1
|
| Manager Responsible: | Assistant Deputy Minister, Program Operations |
| Due Date: | March 31, 2010 |
| Recommendations | Management Action Plan |
|---|---|
6.3.2 INFC should develop a strategy for the review, analysis and validation of the Outcomes Reports that it will receive from provincial and territorial jurisdictions on or before the September 30, 2009 due date. The strategy will need to consider such items as consolidation and communication of the information received from the provinces and territories, 'lessons learned' regarding any enhancements that might be necessary to the reporting template, and defining 'next steps' for subsequent reports (e.g. timeline). |
6.3.2
|
| Manager Responsible: | Assistant Deputy Minister, Program Operations |
| Due Date: | March 31, 2010 |
7 Audit Opinion
Based on the findings reported, our overall opinion is that the adequacy and effectiveness of the management control framework and the appropriateness of the management practices that have been put in place for the Public Transit Fund Transfer Payment Program are satisfactory to ensure compliance, program effectiveness, and financial integrity.
We found evidence that the governance and accountability framework that has been established for the PTF transfer payment program is manageable and that INFC has developed an appropriate and dedicated team of experienced, stable employees to ensure that: the flowing of funds specified for PTF transfer payment program are in accordance with the signed Agreements; adequate financial and program controls are in place to manage the overall program; and continuous support is provided as a means of building and sustaining Infrastructure Canada's partnerships with the provincial and territorial jurisdictions.
With that being said, we found that the program controls associated with monitoring and reporting, and results and performance need to be strengthened to ensure that program management and senior management are provided with timely, reliable, and complete information that supports decision making and ensure that the results and performance of the PTF funds that have been expended are communicated appropriately through the department's Departmental Performance Report, and other reports.
8 Statement of Assurance
In our professional judgment, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of findings reached and contained in this report. The findings were based on a comparison of the situations as they existed at the time against the audit criteria.
The findings are only applicable for the entity examined. The extent of the examination was planned to provide a reasonable level of assurance with respect to the audit criteria. The evidence gathered meets professional audit standards and is sufficient to provide senior management with proof of the findings derived from the internal audit.
- Date modified: