Summative Evaluation Report - Infrastructure Canada Program First Nations Component - Conclusions
4.1.1 Did the ICP-FN achieve its intended outputs and immediate outcomes?
With a contribution of $30,382,690, as of 7 December 2007, the ICP-FN had financed 97 infrastructure projects that have helped First Nations community recipients to improve the quality of their environment, support the long-term growth of their economy, enhance their community infrastructure, and begin to establish the 21st century infrastructure that they need. Despite the lack of quantitative data to corroborate some qualitative findings, our evaluation team found that all interviewees agreed that they were satisfied with the way the Program was delivered and that it achieved some significant expected outcomes.
The $30,382,690 transferred to and administered by INAC generated an additional investment of $63,849,881 from INAC and First Nations communities. The recipients of the Program were only those First Nations communities who could access funding to pay their share of the project contribution and who had resources related to the management of infrastructure projects as well as the expertise to deal with the federal government. The support provided by the Program served to fund and complete projects that have resulted in the enhancement of existing infrastructures and the construction of new infrastructure within participating First Nations communities. We found that the funded infrastructure projects that we reviewed were in alignment with ICP-FN objectives and INFC priorities and covered the two priorities of the Program, green infrastructure projects and other infrastructure projects.
4.1.2 Did the ICP-FN achieve its intended intermediate outcomes?
On the basis of our case studies, interviews, and the data available from SIMSI (these data are incomplete across projects), we found that completed projects have had some positive effects on participating First Nations communities. This finding is tempered by the modest size of the program relative to infrastructure needs in First Nations communities. In correspondence to the intermediate outcomes of the logic model, we found modest improvements with respect to:
- the quality of their environment,
- long-term economic growth opportunities,
- community infrastructure, and
- the use of new approaches as well as best technologies and practices to improve or renew existing infrastructure.
Specifically, interviewees told us of specific infrastructure projects have led to:
- improved water and wastewater services,
- improved sewer systems and solid-waste management,
- improvements in health and safety,
- safer and more efficient movement of people and goods in and out of their communities,
- increased tourism opportunities and the use of modern infrastructure construction methods and best practices, and
- reduced operating and maintenance costs associated with water distribution, sewage treatment, or waste management.
We also found that, although the infrastructure in First Nations communities has shown improvement as a result of the ICP-FN , more work needs to be done to accommodate the current and future needs of these communities with regard to water system upgrades and repairs, affordable housing, road upgrades and repairs, waste management, landfill and recycling, community centres, and multi-purpose buildings.
4.1.3 Was the ICP-FN delivered efficiently and effectively?
We found a lack of unanimity among interviewees with regard to the cost-effectiveness of the ICP-FN . While it was originally thought that working through INAC's Capital Facilities Program and making use of INAC regional offices would be an effective way to deliver the Program to participating First Nations communities, those we interviewed at INFC and INAC did not believe that the model used for the Program's delivery was the most effective and efficient. However, all interviewees agreed that they were satisfied with the way the Program was implemented. INAC received over $30 million for funding infrastructure projects as well as $334,000 for operating expenses, which represents 1.1% of the Program's total budget. This contribution is significantly less than the 3% in administration costs associated with newer programs at INFC. These amounts were used to cover salaries and operation and maintenance costs. However, given the lack of detailed data on the nature of administrative expenses incurred, it is not possible to assess whether or not this amount was sufficient to support effective program delivery.
The INFC and INAC personnel whom we interviewed felt that better results would have been obtained if a different process had been followed. They believed that the selection and approval process for projects, the requirements of the reporting system, and the performance monitoring associated with the Program were the main obstacles to the effectiveness of the Program's delivery model. We found that the model used to deliver the Program revealed weaknesses inherent in INAC 's internal capacity to manage and monitor the Program as well as weaknesses with regard to measuring project performance. As corroborated by our analysis of SIMSI data, the model also shows insufficient capacity with respect to project management and performance management at the recipient level.
Lessons Learned for INAC
- For consistency in program delivery ensure the following:
- adequate capacity to deliver, and
- training of staff both at HQ and in the regions.
- To ensure adequate reporting, implement an auditable and continually improved system for the following:
- project tracking and monitoring, and
- collecting and reporting on results with attention to intermediate and long-term outcomes.
Lessons Learned for INFC and delivery partners
- For complex horizontal initiatives involving multiple federal departments and agencies, partners should prepare an annual performance report on each province, territory, or region they serve in order to gather project information and show program results at the national level.
- Delivery partners should confirm that cost-effective oversight, internal controls, performance measurement, and reporting systems are in place, at the recipient level, to support the management of transfer payments to recipients.
- Program objectives should be concrete and clearly defined in relation to program scope and size.
- A horizontal performance measurement framework and supporting indicators that are realistic, achievable to collect, agreed upon, and congruent with program scope and resources should be developed.
- INFC should monitor that partners are meeting their reporting obligations including sharing information on performance and results as required by the ICP Governance and Accountability Framework.
- Identify and share lessons learned and best practices in the delivery of horizontal initiatives with multiple delivery partners.
- Date modified: