Evaluations - Evaluation of Infrastructure Canada's Economic Action Plan Initiatives - Annex 4

Degree of alignment between program objectives and the current departmental strategic outcome 3: "Construction-ready infrastructure projects are provided with federal funding support."

Degree of consistency in program objective statements among strategic documents

Overall comprehensiveness of initiative governance (e.g. direction setting, control, roles and responsibilities)

Clarity and understanding of roles and responsibilities of INFC and delivery partners

Effectiveness of decision-making bodies (e.g. PRP)

Nature, use and utility of:

• risk assessment and monitoring tools
• performance reporting and monitoring tools
• delegation of authority

Perceptions regarding the use of delegated authorities including risk-based decision making mechanisms (e.g. appropriate level, timely processes, managed risk)

Timeliness and nature of course corrections and management actions related to key reviews and audit observations.

% of funded projects identified as low/med/high risk that were substantially completed/not completed before March 31, 2011 or March 31, 2010

Clarity and understanding of eligibility and project selection criteria (provincial differences)

Length of time between project application and funds committed/approved

Degree of satisfaction, by partner and by province with:

• level of engagement/consultation and communication
• funding, approval and monitoring process
• how the changes were implemented
• level of cooperation among partners
• roles and responsibilities

Perceptions of the capacity of partners and ultimate recipients to respond to accelerated delivery approaches (e.g. need for speed)

Extent to which:

• changes required were identified
• the need to manage the change was recognized (e.g. identification of the person leading the change; HR Plan)

Extent to which risk levels changed (e.g. use of increased Ministerial authority)

Extent to which the right human resources were available and the necessary capacity was built within INFC to overcome barriers to deliver the initiative (e.g. recruitment, training, tools, internal communications)

Perceptions of initiative staff regarding people management:

• they were part of the solution
• they were engaged and informed
• there was clear direction
• they were able to manage the workload
• the division of responsibilities was appropriate and effective
• they were comfortable in their new roles (e.g. risk-based decision-making)
• they had the right skill sets

Results of preliminary analysis from all lines of evidence, in the following areas:

• governance and accountability
• delivery model (e.g. streamlined approvals, exception to environmental assessments)
• risk management
• performance monitoring and reporting
• role of partners

Results of preliminary analysis from all lines of evidence, in the following areas:

• leadership
• people management and HR
• organizational culture/risk tolerance
• stakeholder engagement/partnerships

# and % of initiative projects that met, exceeded or did not meet the leveraging target (e.g. 40% of ISF projects met the target of 75% leveraged funds)

% of projects that leveraged funding from other sources (e.g., PT, municipality, private, not-for-profit) (organized by $ amount and categories by provinces)

# and nature of agreements by province

Average length of time between receipt of project application and funds committed/approved

• compared to other INFC programs

Trend in funding committed by month from April 2009 to January 29, 2011 (ISF and BCF-CC Top-Up) or to March 31, 2010 (for NRT)

% and dollar value of approved projects substantially completed by the initiative due date: March 31, 2011 for ISF and BCF-CC Top-Up; March 31, 2010 for NRT shown by: by province, by materiality, by category, by risk ranking at time of funds approved

% of approved projects requiring extension beyond March 31, 2011 or March 31, 2010

Distribution of projects by investment category

Perceptions and examples of challenges and unintended impacts (standard evaluation question for interviews)

Comparison of delivery mechanisms across the three initiatives (3^rd party arrangements vs. direct delivery) regarding the need for speed

Perceptions of whether the program could be improved or delivered differently to improve efficiency (need for speed)

Ratio of administration costs vs. total initiative costs