Internal Audit of Shared Information Management System for Infrastructure (SIMSI)

In May 2005, the departmental Audit and Evaluation Committee at Infrastructure Canada (INFC) approved the conduct of an assurance audit of the Shared Information Management System for Infrastructure (SIMSI).

The audit objective was to assess the effectiveness of the management control framework related to building, operating and managing the design and development of SIMSI. The audit report distinguishes between two streams, the first one being the SIMSI Operations and Enhancements - which include the operation and enhancement of the system that collects data to address the project tracking needs of the Infrastructure Canada Program (ICP), the Municipal Rural Infrastructure Fund (MRIF), the Canada Strategic Infrastructure Fund (CSIF), the Border Infrastructure Fund (BIF), and in the future, the needs of the Gas Tax Fund (GTF) and the Public Transit Fund (PTF). For the first stream, audit results were mostly favorable.

The second stream, SIMSI Renewal, – covers the changes and investments required to address a broader part of the departmental activities and mandate including knowledge generation and transfer, strategic partnerships, collaboration and research, and corporate requirements. SIMSI Renewal audit results showed that this stream needed re-assessment. The SIMSI renewal project has been put on hold until a thorough reassessment of the project is completed.

Management expressed agreement with the audit recommendations and prepared an action plan to address them.

Table of Contents

1. Executive summary

Background

The Shared Information Management System for Infrastructure (SIMSI) was built to support the implementing agencies and Infrastructure Canada in their responsibilities to administer the Infrastructure Canada Program (ICP). SIMSI provides the information technology support system to assist management and stakeholders (municipalities and federal and provincial implementing agencies) in managing thousands of projects that will eventually be funded through the program. Effective project approval and expenditure authority to build, operate and manage SIMSI were received from Treasury Board at an initial cost estimate of $10.1 million.

SIMSI is a program management system that focuses on project lifecycles and provides for project registration, status information, milestone monitoring, benefits tracking, payment tracking, due diligence analysis and related documentation throughout the life of a project. It also provides reporting on all facets of the movement of the project through its lifecycle, to Infrastructure Canada, stakeholders, and the general public, through web-enabled technologies. It permits a shared, up-to-date and complete database.

SIMSI was envisioned and built to support the ICP, a $2.05 billion program created in 2000 to enhance infrastructure in Canada's urban and rural communities and to improve quality of life through investments that protect the environment and support long-term community and economic growth. Virtually all ICP funding has been committed and close to 3,000 ICPfunded projects have been announced.

SIMSI is expected to support five subsequent programs:

  • Canada Strategic Infrastructure Fund (CSIF) – a $4 billion fund directed to projects of major federal and regional significance in areas that are vital to sustaining economic growth and enhancing the quality of life of Canadians.
  • Border Infrastructure Fund (BIF) – a $600 million fund that targets some of the busiest Canada-United States border crossing points. The BIF was created in the recognition that Canada's border crossings and their highway approaches are vital for economic growth and prosperity.
  • Municipal Rural Infrastructure Program (MRIF) – a $1 billion program announced in 2003 to support smaller scale municipal infrastructure projects that improve the quality of life, sustainable development and economic opportunities, particularly of smaller communities. Includes a component addressing the infrastructure needs of First Nations communities. Negotiations are underway with each province and territory to establish agreements for the co-management of the fund.
  • The Gas Tax Fund (GTF) – Budget 2005 announced that $5 billion of gas tax would be available over the next five years to support cities and communities according to the terms of the New Deal agreements.
  • The Public Transit Fund (PTF) – The Public Transit Fund is designed to contribute to the Government of Canada's environmental objectives through targeted support for public transit infrastructure in Canadian communities. Building on the current financial support through existing infrastructure programs and the gas tax funds (GTF), the public transit funds provide funding to those communities with transit systems to contribute to shared national outcomes of cleaner air and reduced GHG emissions. $400 million was allocated in 2005-2006.

SIMSI activities are being addressed in two distinct streams:

  1. SIMSI Operations and Enhancements – which include the operation and enhancement of the system that collects data to address the project tracking needs of ICP, MRIF, CSIF, BIF, and in the future, the needs of the Gas Tax / Public Transit.
  2. SIMSI Renewal – which encompasses the changes and investments required to address a broader part of the Departmental activities and mandate including knowledge generation and transfer, strategic partnerships, collaboration and research, and corporate requirements.

Consequently, the audit report distinguishes between SIMSI Operations and Enhancements on the one hand, and SIMSI Renewal, on the other hand, when presenting the audit findings and conclusions.

For the purposes of this audit report, the terms SIMSI Operations and Enhancements; and, SIMSI Renewal, refer to the activities described respectively in 1 and 2 above.

The Rationale for an Audit

On May 20, 2005 the Departmental Audit and Evaluation Committee (DAEC) of Infrastructure Canada (INFC) approved the conduct of an assurance audit of the Shared Information Management System for Infrastructure (SIMSI) as part of the audit plan.

Audit Objectives and Scope

The audit objective for the assurance audit of SIMSI was to assess:

  • The effectiveness of the management control framework related to building, operating and managing the design and development of SIMSI.
  • The effectiveness of the management of the SIMSI project risks associated with the above processes; in particular, focusing on the risk related to investments in SIMSI Renewal and related changing scope and requirements.
  • Governance over SIMSI (current and future enhancements). This involved a review of the mandates of the various governing bodies associated with the design and development of the SIMSI system, as well as the roles and responsibilities of key senior managers for making decisions on SIMSI.
  • Effectiveness of the controls to manage the relationships and expectations of the partners. This involved a review of the effectiveness of partner agreements and governance.
  • Resource base. This involved an identification of SIMSI's costs (past, current and forecasted) and human resources (internal and contracted). The processes in place to monitor, control and report on costs were assessed for effectiveness.

Audit Period

The audit was conducted between January 17, 2006 and March 31, 2006 and covered an examination of current practices at the time of the audit field work which ended March 17, 2006.

Audit Conclusion

In our opinion, based on our professional judgment as auditors, sufficient and appropriate audit procedures have been conducted, in accordance with both the Treasury Board Policy on Internal Audit and the Institute of Internal Auditors Standards for the Professional Practice of Internal Auditing, and the evidence gathered supports the accuracy of the conclusions in this report. Criteria were developed to provide a framework in assessing the extent to which SIMSI was meeting the audit objectives. The control framework Criteria of Control (CoCo model), from the Canadian Institute of Chartered Accountants, was used as the basis to develop the criteria. They were reviewed for appropriateness and approved by the Head of Internal Audit prior to executing the audit plan. Consequently, the evidence gathered provides senior management with reasonable assurance of the accuracy of the conclusions drawn from this audit.

Governance and Accountability structures

SIMSI Operations and Enhancements

The governance and accountability framework that was effective in the past no longer adequately serves the new more complex SIMSI environment with its more demanding programs and involvement of multiple organizational units within INFC. A more formalized and broader Governance and Accountability structure is needed to ensure that all stakeholders have appropriate roles in support of setting strategic direction, planning and priority setting, resource allocation and the management of requirements and expectations.

SIMSI Renewal

The governance and accountability structure associated with SIMSI Renewal is not effective in supporting the management of the risks associated with the design, development and implementation of SIMSI Renewal.

  • A functional decision-making structure required to direct SIMSI Renewal involving key senior managers has not been established.
  • Many of the key and fundamental documents required to establish the vision and objectives for the SIMSI of the future are still in unapproved draft form including: the IM/IT Strategic Plan that includes components of the IM/IT Human Resource Plan, a Communication Strategy and the SIMSI Renewal Business Case.
  • A formal Project Office to drive Renewal has not been established.

Manage the relationships and expectations of the partners

SIMSI Operations and Enhancements

The management of relationships and expectations is very effective. Stakeholder involvement was evident in the development, operation and change/enhancement environments associated with the programs supported by SIMSI. Stakeholders were involved in identifying business requirements, business rules and program application implementation within SIMSI. There is a forum for changes and enhancements and a mechanism to develop reports specific to one region or general to all. The IM/IT Directorate has made contact with the manager of new Programs to identify requirements and to assist in establishing Provincial participation.

The recommended new Governance and Accountability structure will ensure the continued effectiveness of management of relationships and expectations in the future.

SIMSI Renewal

The management control framework for SIMSI Renewal management of the relationships and expectations of partners is not effective. The audit found that while SIMSI continued to monitor and manage the emerging requirements and expectations of its traditional partners in the program areas:

  • The expected outcomes of the project are not clearly defined or understood by internal stakeholders.
  • Internal stakeholders do not feel they were adequately involved in the development of SIMSI Renewal Project documents and are unsure of how their feedback, when sought, has been addressed.
  • Many of the documents associated with the SIMSI Renewal are still unapproved drafts in part as a result of issues associated with the identification and management of stakeholder expectations.

Risk management

There is no systematic and holistic approach to risk management within SIMSI. SIMSI risk management practices are operational in nature and deal with the risks associated with the operating system and the delivery of changes and enhancements within the established plans. SIMSI risk management practices and strategies will be addressed within the IM/IT Strategic Plan (currently in draft).

Resource base

The processes in place to monitor, control and report on costs for the current SIMSI environment are sufficient and effective. The Task Authorization process in place ensures that the costs of developing and implementing changes are determined and controlled. Performance at an operational level is monitored and reported. These costs relate to ongoing and cyclical activities covered in the CGI contract.

The process to monitor, control and report on costs for the SIMSI Renewal project is dependent on the establishment of an effective governance and accountability structure, the definition and implementation of processes for planning, priority setting and resource allocation and an effective mechanism to determine and manage the expectations of partners.

Recommendations

Recommendations have been grouped either under SIMSI Operations and Enhancements or under SIMSI Renewal. Some recommendations, however, apply to both environments. These recommendations are: Recommendation 2 (risks), Recommendation 3 (communication), Recommendation 4 (risks) and Recommendation 5 (business continuity) below.

SIMSI Operations and Enhancements

It is recommended that:

  1. The Assistant Deputy Minister, Corporate Services, establish a new Governance and Accountability Structure, Priorities and Resource Allocation processes and Policy Framework to address the requirements of the new broader and more complex SIMSI Operations and Enhancement environment.
  2. The Chief Information Officer implement formal risk management strategies and practices that will ensure the continuous monitoring of potential SIMSI risks using a risk framework and indicators, and systematically mitigate identified risks.
  3. The Director, Application Services, implement measures to improve communication practices within the SIMSI team and between SIMSI and its partners and stakeholders within INFC.
  4. The Director, Application Services, more clearly define and document the SIMSI risks with respect to human resources and develop an action plan, building on current HR initiatives, to address the risks.
  5. The Director, Application Services, undertake a periodic review of business continuity requirements to ensure that SIMSI business continuity plans are up-to-date and appropriately address client needs.
  6. The Director, Application Services, ensure that the SIMSI issues identified in the Mid-Term Evaluation Report, September 2005, are addressed.
  7. The Director, Application Services, ensure that the issues identified in the SIMSI Vulnerability Assessment and SIMSI security requirements are addressed.
  8. The Director, Application Services, ensure that data integrity issues continue to be investigated and resolved and that regular status reports of data integrity activities be communicated to affected stakeholders.
  9. The Chief Information Officer define and collect performance information that measures the degree to which SIMSI is effectively supporting business requirements and provides feedback of SIMSI performance in the planning process.

SIMSI Renewal

In addition to recommendations 2, 3, 4 and 5 above, which apply also to SIMSI Renewal, it is recommended that:

  1. The Assistant Deputy Minister, Corporate Services, establish a SIMSI Renewal Steering Committee to provide department level oversight and direction to the SIMSI Renewal Project. The committee should include senior management members from all areas of the department.
  2. The Chief Information Officer establish a SIMSI Project Management Office with a clearly identified and accountable SIMSI Renewal Project Manager.
  3. The Chief Information Officer ensure that SIMSI Renewal roles, responsibilities and accountabilities are clearly defined, approved by the SIMSI Renewal Steering Committee and appropriately assigned.
  4. The Chief Information Officer consider changing the name of the SIMSI Renewal Project to a new name that better reflects the broader departmental objectives of the project.
  5. The Chief Information Officer ensure that a project plan is developed for SIMSI Renewal including: the identification of activities, deliverables, time frames and accountabilities. The plan should be forwarded to the SIMSI Renewal Steering Committee for approval.
  6. The Chief Information Officer ensure that the SIMSI Renewal Project Manager is made accountable for implementation and management of the SIMSI Renewal Plan.
  7. The Chief Information Officer ensure that Performance Management strategies and practices are included in the SIMSI Renewal Project Plan and in the deliverables to be forwarded to the SIMSI Renewal Project Steering Committee for approval.

2. Introduction

2.1 Background

SIMSI was built to support the implementing agencies and Infrastructure Canada in their responsibilities to administer the Infrastructure Canada Program (ICP). SIMSI provides the information technology support system to assist management and stakeholders (municipalities and federal and provincial implementing agencies) in managing thousands of projects that will eventually be funded through the program. Effective project approval and expenditure authority to build, operate and manage SIMSI was received from Treasury Board at a cost estimate of $10.1 million.

SIMSI is a program management system that focuses on project lifecycles and provides for project registration, status information, milestone monitoring, benefits tracking, payment tracking, due diligence analysis and related documentation throughout the life of a project. It will also provide for reporting on all facets of the movement of the project through its lifecycle, to Infrastructure Canada, stakeholders, and the general public, through web-enabled technologies. It permits a shared, up-to-date and complete database.

SIMSI is made up of the following sub-systems:

  • Application Capture and Registration
  • Activities Tracking
  • Cost Monitoring
  • Benefits Verification
  • Due Diligence
  • Reporting
  • Ancillary sub-systems and applications

The system architecture is based on industry-standard, open-system hardware and software products. It is a commercial-off-the-shelf (COTS) product. An outsource arrangement was competed and contracted with CGI to develop, build, install and operate SIMSI. The contractor provides helpdesk and end-user support services and end-user training. Existing arrangements are expiring June 2006. The procurement process is underway to re-tender the current outsourcing arrangements.

SIMSI was planned as a finite system, with a fixed life of six years, whose demise was to be coincidental with the sunset of the ICP on March 31, 2007. SIMSI proved to be an extremely successful model for infrastructure program operations management in a multi-jurisdictional environment. SIMSI was showcased twice at the Government Technology Show (GTEC) winning the gold medal for service delivery.

With the introduction of new infrastructure programs with longer lives, the life of SIMSI has been extended. In that context, to make the system sustainable for an extended period, while meeting the requirements of the new infrastructure and other programs introduced since the ICP, including new requirements related to due diligence, monitoring, reporting, etc., additional enhancements to the system will be required.

SIMSI is expected to support five new programs:

  • Canada Strategic Infrastructure Fund (CSIF) – a $4 billion fund directed to projects of major federal and regional significance in areas that are vital to sustaining economic growth and enhancing the quality of life of Canadians.
  • Border Infrastructure Fund (BIF) – a $600 million fund that targets some of the busiest Canada-United States border crossing points. The BIF was created in the recognition that Canada's border crossings and their highway approaches are vital for economic growth and prosperity.
  • Municipal Rural Infrastructure Program (MRIF) – a $1 billion program announced in 2003 to support smaller scale municipal infrastructure projects that improve the quality of life, sustainable development and economic opportunities, particularly of smaller communities. Includes a component addressing the infrastructure needs of First Nations communities. Negotiations are underway with each province and territory to establish agreements for the co-management of the fund.
  • The Gas Tax Fund (GTF) – Budget 2005 announced that $5 billion of gas tax would be available over the next five years to support cities and communities according to the terms of the New Deal agreements.
  • The Public Transit Fund (PTF) - The Public Transit Fund is designed to contribute to the Government of Canada's environmental objectives through targeted support for public transit infrastructure in Canadian communities. Building on the current financial support through existing infrastructure programs and the gas tax funds (GTF), the public transit funds provide funding to those communities with transit systems to contribute to the shared national outcomes of cleaner air and reduced GHG emissions. $400 million was allocated in 2005-2006.

The SIMSI Renewal project is a complex large-scale project, and since the estimated investment in the renewal of SIMSI will exceed $10 million, the department must seek Treasury Board approval to proceed with this project. While the business case has not been finalized, the submission will seek approval for spending in the range of $25 – 40 million over a five year period.

SIMSI activities are being addressed in two distinct streams:

  1. SIMSI Operations and Enhancements – which include the operation and enhancement of the system that collects data to address the project tracking needs of ICP, MRIF, CSIF, BIF, and in the future, the needs of the Gas Tax / Public Transit.
  2. SIMSI Renewal – which encompasses the changes and investments required to address a broader part of the Departmental activities and mandate including knowledge generation and transfer, strategic partnerships, collaboration and research, and corporate requirements.

Consequently, the audit report distinguishes between SIMSI Operations and Enhancements on the one hand, and SIMSI Renewal, on the other hand, when presenting the audit findings and conclusions.

For the purposes of this audit report, the terms SIMSI Operations and Enhancements; and, SIMSI Renewal, refer to the activities described respectively in 1 and 2 above.

The Rationale for an Audit

On May 20, 2005 the Departmental Audit and Evaluation Committee (DAEC) of Infrastructure Canada (INFC) approved the conduct of an assurance audit of the Shared Information Management System for Infrastructure (SIMSI).

2.2 Audit Objectives and Scope

Objectives

The audit objective for the assurance audit of SIMSI is to assess:

  • The effectiveness of the management control framework related to building, operating and managing the design and development of the SIMSI system.
  • The effectiveness of the management of the SIMSI project risks associated with the above processes; in particular, focusing on the risk related to investments in SIMSI Renewal and related changing scope and requirements.

Assessment of the management control framework included the review of the following components:

  • Governance over SIMSI (current and future enhancements). This involved a review of the mandates of the various governing bodies associated with the design and development of the SIMSI system, as well as the roles and responsibilities of key senior managers for making decisions on SIMSI.
  • Effectiveness of the controls to manage the relationships and expectations of the partners. This involved a review of the effectiveness of partner agreements and governance.
  • Resource base. This involved an identification of SIMSI's costs (past, current and forecasted) and human resources (internal and contracted). The processes in place to monitor, control and report on costs were assessed for effectiveness.

Scope

The scope of the audit included an assessment of the effectiveness of the management control framework in place to manage the SIMSI project risks associated with the design and development of the SIMSI system for the current program (ICP) as well as the five additional programs of SIMSI (CSIF, BIF, MRIF, Transit Fund and the Gas Tax Fund).

The PTF and Gas Tax Fund were at a very early stage of discussions to determine if SIMSI would be used and if so, how. Consequently, there was no SIMSI activity for these two programs that could be reviewed within the audit.

The current SIMSI environment and SIMSI Enhancements share the same management control framework (MCF)1 and have been reviewed together. SIMSI Renewal shares some MCF elements with the current SIMSI, as well as certain unique MCF requirements

Interviews were conducted with INFC technical and management staff as well as with SIMSI partners and stakeholders. For those partners and stakeholders from outside the National Capital Region, consultations were done over the phone. Internal and external interviews supplemented the review of internal process documents to verify that processes worked as intended and to obtain additional information to assess how the relationships and expectations of the partners are being managed.

2.3 Audit approach

Audit criteria

This audit was conducted in accordance with both the Treasury Board Policy on Internal Audit and the Institute of Internal Auditors Standards for the Professional Practice of Internal Auditing.

Criteria were developed to provide a framework in assessing the extent to which SIMSI was meeting the audit objectives. The control framework Criteria of Control (CoCo model), from the Canadian Institute of Chartered Accountants, was used as the basis to develop the criteria. They were reviewed for appropriateness and approved by the Head of Internal Audit prior to executing the audit plan. They are presented below, organized by the audit objective they support.

Information Gathering

Interviews were conducted with INFC technical and management staff as well as with SIMSI partners and stakeholders. For those partners and stakeholders from outside the National Capital Region, consultations were done over the phone. Internal process documents were reviewed to verify that processes worked as intended and to obtain additional information to assess how the relationships and expectations of the partners were being managed.

Audit Period

The audit was conducted between January 17, 2006 and March 31, 2006 and covered an examination of current practices at the time of the audit field work which ended March 17, 2006.

Scale used to Assess Controls

In stipulating the adequacy of current practices against the audit criteria, the following scale was used as a gauge of adequacy of controls. The describing text was at times modified to better represent the control being assessed.

  1. Not effective at all – Significant management attention is needed to improve these practices.
  2. Somewhat effective – Some parts of this element are working well, but many deficiencies exist.
  3. Effective – Most parts of this element are working as intended, but more work is needed in some areas.
  4. Very Effective – No action is required. Everything is working as intended.

[1]A management control framework consists of elements of an organization (including its resources, systems, processes, culture, structure and tasks) that, taken together, support managers and staff in the achievement of the organizational objectives. Control is effective to the extent that it provides reasonable assurance that the organization will manage its risk and therefore achieve its objectives reliably.

3. Findings

3.1 Mangement Control Framework - Current SIMSI & SIMSI Enhancements

The first audit objective was to assess the effectiveness of the management control framework related to building, operating and managing the design and development of SIMSI.

A summary of the ranking of the controls that support the audit criteria for the management control framework are presented in the table below. The detailed analysis of the findings and recommendations is presented following the table.

The effective ness of the management control framework related to building, operating and managing the design and development of SIMSI.

Criteria Very Effective Effective Somewhat Effective Not at all effective

Governance and accountability structures are in plce for the management of SIMSI.

  • Accountabilities across the entity between stakeholders, including partners, are clearly defined and managed.
  • Processes, activities and structures, which establish clear governance, objectives, roles and responsibilities and appropriate working and reporting arrangements ensure proper coordination between all parts of the organization.
  • Governance supports strategic and operational decision-making, efficiency and effectiveness of operations and the mitigation of risk.
     
The Policy Framework is adequate to define and monitor internal and external requirements associated with the operation, management, design and development of the SIMSI system.      

Planning, Priority Setting and Resource Allocation processes are in place to set strategic direction and operational plans, objectives and priorities and to allocate financial, human and other resources in accordance with these plans.

  • The controls define the required resources and competence and serve to ensure that sufficient resources exist to meet SIMSI objectives, consider results of risk assessment and historical performance, and provide critical support for management in directing and monitoring operations.
  • The controls provide employees, 3rd party service providers and key partners with a clear understanding of priorities and resource boundaries.
     

Corporate Values and Ethics are explicit, well-communicated, well-understood and set the standard to which all employees must adhere.

  • Corporate values include effective and fair human resource management, integrity and professionalism.
  • Mutual trust and shared values provide a guide for individual, group or team decision making and action.
     

Information and Communication processes, both formal and informal are adequate, in terms of quality, quantity and timing, to set direction, establish expectations and monitor performance and employee satisfaction.

  • The processes are adequate, in terms of quality, quantity and timing, to manage the relationships and expectations of SIMSI partners and key stakeholders.
     

The Work Environment and Employee Recognition program supports a healthy and well-balanced work environment.

  • The work environment can impact employee commitment strengthening employee loyalty, diminish absenteeism and affect the commitment to other operational controls.
     
The Training and Capacity Building processes and practices are sufficient to perform tasks, achieve objectives and to ensure the capacity to address future requirements associated with the SIMSI.      
  • The Physical Safeguards are adequate to ensure the physical safety, security and appropriate management of the organization's human, information and physical assets.
     
  • Knowledge and Information Management practices and processes ensure that individuals have access to and are supported by complete and accurate information.
     

Business Continuity Plans that reflect the degree of risk of the SIMSI system are complete, up-to-date, tested and appropriately communicated to key stakeholders and those directly affected.

  • The relationships and expectations of partners are considered in the SIMSI Business Continuity planning process.
     

Operational Control and Project Management

Operational control processes and practices (segregation of duties, limits, approvals and authorizations) are in place to ensure that all operational activities are conducted appropriately and in compliance with stated directives and established standards.

The Project Management regime addresses scope management, quality management, communications management, human resource management, risk management, financial management, problem and issue management and change control management.

     

Financial Integrity Controls and Procurement Management

There are adequate controls within SIMSI that ensure that financial transactions are accurate, authorized and appropriate and that audit trails to track changes to information are maintained.

  • There is an adequate procurement strategy to identify SIMSI operational and renewal requirements and risks; and there are adequate processes and procedures in place to manage procurement activities.
     
Information Technology Controls are in place to ensure the integrity and security of its information technology, as well as the completeness, accuracy and availability of data.      

Environmental Scanning practices such as the environmental scanning of economic, political, demographic factors as well as the monitoring of key indicators such as client satisfaction provide a solid and up-to-date understanding of the external factors that exist as well as their impact on the operational objectives, risks and priorities.

  • The monitoring of changes to the regulatory and legislative frameworks as well as the broad financial conditions facing key suppliers is also addressed in this element.
  • The existence and effectiveness of these monitoring mechanisms contribute to management of the relationships and expectations of key partners and stakeholders.
     

Performance Management and Continuous Improvement process, practices and tools adequately address:

  • Cyclical and regular gathering, tracking, reporting and communicating of progress;
  • The establishment and communication of performance objectives and corresponding performance targets and indicators;
  • The use of performance information to ensure results and products are aligned with business requirements (Scope, Quality, Product management); and
  • The feedback of performance into the planning process.
     

Managerial Oversight

Operations and staff are adequately and effectively monitored with engaged and available managers providing direction and support based on the complexity of activities, the degree of delegation and the level of employee competence.

  • Management directly and tangibly reflects the corporate values articulated in formal policies and procedures.
     

Third-Party Oversight

There are mechanisms, such as audits and other independent formal reviews, which permit the independent verification of the development, maintenance and operation of SIMSI.

     

Governance and Accountability

Effective – Most parts of this element are working as intended, but more work is needed in some areas.

The processes, activities and structures, which establish clear governance, objectives, roles and responsibilities and appropriate working and reporting arrangements (including committee structures) that were effective in the past no longer adequately serve the new more complex SIMSI environment.

The current Governance and Accountability structure was established when SIMSI was organizationally in the same Branch, (Program Operations) as the programs it supported. The structure no longer adequately reflects or addresses the increased numbers of programs supported, the addition of the Cities and Communities programs (GTF and the PTF) and the recent reorganization of SIMSI into Corporate Services Branch.

A more formalized and broader Governance and Accountability Structure, revised Priorities and Resource Allocation processes and a review of the Policy Framework are needed to ensure that all stakeholders have appropriate roles in support of setting strategic direction, planning and priority setting, resource allocation and the management of requirements and expectations.

It is recommended that:

  1. The Assistant Deputy Minister, Corporate Services, establish a new Governance and Accountability Structure, Priorities and Resource Allocation processes and Policy Framework to address the requirements of the new broader and more complex SIMSI Operations and Enhancement environment.

Policy Framework

Effective – Most parts of this element are working as intended, but more work is needed in some areas.

The policy framework associated with the programs (ICP, MRIF, BIF, CSIF) is well established and is adequate to define and monitor the requirements for the operation, management, design and development of SIMSI in its current support of these programs.

Once a new governance and accountability structure and new priorities and resource allocation processes have been sufficiently defined, there will be a requirement to review the degree to which the Policy framework is still adequate and effective.

Recommendation: None – The Policy Framework is included in Recommendation 1.

Priorities and Resource Allocations

Effective – Most parts of this element are working as intended, but more work is needed in some areas.

As was the case in Governance and Accountability, the planning, priority setting and resource allocation processes in place to set operational plans, objectives and priorities and to allocate financial, human and other resources were adequate; however, with the recent addition of new programs, the involvement of more INFC organizations and the movement of the IM/IT function (including SIMSI) to Corporate Services, new Priorities and Resource Allocation processes will be required.

There is no systematic and holistic approach to risk management within SIMSI. SIMSI risk management practices are operational in nature and deal with the risks associated with the operating system and the delivery of changes and enhancements within the established plans. SIMSI risk management practices and strategies are to be addressed within the IM/IT Strategic Plan (currently in draft).

It is recommended that:

2. The Chief Information Officer implement formal risk management strategies and practices that will ensure the continuous monitoring of potential SIMSI risks using a risk framework and indicators and systematically mitigates identified risks.

Corporate Values and Ethics

Very Effective – No action is required.

There were no indications of issues related to values and ethics identified in the interviews conducted or documentation collected. All interviewed, including external clients, indicated that, while there were communications issues horizontally within the team (see paragraph below on Information and Communication), the SIMSI team worked in an environment of integrity, professionalism, mutual trust and had shared values.

Recommendation: None.

Information and Communication

Effective – Most parts of this element are working as intended, but more work is needed in some areas.

Communication processes, both formal and informal within the SIMSI operational environment are well established and functioning effectively.
Stakeholder involvement was evident in the development, operation and change/enhancement environments associated with the programs supported by SIMSI. Stakeholders were involved in identifying business requirements, business rules and program application implementation within SIMSI. There is a forum for changes and enhancements and a mechanism to develop reports specific to one region or general to all. The IM/IT Directorate has made contact with the manager of new Programs to identify requirements and to assist in establishing Provincial participation.

While all external stakeholders interviewed were positive about information and communication, many interviewed felt that the communication horizontally within IM/IT and between IM/IT and other INFC organizational units could be improved, citing examples of where communication had broken down in the past.

It is recommended that:

3.The Director, Application Services, implement measures to improve communication practices within the SIMSI team and between SIMSI and its partners and stakeholders within INFC.

Work Environment and Employee Recognition

Very Effective – No action is required.

The work environment and employee recognition was not addressed in sufficient detail to provide an audit opinion. However, there were no indications of issues in this area which would have an impact on SIMSI. The Infrastructure Canada internal website has an extensive HR component and there is emphasis on activities that relate to a healthy and well-balanced work environment.

Recommendation: None.

Training and Capacity Building

Very Effective – No action is required. Everything is working as intended.

The current training and capacity-building practices have been adequate to ensure people have the skills necessary to support the achievement of SIMSI objectives.

A combination of a core internal group augmented by contracted resources, are employed to address human resource requirements. Employees are hired based on a matching of the core competencies required with the skills and competencies of the individuals. Additional SIMSI specific training is provided to new employees and ongoing training / professional development requirements are discussed during the annual performance review process.

A formal IM/IT Human Resource Strategy is being developed and a draft IM/IT Human Resource Plan is completed but not yet approved. These documents will be required to assist IM/IT manage training and capacity-building activities in what will be an increasingly more complex and dynamic environment.

Recommendation: None.

Physical Safeguards

Very Effective – No action is required. Everything is working as intended.

Physical security with respect to SIMSI is provided as a part of the facility management / operations contract with CGI. SIMSI hardware is housed in a secure CGI facility.

Recommendation: None.

Knowledge and Information Management

Effective – Basic requirements are met, but more work is needed in some areas.

The documentation processes associated with the development, maintenance and enhancement of SIMSI have been adopted from the contractor, CGI. The processes ensurethat required information is retained, available and complete. Documentation maintained both by IM/IT and by CGI is sufficient to ensure that individuals have access to and are supported by the information they need to achieve their objectives.

Files reviewed during the audit provided evidence that knowledge and lessons learned from previous projects are systematically applied to new projects or initiatives.

The combination of a small core of internal resources augmented with contractors results in much of the corporate knowledge being held by a few key employees. Formal knowledge management controls such as succession planning and staff rotation are also difficult given the nature of the work and the small number of people involved. While the use of external resources does mitigates some of the risk there remains concern that the loss of one of these key employees will affect SIMSI's ability to achieve its operational or enhancement objectives.

It is recommended that:

4.The Director, Application Services, more clearly define and document the SIMSI risks with respect to human resources and develop an action plan, building on current HR initiatives, to address the risks.

Business Continuity Plans

Effective – Most parts of this element are working as intended, but more work is needed in some areas.

The business continuity processes in place for the current SIMSI are considered adequate based on an assessment of the impacts of a disruption of service and the length of time the disruption could be tolerated. An alternate "cold site" is maintained.

SIMSI Enhancement and SIMSI Renewal activities will involve new and different INFC business and organizational units that may have different and evolving business continuity needs.

It is recommended that:

5. The Director, Application Services, undertake a periodic review of business continuity requirements in order to ensure that SIMSI business continuity plans are up-to-date and appropriately address client needs.

Operational control and project management

Very Effective – No action is required. Everything is working as intended.

There is evidence that the project management regime for the current SIMSI environment, which is being applied to SIMSI Enhancements, adequately addresses scope management, quality management, communications management, human resource management, risk management, financial management, problem and issue management and change control management.

Recommendation: None.

Financial Integrity Controls and Procurement Management

Effective – Most parts of this element are working as intended, but more work is needed in some areas.

SIMSI is not a financial system but contains financial information related to the projects and the programs and is used for tracking and overall budget control. All regions contacted indicated that financial information was captured first in a departmental or organization financial system then re-entered, as required, into SIMSI. All indicated that there was a reconciliation of financial information between the systems involved.

The Infrastructure Canada Program Mid-term Evaluation Final Report, September 2005, noted differences between financial data provided in SIMSI and the data provided by the financial officers contacted. It observed that while reconciliations of SIMSI to other financial systems were occurring there remain questions with respect to the degree of duplication and the possible decrease in efficiency resulting from the use of parallel systems. The SIMSI Data Quality Initiative has identified the need to reconcile data at least twice each year and the Infrastructure Canada Mid-term Evaluation Final Report proposes more frequent monitoring of financial commitments and reconciliations. The mid-term evaluation also proposes that the current practice of using parallel systems for financial data be revisited for future programs.

It is recommended that:

6. The Director, Application Services, ensure that the SIMSI issues identified in the Midterm Evaluation Report, September 2005, are addressed.

Information Technology Controls

Somewhat effective – Some parts of this element are working well, but many deficiencies exist.

The current SIMSI system has somewhat effective controls to ensure the integrity and security of its information technology and the completeness, accuracy and availability of data.

  • The system is reliable.
  • The system has sufficient controls in place to ensure that basic IT security is addressed.
  • Access is controlled by user IDs and passwords. Each user has a profile which defines access rights.
  • There are procedures in place to mitigate some of the risks associated with the accessing of the production environment by developers.

An independent Vulnerability Assessment (VA) of the INFC computing environment including SIMSI was completed in August 2005. It is the intention of the department to address the VA recommendations related to SIMSI after the tendering process for the SIMSI contract has been completed and a new contract established. The VA is also a first step towards addressing the Government of Canada Information Technology Security Zones Baseline Security Requirements (ITSD-02) to be implemented by 2007.

While SIMSI system developers have physical access to the production and user testing environments, their ability to access these areas is restricted by their user profiles to "read only." Certain officers have been given the authority to modify information in the production environment based on a Task Authorization. The changes are controlled through a formal process which includes a sound audit trail and the ability to undo the changes if necessary.

The Help desk is not viewed as helpful outside of issues related to access – IDs and passwords.

  • Interviews indicated that external clients deal directly with the SIMSI Technical Support Officer or the Manager Information Services when they have problems or want something addressed.

Data integrity issues were identified by some interviewed, within the Infrastructure Canada Program Mid-term Evaluation Final Report, September 2005 and in Data quality reviews conducted by INFC in 2002 and 2003. A number of areas of concern, including completeness of data; consistency of information entered (particularly with respect to open text fields); and frequency of updates were identified. Data issues identified within the mid-term evaluation and identified within the audit included:

  • Data is not consistently entered into the SIMSI system across jurisdictions, resulting in questions being raised as to the validity and reliability of any reporting or monitoring activities undertaken at the National level. Data is entered into the system, at times by the Regional Development Agencies (RDAs) and at other times by the provinces (with the exception of Ontario and Quebec).
  • Definitions of the information that is included in the system are not consistent across the country. As a result, it is difficult to determine to what extent the data in the system is reliable on a national level. Workshops conducted as part of the SIMSI data quality initiative revealed that there are inconsistencies in the ways in which RDAs are interpreting key aspects of the program, including for example, the definition of project start date and the method to be used to calculate the achievement of green and rural targets.
  • The degree to which this data is actually used is not clear. Within any given jurisdiction, some individuals may use the data on an on-going basis while others may not use it at all.
  • A significant portion of the information requested is deemed "optional" and therefore is not required to be entered into the system. This results in incomplete data of little use for monitoring and reporting purposes at the national level.
  • The mid-term evaluation identified that it was difficult to ascertain the actual project completion status of ICP projects since available SIMSI data did not appear to reflect up-to-date project status information. Interviewees stated that while projects are completed, this information is not inputted into the system, as once projects are closed, changes cannot be made to the SIMSI project information.

Data quality issues are being addressed by INFC at this point in time with the introduction of a SIMSI Data Quality Initiative. It is expected that this initiative will resolve the current issues surrounding the accuracy and timeliness of information in the database and introduce a new procedure to ensure data accuracy in the future.

It is recommended that:

7. The Director, Application Services, ensure that the issues identified in the SIMSI Vulnerability Assessment and SIMSI security requirements are addressed.

8. The Director, Application Services, ensure that data integrity issues continue to be investigated and resolved and that regular status reports of data integrity activities are communicated to affected stakeholders.

Environmental Scanning

Very Effective – No action is required. Everything is working as intended.

In the context of the current SIMSI and SIMSI Enhancements, client satisfaction and changes in the regulatory and legislative framework of INFC are monitored.

  • Monitoring of the relationships and expectations of key partners and stakeholders is being done but is made more difficult by the varying use of SIMSI province to province and program to program.
  • The SIMSI User Group Monthly Conference Calls provide a forum for exchange of information as does the semi-annual Meeting of the Coordination Committee of Assistant Deputy Ministers responsible for Infrastructure Programs.

Recommendation: None.

Performance Management and Continuous Improvement

Effective – This element is working as intended; however, there are some areas that could be strengthened.

There are processes in place to establish and monitor the performance targets for SIMSI operations. Information collected and reported includes: Statement of system availability, change management, data statistics, and network usage.

File reviews undertaken during the audit found documentation of project status reporting, risk management (related to specific Task Authorizations and system releases) and evaluation reports reflecting performance, costs and schedules for the development, changes and enhancements of SIMSI.

The Infrastructure Canada Program Mid-term Evaluation Final Report, September 2005 raised concerns with respect to the reliability of the data and its potential impact on the summative evaluation of the ICP and INFC ability to measure program success in the future. It recommended that:

  • Clear links should be made between the performance measures outlined in the Resultsbased Management and Accountability Frameworks (RMAF) documents and the data being collected through SIMSI in order to ensure that the correct indicators are being tracked.
  • INFC ensure that data requirements are documented and distributed to those responsible for data collection and entry. This documentation should be available within the SIMSI system and in foundational documents and include clear definitions of the information that is being requested to ensure consistency across jurisdictions.

The mid-term evaluation saw these activities being addressed within the SIMSI Data Quality Initiative.

It is recommended that:

9. The Chief Information Officer define and collect performance information which measures the degree to which SIMSI is effectively supporting business requirements and which would provide feedback of SIMSI performance into the planning process.

Managerial Oversight

Very Effective – No action is required. Everything is working as intended.

The interviews conducted and documentation reviewed indicate an environment with sufficient and appropriate managerial oversight. The size of the organization (relatively small) results in a higher degree of awareness and engagement by the Director, Application Services, the Director, Operational Support and Web Services, and the Chief Information Officer. A weekly meeting of the two Directors, the contractors and the SIMSI project managers ensures a strong awareness of the operation by managers.

Recommendation: None.

Third-party review

Very Effective – No action is required. Everything is working as intended.

The audit found that independent formal reviews such as the Review of SIMSI March 2003, the Operational Review April 2005 (which included the IM/IT organization and SIMSI), the INFC Mid-Term Evaluation September 2005, an Independent Environmental Scan of Grants and Contribution Systems December 2005 and cyclical reviews of CGI performance have been used to guide improvements in the development, maintenance and operation of SIMSI.

The Review of SIMSI, March 2003, provided a basis for defining many of the elements found in the SIMSI Enhancements initiative, as well as the SIMSI Renewal documents.

Recommendation: None.

3.2 Management Control Framework – SIMSI Renewal

The second audit objective was to assess the effectiveness of the management of the SIMSI project risks in particular, focusing on the risk related to investments in SIMSI Renewal and related changing scope and requirements.

A summary of the ranking of the controls that support the audit criteria for the management control framework are presented in the table below. The detailed analysis of the findings and recommendations is presented following the table.

The effectiveness of the management of the SIMSI project risks in particular, focusing on the risk related to investments in SIMSI Renewal and related changing scope and requirements.

Criteria Very Effective Effective Somewhat Effective Not at all effective

Governance and accountability structures are in place for the management of SIMSI.

  • Accountabilities across the entity between stakeholders, including partners, are clearly defined and managed.
  • Processes, activities and structures, which establish clear governance, objectives, roles and responsibilities and appropriate working and reporting arrangements ensure proper coordination between all parts of the organization.
  • Governance supports strategic and operational decision-making, efficiency and effectiveness of operations and the mitigation of risk.
     
The Policy Framework is adequate to define and monitor internal and external requirements associated with the operation, management, design and develop of the SIMSI system, as well as, provide direction for the SIMSI renewal project.      

Planning, Priority Setting and Resource Allocation processes are in place to set strategic direction and operational plans, objectives and priorities and to allocate financial, human and other resources in accordance with these plans.

  • The controls define the required resources and competence and serve to ensure that sufficient resources exist to meet SIMSI objectives, consider results of risk assessment and historical performance, and provide critical support for management in directing and monitoring operations.
  • The controls provide employees, 3rd party service providers and key partners with a clear understanding of priorities and resource boundaries.
     

Information and Communication processes, both formal and informal are adequate, in terms of quality, quantity and timing, to set direction, establish expectations and monitor performance and employee satisfaction.

  • The processes are adequate, in terms of quality, quantity and timing, to manage the relationships and expectations of SIMSI partners and key stakeholders.
     
The Training and Capacity Building processes and practices are sufficient to perform tasks, achieve objectives and to ensure the capacity to address future requirements associated with the SIMSI renewal project.      
Knowledge and Information Management practices and processes ensure that individuals have access to and are supported by complete and accurate information.      

Business Continuity Plans that reflect the degree of risk of the SIMSI system are complete, up-to-date, tested and appropriately communicated to key stakeholders and those directly affected.

  • The relationships and expectations of partners are considered in the SIMSI Business Continuity planning process.
     

Environmental Scanning practices such as the environmental scanning of economic, political, demographic factors as well as the monitoring of key indicators such as client satisfaction provide a solid and upto-date understanding of the external factors that exist as well as their impact on the operational objectives, risks and priorities.

  • The monitoring of changes to the regulatory and legislative frameworks.
  • The existence and effectiveness of these monitoring mechanisms contribute to management of the relationships and expectations of key partners and stakeholders.
     

Performance Management and Continuous Improvement process, practices and tools adequately address:

  • Cyclical and regular gathering, tracking, reporting and communicating of progress;
  • The establishment and communication of performance objectives and corresponding performance targets and indicators;
  • The use of performance information to ensure results and products are aligned with business requirements (Scope, Quality, Product management); and
  • The feedback of performance into the planning process.
     

Governance and Accountability

Not effective at all – Significant management attention is needed to improve these practices.

The governance and accountability structure associated with SIMSI Renewal is not effective in supporting the management of the risks associated with the design, development and implementation of SIMSI Renewal.

  • A functional decision-making structure required to direct SIMSI Renewal involving key senior managers has not been established.
  • Many of the key and fundamental documents required to establish the vision and objectives for the SIMSI of the future are still in unapproved draft form including: the IM/IT Strategic Plan that includes components of the IM/IT Human Resource Plan, a Communication Strategy and the SIMSI Renewal Business Case.
  • A formal Project Office to drive Renewal has not been established.

It is recommended that:

10. The Assistant Deputy Minister, Corporate Services, establish a SIMSI Renewal Steering Committee to provide department level oversight and direction to the SIMSI Renewal Project. The committee should include senior management members from all areas of the department.

11. The Chief Information Officer establish a SIMSI Project Management Office with a clearly identified and accountable SIMSI Renewal Project Manager.

12. The Chief Information Officer ensure that SIMSI Renewal roles, responsibilities and accountabilities are clearly defined, approved by the SIMSI Renewal Steering Committee and appropriately assigned.

13. The Chief Information Officer consider changing the name of the SIMSI Renewal Project to a new name which better reflects the broader departmental objectives of the project.

Policy Framework

Somewhat effective – Some parts of this element are working well, but many deficiencies exist.

The policy framework with respect to SIMSI Renewal will build on the existing policy framework which establishes the requirements for its external clients as well as its traditional internal clients. Continuous monitoring of the policy framework will be required to ensure that SIMSI Renewal defines and monitors the requirements of new internal clients, Government of Canada initiatives and a changing department.

Recommendation: None – Policy Framework activities are included in other recommendations.

Planning, Priority-Setting and Resource Allocation

Not effective at all – Significant management attention is needed to improve these practices.

While some work has been done, the planning, priority setting and resource allocation processes to set strategic direction and to manage the risks associated with the SIMSI Renewal Project are inadequate. The IM/IT Branch has not put the ideas, plans and work completed into a business case to be circulated to senior departmental managers for review.

  • The expected outcomes of the project are not clearly defined and approved.
  • The expected outcomes are not clearly understood by internal stakeholders.
  • Internal stakeholders do not feel they were adequately involved in the development of SIMSI Renewal Project documents and are unsure of how their feedback, when sought, has been addressed.
  • The Project Charter, Business Case and TB Submission documents are still in draft and are not approved.

There is no systematic and holistic approach to risk management within SIMSI. SIMSI risk management practices are operational in nature and deal with the risks associated with the operating system and the delivery of changes and enhancements within the established plans. SIMSI risk management practices and strategies will be addressed within the IM/IT Strategic Plan (currently in draft).

It is recommended that:

14.The Chief Information Officer ensure that a project plan is developed for SIMSI Renewal including: the identification of activities, deliverables, time frames and accountabilities. The plan should be forwarded to the SIMSI Renewal Steering Committee for approval.

15. The Chief Information Officer ensure that the SIMSI Renewal Project Manager is made accountable for implementation and management of the SIMSI Renewal Plan.

Information and Communication

Somewhat effective – Some parts of this element are working well, but many deficiencies exist.

The quality, quantity and timing of SIMSI Renewal information and communication were considered poor by internal potential stakeholders outside of IM/IT and Program Operations.

The Chief Information Officer is currently developing a Communication Strategy for SIMSI Renewal.

Recommendation: None – The SIMSI Renewal Communication Strategy and Communication Plan would be included in the SIMSI Project Plan and deliverables. The SIMSI Renewal Project Steering Committee will facilitate information and communication activities. Communication practices will also be addressed by Recommendation 3.

Training and Capacity-Building

Somewhat effective – Some parts of this element are working well, but many deficiencies exist.

The current training and capacity-building practices have been adequate to ensure people have the skills necessary to support the achievement of SIMSI objectives for both the current SIMSI operational environment and for SIMSI Enhancements.

SIMSI Renewal has not sufficiently progressed to establish its future requirements. Factors that might add risk to training and capacity building include: what skills will be needed for new clients and applications; the re-tendering of the SIMSI contract (currently with CGI) and the increased complexity of the growing department.

Recommendation: None – the SIMSI Renewal Planning Process would include an HR Strategy and Plan. The SIMSI Renewal Project Manager and Project Management Office would monitor the emerging training and capacity building issues.

Knowledge and Information Management

Effective – Most parts of this element are working as intended, but more work is needed in some areas.

SIMSI Renewal is being undertaken at a time when IM practices are being reviewed within the department. A new draft departmental IM Policy is being circulated for comments; a Responsibility and Accountability matrix for information management has been drafted; and an IM Capacity Check is scheduled for April and May 2006.

Recommendation: None – The SIMSI Renewal Project Manager and Project Management Office would be responsible for the knowledge and information management.

Business Continuity

Effective – Most parts of this element are working as intended, but more work is needed in some areas.

SIMSI Enhancement and SIMSI Renewal activities are changing and will change the types of business activities, the criticality of SIMSI to the business functions and the number and types of partners involved. Periodic business impact analysis will be required to determine if and when more rigorous business continuity processes should be implemented.

Recommendation: None – Business continuity planning for SIMSI Renewal will be addressed by Recommendation 5.

Environmental Scanning

Somewhat effective – Some parts of this element are working well, but many deficiencies exist.

SIMSI renewal has been less successful at environmental scanning than SIMSI operations and SIMSI Enhancements.

  • Internal potential clients expressed concerns with the way their requirements were being addressed and were unsure of how their input was being considered.
  • Many of the documents associated with the SIMSI Renewal are still unapproved drafts in part as a result of issues associated with the identification and management of stakeholder expectations.

Recommendation: None – The recommendations with respect to Governance and Accountability and Planning, Priority Setting and Resource Allocation will address the issues associated with environmental scanning.

Performance Management and Continuous Improvement

Not effective at all – Significant management attention is needed to improve these practices.

The SIMSI Renewal Project has not reached the point of defining performance management and continuous improvement strategies and practices. However, in order to ensure that these elements are given appropriate consideration and attention they should be defined and approved early in the lifecycle of the project.

It is recommended that:

16. The Chief Information Officer ensure that Performance Management strategies and practices are included in the SIMSI Renewal Project Plan and in the deliverables to be forwarded to the SIMSI Renewal Project Steering Committee for approval.

3.3 Resource Base

Effective – Most parts of this element are working as intended, but more work is needed in some areas.

The processes in place to monitor, control and report on costs for the current SIMSI environment are sufficient and effective. The Task Authorization process in place ensures that the costs of developing and implementing changes are determined and controlled. Performance at an operational level is monitored and reported. These costs relate to ongoing and cyclical activities covered in the CGI contract.

The process to monitor, control and report on costs for the SIMSI Renewal project is dependent on the establishment of an effective governance and accountability structure, the definition and implementation of processes for planning, priority setting and resource allocation and an effective mechanism to determine and manage the expectations of partners.

Recommendation: None.

3.4 Risk Management

Somewhat effective – Some parts of this element are working well, but many deficiencies exist.

SIMSI risk management practices are operational in nature and deal with the risks associated with the operating system and the delivery of changes and enhancements within the established plans. There is no systematic and holistic approach to risk management within SIMSI that includes:

  • A formal risk management process;
  • Systems and practices in place to mitigate identified risks, and that work as designed and are approved by senior management; and
  • Ongoing monitoring of potential risks using a risk framework and risk indicators.

SIMSI Risk management practices and strategies are to be addressed within the IM/IT Strategic Plan (currently in draft).

Recommendation: None – Addressed in Recommendation 2 and in Recommendation 4.

4. Audit Conclusion

4.1 Governance and Accountability Structures

SIMSI Operations and Enhancements

The governance and accountability framework that was effective in the past no longer adequately serves the new more complex SIMSI environment with its more demanding programs and involvement of multiple organizational units within INFC. A more formalized and broader Governance and Accountability structure is needed to ensure that all stakeholders have appropriate roles in support of setting strategic direction, planning and priority setting, resource allocation and the management of requirements and expectations.

SIMSI Renewal

The governance and accountability structure associated with SIMSI Renewal is not effective in supporting the management of the risks associated with the design, development and implementation of SIMSI Renewal.

  • A functional decision-making structure required to direct SIMSI Renewal involving key senior managers has not been established.
  • Many of the key and fundamental documents required to establish the vision and objectives for the SIMSI of the future are still in unapproved draft form including: the IM/IT Strategic Plan that includes components of the IM/IT Human Resource Plan, a Communication Strategy and the SIMSI Renewal Business Case.
  • A formal Project Office to drive Renewal has not been established.

4.2 Manage the Relationships and Expectations of the Partners

SIMSI Operations and Enhancements

The management of relationships and expectations is very effective. Stakeholder involvement was evident in the development, operation and change/enhancement environments associated with the programs supported by SIMSI. Stakeholders were involved in identifying business requirements, business rules and program application implementation within SIMSI. There is a forum for changes and enhancements and a mechanism to develop reports specific to one region or general to all. The IM/IT Directorate has made contact with the manager of new Programs to identify requirements and to assist in establishing Provincial participation.

The recommended new Governance and Accountability structure will ensure the continued effectiveness of management of relationships and expectations in the future.

SIMSI Renewal

The management control framework for SIMSI Renewal management of the relationships and expectations of partners is not effective. The audit found that:

  • The expected outcomes of the project are not clearly defined or understood by internal stakeholders.
  • Internal stakeholders do not feel they were adequately involved in the development of SIMSI Renewal Project documents and are unsure of how their feedback, when sought, has been addressed.
  • Many of the documents associated with the SIMSI Renewal are still unapproved drafts in part as a result of issues associated with the identification and management of stakeholder expectations.

4.3 Risk Management

There is no systematic and holistic approach to risk management within SIMSI. SIMSI risk management practices are operational in nature and deal with the risks associated with the operating system and the delivery of changes and enhancements within the established plans. SIMSI risk management practices and strategies will be addressed within the IM/IT Strategic Plan (currently in draft).

4.4 Resource Base

The processes in place to monitor, control and report on costs for the current SIMSI environment are sufficient and effective. The Task Authorization process in place ensures that the costs of developing and implementing changes are determined and controlled. Performance at an operational level is monitored and reported. These costs relate to ongoing and cyclical activities covered in the CGI contract.

The process to monitor, control and report on costs for the SIMSI Renewal project is dependent on the establishment of an effective governance and accountability structure, the definition and implementation of processes for planning, priority setting and resource allocation and an effective mechanism to determine and manage the expectations of partners.

5. Action plan to Implement Internal Audit Recommendation

Purpose of Action Plan: it is through the Action Plan that the auditee responds to the audit exercise by indicating whether he/she agrees with each audit recommendation, providing the rationale for any disagreement, and committing to take action in implementing the agreed-to recommendations. The Action Plan forms part of the Final Audit Report (in an annex). The Head, Internal Audit is responsible to follow-up on the implementation of the Action Plan and to report on progress periodically to the Departmental Audit and Evaluation Committee (DAEC).

Report Title: Audit Report
Audit of Shared Information Management System for Infrastructure (SIMSI)

Date of Complete Draft Report: May 2006

Author: Auditors from Interis, under the supervision of the Head, Internal Audit

Auditee: INFC Corporate Services - Information Management/Information Technologies Directorate

Overall Management Response

SIMSI Operations and Enhancements:
Agreement with the audit recommendations.

SIMSI Renewal:
Agreement in principle with the audit recommendations, however given the other priorities in the IM/IT area and the resources available, this project has been put on hold for six months. A thorough reassessment of the project will be completed by December 2006.

SIMSI Operations and Enhancements With Recommendations 2, 3, 4 and 5 also applying to SIMSI Renewal It is recommended that:

Audit Recommendations
(Recommendations listed in the order they appear in the audit report)
Management Response
(Agreement or not, with rationale for any disagreement)
Action Plan
(Action, Responsible Officer, Deadline)
1. The Assistant Deputy Minister, Corporate Services, establish a new Governance and Accountability Structure, Priorities and Resource Allocation processes and Policy Framework to address the requirements of the new broader and more complex SIMSI Operations and Enhancement environment. Agree. The ADM, Corporate Services will table a new Governance and Accountability Structure at Exec Committee by September 1, 2006.
2. The Chief Information Officer implement formal risk management strategies and practices that will ensure the continuous monitoring of potential SIMSI risks using a risk framework and indicators, and systematically mitigate identified risks. Agree. A risk-management framework for SIMSI will be elaborated. Within this framework, and building upon the initial risk-assessments carried out for SIMSI, formal risk-management strategies and practices will be defined and implemented. The Director, Application Services to undertake this task and CIO will approve the framework by end of October 2006.
3. The Director, Application Services, implement measures to improve communication practices within the SIMSI team and between SIMSI and its partners and stakeholders within INFC. Agree. The Director, Application Services, will establish and implement a communication strategy that will formalize existing communication practices and augment them, based on an assessment of identified gaps. This strategy is to be reviewed and accepted by key participants by end of July 2006.
4. The Director, Application Services, more clearly define and document the SIMSI risks with respect to human resources and develop an action plan, building on current HR initiatives, to address the risks. Agree. The Director, Application Services will develop an action plan and table it at Exec Committee by end of June 2006.
5. The Director, Application Services, undertake a periodic review of business continuity requirements to ensure that SIMSI business continuity plans are up-to-date and appropriately address client needs. Agree. The Director, Application Services will be reviewing and updating the existing business continuity plan for SIMSI, in accordance with an established schedule. The ADM, Corporate Services will approve the updated business continuity plan by end of November 2006.
6. The Director, Application Services, ensure that the SIMSI issues identified in the Mid-Term Evaluation Report, September 2005, are addressed. Agree (Note: Management Response was tabled at DAEC on April 5, 2006). Please see response to Recommendation 8 below concerning data quality issues.  In addition, lessons learned from the implementation of the SIMSI ICP are being incorporated in the current development effort. A comprehensive review to assess progress on data integrity issues will be undertaken during summer 2006 with the results and action plans presented to the INFC ADM IM/IT Committee by early fall 2006.
7. The Director, Application Services, ensure that the issues identified in the SIMSI Vulnerability Assessment and SIMSI security requirements are addressed. Agree. This activity must be undertaken as part of departmental compliance to MITS (Management of IT Security) by end of December 2006. The Director, Application Services will work with the IT Security Coordinator to address issues identified in the Vulnerability Assessment by December 2006.
8. The Director, Application Services, ensure that data integrity issues continue to be investigated and resolved and that regular status reports of data integrity activities be communicated to affected stakeholders. Agree. During summer 2005, plans to improve data quality were jointly developed between INFC and the regions and were eventually presented to the ADM Coordination Committee (ACC), which provided strong support for the entire initiative. Implementation has since been ongoing in close collaboration with the regions with progress status updates provided to the ACC. An internal assessment of progress to-date was completed in February 2006 and the results and future action plans will be finalized and presented to the INFC ADM IM/IT Committee for approval by end of June 2006. Any outstanding data integrity issues will be pursued until they are resolved.
9. The Chief Information Officer define and collect performance information that measures the degree to which SIMSI is effectively supporting business requirements and provides feed-back of SIMSI performance in the planning process. Agree. A comprehensive review will be undertaken to address this issue. The terms of reference of the review will be presented to the ADM, Corporate Services for approval by end of July 2006.

SIMSI Renewal In addition to recommendations 2, 3, 4 and 5 above, which also apply to SIMSI Renewal, It is recommended that:

Audit Recommendations
(Recommendations listed in the order they appear in the audit report)
Management Response
(Agreement or not, with rationale for any disagreement)
Action Plan
(Action, Responsible Officer, Deadline)
10. The Assistant Deputy Minister, Corporate Services, establish a SIMSI Renewal Steering Committee to provide department level oversight and direction to the SIMSI Renewal Project. The committee should include senior management members from all areas of the department. Agree. Given the other priorities in the IM/IT area and the resources available, the SIMSI Renewal project has been put on hold for six months. Its scope, objectives, timeframe, costs, and governance will be re-assessed before December 31, 2006.
11. The Chief Information Officer establish a SIMSI Project Management Office with a clearly identified and accountable SIMSI Renewal Project Manager. Agree. Project on hold for six months.
12. The Chief Information Officer ensure that SIMSI Renewal roles, responsibilities and accountabilities are clearly defined, approved by the SIMSI Renewal Steering Committee and appropriately assigned. Agree. Project on hold for six months.
13. The Chief Information Officer consider changing the name of the SIMSI Renewal Project to a new name that better reflects the broader departmental objectives of the project. Agree. Project on hold for six months.
14. The Chief Information Officer ensure that a project plan is developed for SIMSI Renewal including: the identification of activities, deliverables, time frames and accountabilities. The plan should be forwarded to the SIMSI Renewal Steering Committee for approval. Agree Project on hold for six months.
15. The Chief Information Officer ensure that the SIMSI Renewal Project Manager is made accountable for implementation and management of the SIMSI Renewal Plan. Agree. Project on hold for six months.
16. The Chief Information Officer ensure that Performance Management strategies and practices are included in the SIMSI Renewal Project Plan and in the deliverables to be forwarded to the SIMSI Renewal Project Steering Committee for approval. Agree. Project on hold for six months.

Contact:

Infrastructure Canada
(613) 948-1148
Toll Free Number: 1-877-250-7154
TTY: 1-800-465-7735
info@infc.gc.ca

Date modified: