Internal Audit Report - SIMSI management control framework have procedures or controls

Final Report–June 2009

Table of Contents

1. Executive Summary

The Shared Information Management System for Infrastructure (SIMSI) is a secure, bilingual, web-based program management system that allows users to register projects online, monitor project status, and access benefits and payment information. SIMSI is used as a management tool by Infrastructure Canada and its federal and provincial/territorial partners in their oversight role for the current suite of infrastructure programs. It was designed to provide data for project registration, status, milestone monitoring, benefits tracking, payment tracking, due diligence analysis and related documentation throughout the life of a project. It also provides reporting on all facets of the movement of the project through its lifecycle, to Infrastructure Canada, stakeholders, and the general public, through web-enabled technologies.

The objectives of the audit were to provide assurance that:

  • High-risk areas with the SIMSI management control framework have procedures or controls in place to mitigate the risks; and
  • Recommendations made in the 2006 internal audit report have been implemented.

The audit scope included the management control framework over the SIMSI operational, warehouse and business intelligence databases.

The examination phase of this audit commenced in February 2009 and concluded in mid-April 2009. The examination covered relevant management activities since the previous audit in 2006 and employed various techniques including interviews, review of management documentation, and walkthroughs of control procedures with staff who were responsible for executing them.

We found evidence of good financial integrity controls and procurement management controls with respect to the administration of contract work between INFC and its supplier, CGI. We also found that controls were adequate in the following areas:

  • Information and communications processes;
  • Training and capacity planning;
  • Operational performance management; and
  • Managerial oversight of staff.

However, we found that controls were not in place to monitor and measure performance with respect to business needs. Controls were in place but weak in the following areas:

  • Governance and accountability structures;
  • Strategic and operational planning;
  • Software release and testing processes;
  • Data quality control;
  • Information Technology controls related to information architecture; and
  • Business Continuity Planning.

We found that four of nine recommendations made in the 2006 SIMSI internal audit had been fully implemented. Recommendations made in this audit effectively replace those 2006 recommendations that remain unfulfilled.

Original signed by

Chief Audit Executive
Office of Infrastructure Canada

Date

2. Background

The Shared Information Management System for Infrastructure (SIMSI) was built shortly after the announcement of the Infrastructure Canada Program (ICP) in 2001 to support the implementing agencies and Infrastructure Canada in their responsibilities to administer the ICP and subsequent programs. SIMSI provides the information technology support system to assist management and stakeholders (municipalities and federal and provincial/territorial implementing agencies) in managing thousands of projects that will eventually be funded through the program.

The administration of the various programs is organized into three distinct program segments:

  • Directed Programs (large infrastructure programs determined by federal and provincial ministers);
  • Application Based Programs (community-based programs); and
  • Other Transfer Payment Programs (funding flows directly to the recipients before the expenditures for which it is intended are incurred, e.g. Gas Tax Fund)

All application processing is carried out by a web server that makes calls to an Oracle database. It contains all the SIMSI applications using different schemas (known as the SIMSI Operational Databases). The reporting subsystem of SIMSI uses an Oracle Enterprise Data Warehouse (EDW) that is updated from the SIMSI Operational Databases on a nightly basis. Recently, the reporting subsystem was changed to make use of the Cognos 8 suite of Business Intelligence tools.

The Information Management/Information Technology (IM/IT) Directorate of Infrastructure Canada implemented key stewardship processes in support of the Management Accountability Framework. These include a risk management process, key human resources processes, and detailed contract management processes. Compliance with the Management of Information Technology Security standards is being addressed with the creation of an IT Security position within the directorate. This position is responsible for ensuring that all departmental applications and technical solutions are engineered within the standards.

In May 2005 the Departmental Audit and Evaluation Committee (DAEC) of INFC approved the conduct of an assurance audit of the management control framework of SIMSI as part of the audit plan for that year. The audit was performed in 2005/2006 using a management control framework developed by the consultant (Interis) based on the Criteria for Control (CoCo) Model published by the Canadian Institute of Chartered Accountants. The final audit report issued in May 2006 made 16 recommendations overall; nine of these concern the management of SIMSI and enhancements to SIMSI.

In June 2008, Deloitte was contracted to provide a risk assessment of the IM/IT function of SIMSI. The study attributed high risk to IM/IT governance, funding and allocation, HR capacity, project management processes and in particular, data quality. Deloitte recommended that an audit of SIMSI was warranted.

Accordingly, the Departmental Audit Committee approved an audit of SIMSI in the Risk-based Audit Plan for 2008/2009. The audit was to provide assurance that high risks within the management control framework were being mitigated and to follow-up on recommendations made in the 2005/2006 internal audit.

3. Audit Objectives

The objectives of the audit were to provide assurance that:

  • High-risk areas with the SIMSI management control framework have procedures or controls in place to mitigate the risks; and
  • Recommendations made in the 2006 internal audit report have been implemented.

4. Audit Scope

The audit scope included the management control framework over the SIMSI operational, warehouse and business intelligence databases. The examination included an assessment of the mitigation of ten areas of the management control framework that were assessed as high risk during the planning phase of this audit. These areas were stated as the following audit criteria:

  1. Governance and Accountability structures are in place for the management of SIMSI.
  2. Planning, Priority Setting and Resource Allocation processes are in place.
  3. Information and Communication processes are in place, both formally and informally.
  4. Training and Capacity Building processes and practices enable employees to perform their tasks, achieve their objectives and build capacity to manage future SIMSI requirements.
  5. Business Continuity Plans that reflect the degree of risk of the SIMSI system are complete, up-to-date, tested and appropriately communicated to key stakeholders and those directly affected.
  6. Operational Control and Project Management Processes are in place.
  7. Financial Integrity Controls and Procurement Management for SIMSI are in place.
  8. Information Technology Controls are in place to ensure the integrity and security of its information technology, as well as, the completeness, accuracy and availability of data.
  9. Performance Management and Continuous Improvement processes, practices and tools are in place.
  10. Managerial Oversight: Managers are engaged and available to their staff, monitoring their performance and providing them with direction and support based on the complexity of activities, the degree of delegation and the level of employee competence.

The audit also examined management action in response to 9 of 16 recommendations made in the 2006 internal audit report. The scope excluded follow-up on 7 of the 16 recommendations relating to "SIMSI Renewal" because this project was discontinued shortly after the 2006 internal audit was completed and consequently management action plans were not developed.

5. Audit Approach

The audit was conducted in accordance with the Government of Canada internal auditing standards.

The examination phase of this audit commenced in February 2009 and concluded in mid-April 2009. The examination covered relevant management activities since the previous audit in 2006 and employed various techniques including interviews, review of management documentation, and walkthroughs of control procedures with staff who were responsible for executing them.

The criteria used to assess the SIMSI Management Control Framework were discussed with and agreed to by the Chief Information Officer before the conduct of the detailed audit procedures.

Our findings, as presented in this report, were assembled from our assessments of each of the above ten criteria and consolidated into a set of recommendations that address the governance and control risks for six criteria. Recommendations were not offered for criteria 3, 4, 7 and 10; based on our assessment that residual risk was low for these areas.

Our assessment of the progress of recommendations made in the 2006 SIMSI audit report is presented in Appendix A. Our assessment of the progress on each recommendation was based on our findings for each of the audit criteria.

For purposes of this report, the residual risk rankings associated with findings and previous audit recommendations use a low, moderate, high three-point scale and are subjectively judged based on our knowledge of the SIMSI management control framework gathered during the audit. The subjective criteria are:

High – Threats/Opportunities have very significant impact on INFC's objectives, are imminently likely and no or uncertain mitigation measures are in place.

Moderate - Threats/Opportunities have significant impact on INFC's objectives, have a longer-term likelihood and reliable mitigation measures are planned or are being established.

Low - Threats/Opportunities do not have a significant residual risk to INFC's objectives.

6. Audit Findings

6.1 Governance and Accountability Structures

Observation
Moderate Risk

Processes, activities and structures establishing clear governance, objectives, roles and responsibilities and appropriate working and reporting arrangements, were put in place. However, over time there has been less participation by internal business members and little strategic direction, resulting in a lower quality of governance and accountability.

The SIMSI Steering Committee was originally established to support strategic and operational decision-making, efficiency and effectiveness of operations and the mitigation of risk. Recently, the role of the Steering Committee has been that of a communications forum with the federal delivery partners and internal business units.

Accountabilities of stakeholders, including partners, were not clearly defined and communicated. Partners, regions, provinces/territories were unclear about their roles and responsibilities with respect to SIMSI. In theory, all the stakeholders (internal and external) were represented in the governance structure but the actual commitment and participation of the stakeholders was not adequate for governance. The program management areas were not consistently participating nor were they taking sufficient ownership of SIMSI's business requirements and data, even though it was supposed to be the main business application for INFC's program operations. Although the role of the Steering Committee was documented in the terms of reference, the roles and responsibilities of the various stakeholders in SIMSI were not clear. There needs to be clarification with the business community on

  1. who is accountable for the development of business requirements for IT systems and
  2. who has responsibility for "ownership" of IT systems that support program delivery.

There was a risk that SIMSI will not meet the business requirements of INFC, especially in the area of reporting. There was the risk that additional "tools" (e.g. excel spreadsheets) will be used instead of SIMSI and that may lead to reporting inconsistencies with SIMSI. There was a risk that more stakeholders, including partners, will stop using SIMSI.

Recommendations Management Action Plan

1.1 IM/IT should present to the Executive Committee a proposal to clarify the mandate of the SIMSI Steering Committee in terms of its capacity to make both strategic and tactical business decisions and to structure it so that it can fulfill its reconfirmed mandate. After the Executive Committee has approved the mandate, IM/IT should lead the implementation.

 1.1 Agreed. IM/IT will update the SIMSI Steering Committee terms of reference to reflect a stronger mandate and membership. It will be presented for approval and support from the Executive Committee. Report of its activities will be tabled periodically to the Executive Committee.
Manager Responsible:

Chief Information Officer
Due Date:

September 30, 2009

6.2 Strategic and Operational Planning

Observation
Moderate Risk

Strategic plans were not in place to define the required resources to meet SIMSI objectives, consider the results of risk assessments, provide for business performance management, or provide critical support and communications for management in directing and monitoring operations. Additionally, strategic plans that provide employees, CGI and key partners with a clear understanding of priorities and resource limitations were not in place.

SIMSI strategic planning and priority setting processes were not adequate to set strategic direction and operational plans, objectives and priorities. The allocation of SIMSI financial, human and other resources was not linked to strategic planning and was basically driven by the new Economic Action Plan initiatives and constrained by the capacity and resources available.

INFC was very reactive to the political environment within which they were operating. Strategic planning was not an activity that the organization had addressed well. Several interviews suggested that the business community within INFC had little available capacity to work with IM/IT on developing a SIMSI strategic direction. As the use of SIMSI was not mandatory, certain areas of the business community had little interest in it and were not SIMSI users.

While operational plans were in place, they were not linked to strategic planning. Operational plans assigned available resources to meet SIMSI tactical, short-term objectives and provided support for management in directing and monitoring operations. Operational plans that provided employees, CGI and key partners with an understanding of INFC's short-term priorities and resource boundaries were in place. However, partners were not clear as to whether INFC had adequately considered the priorities and resources of the partners. Also, there was no evidence that users and other stakeholders influence operational planning.

There was a risk that SIMSI may not address the requirements of the business community and external stakeholders, and may not be able to provide the required information, particularly related to program performance reporting. There was also the risk that changes required to keep SIMSI operational may not be made and the system may become obsolete and unsustainable.

There was also a risk that operational costs may increase as application maintainability decreases due to complexity of systems structure and program code, integration of multiple versions of underlying technologies and deteriorating documentation.

Recommendations Management Action Plan

2.1 IM/IT should, in collaboration with both internal and external stakeholders, complete a strategic plan for SIMSI, in line with the current strategic direction for INFC, and present it to the Executive Committee for approval.

2.1 Agreed. A strategic direction document for SIMSI was drafted and discussed at the SIMSI Steering Committee. A final version will be tabled for feedback at the Executive Committee by September 30, 2009. This will be followed by work on the information architecture and an analysis of how SIMSI can effectively support program management (see recommendation 5.1) with a view to producing a final strategic document for approval by the Executive Committee by December 2009.
Manager Responsible:

Chief Information Officer
Due Date:

December 31, 2009

6.3 Operational Control and Project Management

Observation
Moderate Risk

Operational control processes and practices were in place to ensure that all operational activities were conducted appropriately and in compliance with stated directives and established standards. Historically, the production environment has been stable. However, controls have recently been relaxed in the area of Release Management resulting in increased risk of environmental corruption. Under very tight deadlines, software releases were not always properly tested. Under time pressure, release process waivers were signed with CGI to bypass contracted terms and conditions. Also, there have been waivers on service level agreements with the recent system products that have been released to the production environment.

Project Management controls have recently been relaxed in the area of requirements management and user acceptance testing, resulting in increased risk that released products may not meet user requirements. Interviews with staff indicated that the mapping of developed functionality to business requirements (traceability) was not well done for most of the applications. As recent development timeframes tightened, there was less time to perform regression testing and correct any defects. As such, best practices in project management and established standards in operation were bypassed and key development activities such as user acceptance tests were compromised in attempts to meet deadlines.

There was a risk that the overall operations environment may experience difficulties and fail because key activities were not carried out or were inadequate.

Recommendations Management Action Plan

3.1 IM/IT should review Release Management procedures and establish a mandatory process that protects the integrity of the development, testing and production environments. Any deviation from this process should be authorized by the Chief Information Officer.

3.1 Agreed. IM/IT will table procedures and process in support of an effective Release Management.
Manager Responsible:

Director, Application Development and Information Management
Due Date:

September 30, 2009
Recommendations Management Action Plan

3.2 IM/IT should strengthen control on Requirements Management by establishing a process that documents requirements traceability for new applications and their maintenance releases.

3.2 Agreed. A process that established Requirements Management existed but will be reinforced by IM/IT for all significant future releases.
Manager Responsible:

Director, Application Development and Information Management
Due Date:

September 30, 2009
Recommendations Management Action Plan

3.3 IM/IT should ensure that appropriate managers from relevant INFC business units participate in and authorize (sign-off) user acceptance tests of all SIMSI application products prior to their release to production.

3.3 Agreed. Acceptance testing sign off by the end user will be made more explicit in the current sign-off policy.
Manager Responsible:

Director, Application Development and Information Management
Due Date:

September 30, 2009

6.4 SIMSI Data Quality

Observation
Moderate Risk

Business and IT controls were not adequate to ensure the quality (validity, completeness, accuracy, timeliness) and consistency of data.

System and application processes were in place to control data integrity (e.g. back-up and recovery), but it was difficult to ensure the accuracy, completeness and timeliness of data through application controls because these problems mainly occur at data entry. Application controls over data quality are limited to data entry edit checks and exception reporting.

At the point of data entry, concerns were mainly about consistency of data definition for data elements across the various business programs and with different interpretations, terminologies and usage among regions. INFC business units carefully monitored the large projects and the related data. Projects monitored by federal partners and provinces/territories were less likely to have consistent data and data updates were not timely. It was not clear what information must be reported for each of the programs and within this, what information was critical to the business.

There were a significant number of potential users internal to INFC who were not using SIMSI. For example, Finance did not rely on SIMSI information to validate information in their financial systems.

Of all the Infrastructure Programs in SIMSI, the Gas Tax Fund (GTF) was particularly problematic since eleven of the thirteen provincial/territorial partners were not using SIMSI on an on-going basis. The remaining two provinces (BC and Manitoba) depended on SIMSI and were actively using it. The rest either had their own applications, used excel spreadsheets, or MS Access databases as they had concerns about SIMSI data quality and doubted that SIMSI could meet their business needs.

INFC GTF program management, as well as some other INFC areas, were using other tools such as MS Excel spreadsheets and MS Access databases in addition to SIMSI. The GTF implementation demonstrates that a mixed use of SIMSI and other data tools increases the risk of duplication of effort and data error in the implementation of Infrastructure Programs.

There was also little motivation for SIMSI users to update SIMSI beyond the project approval stage, once funds were authorized. Users were never required to keep SIMSI up to date, resulting in timeliness and/or completeness issues in the data. It is usually data associated with later stages of project development such as the project monitoring and financial data that were not entered as the project progresses. There was the risk INFC may not be able to report complete, accurate program results in a timely manner.

The Project Charter for the Data Quality – Communities Programs was issued recently and a project team has been established. The initial work of the group was focused on the quality of data in the Communities Programs, to determine the critical set of data for business purposes, to develop quality criteria and good data definitions and to acquire and use tools to test the quality of information. Previously, no one had responsibility for analysis and review of data quality and consistency on an ongoing basis.

Recommendations Management Action Plan

4.1 IM/IT should propose a more formal data quality monitoring function for SIMSI in the context of a department-wide data quality and information management initiative. In the meantime, IM/IT should continue with its special data quality project.

4.1 Agreed. IM/IT will propose a formal data quality monitoring function based on measurable deliverables and milestones for approval by the Executive Committee.
Manager Responsible:

Director, Application Development and Information Management
Due Date:

November 30, 2009

6.5 SIMSI IT Controls

Observation
Moderate Risk

SIMSI information technology controls were in place to ensure the integrity and logical security of its information technology and to ensure the confidentiality, availability and integrity of its information assets (software and data).

However, no assurance was provided that the data elements in SIMSI, already plethoric and growing with new releases, were all necessary for program management. There was no documentation coordinated by IM/IT as to which elements are more critical in terms of business impact.

The SIMSI application was designed and implemented six years ago for one business program and was running beyond its expected lifetime. With this short-term view, design and coding decisions were made that were expedient but not strategically optimal for the longer term. Data modeling during system design was based on a single business program. This design has been re-used and patched each time a new business program was introduced into SIMSI.

Management also expressed concern that some of the code was becoming increasingly obsolete and would be difficult to sustain, requiring specialized IT resources to maintain it.

This finding, along with data quality and release management issues previously presented, are symptoms of an existing information architecture that is not likely to best serve current business requirements. In the systems sense, information architecture is the analysis and design of the data stored by SIMSI, concentrating on entities, their attributes, and their interrelationships. It refers to the modeling of data for the individual databases and to the corporate data models that INFC should define to coordinate the definition of data in several databases.

Recommendations Management Action Plan

5.1 Consistent with IM/IT strategic direction, IM/IT should lead business management in the examination of the current information architecture for SIMSI and determine its effectiveness for program management.

5.1 Agreed. A review of SIMSI information architecture will be done including an analysis on how SIMSI can effectively support program management. This new document will be created, discussed at the SIMSI Steering Committee with a final version to be tabled for approval and support at the Executive Committee.
Manager Responsible:

Chief Information Officer
Due Date:

December 31, 2009

6.6 Business Continuity Plan

Observation
Moderate Risk

INFC had a Business Continuity Plan (BCP) Program policy in place that required all business units to contribute to the development, implementation, maintenance and testing of BCPs. A Business Impact Analysis (BIA) for the Department, which provides the framework for a BCP, was in progress but not yet completed.

SIMSI was not "critical" as defined by Government of Canada BCP policy. The SIMSI risk management documentation did not identify any risks and impacts of SIMSI not being available. The tolerable outage time was established and set at seven days. It was not clear whether the business community had set or accepted this outage time and this should be addressed by the BIA.

The contractor (CGI) providing SIMSI operations had a Disaster Recovery Plan (DRP) prepared in conjunction with IM/IT, as per the contracted terms and conditions but the DRP had not yet been tested. Also, it is important to ensure that, once the BIA is complete, BIA findings that impact response, recovery and restoration requirements are used to adjust and update CGI's DRP. Following this, the development of a BCP for the Department needs to include a component that addresses business dependencies on SIMSI.

There is a risk that an incident will occur that requires the DRP to be activated. Without adequate testing there was the potential that the plan may not work effectively, resulting in delays to recovery beyond the contracted outage time of seven days. Also, without the completed BIA, the impact of a SIMSI outage on the business community had not been adequately assessed and the need for business contingency planning on a system outage remained unclear.

Recommendations Management Action Plan

6.1 IM/IT in conjunction with CGI should conduct a sufficient test of the "Infrastructure Canada Disaster Recovery Plan" developed by CGI in order to verify that procedures in the plan will adequately address the response, recovery and restoration of the SIMSI system.

6.1 Agreed. The Disaster Recovery Plan will be tested during this fiscal year.
Manager Responsible:

Director, Application Development and Information Management
Due Date:

March 31, 2010
Recommendations Management Action Plan

6.2 The Department BCP Coordinator should complete the Business Impact Analysis that is underway in order to provide a timely update to risks associated with SIMSI outage and the impacts on SIMSI business processes and partner relations and expectations.

6.2 Agreed. The BCP Coordinator will engage the Department to complete a Business Impact Analysis this fiscal year and a Business Continuity Plan for the Department by December 2010.
Manager Responsible:

Director, Finance and Administration Division (Department BCP Coordinator)
Due Date:

March 31, 2010 (for BIA)

7 Audit Opinion

SIMSI Management Control Framework

Based on the findings reported, our overall opinion on the management control framework for SIMSI is that it was not satisfactory.

We found evidence of good financial integrity controls and procurement management controls with respect to the administration of contract work between INFC and its supplier, CGI. We also found that controls were adequate in the following areas:

  • Information and communications processes;
  • Training and capacity planning;
  • Operational performance management; and
  • Managerial oversight of staff.

However, we found that controls were not in place to monitor and measure performance with respect to business needs. Controls were in place but weak in the following areas:

  • Governance and accountability structures;
  • Strategic and operational planning;
  • Software release and testing processes;
  • Data quality control;
  • Information Technology controls related to information architecture; and
  • Business Continuity Planning.

Implementation of 2006 Internal Audit Recommendations

We found that four of the nine 2006 audit recommendations, that were within the scope of this audit, had been fully implemented. Based on our assessment, presented in Appendix Ain section 7, sufficient progress had been made in the fulfillment of recommendations 2, 3, 4, and 7. Further progress is required for the remaining recommendations. Future progress on these recommendations should be followed by INFC Internal Audit as part of the recommendations made in this audit. The table below summarizes our assurance gap in the 2006 audit recommendations and makes a follow-up linkage to the recommendations of this audit.

Table 1. Implementation of 2006 Internal Audit Recommendations

2006 Audit Recommendation Implementation Gap 2009 Audit Rec.
1. The Assistant Deputy Minister, Corporate Services, establish a new Governance and Accountability Structure, Priorities and Resource Allocation processes and Policy Framework to address the requirements of the new broader and more complex SIMSI Operations and Enhancement environment. Governance processes, activities and structures establishing clear objectives, roles and responsibilities and appropriate working and reporting arrangements were put in place. However, management should clarify the mandate of the SIMSI Steering Committee in terms of its capacity to make both strategic and tactical business decisions. A mechanism should be established for the Committee to enable it to fulfill its mandate. 1.1
5. The Director, Application Services, should undertake a periodic review of business continuity requirements in order to ensure that SIMSI business continuity plans are up-to-date and appropriately address client needs. A BIA and BCP for INFC remain to be completed. Also the DRP for the SIMSI installation at CGI needs to be tested. 6.1
6.2
6. The Director, Application Services, should ensure that the SIMSI issues identified in the ICP Mid-term Evaluation Report, September 2005, are addressed.
The Evaluation recommendations pertaining to SIMSI are presented in Appendix A, recommendation 6.		 The ICP Mid-term evaluation report made five recommendations (see Appendix A) regarding improvements to SIMSI data quality, consistency, data entry control and linkage to financial systems. These recommendations have not been fulfilled but a new Data Quality project team has recently been established to address these issues. Also, our recommendation to re-assess SIMSI information architecture should address ICP Evaluation recommendations 11a and 11c, which call for a simplification of data collection and a mapping of data entry processes. 4.1
5.1
8. The Director, Application Services, should ensure that data integrity issues continue to be investigated and resolved and that regular status reports of data integrity activities are communicated to affected stakeholders. The recommendation to continue to investigate and resolve "data integrity" issues (we interpret this to mean "data quality" issues) has not been adequately implemented. However we expect that the establishment of the new Data Quality project team will fulfill this recommendation. 4.1
9. The Chief Information Officer should define and collect performance information which measures the degree to which SIMSI is effectively supporting business requirements and which would provide feed-back of SIMSI performance into the planning process. There are still no formal performance management processes in place for SIMSI with respect to its business requirements. We expect that our recommendation to clarify SIMSI governance and to complete a strategic plan for SIMSI will include definition of performance information which measures the degree to which SIMSI is effectively supporting business requirements. This measurement is vital to support the future planning process for SIMSI. 1.1
2.1

8 Statement of Assurance

In our professional judgment, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of findings reached and contained in this report. The findings were based on a comparison of the situations as they existed at the time against the audit criteria.

The findings are only applicable for the entity examined. The extent of the examination was planned to provide a reasonable level of assurance with respect to the audit criteria. The evidence gathered meets professional audit standards and is sufficient to provide senior management with proof of the findings derived from the internal audit.

APPENDIX A: FOLLOW-UP ON 2006 AUDIT REPORT RECOMMENDATIONS

2006 Audit Recommendation Assessment Level of Risk
1. The Assistant Deputy Minister, Corporate Services, establish a new Governance and Accountability Structure, Priorities and Resource Allocation processes and Policy Framework to address the requirements of the new broader and more complex SIMSI Operations and Enhancement environment.

Governance processes, activities and structures establishing clear objectives, roles and responsibilities and appropriate working and reporting arrangements were put in place.

The SIMSI Steering Committee was originally established to support strategic and operational decision-making, efficiency and effectiveness of operations and the mitigation of risk. Over time, there has been less participation by members, little strategic direction, resulting in a lower quality of governance and accountability.

There was insufficient buy-in and support from both the INFC and Partner business community in order to manage SIMSI strategically. The SIMSI application was not being sufficiently acknowledged by the business community as the INFC business application of choice.

Accountabilities of stakeholders, including partners, were not clearly defined and communicated.

 Moderate Risk
2. The Chief Information Officer should implement formal risk management strategies and practices that will ensure the continuous monitoring of potential SIMSI risks using a risk framework and indicators and systematically mitigates identified risks.

Completed.

Formal risk management strategies and practices have been implemented.

 Low Risk
3. The Director, Application Services, should implement measures to improve communication practices within the SIMSI team and between SIMSI and its partners and stakeholders within INFC.

Completed.

The Director, Application Services, has completed a number of measures with respect to communication practices. These include the following measures:

  • The SIMSI Steering Committee and User Group were actively communicating information between SIMSI and its partners and stakeholders within INFC. The weakness is the mainly one-way communication from IM/IT to partners and stakeholders.
  • Efforts were being made by an internal IT staff member tasked with building the relationships between the INFC and active and potential internal users.
  • A good communication structure (meetings and reporting) has been set up between SIMSI team and CGI contractor.
  • SIMSI Corner (intranet) provided internal resources with information such as training.
  • A SIMSI Communication Strategy was approved September 2007 by the SIMSI Steering Committee. It needs to be updated to reflect changes in business programs.
 Low Risk
4. The Director, Application Services, should more clearly define and document the SIMSI risks with respect to human resources and develop an action plan, building on current HR initiatives, to address the risks.

Completed.

A high level IM/IT Human Resources Strategic Plan was developed originally in 2006/07 and was updated in 2007/08. The plan set priorities, and identified a number of key actions and strategies to improve HR management practices. The plan included recommendations against each of the key results in order to improve the ability of IM/IT to maintain and manage its human resources. It will take some effort and time to address the various recommendations especially in the current environment when IM/IT is responding to demands of the Economic Action Plan.

 Low Risk
5. The Director, Application Services, should undertake a periodic review of business continuity requirements in order to ensure that SIMSI business continuity plans are up-to-date and appropriately address client needs.

A Disaster Recovery Plan (DRP) for INFC has been prepared by CGI, in collaboration with INFC. It contains the minimum elements as described in the contract with CGI. It was approved and issued in November, 2008.

However, the plan has not been tested. A test is required to verify that procedures in the plan will adequately address the response, recovery and restoration of the SIMSI system.

During this audit, a Business Impact Analysis for INFC was underway. It will provide an update to risks associated with SIMSI outage and the impacts on SIMSI business processes, partner relations and expectations.

 Moderate Risk

6. The Director, Application Services, should ensure that the SIMSI issues identified in the ICP Mid-term Evaluation Report, September 2005, are addressed.

The recommendations pertaining to SIMSI are:

  • Rec. 4a: Use SIMSI Data Quality initiatives to enable reporting of project outcomes at a national level;
  • Rec 11a: Simplify the amount of data collection in SIMSI by including only data that is critical to project monitoring and reporting;
  • Rec 11b: Ensure consistent collection across jurisdictions by documenting data requirements and distributing the documentation to those responsible for data collection and entry;
  • Rec 11c: Map the processes used to enter data into SIMSI in order to identify and eliminate inefficiencies in the data collection for SIMSI;
  • Rec 11d: Link SIMSI to financial systems.

The SIMSI Data Quality initiative should be able to address the recommendations resulting from the ICP Mid-term Evaluation Report. The activities related to SIMSI Data Quality initiatives are described in more detail as part of the response to Recommendation 8. It should be noted that these efforts will eventually address the ICP data but initially they will be directed to the communities based programs and then will review data collected and used in SIMSI for all Infrastructure Programs. A related initiative was underway to look at potential integration between SIMSI and a new INFC financial system.

Through the development of processes and procedures (task authorization), controls for contract financial transactions and procurement/contract management were in place. The controls for contract financial transactions were ensuring that these transactions were accurate, authorized and valid; that audit trails were maintained and INFC IM/IT tracked changes to information.

IM/IT put in place a procurement strategy to identify SIMSI requirements and risks. They set up processes to manage the procurement activities that resulted in the CGI contract. In addition there was a strategy and processes for the management of the contract.

 Moderate Risk
7. The Director, Application Services, should ensure that the issues identified in the SIMSI Vulnerability Assessment and SIMSI security requirements are addressed. (In essence the recommendation was to make SIMSI compliant with Treasury Board policy on Management of Information Technology Security, MITS).

Completed.

The Technology Refresh Infrastructure Project (TRIP), which was designed to be MITS compliant, has been completed. A security and accreditation process, including security provisions, was in place and was performed for SIMSI's migration to TRIP at CGI to ensure that the Infrastructure Canada component of TRIP was compliant with MITS. At that time, a Threat and Risk Assessment (TRA) was performed by CGI.

Also, The CGI contract had security requirements that are based on MITS.

 Low Risk
8. The Director, Application Services, should ensure that data integrity issues continue to be investigated and resolved and that regular status reports of data integrity activities are communicated to affected stakeholders. A new plan to initially focus on community-based funding programs and the SIMSI data related to these programs was underway with the creation of an INFC Data Quality project team. Its progress on addressing data quality issues will be provided periodically to the INFC Executive Committee who has initiated the project and provided membership from the different INFC sectors. Moderate Risk
9. The Chief Information Officer should define and collect performance information which measures the degree to which SIMSI is effectively supporting business requirements and which would provide feed-back of SIMSI performance into the planning process.

There were still no formal performance management processes in place for SIMSI with respect to its business requirements. Until the SIMSI governance and planning issues are addressed, and the intermediate and long range business needs are better defined, performance management and continuous improvement can not be adequately addressed.

Management pointed out that some progress has been achieved to further involve the clients as business requirements are being identified and then presented as part of functional designs for SIMSI enhancements. A new policy for systems signoff was approved at a recent Executive Committee. The accelerated pace for enhancements in support of the many new Infrastructure Programs has resulted in much more dialogue and participation by the client community in the early stages of application development.

There has also been progress in performance monitoring and reporting with respect to SIMSI operations and development. This performance was formally in place and was managed as part of the supplier contract with CGI.

 Moderate Risk
Date modified: