Preamble:

Internal Audit (IA) is responsible for providing an independent, objective appraisal activity for the purpose of advising and assisting the management, staff and Board of Governors of the University of Waterloo in the achievement of the institution’s goals and objectives. This document provides information on one type of IA’s activities: reviews of major computer systems development projects. Information on other audit services (operational audits, horizontal audits, special investigations, and external audit assistance) will soon be available on an IA web page. Focus is on significant computer projects, which includes installation of purchased software as well as systems developed at the University. This document was developed by representatives from IA and Information Systems and Technology (IST), and it has been reviewed by senior administrators responsible for areas where major computer systems have been or are about to be implemented.

Purpose of this document:

• Provide a general framework for the above referenced reviews and clarify roles.
• Facilitate planning and coordination with all parties involved in the projects, including: owners of the information technology components, users of the systems, IST staff, and internal auditors.
• Help ensure that the reviews are completed in a timely, and efficient and effective manner.

IA’s involvement in computer system development projects primarily focuses on providing independent assessments on whether appropriate controls (see further information below and definitions in Appendix A also) are incorporated in the systems and suitable project controls are employed. During the reviews IA, would:

• provide advice and assistance to project teams in the management of various risks and controls in the systems.
• provide timely recommendations for any identified control weaknesses. This is one of the main benefits of IA’s computer system development reviews. Coordination with the activities of the project teams is necessary for early detection of control weaknesses. If control weaknesses are not identified until late in the system development activities, expensive changes may be necessary to improve controls, or there may be risk exposures that could have been avoided.
• provide suggestions on possible operational improvements based on observations, previous audit work, contacts with other audit professionals, etc.

General approach and responsibilities:

IA reviews support and encourage a risk management based approach to controls (see definitions in Appendix A). With the movement away from hierarchical "command and control" organization structures to "empowered" organizational structures, it is more important than ever to foster a risk management and control culture at the University. In this environment, it is essential that accountability for information technology risk management and control be clearly established. This should generally rest with the managers who are the "owners" of the information technology components. Responsibility for operation of the controls can be delegated to others (e.g., as operational users, custodians, and third party or internal service providers), but the referenced managers remain accountable for the overall management of risk for their components. IA reviews can assist management in fulfilling these responsibilities.

Scope:

The information provided in this document is not intended to be a detailed "to do" list since each system development project is unique, and, each requires a preliminary determination of the level and type of audit activity. An overview of the reviews is provided below and further details are provided in the next section and in Appendixes B and C (page 1 and page 2).

For most major computer system development reviews IA will:

• meet with project team representatives to obtain an understanding of how the project will be managed and the system and business processes. Audit review procedures will also be discussed/confirmed.
• review the assessment of risk and the adequacy of controls to mitigate significant risks.
• review the deliverables at major stages in the project’s life cycle to help evaluate risks and controls.
• perform certain tests at the appropriate stages in the systems development activities to ensure that controls are in place that mitigate significant risks in a cost-effective manner.

• provide advice and recommendations to the project team to help them meet project and organizational objectives.

IST and IA have developed a schedule (see Appendix B) listing some common system development stages, deliverables that may be of use during IA reviews, and likely audit steps and IA deliverables. Additional information is also provided below under headings that relate to the phases identified in Appendix B.

Throughout all stages of a project, IA would:

• review the evolving project plans.
• perform a general assessment of project controls.
• review minutes of the project teams and management committees to determine risk control issues. If warranted, IA would also attend the meetings to obtain first-hand knowledge and/or assist in risk and control discussion.

IA will periodically review IST plans for upcoming computer system development projects. Based on discussions with the project management, IA will make a preliminary assessment of the level of audit review based on the type of system and the possible business risks associated with the changes being made. IA will incorporate major computer system development reviews into their annual audit plans, which are approved by senior management and the Board of Governors’ Audit Committee.

As business cases for individual projects are developed, IA would review them to become familiar with the scope and objectives of the planned activities. Comments and suggestions may be provided to project staff based on the review.

Analysis and design

During these stages IA would:

• Review the project pre-planning and base line process documents. IA would examine these documents to become familiar current systems and procedures and how these will be improved.

• Review the project plans, preliminary analysis and detailed design documents (and other deliverables during the design phase). IA would look for evidence that audit requirements are included in the project plan, and that organizational objectives and user requirements are reasonably met. Due to rapidly changing technology, there may be occasions when training of IA staff may be necessary, or when it is more practical to obtain external expertise for the review. In these circumstances, the cost and benefits of the alternatives will be determined.

• Review the project teams considerations for risks and controls in the system either in the system design documents or preliminary risk and control analysis documents (see further information in the next section and in Appendix C - page 1 and page 2). Where possible, IA will provide advice on the controls being considered or designed.

During this stage IA will:

• Review the request for proposal (or development plans if the system will be developed internally) and contracts with software suppliers and consultants.
• Review the records and results for any prototyping activities.
• Review the fit gap analysis.

• A review of the risk/controls assessment documents prepared by the project teams (see examples of the suggested documentation in Appendix C - page 1 and page 2) will either be continued (if prior work has already been done) or started. IA will normally recommend a standard risk/control evaluation methodology adopted from our external auditors to facilitate efficient involvement in the audit reviews by both IA and project staff. If necessary, IA will assist in the preparation of the documents. Risk and control sheets should be started as early as possible in the systems development life cycle. Risk management is an ongoing process and the risk/control assessment documents should be updated as the system evolves and the details about the software, processes, and computer environment become known. The risk assessment should be sectionalized by the major processes in the system. In addition, there should normally be a section on project risks, computer environment risks, and file conversion risks. Each section should be introduced by a high level chart or written narrative of the processes involved. The risk assessment should document:

• the significant business risks associated with the project;
• the controls that would mitigate these risks;
• any decisions by management to accept identified risks, along with the rationale for doing so; and
• references to specific systems tests or other evidence that the controls are operating as intended.

• A review of the security documents and plans, including configuring the various security features of purchased software may also be started at this stage, depending on the progress made by the project teams in this area. Security measures should prevent unauthorized access, manipulation of data, and disruption of service. Data should be classified by functional managers as to sensitivity and confidentiality to determine the appropriate level of security. IA would assess whether security is reasonable given the data classifications. IA would review user privileges to ensure they are consistent with job functions and that access is appropriately authorized by the data owners. In client-server environments, IA would review the network topology and any other measures to secure data at the workstation and during transmission.
• A review of the problems/issues tracking records would also be done to ensure that problems are resolved in a reasonable and timely manner.

At this stage IA will:

• Review the functional testing plans and the results of the tests. The tests and plans should be referenced to the functional requirements and the system controls identified in the risk analysis. IA would look at the plans to ensure they employ a reasonable methodology and are complete. The tests would be reviewed, possibly on a sample basis, for evidence that required controls function and that problems are identified for follow-up.
• Review measures to monitor and manage performance. IA would look at network and server configurations and ongoing performance monitoring procedures for reasonability. Prior to implementation, IA would look for evidence that the network, server and application have been load tested.

• Review the conversion of data from legacy systems to the new system. IA would want to review the plans to ensure that conversion methodology is reasonable, that steps are clear and complete, that data is secured during the process, and that there is appropriate field mapping documentation. Tests of the conversion records will normally be done using computer auditing tools (e.g., computer analysis and sampling software) override tests of conversion team work, or manual verification techniques. The extent and type of IA testing will depend on the extent and type of testing done by the conversion team and the significance of data elements.
• Continue with the review of the risk and controls analysis documents. At this point, all the security features of the system should have been installed and related details should have been incorporated into the risk/controls documents.
• Continue with the review of the problems/issues records (see build/buy section above).
• Develop and issue an interim report on audit findings and recommendations.

At this stage IA will:

• Review any post-implementation evaluations to ensure there is a reasonable assessment of whether the system met organizational objectives and user needs. IA would also look for some documentation of any challenges and how they might be avoided in the future.
• Review change management methodology to ensure there is a quality assurance process for upgrades and modifications. IA would look for a planned approach for migration to production that ensures only authorized modifications are put into production. IA would also look for evidence of testing in a test environment, and that relevant risks and controls have been assessed for any business process changes.
• Examine the problems and issues records to ensure that all items have been properly cleared.
• Prepare a final report on the findings and recommendations during the review.

Return to home page