ARCHIVED - Final Audit Report - Audit of the Management of Information Technology (IT) Service Level Agreements

March 2011

Management Response and Action Plan

Table of Contents

• Executive Summary
• 1. Introduction
  • 1.1 Background
  • 1.2 Audit Objective
  • 1.3 Scope and Approach
  • 1.4 Statement of Assurance
• 2. Findings, Recommendations and Management Responses
  • 2.1 Governance
    • 2.1.1 Steering Committees
    • 2.1.2 Strategic Direction
    • 2.1.3 Supplier-Management Relationship
    • 2.1.4 Signed Agreements
      
  • 2.2 Risk Management
  • 2.3 Internal Controls
    • 2.3.1 Internal Controls
    • 2.3.2 Management Processes
      
  • 2.4 Monitoring Performance
• 3. Conclusion
• Appendix A - Line of Enquiry and Criteria
• Appendix B - List of Key IT Outsourced Services

Executive Summary

The Information Technology Shared Services Initiative is a government-wide coordinated approach to managing information technology (IT) services more efficiently. Public Works and Government Services Canada (PWGSC) was tasked to create a Government of Canada IT Shared Services Organization. Since 2005, Health Canada has taken a leadership role in becoming a Wave I department working in partnership with PWGSC to demonstrate success in promoting IT Shared Services. In 2007, Health Canada signed a partnership agreement with PWGSC to actively participate in the design and development of the "Shared Services Initiative". The initial partnership in 2007 was documented in the Executive Strategic Sourcing Agreement which outlined the shared services principles to be achieved. The second phase of the IT services transfer included the Distributed Computing Environment (DCE) services transfer. By the end of February 2008, Health Canada and PWGSC had agreed upon an overall DCE strategy, a governance structure as well as an email/office application "proof of concept".

During 2008, Health Canada also completed "The Way Forward Initiative" to set the current enterprise model for IT bringing an increased focus on stewardship and further development of a shared services model. The third phase in the IT shared services initiative was Data Centre Services. This resulted in a partnership with PWGSC for data centre operations and network management. Later in 2008, it was agreed to use a Memorandum of Understanding as an umbrella document to better capture the Strategic Partnership between the two departments. Currently, there are 27 Service Level Agreements approved and 5 in negotiations. In the fiscal year 2009-2010, outsourced services represented total expenditures of $36 million. While they are major components of the infrastructure, applications and support that are provided by PWGSC, Health Canada remains responsible for the integrity, security and access control to its systems.

The objective of the audit was to assess the Information Technology service level agreements, where another government department is providing IT services to Health Canada, in relation to how they are governed, risk managed, controlled and monitored. 

The IT governance structure at Health Canada consists of several key IM/IT oversight bodies. The governance structure provides a forum for discussions, service definition, assignment and clarification of roles, responsibilities, authorities and accountabilities.

The service agreements are aligned with the Government of Canada's direction for Shared Services, as well as Health Canada's strategy for Enterprise IT. Stewardship of IM/IT resources is also identified as vital to the business of the Department in ensuring that IT provides value for money and that IT Shared Services meet the expected outcome of effective client service in support of the Department's program delivery. Health Canada continues to evaluate its services delivered through the IT Shared Services Initiative to optimize the services provided and become a better service partner.

A review of Health Canada - Memorandum of Understandings, Service Level Agreements, Recovery Agreements and Project Charters - noted that there are definitions of roles, responsibilities, accountabilities and authorities with services and deliverables specified. Of the 27 agreements for 2010-2011, five are still in negotiation, six months into the fiscal year. Of positive note is the recently negotiated agreement for Managed Storage Services which has resulted in a significant reduction over last year's costs. PWGSC and Health Canada continue to meet to optimize the service offerings associated with the agreements and secondly, to implement changes in the agreements which results in clearer service targets and improved service to clients.

Since the inception of IT Shared Services, the Information Management Service Directorate (IMSD) has been successful in developing and evolving the operational controls needed for managing the growing number of IT Shared Services. For the most part, the IT Shared Services that the Department has are meeting the business requirements; although, some product stability/interoperability issues have arisen with the implementation of the Business Enterprise Enabler (BEE) and the vendor along with PWGSC continue to investigate. Meanwhile, Health Canada continues to benefit from the Web 2.0 functionality.

After five years of actively participating in the Government of Canada's Shared Services Initiative, it is an opportune time for the Department to evaluate the full cost over the lifecycle of each agreement based on lessons learned to date. A forward looking strategy should be developed to identify other IT services that could be provided by PWGSC in addition to identifying those which could be delivered in-house or by another interdepartmental shared service initiative. The strategy should be well documented and quantified with strong business analyses to demonstrate efficiencies to be gained via each delivery mechanism chosen and should be aligned with the Government of Canada's direction on IT Shared Services. 

Lastly, the Department should finalize the accounting (both assets tagged to Health Canada and associated "shared" costs) for the Oracle Beehive initiative based on the signed Memorandum of Understanding.

Management has agreed with the three recommendations and has developed an action plan which will serve to further strengthen the management practices for administering the IT service level agreements.

1. Introduction

1.1 Background

A business impact analysis is a fundamental first step in the business continuity process. It is a key methodology used by organizations to determine critical services and priorities. The business impact analysis evaluates the potential impacts of disruptions on the organization to help management identify critical services and prioritize those services for recovery based on the maximum allowable downtime. According to the Government of Canada BCP standard, senior management approval of business impact analyses must be completed before continuity plans are developed; it signifies that all critical services have been identified and that the Department is ready to support the continuity of those services.

During 2008, Health Canada also completed "The Way Forward Initiative" to set the current enterprise model for IT bringing an increased focus on stewardship and further development of a shared services model. The third phase in the IT shared initiative was Data Centre Services. This resulted in a partnership with PWGSC for data centre operations and network management. Later in 2008, it was agreed to use a Memorandum of Understanding as an umbrella document to better capture the Strategic Partnership between the two departments. While major components of the IT infrastructure, applications and IT support are provided by PWGSC, Health Canada remains responsible for the integrity, security and access control to its systems.

The first services outsourced included desktop support, data centre services and distributed computing as well as network management. In 2008, Health Canada completed "The Way Forward Initiative" to set the current enterprise model for IT bringing an increased focus on stewardship and further development of a shared services model. Currently, there are 27 IT service level agreements approved or in negotiations. In the fiscal year 2009-2010, partnered services represented total expenditures of $36 million in Information Management Services Directorate. (See Appendix B)

As the number of common IT Shared Services grows, it is anticipated that government will achieve more efficient and cost-effective operations through economies of scale and standardization; clearer accountabilities; timely decision-making based on consistent management information; and common, more consistent levels of service across the government.

1.2 Audit Objective

The objective of the audit was to assess the Information Technology service level agreements, where another government department is providing IT services to Health Canada. The service level agreements were assessed in relation to how they are governed, risk managed, controlled and monitored.

1.3 Scope and Approach

The audit scope included the 2009-10 fiscal year, as well as current year shared service activities. The audit examined the execution of service level agreements to determine the extent that they are: achieving business requirements in accordance with contractual terms, conditions and deliverables; are compliant with Health Canada standards for service delivery and security; and are measured for performance and the resulting metrics reported to executive management on a regular basis. The audit's examination phase concentrated its detailed testing efforts on a directed sample of six outsourced agreements. Together, these represent 88 percent of the IT Shared Services expenditures for 2009-10. The criteria forming the basis for the directed sample are:

• Materiality - highest value agreements
• Longevity - longest running agreements
• IT Security- agreement having an impact on the security posture of the Department

1.4 Statement of Assurance

In the professional judgment of the Chief Audit Executive, sufficient and appropriate procedures were performed and evidence gathered to support the accuracy of the audit conclusion. The audit findings and conclusion are based on a comparison of the conditions that existed as of the date of the audit, against established criteria that were agreed upon with management. Further, the evidence was gathered in accordance with the Internal Auditing Standards for the Government of Canada and the International Standards for the Professional Practice of Internal Auditing.

2. Findings, Recommendations and Management Responses

2.1 Governance

2.1.1 Steering Committees

Audit Criterion: Steering committees are in place to provide authority, responsibility and accountability for decisions related to IT Shared Services.

Governance should be the foundation of clear accountability, enhanced transparency and sound financial management. Effective management practices, mechanisms and tools facilitate corporate oversight responsibilities and provide a balanced approach to risk management and internal control.

The IT governance structure at Health Canada consists of several key IM/IT oversight bodies: the Executive Management Committee; its Sub-committee on Operations; and the Information Management Services Directorate's Executive Committee. As well, there is the IT Shared Services Management Board and the senior-level interdepartmental IT Shared Services Council. During the time of the audit, the Information Management Accountability Board provided oversight for the use of IT Shared Services with the exception of one of Health Canada's newer service agreements, the Business Enterprise Enabler, which is governed as described below.

A business impact analysis is a fundamental first step in the business continuity process. It is a key methodology used by organizations to determine critical services and priorities. The business impact analysis evaluates the potential impacts of disruptions on the organization to help management identify critical services and prioritize those services for recovery based on the maximum allowable downtime. According to the Government of Canada BCP standard, senior management approval of business impact analyses must be completed before continuity plans are developed; it signifies that all critical services have been identified and that the Department is ready to support the continuity of those services.

2.1.2 Strategic Direction

Audit Criterion: Service level agreements are aligned with the IT strategies and directions of Health Canada.

The IT Shared Service agreements are aligned to the Government of Canada's direction and the Policy on Information Technology, as well as Health Canada's own strategy for Enterprise IT. Stewardship of IM/IT resources is also identified as vital to the business of the Department to ensure that IT continues to provide value for money and that IT Shared Services meet the expected outcome of effective, consistent client service in support of Health's program delivery. To effectively achieve its enterprise plans, the Information Management Services Directorate (IMSD) must measure and manage actual performance of business processes in a highly coordinated manner. To support senior management in their responsibility for the effectiveness of systems of internal control, an approved rationale for meeting business requirements and benefit analysis normally sets the basis for ongoing measurement of performance. 

In the Audit of IM/IT Governance, tabled at the Departmental Audit Committee in December 2009, IMSD noted in its management response that cost/benefit studies for Desktop Support Services, the National Service Desk and storage had been performed prior to entering into Service Level Agreements. Additionally, to ensure fair pricing and best practice in services, IMSD reports that they used the Gartner industry benchmarks to negotiate these agreements. Gartner benchmarks are internationally recognized IT standards that compare the costs and resources of performing a given scope of IT activities with a cohort of similar organizations.

In addition to these initial studies, IMSD stated in their management response that by March 2010, they would undertake year end reviews of all Service Level Agreements (SLAs) to ensure continued benefits and cost savings. However this was only completed for National Service Desk and Storage. IMSD reported that a mid-year review was undertaken and as a result, the agreement for the National Service Desk was continued with planned cost savings; the Storage agreement was reduced considerably from $6.7M to $4M.

A recent study of Help Desk costs conducted by IMSD determined that there were no discernable cost savings in comparing the delivery of this service between the existing IT Shared Services arrangement, using a private firm, or simply in-sourcing the service altogether. As a result, IMSD made the decision to continue its strategic partnership with PWGSC. However, for several other service level agreements, this type of comprehensive analysis has yet to be completed.

Health Canada should continue to evaluate and review the options of delivering services through its IT Shared Services partner, compared to services delivered in-house. For example, when comparing options, Health Canada should take into account other benefits of outsourcing aside from the direct costs, such as: reduced overhead, minimal capital expenditures, reduced management burden from day to day process problems, access to specialised skills, less staffing and training to be completed.

Recommendation 1

It is recommended that the Assistant Deputy Minister, Corporate Services Branch develop an overall strategic plan for IT service level agreements which should be supported by rationales for each of the agreements. The plan should be tabled at Executive Committee for consideration and approval.

Management Response

2.1.3 Supplier-Management Relationship

Audit Criterion: The supplier-management relationship should be formalized including a process to document incidents and a process to resolve disputes.

The supplier-management relationship is formalized in the negotiated and signed agreement; in the case of the DCE/BEE, there is a business arrangement created via a Memorandum of Understanding for the deployment and support of the software. Within IMSD, there are meetings that are held regularly with the supplier representatives as well as with senior management. Meetings at the operational levels occur weekly, or on an ad hoc basis when the urgency for incident and problem resolution is required. In addition to the regular meetings, there is a direct contact line between the PWGSC Client Relations Manager and the Health Canada Manager of Shared Services. On a yearly basis Health Canada and PWGSC executives review the Department’s portfolio and a client survey is completed for PWGSC as well as a formal comments section requesting information with regards to the client experience.

One of the key forums for supplier relationship management is the weekly Business Management and Shared Services meeting, where status reports are presented on IT Shared Services. During these meetings service level shortfalls are identified, but procedures on how to resolve shortfalls vary. While there are some minutes of the meetings and plans of action, there is no formalized central information system that keeps track on a continuous basis of all the service level agreement incidents. These records should provide sufficient information to permit management to address the number of issues by service level agreement, the timeliness of issue resolution and the age of issues remaining unresolved. A yearly analysis may then provide better support to the actual service levels received and help management with the negotiation for renewal or support the business case for an alternate solution (See recommendation 2).

2.1.4 Signed Agreements

Audit Criterion: Service level agreements are defined, agreed and signed-off upon based on requirements and IT capabilities.

A service level agreement typically defines the service to be rendered, the service level to be delivered, scope of work, metrics, assumptions, schedule, costs, billing arrangements and responsibilities for contract and vendor management. The terms and conditions are agreed-upon using a process of negotiation however; Health Canada notes it has been difficult to have its own clauses, service levels or performance metrics.

A review of the Departments’ Memorandum of Understandings, Service Level Agreements, Recovery Agreements and Project Charters noted that there are definitions of roles, responsibilities, accountabilities and authorities with services and deliverables specified. The review did find that five out of twenty-seven Service Level Agreements for 2010-2011 have yet to be approved. These service level agreements are currently being negotiated between parties. At issue; the absence of detailed service descriptions, defined and accepted service levels, monthly performance reporting. However progress continues to be made and both parties continue to negotiate their position. As part of these negotiations, Health Canada requests the required information to ensure that the services satisfy the business requirements, service levels meet client expectations and that the service performs at the agreed upon level. With this information, Health Canada can make an informed business decision as to the value provided by the services as well as select additional shared services that will leverage the Health Canada program delivery. Since DCE/BEE is an evolving service, the current Service Level Agreement scoping, accountability and pricing are being reviewed with PWGSC.

It is reported that it takes on average over seven months to negotiate a service level agreement and obtain the information required by Health Canada, explaining why service level agreements are often expired while services continue to be delivered. Both IT Shared Services and Health Canada continue to meet to reduce the agreements’ costs, to implement changes in the agreements resulting in clearer service targets, and an improved service to clients in many cases. Of the 27 agreements for 2010-2011, five are still in negotiation, six months into the fiscal year.

The results for the six agreements sampled and analyzed as a part of the examination work for the audit shows that half of these are in negotiation for 2010-2011 (See Recommendation 1).

Health Canada continues to encourage its partners to provide the requested information to make informed business decisions about the services and deliver the highest level of performance while it continues to receive services without pre-negotiated agreements which could result in situations where more efficient alternatives could not have been implemented.

2.2 Risk Management

Audit Criterion: Risk management controls continually identify and mitigate risks relating to third party ability to provide effective service delivery in a secure and efficient manner.

Risk management for IT Shared Services is largely an operational risk. The initiative is complex and has inherent risks that need to be well-managed; such as dependency on PWGSC as the service provider to support mission critical services. To mitigate the risks, it is the general practice that specific shared IT services are subject to an independent assessment and that the report is made available to Health Canada.  As was the case with BEE, two Independent Validation and Verification Audits were conducted by KPMG and all parties were provided with the results of the audit.

Overall, while security is stated in the service level agreements, indications are that IMSD management is not receiving sufficient evidence to indicate that the security standards are in conformance. Such evidence could be the result of any applicable Threat and Risk Assessment, a Business Impact Analysis, a Privacy Impact Assessment, a Vulnerability Assessment, security tests, product evaluations, self-assessments, audits and/or security reviews. Any security decisions made by the other government departments can impact Health Canada, although they too are bound to meet or exceed the same minimum security standards. However, the Department does remain accountable for the security of the programs and services under its authority.

An example where controls could be strengthened and risks better managed is with the DCE/BEE project. According to Treasury Board policies, Health Canada should have systems or services certified and accredited before approving them for operation. The purpose of certification is to verify that the security requirements, controls and safeguards work as intended, while accreditation signifies that management has authorized the system to operate and has accepted the residual risk of operating the system based on the certification evidence. 

One of the deliverables included in the Recovery Agreement for DCE/BEE was completion of a Certification and Accreditation Report. The Report, produced by an accepted independent organization, noted risks and issues that should have been presented to Health Canada and mitigated prior to DCE/BEE being migrated into production. Lastly, the Certification and Accreditation Report was dated the 1^st of June 2010 yet DCE/BEE was introduced in March 2010 and the audit team was unable to find an "interim authority to operate" from March 2010 to June 2010, which would have resulted in management being given the opportunity to better understand and accept specific risks prior to the system being introduced (See recommendation 2).

2.3 Internal Controls

2.3.1 Internal Controls

Audit Criterion: Internal controls to manage service level agreements are in place.

Since the inception of IT Shared Services, the Information Management Service Directorate (IMSD) has been successful in developing and evolving the operational controls needed for managing the ever growing number of IT Shared Services. For 2010-2011, there are 27 service level agreements approved or in negotiation. Management of this activity is extremely important as many of the agreements in place for IT services cover "mission critical systems" - systems Health Canada relies on daily to do its business. Moreover, the cost to the Department to have these services provided is approximately $32.2 million annually. 

The responsibility for managing IT Shared Services is shared within IMSD. Reporting to the Chief Information Officer, the Computing and Network Services Centre is responsible for the overall management of the service level agreements while the Business Management and Shared Services unit acts as the "single point of contact" and manages the shared services agreements. One of the Department's service level agreements, the Distributed Computing Environment (DCE) / Business Enterprise Enabler (BEE), is managed by the Office Automation Services Division since this service activity consists of a multi-year software implementation project and is subject to a different governance and management control framework.

There are also subject matter experts' and business owners' responsibilities carried out in other areas of IMSD. For example, the Business Management Services Division acts as an extension to the office of the Chief Information Officer by providing horizontal coordination of administrative services across all IMSD divisions. 

Interviews with the IMSD management team noted that they generally follow the Information Technology Infrastructure Library's best practices for service level management and have developed key performance indicators. The Information Technology Infrastructure Library gives detailed descriptions of a number of important IT practices and provides comprehensive checklists, tasks and procedures that any IT organization can tailor to its needs.

IMSD identifies, solicits, evaluates, monitors and manages the negotiations for the service level agreements. Subject matter experts will review each agreement before submitting any new or renewed agreement for final approval by the Chief Information Officer. To settle the Department's financial obligations, reviews of invoices are conducted before being submitted for approval as per section 34 of the Financial Administration Act (FAA) by the Executive Director of IMSD. Settlements are made using the Interdepartmental Settlement process and a full financial reconciliation of payments is carried out quarterly and annually by the Business Management and Shared Services division.

These key business processes require the adherence to relevant legislation and regulatory requirements, as well as monitoring for compliance of the outsourced contracts and the performance of each contract in meeting the stated business requirements. There are measurements of on-going activities, reviews by management, approval of the service level agreements and some reporting by PWGSC and Health Canada.

While many elements of good practices were observed in the service level management control framework, some are not documented as there are no formal policies or guidelines to define and support the IT shared service management activities. Considering that these services account for one third of IMSD'S budget, the Directorate would benefit from formally documenting the management control framework aligned with the Information Technology Infrastructure Library (ITIL). ITIL was developed by the Office of Government Commerce in the United Kingdom and is the most widely accepted approach to IT service management as it provides a cohesive set of best practice guidance drawn from the public and private sectors across the world. It describes the organization of IT service management resources to deliver business value and documents, for example, expectations around: processes; functions; security management; risk management; change management and performance monitoring of service agreements.

Formalizing the internal control framework for the IT Shared Services in line with ITIL would provide long term sustainability to the good practices that have been developed by IMSD over the years as well as reducing the risks identified in the previous section. The internal control framework should also include reasonable assurance that the service provider meets the control and security requirements of Health Canada.

Recommendation 2

It is recommended that the Assistant Deputy Minister, Corporate Services Branch formalize an internal control framework for managing the service level agreements.

Management Response

2.3.2 Management Processes

Audit Criterion: Health Canada regularly reviews service level agreements to ensure that they are effective in meeting current business requirements and changes in requirements should be incorporated.

Service level management processes in IMSD have been subject to continuous improvements over the years. While a formal integrated system of measurement is not maintained, service level agreements are being re-evaluated to ensure alignment of IT and business objectives. Currently in 2010-2011, the main focus is on IT cost reductions and or value, resulting in business analysis being done and measures taken to consider the implementation of additional shared services (See recommendation 1).

Audit transaction testing results indicated that transaction processes for payment were generally compliant with Section 34 certification requirements. However, an important area where the Department could strengthen processes is to maintain sufficient and appropriate documentation on file as evidence that services have in fact been received in accordance with the service level agreements. An audit trail containing indicators of services received would help substantiate claims that these documents were reviewed prior to performing the section 34 certification. Health Canada may be paying in accordance with the service level agreements contract, but may not be receiving all the services intended by the service level agreements (See Recommendation 2).

For the most part, the IT Shared Services that the Department employs are meeting the business requirements with the exception of certain functionality in the DCE/BEE offering. In August 2010, due to product instability and interoperability issues between the Microsoft Outlook Calendar and the Lotus Notes Calendar, senior management decided to suspend a portion of the initiative (the use of BEE for e-mail/calendaring) until appropriate modifications to the system could be made to co-exist with Lotus Notes. BEE continues to be the Department's Web 2.0 tool of choice for users throughout. 

Health Canada made an investment in rebuilding its Distributed Computing Environment of which BEE remained the cornerstone from the outset. The partnership, as defined in the Memorandum of Understanding and signed in 2008, stated that Health Canada and PWGSC would "share costs" associated with the product deployment and operations. Health Canada purchased the hardware for the project and the IT Shared Services partner invested into the architectural and engineering cycles to build the BEE infrastructure for the Department. These hardware and software assets are deployed on PWGSC's premises and are managed by them. Currently, the Department is in the process of determining the inventory and its value for assets tagged to Health Canada in preparation for a formal asset transfer to PWGSC.

Recommendation 3

It is recommended that the Assistant Deputy Minister, Corporate Services Branch prepare a "bill of materials" for the DCE/BEE project and develop a recovery agreement in accordance with the Memorandum of Understanding.

Management Response

2.4 Monitoring Performance

Audit Criterion: An established process monitors service delivery to ensure that the supplier continues to adhere to the agreements and that performance is competitive with alternative suppliers and market conditions.

The Helpdesk service level agreement offers the best available reporting on its activities. Management goes as far as challenging the key figures provided by the service provider on performance by redefining first level resolution success rate. As for the other service level agreements, some monitoring and performance measurements do exist, however, they are often not standardized across all business lines and therefore can be difficult to interpret and measure.

The reports on the services provided do not necessarily support Health Canada's business objectives since they are generated to show compliance with the Service Level Agreement metrics and not necessarily to demonstrate business value. As such, Health Canada has a separate team for service monitoring and reporting. The business unit is often creating its own suite of reports to monitor the IT Shared Services in an attempt to provide better service (See Recommendation 2).

3. Conclusion

Health Canada continues to play a strong leadership role in the Government of Canada's IT Shared Services Initiative. While its five year participation with this arrangement has been consistent, the Department needs to continually seek value from PWGSC, evaluate opportunities to improve the service, and reduce costs. Furthermore, constant review of the service offerings delivered from the provider needs to continue in order to compare the full costs of in-sourcing versus that of shared services. This type of review will also lend itself to making modifications on how the Department delivers its services to clients using shared services.

The strategy should be well documented and quantified with business analysis including full costs to demonstrate efficiencies to be gained via each delivery mechanism chosen. The strategy should be transparent so that senior management is able to make the necessary decisions around delivery of IT services. 

The partnership between Health Canada and PWGSC is considered a success as evident in the results achieved to date and the collaborative effort between the two organizations in the scoping and planning of subsequent activities. As stated in the Memorandum of Understanding of October 16, 2008, the success to date coupled with a continued commitment by both parties sustains the belief that the joint partnership will continue to be a success in the future.

Appendix A - Line of Enquiry and Criteria

Appendix B - List of Key IT Outsourced Services

¹ Information not audited - Source: Information Systems Business Management, IMSD