Final Report - Audit of the Project Management Framework - October 2014

Table of Contents

Executive summary

The focus of the audit was on Health Canada's (the Department) project management framework. The management of projects is key to providing value for money and demonstrating sound stewardship in program delivery. In June 2007, the Treasury Board Ministers approved the Policy on the Management of Projects. The revised policy represents a significant change in how government manages projects.

The objective of the audit was to assess the effectiveness of the project management framework and controls to support the delivery of projects, including compliance with the Policy on the Management of Projects. The audit was conducted in accordance with the Internal Auditing Standards for the Government of Canada and the International Standards for the Professional Practice of Internal Audit. Sufficient and appropriate procedures were performed and evidence gathered to support the accuracy of the audit conclusion.

While the current processes were found to be well-established and in compliance with the Treasury Board policy, the audit identified some areas where the project management process could be enhanced. Specifically, the project management framework should be updated. As well, the process would be strengthened by adding an oversight reporting requirement related to project closure.

The IPMPU reviews the project documentation prior to gating approval. However, it was noted that project management plans did not always include all required information and that project costing was a common area where improvements can be made. Further, some functions and branches did not always use the departmental templates. Harmonized templates would reduce the administrative burden for project managers and better support gating decisions. Lastly, the IPMPU should use any risks identified in the OPMCA to further strengthen the Department project management practices.

Management agrees with the four recommendations made in the audit report and has prepared a management action plan that will serve to further strengthen the project management process at Health Canada.

A - Introduction

1. Background

In June 2007, the Treasury Board Ministers approved the Policy on the Management of Projects. This policy replaced the Project Management Policy, the Policy on the Management of Major Crown Projects and the Project Approval Policy for departments and agencies, as defined in Section 2 of the Financial Administration Act. This new policy represents a significant change in how government manages projects. As a result, the Treasury Board Secretariat (TBS) adopted a phased implementation approach, starting in 2007, which began with a group of departments that agreed to participate in a pilot. Groups of remaining departments were brought onboard, so that by April 1, 2012, all departments and agencies were to have the systems and processes in place to meet the policy's requirements.

Project management is defined in the policy as the systematic planning, organizing and control of allocated resources to accomplish identified project objectives and outcomes. Project management is normally reserved for focused, non-repetitive, time-limited activities with some degree of risk, and for activities beyond the usual scope of program (operational) activities.

The policy notes that the management of projects is key to providing value for money and demonstrating sound stewardship in program delivery. Moreover, a comprehensive approach to managing projects, which is integrated across the Department and is appropriate for the level of project risk and complexity, will enhance the likelihood of realizing project outcomes. The approach is designed so that accountability for outcomes is clear, appropriate controls are in place to minimize risk and limit project duplication and overlap, key project stakeholders are consulted and outputs and outcomes are monitored and reported.

The policy is framed by the principles set out in the Policy Framework for the Management of Assets and Acquired Services and is supported by the Standard for Organizational Project Management Capacity and the Standard for Project Complexity and Risk. These policy instruments emphasize the people, systems and processes required to effectively manage projects. The policy allows departments the flexibility to tailor project management processes and oversight to business operations. Additionally, it aligns Treasury Board (TB) oversight with an assessed level of project risk and a holistic understanding of the capacity of an organization to manage its planned portfolio of projects, rather than focusing solely on project costs.

The TB Policy on the Management of Projects delegates project authority to departments, commensurate with the project management capacity of the department relative to the level of project risks. This requires that the complexity and risk of each project be assessed to determine if it is within the organization's capacity to manage projects. The Organizational Project Management Capacity Assessment (OPMCA) is a 92-question self-assessment used to determine an organization's capacity to manage projects. Individual project complexity and risk is assessed using the government's Project Complexity and Risk Assessment (PCRA) tool. This 64-question self-assessment is completed by the project manager. Both OMPCA and PCRA scores are submitted and reviewed by TBS. The project's score is then compared with the organization's capacity score to determine authority. Project PCRAs with scores that exceed the organization's OPMCA capacity score require TB approval and expenditure authority in order to proceed, while projects that are equal to or less than the organizational capacity score are likely within the approval limit and can be subject to either minimal or no TB oversight.

Health Canada project management

Each year, Health Canada (the Department) manages a range of projects to support the delivery of health programs and services to Canadians. The Department has an approved Investment Planning and Reporting Framework and related Project Management Guide (November 2012) to support this project management work.

Figure 1: Risk-based Project Approval and Oversight

figure

Text description

Figure 1 illustrates Health Canada's risk-based project and approval framework in relation to organizational project management capacity and project complexity and risk.

The figure shows that Health Canada has delegated authority for projects that are at level 1 or 2 for both organizational project management capacity and project complexity and risk.

Projects that obtain a score of more than 2 require Treasury Board Secretariat oversight, although TBS oversight can be extended to any project.

In March 2012, the Department's organizational project management capacity was approved at a capacity class of 2 (Tactical) on a scale from 0 to 4 (see Appendix D). At a capacity class of 2, departments are considered to have a tactical project management capacity that includes successful delivery of projects, adjusting operations and meeting planned objectives. According to TB policy, projects with costs exceeding $1M require a PCRA. For the Department, projects that obtain a complexity and risk level above 2 require TBS oversight; however, TBS can choose to review any project information for approval, even if it is within departmental authority.

As of February 28, 2014, the Department was managing 61 active level-2 projects, with a total planned project cost of $301M (see Appendix C). Of the 61 active projects, 14 are less than $1M. The Health Canada Investment Planning framework includes these investments, as all projects over $250, 000 go through the gating process.

2. Audit objective

The objective of the audit was to assess the effectiveness of the project management framework and controls to support the delivery of projects, including compliance with the Policy on the Management of Projects.

3. Audit scope

The scope of the audit included the Department's project management framework and its controls for managing projects at both the portfolio level and project level. Only projects initiated after the effective date of the Policy on the Management of Projects (April 1, 2012) were included. The audit scope excluded projects with an estimated value of less than $250,000, as these projects are not subject to the departmental project management framework.

4. Audit approach

The audit examined the design of the Department's project management framework for compliance with the Policy on the Management of Projects and the Policy on Investment Planning - Assets and Acquired Services. It also examined the governance structures, risk management practices and controls aimed at ensuring that projects are implemented successfully and in accordance with plans, including set timelines and budget. A sample of 21 projects was examined to assess compliance with the organization's project management framework. A second smaller sample of seven projects was drawn from the larger sample for a more detailed analysis, to assess the overall quality of the information contained in the project's documents. Finally, the audit also examined departmental support, methods and tools available to project managers, and the effectiveness of oversight and accountability mechanisms for monitoring progress and reporting results achieved.

The audit criteria, outlined in Appendix A, were derived from the following sources:

  • the TBS policy suite for the management of assets and acquired services;
  • the Office of the Comptroller General, Internal Audit Sector's Audit Criteria Related to the Management Accountability Framework: A Tool for Internal Auditors (2011); and,
  • Project Management Institute - A Guide to the Project Management Body of Knowledge (PMBoK® Guide, fifth edition).

Methodologies included a review of documentation, policies, standards, guidelines and frameworks; interviews with key personnel involved in project management; and a detailed analysis of a sample of projects for compliance with TB policy and the departmental project management framework.

The audit was conducted in accordance with the Government of Canada's Policy on Internal Audit, examined sufficient relevant evidence and obtained sufficient information and explanations to provide a reasonable level of assurance in support of the audit conclusion.

5. Statement of conformance

In the professional judgment of the Chief Audit Executive, sufficient and appropriate procedures were performed and evidence gathered to support the accuracy of the audit conclusion. The audit findings and conclusion are based on a comparison of the conditions that existed as of the date of the audit, against established criteria that were agreed upon with management. Further, the evidence was gathered in accordance with the Internal Auditing Standards for the Government of Canada and the International Standards for the Professional Practice of Internal Auditing. The audit conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.

B - Findings, recommendations and management responses

1. Governance

1.1 Project management framework

Audit criterion: A department-wide project management framework is documented and maintained to support accountability for projects.

The Treasury Board (TB) Policy on the Management of Projects requires deputy heads to have a department-wide governance and oversight mechanism in place that is documented and maintained. The mechanism is to be used to manage the initiation, planning, execution, control and closing of projects. In addition, the mechanism is to be designed so that opportunities can be considered for integrating projects across the department and the Government of Canada.

In November 2012, Health Canada (the Department) developed a Project Management Guide, which describes the Department's project management expectations. The guide also outlines the information and documents needed to meet the gating requirements for projects over $1M (five-gate process) and projects between $250K and $1M (three-gate lite process). The gating process is the basis for controlling and managing projects within the Department and is based on guidance developed by both Treasury Board Secretariat (TBS) and the Project Management Institute (see Appendix E).

The Department's framework is well-established, but some opportunities for improvement were noted. Specifically, the framework should be updated to reflect the organizational changes made since its development and to give the oversight bodies an increased role in the close-out of projects.

As the Department's Project Management Guide was being finalized, Health Canada and the Public Health Agency of Canada (the Agency) entered into a Shared Services Partnership (SSP) Agreement for the delivery of corporate administrative services. Some of the Department's corporate projects are undertaken with the Agency as part of the SSP.  Directorates within the Corporate Services Branch (CSB) that have project delivery responsibilities for these shared or portfolio projects have adapted the processes to accommodate the shared status. A revised common approach, drawing on the best practices being utilized, would be beneficial for all stakeholders and would provide clarity on how common or shared projects are managed.

There have also been changes to the oversight committee structure since the framework's development in 2012. New executive-level committees have been formed and old committees have been renamed or merged with other committees. The framework should be updated to reflect these changes.

Finally, there is room for improvement in the oversight structure defined in the current framework. Currently one of the key oversight committees, the Executive Subcommittee - Finance, Investment Planning and Transformation (EC-FIPT), is responsible for reviewing and approving projects from idea generation (Gate 1) to project initiation (Gate 2) and finally to project planning (Gate 3). Oversight during project execution and project close-out then falls to the sponsoring Assistant Deputy Minister. EC-FIPT monitors the projects using a dashboard and management variance reports (MVR), and requests status updates if required. Given this practice, the oversight committees only receive reports on the final project results and the lessons learned at the discretion of EC-FIPT or the sponsoring branches. Systematic sharing of this information may better enable senior management to refine project management practices.

In reviewing the project management framework, the audit notes that the process is well-defined and implemented, but that it should be updated to reflect organizational changes and to include a more active role for departmental oversight bodies in the close-out of projects.

Recommendation 1

It is recommended the Chief Financial Officer update the framework to reflect the changes in governance and to include a formal role for the oversight committees in the management of the close-out of projects.

Management response

Management agrees with the recommendation.

CFOB will update the HC Project Management and Governance Framework to reflect changes in governance.

CFOB will develop a formal role for the oversight committees in the close-out of projects, in consultation with stakeholders, for endorsement by the governance committees. This will be incorporated into the framework.

1.2 Oversight committees

Audit criterion: Oversight bodies are established to govern the departmental portfolio of projects.

The Department has established a governance process to oversee project management, which is governed by three committees: Executive Committee (EC - chaired by the Deputy Head), the Executive Sub-committee on Finance, Investment Planning and Transformation (EC-FIPT) and the Director General Investment Planning (DG-IP) Committee.

The Executive Committee is the organization's most senior decision-making oversight body. It approves the investment plan and allocates and reallocates resources. EC is supported by the EC-FIPT and ratifies the sub-committee's gating decisions on a quarterly basis. The EC-FIPT is responsible for approving and monitoring projects through the project lifecycle. Its project management mandate is to establish and oversee processes for gate reviews and oversight of investment plan projects; make and review recommendations, including those of third parties; monitor and escalate project issues and risks as required; establish and oversee processes to adhere to related TBS guidelines and policies; and establish and oversee processes for optimal project management and for collaboration within the Department and with relevant stakeholders.

EC-FIPT is chaired by the Associate Deputy Minister, and its members include the assistant deputy ministers from the Chief Financial Officer Branch, Communications and Public Affairs Branch, Corporate Services Branch, First Nations and Inuit Health Branch, Health Products and Food Branch and the Pest Management Regulatory Agency. The Director General of the Resource Management Directorate and the Chief Information Officer are also ex-officio functional lead members of the committee.

The Director General Investment Planning Committee makes recommendations to the EC-FIPT concerning progress through the gating process. It is co-chaired by the Chief Information Officer and the Director General of the Resource Management Directorate. Committee members include director general-level representatives from each branch, as well as functional leads as required. DG-IP's mandate is to provide direction to investment and project owners, to ensure alignment with policies adopted by TB. In addition, DG-IP is responsible for ensuring that the investment plan process is coordinated and reflects the proper linkages between priorities and investments; increasing engagement on investment and project management issues; recommending strategies to identify and develop horizontal business requirements and plans using an enterprise-wide approach; sharing information and lessons learned to identify opportunities for collaboration and improvement; increasing engagement on project management issues; and providing second-level operational oversight to investment projects, including project review and recommendation to EC-FIPT at Gates 1, 2 and 3 for all IP projects where cost is greater than $250,000.

The audit found that oversight bodies approve project baselines for scope, cost and time at the idea generation, project initiation and project planning gates. The committees actively monitor project performance through regular dashboard reports. The dashboard highlights any issues with a project's scope, schedule, costs, risks and overall status. This reporting mechanism gives committees an overview of how the organization is performing in terms of its project portfolio and provides the ability to identify issues in a timely manner. As of March 2014, the dashboard is presented every second month.

Since the introduction of the two new policies, the Department's IP governance committees have had regular meetings to review all aspects of investment planning, from the update of the annual investment plan to the review of financial strategies and affordability, and in particular, to undertake those activities related to effectively managing projects and investments, as outlined in the project management framework. Coincident with the broadening of the mandate of the EC from IP to FIPT, an annual work plan for IP was developed that captured and formalized those practices. The work plan includes monthly reviews of the project dashboards, investment plan recommendations and a bi-annual overall project performance review. This strategic focus brings additional assurance to the oversight process. The overall work plan should continue to be reviewed and updated on an ongoing basis to maintain the strategic focus as it relates to fulfilling its mandate and overseeing key projects that impact the Department.

The Investment Planning and Management of Projects Unit is the secretariat for both the above-mentioned committees.

Based on a review of the terms of reference, agendas and minutes, the audit found that there is an effective governance structure in place to support the oversight of the management of projects at Health Canada.

1.3 Roles and responsibilities

Audit criterion: Project management roles and responsibilities are clearly defined.

Project management in the Department is a shared responsibility between the Chief Financial Officer Branch's Investment Planning and Management of Projects Unit (IPMPU) and the individual project's sponsoring branch.

As noted, the IPMPU acts as the secretariat for project management reporting within the Department. It is responsible for establishing and maintaining the departmental project management framework and its supporting tools and templates. The staff coordinate the gate approval process, ensuring that the committees receive the information they require to make sound project management and investment planning decisions. They provide advice and support to project managers, review project gate documentation and provide a challenge function prior to gate reviews by the oversight committees. Finally, they act as the Department's point of contact with TBS by coordinating responses to information requests and conducting an Organizational Project Management Capacity Assessment (OPMCA) on a regular basis.

Assistant deputy ministers are responsible for ensuring the effective and efficient management of their respective projects and promoting sound project management practices within their branch and across the Department.

Project leads (business owners) manage the overall project, including change management, transitioning to new business processes, communications and preparing project status updates and reviews, with input from functional areas. The project lead is responsible for achieving the defined project objectives within the costs, time and resources allocated, as baselined in the project charter.

Project managers are responsible for achieving the defined project objectives within the approved cost, time and resources. They are responsible for the day-to-day management of the project, including the project team, schedule, budget, stakeholders, risks and issues. They manage the project throughout all stages, including planning, execution and close-out. Depending on the complexity of the project, different major contributors may each have a project manager.

Specific project manager responsibilities are defined in the project charter, since they may vary from project to project. The audit notes that all projects sampled had a project charter and a project management plan that included a section outlining the roles and responsibilities of the project manager.

The departmental project management framework clearly defines the roles and responsibilities for all internal stakeholders involved in project management. This includes, but is not limited to, the IPMPU, assistant deputy ministers, the project lead (business owner) and the project manager. Specific project manager roles and responsibilities are further defined in each project's charter.

2. Risk management

2.1 Departmental portfolio risk management

Audit criterion: Portfolio project management risks are identified, assessed, mitigated and monitored.

The management risk at the portfolio level relates to those organizational risks and/or opportunities that can impact the Department's ability to achieve its project objectives from a portfolio perspective. Risk management at both the portfolio and the project level involves identifying risks, developing mitigation strategies and regular monitoring.

Portfolio risk management

One of the most effective tools available for assessing risk at the portfolio level is the OPMCA. The assessment tool provides the Department with a high-level assessment of its capacity to manage projects. The assessment is based on the risks associated with the management of projects and when completed, it identifies areas of strength and weakness in organization's project management practices. In 2011, Health Canada completed its first assessment and was approved at a capacity class of 2 (Tactical) on a scale of 0 to 4 (see Appendix D for details). This capacity class is acceptable to senior management, given the types of projects the organization undertakes, and it accurately represents a project management capacity that will meet the Department's requirements.

The other benefit of the OPMCA is that it identifies areas potentially requiring improvement and possible risks to managing the portfolio of projects. The roles and responsibilities of the IPMPU, outlined in the Project Management Guide, include performing regular reviews of the capacity assessment in order to identify potential areas of strength and weakness within the organization and reporting on these findings and the recommended action plans for improvement to senior management. The audit found that the IPMPU was in the process of updating the OPMCA and would benefit from using the results of the assessment to support portfolio risk management activities.

The project management framework was designed to manage portfolio risks through the gating process. This process minimizes the financial and resource risks to the organization, since only well-planned strategic businesses cases are approved.

Regular dashboard reporting by the individual projects is another way that the IPMPU is able to mitigate project portfolio risk. The monthly dashboard not only highlights issues with individual projects, but it can also identify trends that could impact the Department's portfolio of projects.

Project risk management

As described in the background section of the report, Project Complexity and Risk Assessments (PCRA) are required for all projects over $1M. The PCRA tool identifies project risks and is used to determine whether the project is within the organization's assessed project management capacity. In the sample selected, all projects had a PCRA score of 2 or lower and were within the Department's assessed capacity.

The IPMPU has taken steps to manage risks at the project level by requiring each project to prepare a risk management plan as part of its project management plan, which is a requirement at Gate 3. Project managers must also maintain a risk log throughout the execution phase of the project and review and update the risk register on an ongoing basis.

The IPMPU has implemented a number of measures and controls to mitigate risks to the Department's portfolio of projects. These include the implementation of the gating process for the approval of projects, along with the creation of the monthly monitoring dashboard and the completion of the capacity assessment. The IPMPU would benefit from using the risk results from the OPMCA to further strengthen the departmental project management capacity.

Recommendation 2

It is recommended that the Chief Financial Officer use the results from the Organizational Project Management Capacity Assessment to further strengthen the project management process.

Management response

Management agrees with the recommendation.

The OPMCA is one of a suite of tools used in the Department to identify potential improvements in project management.

CFOB will include the results of the OPMCA as a source of information to make further improvements in project management practices.

3. Internal controls

3.1 Guidance and supporting tools

Audit criterion: Guidance and supporting tools are developed and up-to-date to support effective project management.

Guidance and supporting tools are important in ensuring that project managers understand and comply with departmental expectations for sound project management and meet the requirements at each gate of the process.

The audit found that the gating process is well-defined and documented, with minor administrative updates required to reflect recent changes to governance committee names and organizational structure. Expectations for each gate are clearly defined, and templates are provided to guide project managers in the development of key documentation. The IPMPU is active in supporting project managers in completing the templates and in assisting with the preparation for gate reviews with the oversight bodies. IPMPU staff meet with project managers in advance of committee meetings to review and challenge the documentation. Despite this support, our detailed review of projects identified some issues with the quality of the project management plans (discussed in section 3.2).

The templates themselves are comprehensive, are generally self-explanatory and guide project managers through an effective planning process that is consistent with the guidance provided by both TB and the Project Management Institute. However, it was noted that the Real Property and Security Directorate (RPSD) and the Information Management Services Directorate (IMSD) have also developed project management templates. In some instances, project managers reported that they had transcribed information from branch templates into the departmental templates to satisfy the gating requirements. As well, the Agency has also developed its own set of templates, adding additional complexity to partnership-related projects. Although all templates were developed with a similar intent, namely to introduce rigour and consistency in the project management process, they were developed independently of each other and often require the same information to be presented in a different manner. This redundancy has created an administrative burden for project managers. The Department and the Partnership would benefit from updating the framework to include minimum content and documentation requirements for all aspects; the templates being used for projects and partnerships would need to contain the minimum content and documentation requirements.

The audit assessed the adequacy of the guidance and tools provided by the IPMPU and found both to support project management capacity, but noted some areas for improvement in the supporting tools.

Recommendation 3

It is recommended that the Chief Financial Officer (in partnership) maintain a set of minimum content and documentation requirements for all aspects of the framework, which consider the common needs and essential control requirements of the different projects while allowing that others may require complementary documentation for their particular projects and partnerships.

Management response

Management agrees with the recommendation.

HC templates are consistent with Treasury Board of Canada Secretariat guidelines and reflect requirements. CFOB will work with the branches and project owners to make sure that the expectations of documentation for gate approvals are being met where different templates are used. In such cases, the branches may need to prepare a cross-walk of their templates to those of HC to meet minimum content requirements.

3.2 Project initiation and planning

Audit criterion: Projects are initiated and planned according to the departmental project management framework.

The project management framework has two project lifecycle models: the full five-gate model for projects valued at $1M or greater and a lite three-gate model for projects with a cost greater than $250K but less than $1M. The five-gate model includes three gates before the project execution phase: idea generation, project initiation and project planning. The three-gate model has only two gates before project execution, as it combines project initiation and project planning. These gating processes are designed to ensure that projects are worthy of investment, that they are aligned with departmental priorities and that sufficient planning, stakeholder engagement and organizational commitment are in place to provide the necessary resources to implement the project.

Overall, projects were generally compliant with the requirements of idea generation (Gate 1) and project initiation (Gate 2), but improvement is required in project costing and certain other elements of project planning (Gate 3).

Gate 1 - Idea generation

This stage is considered to be pre-project work, as it represents the initial presentation of the idea to the Department's senior management to seek the authority to continue with project initiation. At this stage, potential project sponsors present a proposal of the idea and a high-level estimate of the project cost.

The audit sampled seven projects in detail. It found that projects were generally in compliance with the gate requirements and that in most instances, approval for the project idea was obtained before investing further in the project initiation stage.

Gate 2 - Project initiation

The project initiation stage is focused on ensuring that there is sufficient business justification and organizational support for the project before further organizational resources are committed to detailed planning. Key project deliverables for this stage include the project charter, the business case, and a PCRA.

The audit found that projects were generally compliant with the gate requirements. With only minor exceptions, the projects reviewed had completed appropriately a detailed project charter, a business case and a PCRA.

Gate 3 - Project planning

During the project planning stage, project management and technical activities are defined, resourced and sequenced, and the proper management processes are put in place to successfully deliver the product and/or service. This project stage also establishes the official baselines against which the project will be measured. The key project deliverable at this stage is the project management plan, which defines how and when a project's objectives are to be achieved by showing the major products, milestones, activities and resources required. Development of these plans is the responsibility of the sponsoring branch. The PCRA is also refined at this stage, since more is known about the project parameters.

Based on the sample of projects, the analysis found that while all the required documentation was prepared, the quality of information varied by project. More specifically, there should be better information related to project costing, scheduling, change control, quality control, human resources and communications planning. While each of these is important, project costing is one of the most important project management activities to ensure that projects are delivered within the cost expectations documented in the project management plan. Cost can be affected by many factors but some of the more common pressures are scope and scheduling changes (either acceleration or slowdowns) and estimation errors.

Project costing is the responsibility of the sponsoring branch, which identifies major products, milestones, activities and the related resources required; it is also expected to maintain this information over the course of the project lifecycle. To support this activity, the IPMPU provides guidance on the level of detail and accuracy of costing information required at each gate, and further supports the costing activity with comprehensive templates. As well, as noted in the Project Management Gating Guide, stewardship of the project gate review process is further strengthened, since the IPMPU is to provide an independent verification that gate requirements have been satisfied.

Despite having a well-designed process with the appropriate gating controls, the audit found that there was a range of maturity in the costing information provided as part of the gating and approval processes. Costing information varied by project and was often not sufficiently detailed, such that cost estimates were not tied or assigned to specific project activities or work packages, thus making it difficult to understand how they were calculated or when they would be incurred. More detailed costing information would better enable project managers and others to more accurately forecast and monitor project progress and financial performance.

The Chief Financial Officer Branch (CFOB) recognizes that branches need to improve the project costing element of their project management plans. It is exploring a number of areas and activities to strengthen costing overall. This includes assessing changes to costing tools through to evaluating changes in engagement and consulting with project owners and stakeholders to assist them in developing improved detailed costing breakdowns. This multi-pronged approach is expected to improve the quality and completeness of the costing information.

The detailed examination also found that in four of seven projects sampled, the project schedules should have provided more detail related to tasks, critical path and milestones to support effective project monitoring. Three of the seven projects examined also did not have adequate change control plans to define how project changes would be identified, recorded, approved and actioned. Although interviews with project managers indicated that changes were discussed at branch-level oversight committees, these processes were not formally documented and it was unclear when or if significant changes should be brought to the departmental oversight bodies for approval.

A majority of projects also did not include adequate human resource or communications plans in the project management plans. Interviews with project managers indicated that many of these plans had been considered informally but were not documented as required by the project management framework. Finally, it was noted that signed approval of project documentation by branch heads, as required by the departmental project management framework (and TBS for the PCRA), was missing.

In conclusion, project sponsors should more closely develop and manage projects according to the expectations outlined in the departmental project management framework and IPMPU should continue to exercise its verification role to ensure that documentation is sufficient to allow for effective committee decision-making.

Recommendation 4

It is recommended that the Chief Financial Officer continue to develop options to improve project costing and to work with sponsoring branches to ensure that project documentation is more closely aligned with the requirements outlined in the departmental project management framework.

Management response

Management agrees with the recommendation.

CFOB will produce a guidance document on costing details required for each gate. CFOB will work with the branches to ensure that suitable costing information is provided for each gate approval.

3.3 Project execution

Audit criterion: Projects are completed in terms of authorized time, budget and scope.

Project execution is when the project plan is put into action with the purpose of achieving the project's expected results within the planned time and resources. Project managers are tasked with executing the plan, monitoring progress against the plan and identifying and addressing any issues or unforeseen events as they arise.

Twenty-one projects were tested to determine the degree to which they were on-scope, on-time and on-budget relative to the plan approved at Gate 3. As discussed in other sections of the report, projects did not always report against the milestones or costs approved in the project management plans, making it difficult to determine performance against the plan. Further, the deficiencies in project costing discussed earlier meant that cost targets were not always representative of the true project costs and meaningful milestone-based cost targets were not available.

3.4 Project closure

Audit criterion: Projects are closed out.

Project closure is the process of ensuring that all the necessary activities have been performed so that the project can be officially terminated within the organization. Assessing the project's success and identifying lessons learned are two of the most important activities of this stage. Determining whether the project was successful in achieving its goals within the expected time and cost and identifying what worked and what did not are important elements that contribute to the continuous improvement of the Department's project management practices. The key project documents required at this gate are the Project Close-out Report and Benefits Realization Report, as well as the Lessons Learned Report.

As discussed earlier in the report, the current framework does not require EC approval prior to closure. Providing the oversight committees with this type of information, including any lessons learned, would assist them in decision-making for future projects and provide them with summary data on the projects they have approved (addressed by recommendation 1).

Of the seven projects examined in detail, only one had reached the project closure phase at the time of the audit. This project completed its project execution activities in July 2013, but a lessons learned exercise or the assessment of the project's performance against its initial baselines had not yet been conducted.

To ensure the continuous improvement of project management practices at Health Canada, the Department should ensure that project close-out activities for all projects are conducted promptly following project execution. The results should then be reported to the departmental oversight bodies so that they are kept apprised of project results and opportunities for improvement.

3.5 Project monitoring and controlling

Audit criterion: Projects are monitored and controlled, and change requests are managed.

To successfully deliver on a project's objectives within the approved time, cost and scope, project managers must continuously monitor the project's performance and carefully manage change requests. While oversight bodies also have responsibility for monitoring project performance and managing change (discussed above in section 1.3), this section is focused on monitoring and change control at the project manager level.

Monitoring

Over the last three years, the IPMPU has developed and refined its monthly dashboard reporting into an effective tool for monitoring projects. The dashboard uses red, yellow and green indicators to highlight areas of concern for each active project in terms of schedule, cost, scope, risks and overall performance. The information contained in the dashboard is supported by more detailed and comprehensive data submitted to the IPMPU by project managers. The detailed templates developed by the IPMPU contain sections to capture information on the project's costs, key milestones and deliverables, risks, project issues and an overall assessment of the project. The detailed reports generally captured the appropriate information to support effective project management and in interviews, project managers reported that the templates are effective tools for monitoring their projects. However, it was also noted that budget and milestones did not always match those identified in the project management plan approved at Gate 3 (Project planning). This made it difficult to objectively assess the project status relative to its approved plan. Further, detailed cost reporting in the dashboards was focused on the current year and did not always include a full accounting of the entire project lifecycle.

Change control

Change control is an essential aspect of any project management process. Without documentation and approval of changes, projects may suffer from cost or schedule overruns or end up not meeting the needs of the business owner. In order to address this issue, project managers are required to document the change management process in the project management plan.

Once a project receives Gate 3 approval, it is up to the branch to ensure that changes are being properly logged and approved. In our sample review of individual projects, we found that branches had developed processes to review and approve changes made at the project level. Significant changes requiring a rebaselining of the project's schedule, cost or scope are also managed at the branch level and reported to EC-FIPT on the monthly dashboard.

C - Conclusion

The project management framework adopted by Health Canada (the Department) in 2012 has been successful in putting more structure and accountability into the management of projects. However, moderate improvements can still be made to address some of the weaknesses identified during the audit.

The Department's framework establishes clear roles and responsibilities for project management and supports effective oversight and project management practices. The framework requires updating to reflect organizational changes since 2012, including the Shared Services Partnership and changes to the Department's oversight committee structure. Further, a more active role for oversight bodies in the project close-out phases would ensure that senior management is receiving the information required to effectively manage the Department's portfolio of projects and to continuously improve departmental project management practices.

The Investment Planning and Management of Projects Unit (IPMPU) has established effective reporting practices to ensure that oversight bodies receive the information they need to achieve their mandate and to support project managers in their day-to-day management. Departmental-level risks are also being managed, although a more formal assessment and mitigation of risks identified in the Organizational Project Management Capacity Assessment (OPMCA) would further strengthen this area.

Finally, the organization has established a good balance of internal controls to manage and oversee the management of projects. Although these controls are functioning as planned in most cases, the IPMPU should ensure that gating documentation requirements reflect the minimum content and documentation requirements for all aspects of the framework and that the templates being used for projects and partnerships contain the minimum content and documentation requirements while allowing business owners the flexibility to prepare appropriate documentation for particular projects and partnerships.

Appendix A - Lines of enquiry and criteria

Lines of enquiry and criteria
Criteria Title Audit Criteria
Line of Enquiry 1: Governance
1.1 Project management framework A department-wide project management framework is documented and maintained to support accountability for projects.
1.2 Oversight committees Oversight bodies are established to govern the departmental portfolio of projects.
1.3 Roles and responsibilities Project management roles and responsibilities are clearly defined.
Line of Enquiry 2: Risk management
2.1 Departmental portfolio risk management Portfolio project management risks are identified, assessed, mitigated and monitored.
Line of Enquiry 3: Internal controls
3.1 Guidance and supporting tools Guidance and supporting tools are developed and up-to-date to support effective project management.
3.2 Project initiation and planning Projects are initiated and planned, according to the departmental project management framework.
3.3 Project execution Projects are completed in terms of authorized time, budget and scope.
3.4 Project closure Projects are closed out.
3.5 Project monitoring and controlling Projects are monitored and controlled, and change requests are managed.

Appendix B - Scorecard

Scorecard
Criterion Rating Conclusion Rec #
Governance
1.1 Project management framework Needs minor improvement The framework requires revision to include the Shared Services Partnership arrangement and project close-out reporting practices. 1
1.2 Oversight committees Satisfactory The oversight bodies meet regularly, are well-attended and meaningfully address project management issues. N/A
1.3 Roles and responsibilities Satisfactory Roles and responsibilities related to project management are clear and well-defined. N/A
Risk Management
2.1 Departmental portfolio risk management Needs minor improvement Regular reporting of project status and risks are presented in a monthly dashboard. OPMCA results should be used to track risks. 2
Internal Controls
3.1 Guidance and supporting tools Needs minor improvement Templates are comprehensive and support good project management practices. There would be some benefit in harmonizing templates with those of the Agency, RPSD and IMSD, and in updating them to align with framework updates. 3
3.2 Project initiation and planning Gate 1: Satisfactory There is good compliance with idea generation and project initiation phases. N/A
Gate 2: Needs minor improvement Project management plans did not fully comply with the template requirements. N/A
Gate 3: Needs improvement Detailed costing information should be linked to work packages or broken down by period. 4
3.3 Project execution Needs moderate improvement A clear critical path and full costing are needed to enable a better assessment of whether projects are on target in mid-stream. 4
3.4 Project closure Needs minor improvement Close-out activities for all projects should be conducted promptly following project execution. N/A
3.5 Project monitoring and controlling Needs minor improvement Project-level monitoring and change control are generally effective. N/A

Appendix C - Summary of Health Canada's projects

Summary of Health Canada's projects, as of February 28, 2014
Branch Active Projects Planned amount ($,000)
Corporate Services Branch (CSB) 16 $139,664
First Nations and Inuit Health Branch (FNIHB) 11 43,071
Healthy Environments and Consumer Safety Branch (HECSB) 9 38,758
Healthy Products and Food Branch (HPFB) 10 30,412
Communications and Public Affairs Branch (CAPB) 3 25,937
Chief Financial Officer Branch (CFOB) 7 16,193
Regions and Programs Bureau (RAPB) 3 3,526
Pest Management Regulatory Agency (PMRA) 2 3,409
Total 61 $300,970
Sustaining
Project has low risk and complexity. The project outcome affects only a specific service or at most a specific program, and risk mitigations for general project risks are in place. The project does not consume a significant percentage of departmental or agency resources.
Tactical
A project rated at this level affects multiple services within a program and may involve more significant procurement activities. It may involve some information management or information technology (IM/IT) or engineering activities. The project risk profile may indicate that some risks could have serious impacts, requiring carefully planned responses. The scope of a tactical project is operational in nature and delivers new capabilities within limits.
Evolutionary
As indicated by the name, projects within this level of complexity and risk introduce change, new capabilities and may have a fairly extensive scope. Disciplined skills are required to successfully manage evolutionary projects. Scope frequently spans programs and may affect one or two other departments or agencies. There may be substantial change to business process, internal staff, external clients, and technology infrastructure. IM/IT components may represent a significant proportion of total project activity.
Transformational
At this level, projects require extensive capabilities and may have a dramatic impact on the organization and potentially other organizations. Horizontal (i.e. multi-departmental, multi-agency, or multi-jurisdictional) projects are transformational in nature. Risks associated with these projects often have serious consequences, such as restructuring the organization, change in senior management, and/or loss of public reputation.

Appendix E - Health Canada project management lifecycles

Five-stage project lifecycle

figure

Text description

The complete five-stage or five-gate project lifecycle model for projects valued at $1M or greater.

The five-gate model includes three gates before the project execution phase: idea generation, project initiation and project planning. The appendix also outlines the requirements that a project needs to meet in order to pass the gate and move to the next project stage. For example, gating requirements include a project proposal at Gate 1, a preliminary business case at Gate 2, a project management plan at Gate 3, issues and change logs at Gate 4 and a lessons-learned report at Gate 5. The gating requirements are designed to ensure that projects are worthy of investment, that they are aligned with departmental priorities and that sufficient planning, stakeholder engagement and organizational commitment are in place to provide the necessary resources to implement the project.

Health Canada "Full" Project Stages/ Gates

Stage 1

Initiation

Gate 1 Requirements

  • Project Proposal/ Project Brief
  • Initial Costing Estimates @ +/- 15% for Stage 2 and +/- 100% for Stages 3 through 5 (each year of the project for multiyear projects)

Stage 2 Project-Initiation (TB PPA)

Planning

Gate 2 Requirements (PAA)

  • High-Level Business Requirements
  • Preliminary Business Case
  • Full Business Case (if required for TB)
  • Signed Project Charter
  • Approved PCRA
  • Project Brief (if required for TB)
  • Submission (if required)
  • Refined Costing Estimates @ +/- 15% for Stage 3 and +/- 50% for Stages 4 and 5 (for each year of the project for multi-year projects)

Stage 3 - Project Planning (TBEPA)

Definition

Gate 3 Requirements (EPA)

  • Project Management Plan
  • Refined Costing Estimates @ +/-15% (for each year of the project for multiyear projects)
  • Updated and Approved PCRA
  • TB Submission (if required)
  • Independent Review (if required)

Stage 4 - Project Execution

Implementation

Gate 4 Requirements

  • Risk Register
  • Issues Log
  • Change Log
  • Transition Plan

Stage 5 - Project Close-out

Close-out

Gate 5 Requirements

  • Close-out Report
  • Lessons Learned
  • Benefits Realization Report
  • Finalized and Archived Project Documentation
  • Post-Launch Review
  • Benefits Realization

Notes

  • Gate 1 represents EC approval.
  • Gates 2 and 3 approval authority is based on project tier.
  • Gates 2 and 3 may also require a TB Submission based on project tier. Project owners provide monthly status reports based on the established dashboard reporting process.

Three-stage project lifecycle

figure

Text description

The lite three-stage or three-gate model for projects with a cost greater than $250K but less than $1M.

The three-gate model has only two gates before project execution because it combines project initiation and project planning. The requirements that a project needs to meet in order to pass the gate and move to the next project stage include initial cost estimates at Gate 1, a signed project charter at Gate 2 and a project close-out report at Gate 3.

Health Canada "Light" Project Stages/Gates

Stage 1 - Idea Generation

Gate 1 Requirements

  • Project Proposal/Project Brief
  • Initial Costing Estimates @ +/- 15% for Stage 2 and +/- 100% for Stage 3.

Stage 2 - Project Initiation and Planning

Gate 2 Requirements

  • Signed Project Charter
  • Additional Project Information in PowerPoint format, including:
    • Summary of Options Analysis
    • Detailed Costing Estimates for preferred solution for Stage 3 @ +/-15% (each year of the project for multiyear projects)
    • Schedule
    • Risks / Issues

Stage 3 - Project Execution and Close-out

Gate 3 Requirements

  • Lessons Learned
  • Project Close-out Report
  • Post-Launch Review
  • Benefits Realization

Page details

Date modified: