Evaluation of the First Nations and Inuit Health Human Resources Program 2008-09 to 2012-13

Prepared by Evaluation Directorate, Health Canada and Public Health Agency of Canada
September 2013

List of Acronyms

AHHRI
Aboriginal Health Human Resources Initiative
FNI
First Nations and Inuit
FNIHB
First Nations and Inuit Health Branch
FN/I/M
First Nations/Inuit/Métis
HHR
Health Human Resources
IIHCP
Indian and Inuit Health Careers Program
LCO
Land Claim Organization
PAA
Program Alignment Architecture
PSE
Post-Secondary Education

Table of Contents

List of Tables

Executive Summary

The evaluation covered the First Nations and Inuit Health Human Resources sub-sub activity (the program) for the fiscal years 2008-09 to 2012-13. This evaluation was undertaken in fulfillment of the requirements of the Financial Administration Act and the Treasury Board of Canada's Policy on Evaluation (2009).

Evaluation Purpose, Scope and Design

The purpose of the evaluation was to assess the relevance and performance of the First Nations and Inuit Health Human Resources program. The scope covered the Pan-Canadian aspects of the program from April 2008 to March 2013. The methodology used for the evaluation included a document and data review, literature review and key informant interviews.

Program Description

The First Nations and Inuit Health Human Resources program was formed in 2011 through consolidation by the First Nations and Inuit Health Branch (FNIHB) of a number of funding authorities, which included the Aboriginal Health Human Resources Initiative (AHHRI - funding ending in 2015) and the Indian and Inuit Health Careers Program (IIHCP). The objective was to lay the foundation for longer term systemic changes in the supply, demand and creation of supportive working environments for Aboriginal health human resources. The program targeted First Nations, Inuit and Métis people regardless of their status and where they reside, as well as health care providers, post-secondary institutions, and health professional and paraprofessional associations and organizations. The average annual program expenditures over the evaluation period was approximately of $20.4 million (i.e., approximately $102M over the five years).

Evaluation Conclusions and Lessons Learned

Conclusions - Relevance

Continued Need for the Program

There is a continued need to increase the representativeness of First Nations, Inuit and Métis health care workers in the health care system and to address First Nations and Inuit health human resources' needs on-reserve, particularly for rural and remote communities. However, while programs such as the First Nations and Inuit Health Human Resources program can mitigate some of the challenges and barriers to building and maintaining a First Nations/Inuit/Métis health care workforce (e.g., health sciences bridging and access programs) many barriers to enrolment in post-secondary education do not fall under the purview of Health Canada, specifically gaps in education at the primary and secondary school level.

Alignment with Government Priorities

The First Nations and Inuit Health Human Resources program has aligned with an important priority of the federal government since its inception in 2005 to meet the commitments made as part of the 2003 Health Accord and continues to be aligned with departmental outcomes and FNHIB strategic goals and objectives.

Alignment with Federal Roles & Responsibilities

The First Nations and Inuit Health Human Resources program is aligned with federal roles and responsibilities, which are shared among several departments, including Health Canada, Aboriginal Affairs and Northern Development Canada and Employment and Social Development Canada (previously Human Resources and Skills Development Canada). In Health Canada, the program aligned with the roles and responsibilities of FNIHB as described in its 2012 Strategic Plan.

Conclusions - Performance

Achievement of Expected Outcomes (Effectiveness)

Overall, the First Nations and Inuit Health Human Resources program has made progress towards its expected outcomes, including the longer term outcome of "an Aboriginal health workforce delivering health services to Aboriginal individuals, families and communities". There has been progress on increasing enrolments in and graduations from health programs. The program has facilitated progress by mitigating the impacts of some of the challenges and/or barriers faced by First Nation, Inuit and Métis people, in particular through access, bridging and support programs, as well as bursaries and scholarships. However, there are opportunities for improvement in how bursaries and scholarships are delivered by Indspire for its Health Careers stream in Quebec.

Focus was placed on improving the qualifications of community-based workers and health managers in recent years. This also increased technical and cultural competency (where this has been embedded into other projects).

Demonstration of Efficiency and Economy

The program demonstrated efficiency by adjusting its priorities and mix of products and services in accordance with where the needs and opportunities for impact were the greatest and by implementing a regional governance and delivery model to support economical implementation. The First Nations and Inuit Health Human Resources program relied on collaboration and partnerships to leverage resources and support sustainable results.

There appear to be opportunities to work within the larger FNIHB strategic goal related to effective and efficient performance to ensure appropriate linkages with other federal departments to support a population health and whole-of-government approach. Improvements could also be made in performance measurement activities to assess outcome achievement and inform future priorities.

Evaluation Lessons Learned

Formal recommendations are not being proposed given that program funding is expected to end by March 2015. Based on the findings and conclusions outlined in this evaluation report, the department is aware that there are lessons learned that will be valuable should interventions targeting First Nations, Inuit and Métis health human resources be considered in the future.

Undertake applied research to inform priorities and future initiatives

There is currently no baseline information available pertaining to the number of First Nations individuals originating from reserves or Inuit communities who are enrolling in and/or graduating from post-secondary institutions from various health disciplines or on the extent to which they return to their home communities after graduation. This information is challenging to acquire, due to confidentiality issues and the tracking processes in place in educational institutions. However, the acquisition of this type of data would have a number of benefits, which include contributing to the identification of more precise human health resource needs, the development of more appropriate interventions and/or, alternatively, the adaptation/improvement of current ones in addition to supporting evidenced-based discussions with First Nations and Inuit stakeholders on future programming needs.

Furthermore, additional research on one or more of the post-secondary education programs (bridging, access, student support programs, cultural competency) funded by AHHRI may also be useful in informing future initiatives that seek to increase and improve Aboriginal health human resources.

Strengthen performance measurement activities

A strengthened performance measurement system would enable an enhanced assessment of potential causal linkages between the delivery of program activities and some of the longer-term changes. It would also enable an assessment of effectiveness and efficiency in program delivery. While monitoring information is collected, it could be enhanced through standardized definitions for what should be reported. Consistent implementation at both the regional and national levels would be beneficial to ongoing program monitoring and evaluation activities.

Ensure coordination of Branch activities and collaboration with other government departments

Multiple federal initiatives are being implemented to address a range of barriers and challenges in this area. Therefore, it is of benefit to build upon existing strengths in collaborating and partnering internally, with other federal government departments and with external institutions. This should help to ensure activities are targeted and appropriately linked with others who may hold levers to achieving intended outcomes. In this way, future initiatives can target gaps and/or barriers that are not already addressed by other programs funded by the Branch, by federal partners or by other jurisdictions.

Management Response

Formal recommendations are not being proposed given that program funding is expected to end by March 2015. Based on the findings and conclusions outlined in this evaluation report, the department is aware that there are lessons learned that will be valuable should interventions targeted to First Nations, Inuit and Métis health human resources and capacity building be considered in the future.

First Nations and Inuit Health Human Resources Program Evaluation
Lessons Learned Management Response Further Considerations/Comments/Actions
Undertake applied research to inform priorities and future initiatives.

Strengthen performance measurement activities.

Ensure coordination of Branch activities and collaboration with other government departments.

Management accepts the findings, conclusions and lessons learned of this evaluation.

The AHHRI was designed to address the severe underrepresentation of Aboriginal people in the health care workforce and especially in health care and program delivery in First Nations and Inuit communities. While it made some strides in increasing the numbers of Aboriginal health care professionals and training of community-based workers, there remains a significant gap and hence a continued need to address this gap.

The health human resources funding has resulted in a number of successes that demonstrated improvements in increasing enrolment in and graduation from health career related programs; in the qualifications of community-based health care workers and health managers and in the availability of culturally competent curricula in medical and nursing schools.

In order to achieve the successes, it required collaboration especially in regions with their partners (First Nations/Inuit stakeholders, post-secondary education institutions, provinces/territories, health care associations and other government jurisdictions). Most regions established governance structures to assist their planning. This served to improve the regional capacity and First Nation/Inuit capacity in health human resources planning as well as maximizing resources.

FNIHB will apply the many lessons learned from the AHHRI work to inform future initiatives.

A significant portion of the AHHRI work was foundational such as the development of bridging and access programs, the student support programs in post-secondary education, the cultural competency curricula in medical and nursing schools, and the development of the First Nations health managers' competency framework and training. These important accomplishments along with the established partnerships are important to the future success of First Nations health programs.

Further, it is important that Health Canada work with the Aboriginal Affairs and Northern Development Canada, Employment and Social Development Canada, FNIHB programs, and First Nations and Inuit organizations to make sure that education/training programs currently supported by them increase their focus on health careers so that the important gains made by the AHHRI are not lost.

1.0 Evaluation Purpose

The purpose of the evaluation was to assess the relevance and performance of the First Nations and Inuit Health Human Resources sub-sub activity (the program) for the fiscal years 2008-09 to 2012-13.

This evaluation is the first undertaken of the First Nations and Inuit Health Human Resources program as a whole and has been done to fulfill the requirements of the Financial Administration Act and the Treasury Board of Canada's Policy on Evaluation (2009). A mid-term review of the Aboriginal Health Human Resources Initiative component of the First Nations and Inuit Health Human Resources program was completed in 2009 (Health Canada, 2009).

2.0 Program Description

2.1 Program Context

The First Nations and Inuit Health Human Resources program was formed in 2011 through consolidation by the First Nations and Inuit Health Branch (FNIHB) of a number of funding authorities, which included the Aboriginal Health Human Resources Initiative (AHHRI) and the Indian and Inuit Health Careers Program (IIHCP), as these programs have similar objectives.

The AHHRI originated from a commitment made by the federal government at the Special Meeting of the First Ministers and Aboriginal Leaders in September 2004. This commitment, which built upon the 2003 Health Accord, directed provincial, territorial and federal governments to work together with Aboriginal people to advance a health care system that would be more responsive to the needs of Aboriginal people.

At the 2004 Special Meeting, the scope of AHHRI was broadened to be Pan-Aboriginal and include First Nations, Inuit, Métis, and non-status and off-reserve First Nations. The Initiative was also designed to closely align with the Pan-Canadian Health Human Resources Strategy which was one of the primary initiatives of the 2003 Health Accord. Within this larger strategy, a Framework for Collaborative Pan-Canadian Health Human Resources Planning (Health Canada, 2007) was endorsed. The framework promoted a systems-based, collaborative, population needs-based approach to human resource planning. It recognized that the jurisdictional responsibility for health system design and health human resources planning rests with the jurisdictions. As well, it determined the resources available to deliver health care. It affirmed that, because there were a small number of training programs across the country and because of the mobility of the health work force, jurisdictions could not plan in isolation and there was a need to adopt a collaborative Pan-Canadian approach to health human resources planning, including health human resources planning for Aboriginal communities.

The AHHRI was formally announced in Budget 2005 as a new investment of $100 million over five years and then renewed in March 2010 when the Initiative obtained $80M from 2010-11 to 2014-15, which is referred to as its second phase.Footnote 1

The IIHCP has been a more modest program in terms of funding, with approximately $3.7 million in annual ongoing funding, but has had a longer history than AHHRI. It was originally approved by Treasury Board in 1984 (ongoing funding). In its earliest years (1984-87), various Treasury Board documents referred to the program as the Indian and Inuit Health Careers Program, the Indian and Inuit Professional Health Career Program, and the Indian and Inuit Professional Health Careers Development Program.

The IIHCP had objectives very similar to the later AHHRI program. At the outset it had five components: Promotion, Career-Related Employment, Community-based Programs, Institutional Programs, and Bursaries and Scholarships. Bursaries and Scholarships were initially provided directly by Health Canada. In 1998, this was changed so that funds were provided to the National Aboriginal Achievement Foundation (NAAF), now renamed IndspireFootnote 2, to administer bursaries and scholarships. Through bursaries and scholarships, the program was designed to encourage and support Aboriginal participation in educational opportunities leading to professional careers in the health field and provide a learning environment to help Aboriginal students at universities and colleges overcome many of the social and cultural barriers that inhibited their educational achievement. In the long run, achieving these objectives was seen as a means to address Aboriginal under-representation in the health professions and enhance opportunities for Aboriginal people to receive quality health care from people of their own culture.

2.2 Program Profile

The objective of the First Nations and Inuit Health Human Resources program was to lay the foundation for longer term systemic changes in the supply, demand and creation of supportive working environments for Aboriginal health human resources. The program targeted First Nations, Inuit and Métis people regardless of their status and where they reside, as well as health care providers, post-secondary institutions, health professionals and paraprofessional associations and organizations, and associations representing colleges and universities.Footnote 3

Stakeholders for the program included a wide range of internal and external parties. Internal stakeholders included Health Canada national and regional offices. External stakeholders included First Nations, Inuit and Métis communities, bands and tribal councils, national Aboriginal organizations, provincial territorial organizations, land claims organizations, provincial and territorial governments, regional health authorities, health service delivery organizations, health professional and paraprofessional organizations, health service providers, colleges and universities and their associations. Beneficiaries included First Nations, Inuit and Métis individuals, as post-secondary students, health care consumers and health care workers.

As previously stated, AHHRI and IIHCP were brought together in the First Nations and Inuit Health Human Resources program because they had similar objectives. The specific activities carried out within the program before and after the 2011 consolidation of authorities can be found in the following Table 1.

Table 1: Crosswalk of Specific Activities within the First Nations and Inuit Health Human Resources Program for the Evaluation Period (2008-09 to 2012-13)
Specific activities within the First Nations and
Inuit Health Human Resources Program
08/09 09/10 10/11 11/12 12/13
Bursaries and scholarships X X X X X
Promotion of health careers X X X X X
Training for community-based workers     X X X
Post-secondary educational support programs X X X X X
Curricula adaptation X X X X  
Health managers (focused on the development of the core competency framework) X X      
Health managers (focused on the training of health managers)     X X X
Health human resources planning, implementation and integration X X X X X
Capacity for First Nations and Inuit X X X X X
Knowledge translation strategy     X X X

As part of the program evolution, a mid-term review (Health Canada, 2009) commissioned by the program supported ongoing investment in increasing the health care workforce but also called for a re-focus to include training efforts to bring community-based workers' skills and certifications in line with health care workers in the provincial/territorial health care systems.

Currently, the First Nations and Inuit Health Human Resources program can be described as followsFootnote 4:

  • The objective is to increase the number of qualified First Nations and Inuit individuals working in health care delivery.Footnote 5 It administers contribution agreements and direct departmental spending to promote and support competent health services at the community level by increasing the number of First Nations and Inuit individuals entering into and working in health careers and ensuring that community-based workers have skills and certification comparable to workers in the provincial/territorial health care system. This program engages many stakeholders including: federal, provincial and territorial governments and health professional organizations; national Aboriginal organizations; non-governmental organizations and associations; and educational institutions.
  • Key activities supporting program delivery include: health education bursaries and scholarships; health career promotion activities; internship and summer student work opportunities; knowledge translation activities; training for community based health care workers and health managers; and development and implementation of health human resources planning for Aboriginal, federal, provincial, territorial, health professional associations, educational institutions, and other stakeholders.

2.3 Program Logic Model and Narrative

According to the First Nations and Inuit Health Human Resources program logic model, the program is organized under five major themes including: service provision; capacity building; stakeholder engagement and collaboration; data collection, research, and surveillance; and policy development and knowledge sharing.

Through these themes, the following outputs are to be delivered:

  • career fairs and promotional material to increase awareness and translate into increased enrolment in health programs;
  • bursaries and scholarships for First Nations, Inuit and Métis youth pursuing health career studies;
  • bridging, access and supportive programs to address the education gap observed in First Nations, Inuit and Métis populations and increase enrolment and graduation (by reducing the drop-out rate);
  • training for community-based workers (including health managers) to acquire certification aligned with equivalent provincial/territorial standards where applicable;
  • nursing and medicine curricula to prepare non-Aboriginal students, undergraduates and graduates for a culturally appropriate practice when intervening with First Nations and Inuit individuals;
  • tools and other material for use by regions and communities in health human resources planning for subsequent integration in regional processes aimed at identifying health human resource needs and priorities;
  • funded provincial/territorial organizations and land claim organizations to engage/participate in health human resources planning; and
  • a knowledge strategy to improve communications, particularly as it pertains to project and outcome reporting as well as best practices with partners and stakeholders.

Each of these outputs is targeted to specific audiences and intended to contribute to the outcomes shown on the logic model.

The expected immediate outcomes include:

  1. Increased enrolment in health programs; and
  2. Increased awareness of best practices in HHR planning and management.

The expected intermediate outcomes include:

  1. Increased number of First Nations/Inuit/Métis technically competent health service providers;
  2. Increased number of health service providers trained for cultural competency; and
  3. Increased capacity in health human resources planning and management.

In the long-term, the program is expected to contribute to an increase in the proportion of Aboriginal health professionals delivering health services to Aboriginal individuals, families and communities.

2.4 Program Alignment and Resources

The First Nations and Inuit Health Human Resources sub-sub activity (3.3.1.2) contributes to Health Canada's Strategic Outcome 3: First Nations and Inuit communities and individuals receive health services and benefits that are responsive to their needs so as to improve their health status. The program was identified in the Department's Program Alignment Architecture (PAA) under Program Activity 3.3: Health Infrastructure Support for First Nations and Inuit and under Sub-Activity 3.3.1: First Nations and Inuit Health System Capacity.

According to financial dataFootnote 6, program expenditures averaged $20.4 million per year from 2008-09 to 2012-13 (i.e., approximately $102M over the five years). Actual expenditures by Health Canada headquarters and regions are shown on Table 2.

Table 2: Expenditures for the First Nations and Inuit Health Human Resources Program
Year Contributions Salaries/Wages Other Operating and
Minor Capital
Total
2008-09 $22,079,950 $1,575,443 $592,079 $24,247,472
2009-10 $22,156,145 $1,695,453 $683,392 $24,534,990
2010-11 $15,097,499 $1,564,917 $322,672 $16,985,088
2011-12 $20,787,738 $1,275,328 $167,137 $22,230,203
2012-13 $12,647,087 $1,166,923 $179,036 $13,993,046

3.0 Evaluation Description

3.1 Evaluation Scope and Issues

The scope of the evaluation covered the Pan-Canadian aspects of the First Nations and Inuit Health Human Resources program from April 2008 to March 2013.

The evaluation issues were aligned with the Treasury Board of Canada's Policy on Evaluation (2009). The evaluation considered the five core issues under the two themes of relevance and performance, as shown in Table 3. Corresponding to each of the core issues, there were evaluation questions which were tailored to the program and guided the evaluation process.

Table 3: Core Issues and Questions
Core Issues Evaluation Questions
Relevance
Issue #1: Continued Need for Program Is there a continued need for the Program?
  • 1.1: Does the Program continue to address a demonstrable need?
  • 1.2: Is the Program responsive to the needs of Canadians?
Issue #2: Alignment with Government Priorities Does the Program align with Government of Canada priorities?
  • 2.1: Does the Program remain a priority of the federal government?
  • 2.2: Does the Program align to departmental strategic priorities/outcomes?
Issue #3: Alignment with Federal Roles and Responsibilities Is the Program aligned with federal roles and responsibilities?
  • 3.1: Does the Program align with departmental key program activities?
  • 3.2: Do the Program's key stakeholders see the Program's activities as relevant and aligned to its roles and responsibilities?
  • 3.3: Are the Program's activities aligned/congruent with the department's jurisdictional, mandated and/or legislated role?
Performance (effectiveness, efficiency and economy)
Issue #4: Achievement of Expected Outcomes (Effectiveness) Is the Program achieving the outcomes expected as outlined in the Logic Model?
  • 4.1: Has the Program achieved its immediate outcomes?
  • 4.2: Has the program achieved its intermediate outcomes?
  • 4.3: Has the Program achieved its long-term outcomes (also called ultimate outcomes)?
Issue #5: Demonstration of Efficiency and Economy Has the Program been implemented: efficiently?
  • 5.1:How has the Program optimized the overall quantity, quality, and blend of products and/or services to facilitate achievement of the Program's expected outcomes?
  • 5.2: Are there alternative methods which ensure the same achievement of immediate expected results?
Has the Program been implemented: economically?
  • 5.3: Has the Program minimized resources while optimizing outputs?
  • 5.4: Were the Program's resources managed effectively to facilitate the achievement of relevant immediate outcomes?

3.2 Evaluation Approach and Design

The evaluation used an outcome-based approach to assess the progress made towards the achievement of intermediate outcomes. The approach included collaboration with key internal and external stakeholders in the planning and conduct of the evaluation, review of technical data and the evaluation report as well as the development of the management response.

A non-experimental descriptive design was used. The design was developed based on the expectations regarding the availability of data sources for addressing the evaluation questions. An assessment of data by the Evaluation Directorate prior to the initiation of the evaluation confirmed the absence of some key performance measurement data (e.g., Aboriginal status information was not available for many indicators) and thus progress towards the achievement of expected outcomes was in some cases observational in nature.

3.3 Data Collection and Analysis Methods

The evaluation used multiple lines of evidence to increase the reliability and credibility of the evaluation findings. The lines of evidence included a literature review; a document and data review; and key informant interviews to gain the qualitative perspective of program stakeholders.

Literature Review

The overall objective of the literature review was to provide a summary of current literature that could contribute to the assessment of the First Nations and Inuit Health Human Resources program's relevance and performance, mainly by attempting to identify the external influences potentially impacting achievements. The review was conducted in alignment with the specific evaluation questions and related indicators focusing on the following themes:

  • determinants of access to and success in post-secondary education for First Nations, Inuit and Métis populations;
  • factors influencing mobility of First Nations, Inuit and Métis health professionals;
  • factors influencing the retention of aboriginal and non-aboriginal staff (particularly health staff) in First Nations, Inuit and Métis communities; and
  • profile of the First Nations, Inuit and Métis health workforce from 2005 up to now.

A total of 50 documents were retrieved and reviewed.

Document and Data Review

A document and data review was conducted to provide information about each of the outputs in the First Nations and Inuit Health Human Resources program logic model. Approximately 1800 documents and data sources were provided to the evaluation team, of which 1000 were deemed to be relevant. Other program-related documents (e.g., the program framework, the First Nations and Inuit Health Branch strategic plan and budgets) were also reviewed to assess the relevance core issues.

Key Informant Interviews

A total of 32 interviews were held with key informants to supplement information gathered through other methods and to seek their views on First Nations and Inuit Health Human Resources program progress and context. Key informants were selected from the following stakeholder groups:

  • Health Canada headquarters executives and staff (5 interviews);
  • Health Canada regional coordinators (7 interviews);
  • National Aboriginal Organizations, Provincial Territorial Organizations, and Land Claim Organizations representatives (7 interviews);
  • Post-Secondary Institution representatives (11 interviews);
  • Indspire representative (1 interview); and
  • First Nations Health Managers Association representative (1 interview).

Structured interview guides were prepared for each stakeholder group. Key informants received the interview guides prior to the interview. The interviews were conducted by telephone and interview notes were then prepared. Key informants were provided with the notes and given an opportunity to validate them.

Analysis

Evidence from all lines of inquiry was examined according to the key evaluation issues and questions. Data were analyzed by triangulating information gathered from the different sources and methods. Systematic compilation, review and summarization of the data were conducted to illustrate key findings.

For qualitative key informant interviews, the following descriptive scale was used to indicate the approximate number of key informants that made the relevant statement, with "a few" being approximately 10-15% or less of respondents, "some" being more than 15% to approximately 40%, "many" being more than 40% to approximately 60%, "most" being more than 60% to approximately 80%, and "almost all" being over 80%.

3.4 Limitations and Mitigation Strategies

Most evaluations face constraints that may have implications on the validity and reliability of evaluation findings and conclusions. This section (Table 4) illustrates the limitations in the design and methods for this particular evaluation. Also noted are the mitigation strategies put in place to ensure that the evaluation findings can be used with confidence to guide program planning and decision making.

Table 4: Limitations and Mitigation Strategies
Limitation Impact Mitigation Strategy
Literature Review
The design of studies in the literature yielded mostly anecdotal evidence on the success of interventions used to address health human resources issues.
Limited relevance of findings from the literature as information could not be generalized to populations, geographic areas and heritage groups. Use was made of multiple lines of evidence to build up a body of evidence for the evaluation.
Performance data
The project level documentation provided by the program was incomplete as the final 2012/13 reports were not available at the time of the document review.

There was a lack of consistent, standardized performance data on expected outcomes.

There was limited departmental financial data and mapping financial data to all specific outputs or outcomes was not possible (although this could be done for bursaries and scholarships and community-based worker training). 

Limited ability to undertake a full and comprehensive assessment of the extent to which outcomes had been achieved and efficiency and economy realized. Interviews were used to gather opinions on outcomes to complement the evidence from the document review.

A focused assessment of resource utilization was included in the evaluation using only approved data provided by the Branch Senior Financial Officer.

Key Informant Interviews
Stakeholders invited to be key informants were representative of various stakeholder communities and community-based project representatives were not interviewed.Footnote 7
Information gathered in key informant interviews cannot be extrapolated across all stakeholders. A range of stakeholder categories were used to "reflect" the stakeholder community.
Key Informant Bias
While key informants included a range of views and organizations, all were involved to some degree with the program.
Interview findings may represent a bias due to key informant involvement in the program. The interview findings were balanced with other lines of evidence to develop overall findings for the evaluation.

4.0 Findings

4.1 Relevance: Issue #1 - Continued Need for the Program

While the proportion of First Nations, Inuit or Métis occupying health positions has increased between 1996 and 2006, there is still a demonstrable need to further increase the number of First Nation, Inuit and Métis health care workers, including physicians and nurses, paraprofessionals who are community-based workers, and health managers, especially those serving First Nation, Inuit and Métis clients. The ongoing need relates to increasing the qualifications of community-based workers, an important resource to manage and deliver services in more rural or isolated First Nations and Inuit communities.

There are multiple jurisdictions involved in increasing the Aboriginal health work force as well as many challenges and barriers related to increasing this work force. The First Nations and Inuit Health Human Resources program responds to a subset of these challenges and barriers primarily by mitigating some challenges and barriers for those who are in or close to entering post-secondary education institutions and workplaces.

Findings related to "continued need" are organized into three parts:

  1. a general description of the larger issue of First Nations, Inuit and Métis under-representation in the health workforce that the First Nations and Inuit Health Branch has been addressing over the years in collaboration with other stakeholders;
  2. an overview of the challenges and barriers that need to be addressed; and
  3. where and how the First Nations and Inuit Health Human Resources program has responded.

First Nations, Inuit and Métis Under-Representation in the Health Workforce

The need for more First Nations, Inuit and Métis workers in health and social services has been known for a long time. The original rationale for the Indian and Inuit Health Career Program in 1984 was to help address this problem. A call for action came from the 1996 Report of the Royal Commission on Aboriginal Peoples (RCAP, 1996) which recommended that governments and educational institutions train 10,000 Aboriginal workers (including professional and managerial roles) in health and social services by 2006, to meet requirements in all areas of health. The target of 10,000 was expected to address a number of issues including:

  • serious under-representation of Aboriginal people in the medical profession (and other health and social service professions);
  • distribution of these scarce professional resources i.e., many Aboriginal professionals did not practice in Aboriginal communities, and often they were concentrated in the lower ranks of organizations rather than in supervisory, management or policy positions, where health human resources decisions are made;
  • a concentration of Aboriginal personnel in paraprofessional positions, and the need to advance the qualifications of many of them to professional and supervisory roles; and
  • the need to staff new Aboriginal services in urban and rural off-reserve settings.

These issues were reported to have continued into more recent years.Footnote 8 A 2003 Canadian Nurses Association study noted in AHHRI Program Framework documents that less than 0.5% of nurses were of First Nation, Inuit or Métis ancestry, compared to 3% required for equitable representation.  Similar shortages were noted for other health human resources.

A recent study supported by the AHHRI (Lecompte, 2012)Footnote 9 looked at data from the censuses conducted in 1996, 2001 and 2006 and reported that progress had been made with 12,965 First Nations, Inuit and Métis people entering health careers between 1996 and 2006.Footnote 10 The study reported that the "10,000 target" set by the Royal Commission can be interpreted as having been surpassed just about the time that the AHHRI component of the First Nations and Inuit Health Human Resources program really became operational. However, despite these achievements, equitable representation was still not achieved, with Aboriginal people making up 3.8% of Canada's population according to the 2006 census and only representing 2.2% (or 21,815 people) of Canadian workers in health occupations [N.B., the percentage of Aboriginal peoples in health occupations has increased over time (1.16% in 1996 to 1.65% in 2001 to 2.2% in 2006)].Footnote 11

The Lecompte study observed the following increasesFootnote 12:

  • for Métis, health professionals and paraprofessionals working in off-reserve areas increased from 2,895 in 1996 to 10,425 in 2006, with two-thirds of the increase coming in the 2001-2006 period;
  • for First Nations, the increase was from 3,745 to 7,530 from 1996 to 2006;
  • for Inuit, the increase was from approximately 325 to 430 over the same period; and
  • for on-reserve, the numbers of First Nations health care providers grew from 1,435 to 2,550 over the ten year period (1996-2006).

The literature review (including the Lecompte study) indicated that gains made were primarily off-reserve although gains were still significant for First Nation on-reserve but less so for Inuit in Inuit communities. Graduation from secondary schools appeared to be a key factor and varied according to the distance from urban centres, with the lowest educational achievement being observed on-reserve (Caledon Institute of Social Policy, 2006). This supports the perspective of key informants who noted that the continued need is not just a matter of the quantity but also of the location and qualifications of health care workersFootnote 13. This includes health care professionals such as physicians and nurses, as well as community-based workers working on-reserve. This issue is considered significant, resulting in larger shortfalls and greater impacts upon health care delivery in rural and remote areas and in isolated First Nations communities. This situation is similar to other rural areas of Canada that also experience difficulty in recruiting and retaining health professionals. Even in the absence of robust data on the volume, locations and qualifications of health care workers, evidence suggested that shortfalls have continued up to the current date. The shortfalls are also thought to extend to the qualifications of health directors and health managers.

Health Human Resources Needs - Challenges to be Addressed

To understand why these shortfalls may be happening in the first place, it is important to understand the significant challenges identified in the literature review, document review and key informant interviews that need to be addressed. These challenges can be present at several stages in the life events of a First Nation, Inuit or Métis individual (examples are listed below) and highlight the interdisciplinary nature of health human resources needs. This is noteworthy because these challenges limit the extent to which outcomes can be achieved.

Key challenges include:

Early childhood

  • social determinants.

High school student

  • awareness of health careers;
  • choices made in selection of courses (e.g., not understanding the need for mathematics and science credits);
  • quality of mathematics and science courses in rural and isolated communities;
  • lack of mentors and role models;
  • dropping out prior to graduation; and
  • relocation in order to achieve grades 9-12 (as some communities only offer local schooling up to grade 8).

Applicant for enrolment in a post-secondary education institution

  • not graduating or not having the credits (e.g., math and science, to meet the acceptance criteria for a PSE program);
  • financial issues (e.g., lack of funding);
  • relocation from the community; and
  • availability of access and bridging support.

Student enrolled in a post-secondary education institution

  • economic (e.g., poverty, poor quality and lack of housing, lack of financial support, lack of childcare);
  • socio-cultural barriers (e.g., social alienation, low language proficiency, racism, culture shock, differences between traditional/ mainstream culture, lack of community and family connection, family responsibilities); and
  • barriers within the post-secondary education environment (e.g., lack of acknowledgement of non-western ways of knowing/learning, lack of regard for indigenous knowledge, lack of professors to adapt to specific Aboriginal needs).

Health care workers in communities

  • payment structure, leading to practitioner turnover;
  • access to and quality of accommodations in remote and isolated communities;
  • lack of medical equipment and facilities to deal with complex health issues;
  • limited career paths;
  • qualifications; and
  • the isolation factor and level of integration into communities.

Program Response

The AHHRI and the IIHCP components of the First Nations and Inuit Health Human Resources program have responded to the larger problem of under-representation and distribution of First Nations/Inuit/Métis health human resources by focusing upon the mitigation of specific challenges and barriers within the mandate of Health Canada and acting as a catalyst to accelerate progress. For example, the program supported promotional activities to increase the awareness of health careers and the academic requirements to pursue a health career through career days and science fairs. The program also supported Indspire's health career stream of bursaries and scholarships to address financial issues. It should be noted that, despite receiving Aboriginal Affairs and Northern Development Canada post-secondary education funding, recipients still required additional financial assistance from AHHRI through Indspire bursaries and scholarships. Furthermore, in the 2009 horizontal evaluation of the National Aboriginal Achievement Foundation, the vast majority of students (90%) strongly agreed or agreed that if they had not received a scholarship or bursary from the foundation they would have fallen deeper into debt.Footnote 14

However, in a number of cases, the challenges are either being addressed by other programs and/or fall outside of the mandate of Health Canada. For example, as outlined in the previous section, a number of barriers relate to social determinants that can influence school achievement in the early years. These then impact students in high school who may not be aware of health careers options, may not understand the need for prerequisites or may not have access to quality mathematics and science courses in rural and isolated communities to join the health field. For all sorts of reasons, Aboriginal high school students may be dropping out prior to graduation or may not be interested in pursuing their high school education in another community, should their own community not offer schooling beyond grade 8. It was outside the scope of the AHHRI program to try to address these problems at their root cause. Yet, there was widespread recognition by stakeholders that successful sustainable outcomes require that these problems be addressed and that Health Canada has an interest in them. As a result, the AHHRI invested in the development of a post-secondary education bridging and access program to ensure high school graduates were encouraged to tap their potential in the health care field.

Over time, the areas of program response have shifted and become more focused in scope. For example, the framework for the first AHHRI (2005-2010) was developed through a substantial consultative process to ensure programming would build upon the existing systems, processes and organizations, by coordinating with the Pan-Canadian Health Human Resources Strategy and by leveraging investments in health human resources that were already made. The first AHHRI program focussed on increasing the number of Aboriginal health care workers, improving the retention of health care providers in First Nation and Inuit communities, and improving the cultural competence of health care providers serving Aboriginal communities. The midterm review of the AHHRI and stakeholder consultations confirmed the focus of the program as addressing the on-reserve shortage of First Nations, Inuit and Métis health care providers and led to a new focus on providing training opportunities for community-based workers, including health managers, to acquire skills and certification comparable to workers in provincial-territorial health systems.

Most key informants supported the revised objectives such as the focus on community-based workers and health managers, and the continuance of components such as bursaries and scholarships. At the same time, some key informants observed that activities (e.g., cultural competency and summer jobs) that had been successful in addressing needs had to be dropped as priorities shifted within a reduced funding envelope (e.g., overall program funding decreased 40% from 2007-08 to 2012-13). Key informants from the North also noted that AHHRI in general was designed for South of 60 and so did not respond as well (e.g., in terms of project criteria) to the needs of the North and the federal role of supporting territorial governments as compared to a large number of First Nations communities.

4.2 Relevance: Issue #2 - Alignment with Government Priorities

From its inception in 2005 up to recent years, the First Nations and Inuit Health Human Resources program has been aligned to federal government and Health Canada priorities.

The AHHRI component of the First Nations and Inuit Health Human Resources program originated from a commitment made by the federal government at the Special Meeting of the First Ministers and Aboriginal Leaders in September 2004. This original commitment, which stems from the 2003 Health Accord, included a new investment of $100 million over five years for the Initiative. The AHHRI was announced in the 2005 federal budget and then renewed for five years in the 2010 federal budget. The priority of the Initiative was confirmed in approvals for funding in 2005 and 2010. Similarly, there was a series of approvals for funding related to the Indian and Inuit Health Career Program.

4.3 Relevance: Issue #3 - Alignment with Federal Roles and Responsibilities

The First Nations and Inuit Health Human Resources program is consistent with the collaborative role that the federal government has regarding Pan-Canadian health human resource issues, including Aboriginal communities, and is aligned with the roles and responsibilities of FNIHB, as described in its 2012 Strategic Plan. Other departments such as Aboriginal Affairs and Northern Development Canada also hold important levers for effecting changes in the First Nation, Inuit and Métis health care workforce, and have their own roles and responsibilities upon which the program depends.

The AHHRI, now part of the First Nations and Inuit Health Human Resources program, originates from the Framework for Collaborative Pan-Canadian Health Human Resources Planning, which recognized that the jurisdictional responsibility for health system design and health human resource planning rests with the jurisdictions. However, factors such as the small number of training programs and the mobility of the health work force suggested an imperative to implement a collaborative Pan-Canadian approach to health human resource planning, including for Aboriginal communities. Therefore, at a high level, the federal government role and responsibilities may best be described as collaborative and Pan-Canadian. The emphasis on partnerships and collaboration in the AHHRI component of the program suggests that it has been aligned with this role and responsibility.

Looking more closely at Health Canada's roles and responsibilities, the First Nations and Inuit Health Human Resources program is part of the department's PAA and ultimately maps to Strategic Outcome 3 that states: "First Nations and Inuit communities and individuals receive health services and benefits that are responsive to their needs so as to improve their health status". The need for a First Nations and Inuit health care work force has been consistently voiced by these communities and is articulated as an outcome of the sub-sub-activity.

Findings from the literature review noted that the proportion of Aboriginal students completing some form of post-secondary education is the same as in the general population when considering only those Aboriginal students who have completed high school (Caledon Institute of Social Policy, 2006). This suggests that one of the most important barriers to access to and success in post-secondary education relates to academic preparation preceding entry into a post-secondary institution. Furthermore, academic preparation is itself likely to be influenced by many of the barriers described in section 4.1. This suggests a complex interface between Health Canada (whose mandate is focused on the health of First Nations and Inuit individuals through the delivery of various programs, some of which is targeted to children and to Aboriginal individuals experiencing mental health and addiction issues), Aboriginal Affairs and Northern Development Canada (which is responsible for providing formal education services on-reserve); Employment and Social Development Canada (previously Human Resources and Skills Development Canada, which has a role to ensure horizontal integration of federal health human resources skills development) and the provinces/territories (which are responsible for educating Aboriginals off-reserve).

The First Nations and Inuit Health Branch's 2012 Strategic Plan (Health Canada, 2012) notes that the Branch is indeed part of a complex environment for First Nations and Inuit health:

  • "Healthcare for First Nations and Inuit in Canada is delivered in the context of a complex, dynamic and interdependent health system government by federal, provincial, territorial and First Nations and Inuit jurisdictions.
  • As a result, the need for the First Nations and Inuit Health Branch to work collaboratively with other jurisdictions and partners with health and other expertise is fundamental, since many of the necessary levers to effect change are not held by the First Nations and Inuit Health Branch or even by the federal government."

Health Canada demonstrates its role within this complex environment in two key ways. First, Health Canada funds primary care in 85 remote/isolated First Nations communities and public health nursing, health promotion/disease prevention programming and environmental health services and home and community care in over 600 communities. The First Nations and Inuit Health Human Resources program is directly aligned with these responsibilities of FNIHB, First Nations and Inuit, and other jurisdictions in delivering health services in these communities. 

Second, Health Canada provides eligible First Nations and Inuit, regardless of where they live, with supplementary health benefits for certain medically required services where these individuals do not have coverage from other public or private programs. The AHHRI component of the program has focused more on First Nations, Inuit and community-based workers and thus it has aligned more closely with the primary role of FNIHB in this manner.

The FNHIB 2012 Strategic Plan identified four strategic goals: 1) high quality health services; 2) collaborative planning and relationships; 3) effective and efficient performance; and 4) supportive environment in which employees excel. The First Nations and Inuit Health Human Resources program is identified as an initiative "supporting First Nations and Inuit in their aim to influence, manage and/or control health programs and services that affect them" which is an objective identified under the second strategic goal, i.e., collaborative planning and relationships. However, the range of barriers noted earlier suggests that more emphasis may be required on creating appropriate linkages between the First Nations and Inuit Health Branch`s programs and services, those of other federal departments, provinces, territories, and First Nations and Inuit to support a population health approach and a whole-of-government approach on the social determinants of health, which is an objective of the third strategic goal (i.e., effective and efficient performance).

Almost all key informants across all categories felt the federal government and Health Canada continue to have important and relevant roles to play in the development of the First Nations, Inuit and Métis health human resource workforce. Respondents noted the federal government's responsibilities for both health care for First Nations and Inuit people and for health human resource planning as one of the enablers of that system. The federal government's role as a funder/investor and partner to stimulate change across the health system was also emphasized.

4.4 Performance: Issue #4 - Achievement of Expected Outcomes (Effectiveness)

4.4.1 To what extent have the intermediateFootnote 15 outcomes been achieved?

Intermediate Outcome #1: Increased Number of First Nations, Inuit and Métis Health Service Providers Technically Competent

To contribute to this outcome, there has been demonstrable progress in increasing enrolment in and graduation from post-secondary health programs. Similarly, there has been progress in increasing the qualifications of First Nations and Inuit community-based workers and health managers.

The Intermediate Outcome of "Increased number of First Nations, Inuit and Métis health service providers (including community-based workers and health managers) technically competent" is supported through achievement of the immediate outcomes related to enrolment in and graduation from health programs and by the qualifications of health service providers. The discussion that follows considers the three approaches supported by the First Nations and Inuit Health Human Resources program, namely, (1) post-secondary education, (2) community-based workers, and (3) health manager qualifications and certifications.

Post-Secondary Education

Although data is limited since most post-secondary institutions do not track the First Nation, Inuit or Métis status of their students, there is evidence suggesting that enrolments in health programs increased overall under the evaluation period.

There has also been progress on increasing the number of Aboriginal post-secondary education graduations, likely resulting in an increase in the number of health care providers who are technically competent. Many key informants give credit to the First Nations and Inuit Health Human Resources program for some of the increase in enrolments and graduations but there is no data to quantify the contribution.

Furthermore, data is not collected on where these graduates choose to work so it is not known whether they moved to under-served communities or into the mainstream workforce, or if they remained in health care or working in other areas.

In terms of enrolment, data obtained from Indspire suggested that 2,594 Aboriginal students were awarded post-secondary education health career bursaries or scholarships between 2006 and 2012. Of this number, 1,132 studied nursing and 483 studied medicine. In the mid-term review, Quebec-based key informants had concerns pertaining to Indspire's lack of a strong presence in Quebec and suggested the limited availability of French language-based documentation and services may have prevented French-speaking Aboriginals from submitting applications.  A subsequent agreement with Indspire included terms and conditions to ensure material was available in both official languages and bilingual staff resources were available.

Some of the post-secondary institution representatives interviewed noted that First Nations, Inuit and Métis student enrolment (and graduation rates) has been relatively stable over the last five years. In these institutions, educational programs are mature and, in many cases, include designated seats for Aboriginal students. According to the Association of Universities and Colleges of Canada (2007), initiatives to promote Aboriginal enrolment were implemented by universities to respond to government prompting, in search for competitive advantages for the educational institution, to counter social inequities and/or to address local demographic realities.

As was the case for enrolment, national information about the total number of Aboriginal students graduating from health programs is not collected. As a result, the number of students in their final year of study being awarded bursaries and/or scholarships, in health programs, was used as a proxy. Using Health Canada data, the number of Aboriginal students awarded health career bursaries and/or scholarships in their final year of study (and presumably graduating) increased from 73 in 2007-2008, to 129 in 2008-2009, to a maximum of 157 in 2009-2010, with a slight decrease to 149 in 2010-2011 and to 142 in 2011-2012 for a total of 650 graduates over the evaluation period. It should be noted that not all First Nations students access Indspire's Bursaries and Scholarships program, so relying on just this data potentially underrepresents the enrolment and success of First Nation students in health studies.

Both the literature and key informants indicated that academic preparation starting early in the K-12 school years has a great impact upon eventual enrolments in health careers. While academic preparation is most likely to be influenced by social determinants and other barriers described earlier, raising awareness of what needs to be done in terms of the appropriate math and science credits (matching the necessary prerequisites) may theoretically contribute to increased First Nations enrolment in post-secondary institutions.

Similarly, key informants reported that successfully completing the first year of post-secondary education was, in their experience, a key factor in continuing to stay enrolled in health programs. Key informants described helpful initiatives relating to innovative access, bridging programs and support programs in post-secondary institutions across Canada, many of which have operated for a number of years. In some post-secondary institutions, there are specific programs for Aboriginal students, and in others, there are specific programs for Aboriginal students in health programs. These programs are designed to increase the number of students enrolling and succeeding in (staying enrolled in) post-secondary studies. There has been a push to build more rungs on the ladder to alleviate barriers to enrolment, and then to mitigate the stress factors of enrolling and staying enrolled (e.g., dislocation, isolation). Some of these programs provide support closer to home (e.g., through more local colleges) or in communities (e.g., through distance learning). Some are dealing with the issues faced by more mature students (e.g., childcare, family responsibilities, community ties, jobs). At one time or another, many of these programs received support from the Aboriginal Health Human Resources Initiative and gave the Initiative the credit for having launched, piloted or enlarged the programs. In fact, AHHRI supported over 240 post-secondary education projects since its inception in 2005.

Findings from the literature indicated that educational achievement is highest in cities and lowest on-reserve, suggesting awareness building activities as well as access, bridging and support programs that address issues faced by Aboriginal people in rural areas or on-reserve should contribute to fulfilling health human resources requirements in areas of greatest need. Furthermore, AHHRI was instrumental in supporting health career awareness through videos, training, and two health career fairs per year reaching over 1,000 secondary students.

The importance of these awareness, bridging and support programs was recognized by post-secondary institutions and their associations. In response to this recognition, for example, the Association of Community Colleges of Canada published a case studies report that included a number of programs related to health careers. The Association of Universities and Colleges of Canada launched an inventory and search tool on its website which lists such programs across Canada and which can be used by First Nations, Inuit and Métis students or their counsellors/advisors to identify those institutions that may best fit their needs (AUCC, 2013).

Although access, bridging and support programs are meeting a need, funding remains an important barrier for students. In terms of increasing enrolments in health programs, Indspire's Health Careers streamFootnote 16 of bursaries and scholarships was considered by a few interviewees as a success story stemming from the First Nations and Inuit Health Human Resources program. However, there appear to be some issues or potential concerns with regard to the distribution of bursaries and scholarships. For example, when comparing the distribution of bursaries and scholarships with the proportion of each identity group in the broader Canadian population, there are significant discrepancies. First Nations and to a lesser extent Inuit recipients awarded bursaries or scholarships are disproportionally lower (about 30% and 3.5% less than the Canadian average) while Métis recipients are disproportionally higher (about 27% more than the Canadian average) which may to some extent reflect the variation in academic readiness reported by the Caledon Institute of Social Policy (2006). Further, the distribution of bursaries and scholarships is less per capita for the Aboriginal population in Quebec compared to other provinces.

Community-Based Workers

There has been demonstrable progress on increasing the qualifications of community-based workers to be comparable to the provincial/territorial standards. Notionally, since this tends to be related more to on-the-job training, these workers are considered to be more likely to stay in/return to their current work places after completion of their programs/training.

Between April 2010 and March 2013, the number of participants across Canada in community-based worker training was anticipated to be at least 1,900. Data on completion of community-based workers training was not comprehensive, with information only being available for 34 out of 100 community-based workers training projects. The number of participants who completed training nationally increased from about 486 in 2010-2011 to 510 in 2011-2012. Of these individuals, 253 and 206 (respectively) increased their qualifications to the existing provincial/territorial standards (for the remainder of those trained, no information was provided or provincial/territorial standards did not exist). Data was incomplete for 2012-2013 given that the regional reports were not available at the time of the evaluation and many of the projects required multi-year training.

Post-secondary institution key informants considered community-based worker training to be a success and indicated that their programs were accredited and led to certification.

Health Manager Training

There has been demonstrable progress on increasing the qualifications of health managers.

Between April 2010 and March 2013, the number of participants across Canada in health manager training was anticipated to be at least 1,000. Data on completion of health managers training was not comprehensive with information being available for only 8 of 35 health manager projects. Although the training for the First Nation health manager was just recently developed, it does seem to show an increase in First Nation participation over time.

The need for a First Nations health manager competency framework was identified by the Assembly of First Nations and FNIHB in 2006. The objective of the framework was to describe the competencies, skills and knowledge of a First Nations health manager in carrying out his/her work and to assist First Nations health managers and their employers in better understanding the complexity of the work. The aim was to provide First Nations communities and organizations with a tool to help improve their awareness of the strengths and challenges of First Nations health managers and assist them to focus on developing other competencies within themselves and being able to recognize these qualities in others.

The framework for health managers training was supported by the program through a contribution to the First Nations Health Managers Association and it covered ten domains that are relevant to technical competency in health management: leadership and governance; professionalism; advocacy, partnerships and relationships; human resource management; financial management and accountability; health service delivery; quality improvement and assurance; planning; communication; and cultural awareness.

Key informants supported the value of the health managers training. Some informants made a linkage between having a professionally run health care centre (with a well-trained health director or health manager) and recruitment and retention of qualified health care workers.

Intermediate Outcome #2: Increased number of health service providers trained for cultural competency

The cultural competency of health service providers was not an area of focus in the second phase of the AHHRI. However, some evidence suggested that the awareness and practices from the first phase have been embedded in some organizations.

This was not an area of focus for the second phase of the AHHRI (2010-2015). Projects initiated in the first phase (2005-2010) were to be completed early during the second phase, and evaluations were to be undertaken in the second year of the renewed program. Cultural competency training began during the first phase of the AHHRI by the Association of Faculties of Medicine and Royal College of Physicians and Surgeons in collaboration with the Indigenous Physicians Association of Canada and this work continues. In addition, medical schools are being encouraged to implement the curricula. Nursing schools developed cultural competency curricula in collaboration with the Aboriginal Nurses Association of Canada.

Awareness and practices from phase one have been embedded in some organizations. For example, several post-secondary institution key informants indicated that their programs were culturally adapted and that faculty was trained for cultural competency. In other cases, the review of project files showed that cultural competency was included in projects through, for example, cultural inclusion and competency workshops in a bridging program for nursing.

Intermediate Outcome #3: Increased capacity in health human resources planning and management

Increasing capacity in health human resources planning and management was supported through the First Nations Health Managers Association.

The work of the First Nations Health Managers Association was considered noteworthy by stakeholders for developing the ten competency domains for health managers training, which was followed by the actual delivery of training. The domains include planning and enablers such as leadership and governance, partnerships, and cultural awareness. As such, the First Nations and Inuit Health Human Resources program's support for the First Nations Health Managers Association can be tied to helping increase awareness, as well as understanding and application of best practices in health human resources planning and implementation.

Initial plans for renewal stated that Health Canada regions would develop strategic health human resources plans with their First Nations and Inuit partners in the first year of renewal (2010-11). Plans also stated that the Aboriginal Health Human Resources Initiative needs-based assessment and planning tools would be implemented by regions within the first year of the renewed program (also identified as one of the risk factors). Most regions developed strategic health human resources plans with their First Nations and Inuit partners or implemented a needs-based assessment and planning toll during the first year of renewal (2010-11). Operational plans were also developed by regions.

Some organizations such as national Aboriginal organizations, provincial/territorial organizations and land claim organizations had health human resources plans supported through the AHHRI and this planning appeared to be driven by the First Nations and Inuit Health Human Resources program's influence or through the sharing of best practices. Some provincial/territorial organizations and land claim organizations reported that they have good planning capacity and some recruitment and retention strategies have been developed.

Some national Aboriginal organizations reported that, with regional-level planning and delivery of the First Nations and Inuit Health Human Resources program, their role, activity and capacity in national health human resource planning had diminished. This was due to Health Canada's regional offices dealing directly with provincial/territorial organizations, land claim organizations and other stakeholders in their regions.

The mid-term program review identified the need for AHHRI to develop a knowledge management and dissemination strategy and to improve on communications with Aboriginal stakeholders to ensure that the goals, objectives, priorities and best practices were understood and shared at the local, regional and national levels. Since 2010, AHHRI has hosted two such knowledge translation fora in Ottawa and Moncton.

4.4.2 To what extent has the longer term outcome been achieved?

Longer-term Outcome: An Aboriginal health workforce delivering health services to Aboriginal individuals, families and communities

While there is indication of progress, additional time and data beyond the scope of this evaluation would be necessary to more fully understand the composition of the health care workforce at the community level, the mobility patterns and predictors of the Aboriginal workforce on- and off-reserve, and the factors influencing Aboriginal health care workers' decisions in choosing work settings (e.g., community compared to urban).

The number of students entering health programs at the post-secondary level and the number of community-based workers and health managers receiving training to improve their qualifications, as detailed under the intermediate outcomes, indicated that the First Nations, Inuit and Métis health workforce increased and that most of them had qualifications comparable, or becoming comparable, to the health workforce at large.

However, what is unclear is the distribution of the First Nations, Inuit and Métis health workforce and where they are providing their services. For example, it is not known if First Nations, Inuit and Métis health workers are choosing to work in Aboriginal communities, serving individuals and families there, or whether they are choosing to work in off-reserve mainstream health facilities located in an area more or less densely populated by individuals of Aboriginal ancestry. There is some general evidence from the literature review that health workers with a rural or remote background are more likely to work in rural or remote areas. However, there was only anecdotal information about the likelihood that health workers return to work in their own communities.

Furthermore, the degree to which Aboriginal health care workers move around is not well understood. Although there is some evidence that mobility may be linked to the level of educational attainment, the literature review did not reveal studies that focussed explicitly on intra-provincial migration of both Aboriginal and non-Aboriginal Canadian physicians or other health workers, and the detailed migration patterns and predictors of such movements.

4.5 Performance: Issue #5 - Assessment of Economy and Efficiency

The program has taken steps to be efficient. It has adjusted its priorities and mix of products and services in accordance with needs and opportunities, and relied on collaboration and partnerships to leverage resources and support sustainable results. It implemented a more regional governance and delivery model to efficiently deliver a quantity, quality and blend of products and services closely aligned with regional needs.

While the program worked to promote complementarity and to minimize potential duplications with other programs, there may be further opportunities to enhance effective and efficient performance by ensuring appropriate linkages with complementary programs.

The Treasury Board of Canada's Policy on Evaluation (2009) and guidance regarding Assessing Program Resource Utilization When Evaluating Federal Programs (2013) defines the demonstration of economy and efficiency as an assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes. This assessment is based on the assumption that departments have standardized performance measurement systems and that financial systems link information about program costs to specific inputs, activities, outputs and expected results.

The data structure of the detailed financial information provided for the program did not facilitate the assessment of whether program outputs were produced efficiently, or whether expected outcomes were produced economically. Specifically, the lack of output/outcome-specific costing data limited the ability to use cost-comparative approaches. In terms of assessing economy, challenges in tracking funding within the broader Health Infrastructure Support for the First Nations and Inuit envelope limited the assessment. Considering these issues, the evaluation provided observations on economy and efficiency based on findings from the key informant interviews and available relevant financial data.

In addition, the findings below provide observations on the adequacy of performance measurement information to support economical and efficient program delivery and evaluation.

Observations on Economy

This section provides a review of resource utilization (i.e., direct and indirect spending; including salary, operating/maintenance and contributions for funded projects) from fiscal year 2008-09 to 2012-13.

First Nations and Inuit Health Human Resources program average expenditures were approximately $20.4 million annually under the evaluation period. However, expenditures were lower in two years: $16.9 million was expended in 2010-2011 (the year that marked the launch of the AHHRI second phase) and $14.0 million was expended in 2012-2013. Reduced spending during 2010-11 was due to the delays experienced in obtaining funds and in transferring them to regions while the decrease reported for 2012-2013 was due to funding reductions.

For the years included in the evaluation, expenditures by regional and national offices varied from 4% of expenditures in the northern regions to 11% of expenditures in Ontario and up to 26% of the total expenditures in headquarters. The high proportion of national expenditures was accounted for by the Indspire bursaries and scholarships and support of the national cultural competency projects.

On average, salaries represented approximately 5% of the amount of grants and contributions administered at the regional level and 14% of the grants and contributions administered at headquarters. Overall, operating expenditures (including salaries) were 11% of grants and contributions expenditures.

Observations on Efficiency

The criteria for program activities (in the Program Framework for Phase 2) stated that: they be designed and/or delivered in partnership with First Nations/Inuit/Métis (as applicable) and their organizations; there be collaboration with partners; and they be sustainable without the need for incremental or ongoing funding. These criteria were viewed as supporting efficiency in terms of promoting the optimal quantity, quality and blend of products and services, minimizing overlap and duplication, and encouraging self-sustaining activities.

In the second phase of the AHHRI, priorities shifted based upon the recommendations from the mid-term review and input from stakeholders. These changes were consistent with the requirement to direct resources towards the key needs and opportunities so the program could influence change in an efficient manner.

The regional governance and delivery model provided the opportunity to plan and deliver a quantity, quality and blend of products and services that would be more closely aligned with regional needs, context, and approaches. Models of aligning activities with regional needs were an efficient way of supporting the achievement of outcomes.

However, some key informants noted a number of process issues affecting efficiencies. For example, the time taken between program launch and moving ahead with projects in both phases of the AHHRI was too long as it related to the project approval process.

Finally, the program worked to promote complementarity and minimize potential duplications with other programs. For example, the AHHRI worked with several FNIHB programs to maximize resources and to coordinate community-based worker training to avoid duplication.

There appear to be opportunities to work within the larger FNIHB's 2012 Strategic Plan (Goal 3: Effective and Efficient Performance) to improve data and further collaborations with programs of other departments.

Observations on the Adequacy of Performance Measurement Data

The lack of complete and standardized performance data to assess outcomes hindered the provision of information to support the evaluation, in particular, data to support the assessment of the achievement of all expected outcomes outlined in the logic model. For example, while there was a standardized performance reporting template for headquarters, the performance data provided was not always complete, varied across regions and did not consistently provide data on outcome measures.

5.0 Conclusions

5.1 Relevance Conclusions

Core Issue #1: Continued Need

There is a continued need to increase the representativeness of First Nations, Inuit and Métis health care workers in the health care system and to address First Nations and Inuit health human resource needs on-reserve, particularly for rural and remote communities. However, while programs such as the First Nations and Inuit Health Human Resources program can mitigate some of the challenges and barriers to building and maintaining a First Nations/Inuit/Métis health care workforce (e.g., health sciences bridging and access programs) many barriers to enrolment in post-secondary education do not fall under the purview of Health Canada, specifically gaps in education at the primary and secondary school level.

Core Issue #2: Alignment with Government Priorities

The First Nations and Inuit Health Human Resources program has aligned with an important priority of the federal government since its inception in 2005 to meet the commitments made as part of the 2003 Health Accord and it is currently aligned with departmental outcomes and FNHIB strategic goals and objectives.

Core Issue #3: Alignment with Federal Roles & Responsibilities

The First Nations and Inuit Health Human Resources program is aligned with federal roles and responsibilities, which are shared among several departments, including Health Canada, Aboriginal Affairs and Northern Development Canada and Employment and Social Development Canada (previously Human Resources and Skills Development Canada). In Health Canada, the program aligned with the roles and responsibilities of FNIHB as described in its 2012 Strategic Plan.

5.2 Performance Conclusions

Core Issue #4: Achievement of Expected Outcomes (Effectiveness)

Overall, the First Nations and Inuit Health Human Resources program has made progress towards its expected outcomes, including the longer term outcome of "an Aboriginal health workforce delivering health services to Aboriginal individuals, families and communities." There has been progress on increasing enrolments in and graduations from health programs. The program has facilitated progress by mitigating the impacts of some of the challenges faced by First Nation, Inuit and Métis people, in particular through access, bridging and support programs, as well as bursaries and scholarships. However, there are opportunities for improvement in how bursaries and scholarships are delivered by Indspire for its Health Careers stream in Quebec.

Focus has been placed on improving the qualifications of community-based workers and health managers in recent years. This also increased technical and cultural competency (where this has been embedded into other projects).

Core Issue #5: Demonstration of Efficiency and Economy

The program demonstrated efficiency by adjusting its priorities and mix of products and services in accordance with where the needs and opportunities for impact were the greatest and by implementing a regional governance and delivery model to support economical implementation. The First Nations and Inuit Health Human Resources program relied on collaboration and partnerships to leverage resources and support sustainable results.

There appear to be opportunities to work within the larger FNIHB strategic goal related to effective and efficient performance to ensure appropriate linkages with other federal departments to support a population health and whole-of-government approach. Improvements could also be made in performance measurement activities to assess outcome achievement and inform future priorities.

6.0 Implications

Formal recommendations are not being proposed given that program funding is expected to end by March 2015. Based on the findings and conclusions outlined in this evaluation report, the department is aware that there are lessons learned that will be valuable should interventions targeted to First Nations, Inuit and Métis health human resources and capacity building be considered in the future.

7.0 Lessons Learned

The lessons learned stemming from the First Nations and Inuit Health Human Resources program pertain to three themes as outlined below and include ways to implement practices and approaches necessary to improve health human resources in First Nations, Inuit and Métis communities.

Undertake applied research to inform priorities and future initiatives

There is currently no baseline information available pertaining to the number of First Nations originating from reserves or Inuit communities who are enrolling in and/or graduating from post-secondary institutions from various health disciplines or on the extent to which they return to their home communities after graduation. This information is challenging to acquire, due to confidentiality issues and the tracking processes in place in educational institutions. However, the acquisition of this type of data would have a number of benefits, which include contributing to the identification of more precise human health resource needs, the development of more appropriate interventions and/or, alternatively, the adaptation/improvement of current ones in addition to supporting evidenced-based discussions with First Nations and Inuit stakeholders on future programming needs.

Strengthen performance measurement activities

A strengthened performance measurement system would enable an enhanced assessment of potential causal linkages between the delivery of program activities and some of the longer-term changes. It would also enable an assessment of effectiveness and efficiency in program delivery. While monitoring information is collected, it could be enhanced through standardized definitions for what should be reported. Consistent implementation at both the regional and national levels would be beneficial to ongoing program monitoring and evaluation activities.

Ensure coordination of Branch activities and collaboration with other government departments

Multiple federal initiatives are being implemented to address a range of barriers and challenges in this area. Therefore, it is of benefit to build upon existing strengths in collaborating and partnering internally, with other federal government departments and with external institutions. This should help to ensure activities are targeted and appropriately linked with others who may hold levers to achieving intended outcomes. In this way, future initiatives can target gaps and/or barriers that are not already addressed by other programs funded by the Branch, by federal partners or by other jurisdictions.

Appendix 1 References

AUCC (2013). Creating Opportunities in Education for Aboriginal Students, Association of Universities and Colleges of Canada, January 2013.

Caledon Institute of Social Policy (2006). Michael Mendelson, Aboriginal Peoples and Postsecondary Education in Canada.

Health Canada (2007). Framework for Collaborative Pan-Canadian Health Human Resources Planning, Federal/Provincial/Territorial Advisory Committee on Health Delivery and Human Resources (ACHDHR), Health Canada, September 2005, Revised March 2007.

Health Canada (2009). Program Review of the Aboriginal Health Human Resources Initiative (AHHRI).

Health Canada (2012). First Nations and Inuit Health Strategic Plan: A shared path to improved health,  First Nations and Inuit Health Branch.

Canadian Heritage (2009). Horizontal Evaluation of the National Aboriginal Achievement Foundation,

Office of the Chief Audit and Evaluation Executive Evaluations Service Directorate.

Lecompte (2012). Emily Lecompte, Aboriginal Health Human Resources: A Matter of Health, Journal of Aboriginal Health, March 2012.

RCAP (1996). Report of the Royal Commission on Aboriginal Peoples.

Appendix 2 Logic Model and Narrative

First Nations and Inuit Health Human Resources Program Logic Model

Objective: To lay the foundation for longer term systemic changes in the supply, demand and creation of supportive working environments for Aboriginal health human resources. More specifically:

  • Increasing the number of First Nations/Inuit entering into and working in health careers; and,
  • Integrating First Nations/Inuit health care workers with the provincial/territorial health delivery systems by ensuring they have comparable skills and certification, including training of Health Managers.

Target Group: First Nations/Inuit/Métis regardless of their status and where they reside; health care providers providing services to First Nations/Inuit/Métis; Post-secondary Educational Institutions delivering health sciences programs; First Nations/Inuit/Métis and non-Aboriginal health professional, and paraprofessional associations and organizations, and associations representing colleges and universities.

Theme Activities and Related Outputs and Outcomes:

Theme 1: Service Provision

  • Related Outputs
    • Careers Fairs & Promotional material
    • Bursaries & Scholarships
    • Post-Secondary Education Initiatives
  • Related Immediate Outcomes
    • Increased enrolments in health programs
  • Related Intermediate Outcomes
    • Increased number of First Nations/Inuit/Métis health service providers (including Community-Based Workers and Health Managers) technically competent
    • Increased number of health service providers trained for cultural competency

Theme 2: Capacity Building

  • Related Outputs
    • Training for Community-Based Workers (including Health Managers)
    • Adapted Curriculum
  • Related Immediate Outcomes
    • Increased enrolments in health programs
  • Related Intermediate Outcomes
    • Increased number of First Nations/Inuit/Métis health service providers (including Community-Based Workers and Health Managers) technically competent
    • Increased number of health service providers trained for cultural competency

Themes 3: Stakeholder Engagement and Collaboration

  • Related Outputs
    • Health Human Resources Plans
  • Related Immediate Outcomes
    • Increased awareness of best practices in HHR planning and management
  • Related Intermediate Outcomes
    • Increased capacity in health human resources planning and management (response coordinated with needs; health human resources practices informed by evidence and best practices)

Theme 4: Data Collection, Research and Surveillance

  • Related Outputs
    • Number of students awarded bursaries and scholarships, numbers of post-secondary education projects, number of career fairs, number of Community-Based Workers and Health Managers trained
  • Related Immediate Outcomes
    • Increased awareness of best practices in HHR planning and management
  • Related Intermediate Outcomes
    • Increased capacity in health human resources planning and management (response coordinated with needs; health human resources practices informed by evidence and best practices)

Theme 5: Policy Development and Knowledge Sharing

  • Related Outputs
    • Knowledge Translation Strategy
  • Related Immediate Outcomes
    • Increased awareness of best practices in HHR planning and management
  • Related Intermediate Outcomes
    • Increased capacity in health human resources planning and management (response coordinated with needs; health human resources practices informed by evidence and best practices)

All activities, outputs and outcomes lead to the longer term outcome of:

  • An Aboriginal health workforce delivering health services to Aboriginal individuals, families and communities

Logic Model Narrative

The objective of the First Nation and Inuit Health Human Resources program is to lay the foundation for longer term systemic changes in the supply, demand and creation of supportive working environments for Aboriginal health human resources. More specifically, the First Nation and Inuit Health Human Resources program aims to:

  • increase the number of First Nations/Inuit entering into and working in health careers; and
  • integrate First Nations/Inuit health care workers with the provincial/territorial health delivery systems by ensuring they have comparable skills and certification, including training of health managers.

The First Nation and Inuit Health Human Resources program aims to attract to health professions First Nations on- and off-reserve, Inuit and Métis with the expectation that once their studies are completed, they opt for a health career in First Nations/Inuit/Métis communities. The First Nation and Inuit Health Human Resources program supports local, regional and national-level projects with variable duration.

Target Groups

The recipients targeted for First Nation and Inuit Health Human Resources program funding are First Nations/Inuit/Métis regardless of their status and where they reside; health care providers providing services to First Nations/Inuit/Métis; post-secondary educational institutions delivering health sciences programs; First Nations/Inuit/Métis and non-Aboriginal health professional and paraprofessional associations and organizations; and associations representing colleges and universities.

Themes

The First Nation and Inuit Health Human Resources program is organized under five major themes including: service provision; capacity building; stakeholder engagement and collaboration; data collection, research and surveillance; and policy development and knowledge sharing.

These themes are represented, respectfully, in the following outputs (described in more detail below):  career fairs; bursaries and scholarships; post-secondary education programs; training for community-based workers (including health managers); adapted curriculum; health human resources plans; and a knowledge translation strategy.

Outputs

Career Fairs and Promotional Material - The First Nation and Inuit Health Human Resources program provides annual non-permanent funding to Indspire in support of two annual career fairs - Blueprint for the Future. The FNIHB headquarters also produces promotional materials to be used at the Blueprint for the Future career fairs and coordinates regional involvement in these events. Other national organizations may also choose to propose group specific health career promotion projects.

Regions also contribute to the promotion of health careers mainly through the organization of local health career fairs. Other activities include booth displays at community events, science camps, role modeling and job shadowing projects.

Bursaries and Scholarships - The First Nation and Inuit Health Human Resources program provides non-permanent funding to Indspire through a national contribution agreement to administer the bursaries and scholarships for First Nations/Inuit/Métis youth pursuing health career studies.

Post-Secondary Education Initiatives - Through the provision of non-permanent funding, regions support the development and enhancement of bridging, access and supportive programs aimed at addressing the education gap observed in First Nations/Inuit/Métis populations which have a higher dropout rate compared to the rest of the Canadian population. These initiatives are expected to increase the enrolment and success (graduation) of First Nations/Inuit/Métis students in health science programs.

Access programs are characterized by the adaptability of the educational program to the needs of the students. They include a supportive component (e.g., math preparatory, research skills, etc.) and may have an extended academic year and flexible schedule so students can meet their other commitments (e.g., family commitments).

Bridging programs are a type of access program typically providing a means for students to gain access to admission in a post-secondary program (e.g., completion of a prerequisite, academic upgrading).

While access and bridging programs target candidates to health studies, support programs target actual students in the field, providing a culturally appropriate welcoming environment aimed at increasing the graduation rate of First Nations/Inuit/Métis students enrolled in health care studies.

Training for Community-Based WorkersFootnote 17 (including Health Managers) - The primary purpose of this output is to provide community-based workers with training that allows them to acquire the certificationFootnote 18 to align with equivalent provincial/territorial standards where applicable. Training is to be conducted in partnership with recognized colleges or technical training institutes, and should take into account which organizations are provided with project / non-permanent funding for the training of community-based workers working in one or more of the eligible community-based certifications (addiction; community / mental wellness; diabetes; early childhood education; personal care; pharmacy technician), as identified in the regional training work plan.

Regions have a key role as being responsible for identifying regional training requirements and priorities feeding into the development and implementation of regional training work plans. Subsequently, regions support regional program staff and/or First Nation/Inuit organizations to deliver funding and implement approved training projects.

Adapted CurriculumFootnote 19 - Non-permanent funding is provided to colleges, universities and First Nation institutions to work at the adaptation of nursing and medicine curriculum to prepare non-Aboriginal students for a culturally appropriate practice when intervening with First Nations and Inuit individuals.

Health Human Resources plans - Projects aimed at developing tools and other material for use by regions and communities in health human resources planning are funded through the First Nation and Inuit Health Human Resources program. These tools are then expected to be integrated in regional processes when working with partners to identify health human resources needs and priorities.

It should also be noted that the ten provincial/territorial organizations and four land claim organizations are receiving First Nation and Inuit Health Human Resources program funding annually to provide them with financial support so that they can have the capacity to engage/participate in health human resources planning. Contribution agreements are signed with these organizations following approval of their work plan.

Knowledge Translation Strategy - A knowledge strategy was identified as necessary to improve communications, particularly as it pertains to project and outcome reporting as well as best practices with partners and stakeholders.

Immediate Outcomes

Increased enrolment in health programs - The 'career fairs' and 'post-secondary education initiatives' support those interested with various supports aimed at increasing the likelihood to be admitted and/or complete health studies. The bursaries and scholarships is another measure used to mitigate the financial barrier to access health studies. Enrolment in adapted curricula, which are aimed at improving cultural competency of health professionals, is also expected to increase enrolment.

The training offered and provided to community-based workers is also expected to be a measure removing another barrier and thus facilitating community-based workers access to training allowing them to 'upgrade' their skills to provincial/territorial standards. In this context, community-based workers training projects are also anticipated to result in increased enrolment in eligible certifications (which are assumed to be captured under 'health programs').

Increased awareness of best practices in health human resources planning and managementFootnote 20 - Increased awareness is expected to stem from the knowledge translation strategy which was considered necessary to ensure the sharing of health human resources best practices gained through the development of health human resources plans (e.g., implementation of specific tools) and other projects, activities in various streams.

Intermediate Outcomes

Increased number of First Nations/Inuit/Métis technically competent health service providers (including community-based workers and health managers) - This intermediate outcome refers to graduation rates, and thus flows from the immediate outcome of increased enrolment in health programs. It will be important to track/examine the reasons why studies are not completed as well as factors explaining variations between enrolment and graduation as this is likely to provide information as to whether efforts are deployed in relevant areas.

Increased number of health service providers trained for cultural competency - The pre-2008 period focused on the development of framework from which curricula were developed, piloted and evaluated by March 2012. The post-2008 period focused on the implementation of the framework and curricula, leading to an increased number of health professionals to improve their cultural competency.

Long Term Outcomes

In the long-term, the First Nation and Inuit Health Human Resources program is expected to contribute to an increase in the proportion of Aboriginal health professionals delivering health services to Aboriginal individuals, families and communities. This outcome is expected to be a long term endeavor given it potentially requires the knowledge and control of environmental influences such as turnover/retention of health staff in communities and workforce distribution.

Page details

Date modified: