First Nations and Inuit Mental Health and Addictions - Cluster Evaluation 2005/06-2009/10

Final Report

September 2012

Acknowledgement

We would like to thank the community representatives and all key informants for their participation in this evaluation, as well as the members of the Evaluation Steering Committee and the Evaluation Advisory Committee.

Table of Contents

• Management Response and Action Plan 2012/2013
• Executive Summary
• 1.0 Evaluation Purpose
• 2.0 Mental Health and Addictions Cluster Description
  • 2.1 Mental Health and Addictions Cluster Profile
  • 2.2 Program Logic Model and Narrative
  • 2.3 Program Authority and Resources
• 3.0 Evaluation Description
  • 3.1 Evaluation Scope
  • 3.2 Evaluation Approach
  • 3.3 Evaluation Design
    • 3.3.1 Community Sample
    • 3.3.2 Evaluation Participants
  • 3.4 Data Collection Methods
  • 3.5 Limitations for the Conduct of the Evaluation and Mitigation Strategies
• 4.0 Key Findings
  • 4.1 Relevance
    • 4.1.1 Core Issue #1: Continued Need for the Program
    • 4.1.2 Core Issue #2: Alignment with Government Priorities
    • 4.1.3 Core Issue #3: Alignment with Federal Roles and Responsibilities
  • 4.2 Performance
    • 4.2.1 Core Issue #4: Achievement of Immediate Outcomes
    • 4.2.2 Core Issue #4: Achievement of Intermediate Outcomes
    • 4.2.3 Core Issue #4: Achievement of Long-Term Outcomes
    • 4.2.4 Core Issue #5: Assessment of Economy and Efficiency
• 5.0 Conclusions
  • 5.1 Relevance
  • 5.2 Performance
• 6.0 Recommendations
• Appendix A - Community Sampling Strategy
• Appendix B - TECHNICAL ANNEX
• References

Management Response and Action Plan 2012/2013

Acronyms

AFN

Assembly of First Nations

BF

Brighter Futures

BHC

Building Healthy Communities

BSFO

Branch Senior Financial Officers

CBRT

Community Based Reporting Template

CSP

Cultural Support Providers

DPR

Departmental Performance Report

HC

Health Canada

HQ

Headquarters

IRS RHSP

Indian Residential Schools Resolution Health Support Program

INAC

Indian Northern Affairs Canada

ITK

Inuit Tapiriit Kanatami

FNIHB

First Nations and Inuit Health Branch

F/P/T

Federal/Provincial/Territorial

MHA

Mental Health and Addictions

MWAC SAP

Mental Wellness Advisory Committee's Strategic Action Plan

NADS

National Anti-drug Strategy

NAO

National Aboriginal Organizations

NAHO

National Aboriginal Health Organization

NAYSPS

National Aboriginal Youth Suicide Prevention Strategy

NGO

Non-governmental Organizations

NNADAP

National Native Alcohol and Drug Abuse Program

NYSAP

National Youth Solvent Abuse Program

PAA

Program Alignment Architecture

RHS

Regional Health Survey

RHSW

Resolution Health Support Worker

RMAF

Results-based Management and Accountability Framework

RNAs

Regional Needs Assessments

RPP

Report on Plans and Priorities

SAIS

Substance Abuse Information System

TB Sub

Treasury Board Submission

YSAC

Youth Solvent Abuse Committee

Executive Summary

This evaluation covers six programs funded under the Mental Health and Addictions (MHA) Cluster, for the period from April 1, 2005 to March 31, 2010. The evaluation was undertaken in fulfillment of the requirements of the Financial Administration Act and the Treasury Board of Canada's Policy on Evaluation (2009).

Evaluation Purpose, Scope and Design

The purpose of the MHA Cluster^{Footnote 1} evaluation was to assess the relevance and performance of the MHA Cluster. The evaluation was also designed to identify gaps, barriers to success and success stories related to the MHA Cluster at the community level to help explain progress made towards the MHA outcomes. This is the first Cluster level evaluation for the MHA Cluster.

In total, more than 165 people in First Nations and Inuit communities and organizations, across Canada, including the three territories, participated in the evaluation through interviews, surveys and focus groups with community leaders, Elders, clients, health directors, community workers, territorial employees and treatment centre directors. Community visits took place with fifteen First Nations and two Inuit communities. The Inuit Tapiriit Kanatami (ITK) and the Assembly of First Nations (AFN) were on the advisory committee and provided comments on earlier drafts of this report.

The MHA Cluster Description

The MHA Cluster funds and supports community-based programming and services that aim to reduce risk factors, promote protective factors, and improve health outcomes associated with the mental wellness of First Nations and Inuit. The Cluster aims to contribute to the improved health status of First Nations and Inuit individuals, families and communities through a strengthened continuum of mental health and addictions programs and services. Strengthening the continuum of services means improving access to a range of appropriate and effective services that are offered by the First Nations and Inuit Health Branch (FNIHB) of Health Canada and delivered by communities as well as those offered by provinces and territories.

The MHA Cluster is comprised of the following programs:

• Brighter Futures;
• Building Healthy Communities;
• Indian Residential Schools Resolution Health Support Program;
• National Aboriginal Youth Suicide Prevention Strategy;
• National Native Alcohol and Drug Abuse Program - Residential Treatment and Community-Based; and
• National Youth Solvent Abuse Program.

Evaluation Conclusions and Recommendations

Overall, the evaluation found that the MHA Cluster is a highly relevant program area that has demonstrated progress towards its expected outcomes and has identified specific actions that could assist with further addressing mental health and addiction needs in First Nations and Inuit communities across Canada.

Relevance of the MHA Cluster

Continued Need for MHA Programs and Services

The evaluation identified a demonstrable need for mental health and addictions programs in First Nations and Inuit communities. There is a greater prevalence of mental health issues and substance abuse in First Nations and Inuit communities, compared to the total Canadian population. Thoroughly addressing these needs is complex, involving mental health and addictions programs, as well as improvements to social determinants of health.

Alignment with Government Priorities

The MHA Cluster is aligned with Government of Canada priorities. These include prioritizing the health of Canadians and supporting communities to develop their own health solutions, strengthening health related programming in First Nations and Inuit communities and improving First Nations and Inuit access to health services through the integration of federal and provincial/territorial programs.

Alignment with Federal Roles and Responsibilities

The MHA Cluster programs are aligned with the federal role and responsibilities to First Nations and Inuit. Consistent with the Government of Canada's policy  to improve First Nations and Inuit health, the MHA Cluster is designed to encourage both individuals and communities to become actively engaged in developing and delivering programs and services which reflect the needs of each unique community.

Performance of MHA Cluster

Immediate Outcome - Increased and Improved Collaboration and Networking

The evaluation concluded that collaboration and networking among MHA programs at the community level and with provincial or territorial mental health and addictions services have increased over the evaluation period and have enhanced the planning and delivery of MHA programs and services. However, the relationships with provincial and territorial governments are still relatively new and require continued focus and attention to promote greater harmonization between federal and provincial/territorial services.

Immediate Outcome - Improved Continuum of MHA Programs and Services in First Nations and Inuit Communities

There has been some improvement in the continuum (i.e., improvement in access and range of services) of programs and services delivered in First Nations and Inuit communities with no significant program overlap, but gaps in MHA programs and services exist specifically in the following areas: access to aftercare services, detoxification and provincial services (e.g., psychiatry, acute care).

Immediate Outcome - Increased Participation of First Nations and Inuit Individuals, Families and Communities in MHA Programs and Services

Participation in MHA Cluster programs and services has increased over the years covered by this evaluation. This may be in part due to an increased range of MHA programs and servicesfundedover the evaluation period (i.e., over the evaluation period more programs, covering a broader range of services, were funded and delivered, including the National Aboriginal Youth Suicide Prevention Strategy and the Indian Residential Schools Resolution Health Support Program). The evaluation identified opportunities to enhance access to programs by addressing the continued stigma attached to seeking mental health and addiction support and ensuring the confidentiality of services particularly in smaller communities. Although the evaluation revealed barriers to participation in programs, not all communities share the same issues or require the same solutions. The evaluation found that any solutions to enhance access need to use a gender-based analysis and address the need for appropriate programming for women who face particular difficulties accessing services (e.g., who have experienced abuse), as well as ensure access to culturally appropriate programs.

Immediate Outcome - Increased Awareness of Healthy Behaviours

While data on the awareness of healthy behaviours is limited, the evaluation found that the awareness of healthy behaviours among program participants had increased, and this increase was often attributed to attendance at MHA programming. Additional data regarding the awareness of healthy behaviours may help to fully capture progress on this outcome in the future.

Intermediate Outcome - Increased Practice of Healthy Behaviours

While data on the practice of healthy behaviours is limited, the evaluation found that the practice of healthy behaviours by program participants had increased, and this increase was often attributed to attendance at MHA programming. Additional data about the uptake of healthy behaviours by participants is required to capture progress on this outcome in a comprehensive way that includes input from the program participants themselves.

Intermediate Outcome - Increased First Nations and Inuit Ownership and Capacity to Combat Substance Abuse, Suicide and Other Mental Health Issues

First Nations and Inuit ownership and capacity to combat substance abuse, suicide and other mental health issues has increased over the evaluation period.As community ownership to combat mental health and addictions issues increases, communities are able to tailor programming to meet their unique needs. While the amount of control divested to communities has increased, the evaluation found that communities would like to see continued attention on the development of equitable partnerships between FNIHB and communities to enhance the federal government's understanding of community needs. As well, all parties would like to increase the use of tripartite partnerships to further increase the role of First Nations and Inuit communities in program design, development and delivery. The evaluation noted that First Nations and Inuit communities with more control over their programs, facilities, and institutions tend to have lower incidences of suicide and substance addictions. The evaluation found that MHA programs and services draw on the cultural practices of communities. The incorporation of cultural practices into mental health and addictions programming is critical to the success of mental health and addictions programming in First Nations and Inuit populations and is evidence of an increase in community ownership of MHA program delivery.

While training has increased and contributed to increased capacity to deliver programs at the community level, there is a need for continued training to address the identified gaps (e.g., cultural awareness training for non-Aboriginal service providers and management training) and address the high rate of staff turnover.

Intermediate Outcome - Improved Access to Quality Well-coordinated Programs and Services for First Nations and Inuit Individuals, Families and Communities

The evaluation found that access to MHA programs had improved and that MHA programs are of high quality. Best practices and evidence-based approaches are followed in the design of MHA programs and examples were found showing how best practices were incorporated into programming delivery. However, the evaluation found that there remains an ongoing need to fully address: (1) staff retention and training; (2) improved access to specialized and culturally sensitive services within the community (e.g., psychiatric services); and (3) the existing gaps (i.e., aftercare services) in the continuum of services available to First Nations and Inuit communities.

Final Outcome - MHA Contributes to the Improved Health Status of First Nations and Inuit Individuals, Families and Communities through a Strengthened Continuum of Mental Health and Addictions Programs and Services

The evaluation revealed evidence to suggest that the MHA programs are contributing to improved health among First Nations and Inuit. However, there was limited data from follow-up or pre-post studies to fully assess the extent of the contribution MHA programs are making to these improvements in health outcomes.

Improving the Cluster's ability to address First Nations and Inuit mental health and addictions needs requires access to a full continuum of provincial, territorial and federal mental wellness programs and services as well as actions to increase the effectiveness of community-based programming, such as strengthened recruitment and retention of culturally competent health professionals and training for community workers.

Assessment of Economy and Efficiency

The evaluation found some improvement in the efficiency of the MHA programs and services over the evaluation period however, due to the limited financial data (i.e., not having expenditure allocations by activity type or object costing or expenditure records by program components and/or sub-components), a robust assessment of the economy and efficiency of the MHA Cluster was not possible. Improved financial data on program expenditures linked to program activities and outcomes would support a more complete assessment of the program's economy and efficiency in the future.

Performance Measurement in Support of Planning and Reporting

Many of the challenges encountered in the design and conduct of this evaluation can be addressed through an improved and implemented Performance Measurement Strategy. Furthermore, improved availability of and access to high quality data (specifically information on program participation, the awareness of and uptake of healthy behaviours by participants, changes to mental health, addictions and overall health) may help to further support evidence-based decision-making in policy, expenditure management and program improvements for the MHA Cluster in the future.

Recommendations

Recommendation 1 - Strengthen the continuum of mental health and addictions services available to First Nations and Inuit individuals, families and communities.

There are gaps in the continuum of mental health and addictions care. To address these gaps, it is recommended that the Program^{Footnote 2} identify and prioritise, with First Nations and Inuit, gaps in mental health and addictions program and service delivery, and develop, implement and monitor a strategy that uses a phased approach to ensure gaps are addressed in the short-, medium-, and long-term. To strengthen the continuum, the program area would need to:

• ensure that the continuum incorporates and provides access to the best practices of traditional, cultural and mainstream treatment approaches;^{Footnote 3}
• identify and work towards closing the gaps in aftercare;
• identify and work towards closing the gaps in access to detoxification and other provincially funded services; and
• develop, implement and monitor a strategy to improve the integration of Mental Wellness programs and services at the community level.

Addressing gaps in the services available and improving access requires extensive partnership building between jurisdictions (i.e., provinces, communities and FNIHB), and sectors (e.g., housing, income support, and education), and is therefore, also supported by the following recommendation on partnership development.

Recommendation 2 - Build and maintain partnerships between FNIHB, First Nations and Inuit communities, National Aboriginal Organizations (NAOs) and provincial/territorial health services.

There is a need for continued focus and attention to be placed on partnership development and collaborative relationships particularly with provincial and territorial governments. The Program should further promote and foster partnership development with relevant First Nations and Inuit stakeholders on initiatives like the Mental Wellness Advisory Committee Strategic Action Plan, Alianait, tripartite agreements, memoranda of understanding, and joint committees like those that currently exist in British Columbia, Saskatchewan and Ontario, respectively.

Recommendation 3 - Address the barriers to accessing services.

The Program should address the key barriers to accessing MHA programs and services as identified in the evaluation. This includes addressing issues of privacy and confidentiality  through the development of guidelines and training  for community staff; addressing the stigma associated with mental illness and addictions; and ensuring culturally-safe and culturally competent programming, by recognizing that the solutions to increasing access to the MHA programs needs to be community driven. The MHA Cluster can then provide resources and support to improve access, so that the communities are able to identify their unique barriers and strengths and address these through appropriate community-based solutions.

Recommendation 4 - Increase training available to mental health and addictions workers.

The Program should, with First Nations and Inuit partners, offer training for mental health and addictions workers. Increased training may help to reduce staff turnover. In addition, the uptake of best practices and evidence-based approaches in the delivery of MHA programs can be improved through training, as workers become exposed to new and evidence-based approaches through the training that they receive. The Program should offer options for training that reflect community needs.

Recommendation 5 - Implement an improved performance measurement strategy.

The Program should work more closely with internal Health Canada partners, community partners and National Aboriginal Organizations to:

• improve the quality and consistency of data collected by the Program in all regions;
• collect information on program participation (numbers participating and frequency of participation), the uptake of healthy behaviours by participants, and changes to mental health, addictions, and overall health; and
• increase timeliness of data availability to have national level aggregated data available within two years of its collection.

In addition, the Program should work with internal Health Canada partners to collect better financial data on program expenditures, which includes the planned and actual expenditures by program and the tracking of reallocations between programs.

An improved and implemented Performance Measurement Strategy would assist with assessing levels of participation in programs and the impact of the programming. This information would also be valuable for planning and reporting purposes and for assisting in decision making at all levels: community, regional, and national.

1.0 Evaluation Purpose

The purpose of the evaluation was to assess the relevance and performance of the Mental Health and Addictions (MHA) Cluster^{Footnote 4} for the period of 2005/06 - 2009/10. The evaluation was also designed to identify gaps, barriers to success and success stories related to the MHA Cluster at the community level to help explain progress made towards the MHA outcomes.

The evaluation is required by the Financial Administration Act and the Treasury Board of Canada's Policy on Evaluation (2009).

2.0 Mental Health and Addictions Cluster Description

The MHA Cluster funds and supports community-based programming and services that aim to reduce risk factors, promote protective factors and improve health outcomes associated with the mental wellness of First Nations and Inuit. The Cluster contributes to the improved health status of First Nations and Inuit individuals, families and communities by strengthening the continuum of mental health and addictions programs and services that these populations have access to (RMAF, 2007).

The First Nations and Inuit Health Branch (FNIHB) oversees the MHA Cluster, one of three clusters in the Community Programs Contribution Program Authority. The MHA Cluster is comprised of the following programs:

• Brighter Futures;
• Building Healthy Communities;
• Indian Residential Schools Resolution Health Support Program;
• National Aboriginal Youth Suicide Prevention Strategy;
• National Native Alcohol and Drug Abuse Program - Residential Treatment and Community-Based; and
• National Youth Solvent Abuse Program.

2.1  Mental Health and Addictions Cluster Profile

The MHA Cluster programs are described below:

• Brighter Futures (BF) program improves the quality of, and access to, culturally appropriate, holistic and community-directed mental health, child development, and injury prevention services at the community level.
• Building Healthy Communities (BHC) program assists communities in preparing for and managing mental health crises such as suicide and substance abuse, and addresses community capacity-building by training caregivers and community members to deliver programs and services within their own communities.
• Indian Residential Schools Resolution Health Support Program (IRS RHSP) provides mental health and emotional supports to eligible former Indian Residential School students and their families before, during and after their participation in Settlement Agreement processes, including: Common Experience Payments, the Independent Assessment Process, Truth and Reconciliation Commission events and Commemoration activities.
• The National Aboriginal Youth Suicide Prevention Strategy (NAYSPS) increases protective factors (e.g., youth leadership) and decreases risk factors (e.g., loss of traditional culture) concerning Aboriginal youth suicide. This includes increasing community capacity to deal with the challenge of youth suicide, enhancing community understanding of effective suicide prevention strategies and supporting communities to reach youth at risk and intervene in times of crisis. NAYSPS targets resources that support a range of community-based solutions and activities which contribute to improved mental health and wellness among Aboriginal youth, families and communities.
• The National Native Alcohol and Drug Abuse Program (NNADAP) is a network of services with two components: (1) Community-based provides prevention, intervention, aftercare, and follow-up services in more than 500 First Nations and Inuit communities across Canada, and (2) Residential treatment is a national network of 49 treatment centres operated by First Nations organizations and/or communities that provide culturally-appropriate in-patient and out-patient treatment services for alcohol and other forms of substance abuse.
• The National Youth Solvent Abuse Program (NYSAP) is a community-based prevention, intervention, aftercare and in-patient treatment program that targets First Nations and Inuit youth who are addicted to or at risk of inhaling solvents. This includes a network of nine national youth solvent abuse centres and community supports. NYSAP treatment centres provide culturally appropriate in-patient and out-patient treatment services to First Nations and Inuit youth. The treatment centres target youth between the ages of 12 to 25 years of age.

2.2  Program Logic Model and Narrative

FNIHB's overarching strategic outcome for its suite of programs, including those delivered through the MHA Cluster, between 2005/06-2009/10, was to improve health outcomes for First Nations and Inuit and reduce health inequalities between First Nations and Inuit and other Canadians (RMAF, 2007). Together with First Nations and Inuit, FNIHB (including its Regional Offices), delivers public health and community health programs that support this strategic outcome.

Within this context, the primary objective of the MHA Cluster is to deliver a continuum^{Footnote 5} of MHA programs and services. MHA Cluster programs and services are delivered at the national, regional, and community levels, and are managed by FNIHB or in partnership with First Nations and Inuit. To achieve this objective and the expected outcomes stated on the MHA Logic Model, the MHA Cluster carries out the following activities:

• collaborate with First Nations and Inuit, F/P/T authorities and organizations.
• deliver mental health and addictions programs and services.
• lead, innovate and incorporate evidence-based practices in mental health and addictions programs.
• educate and create awareness of mental health and addictions-free lifestyles.
• build capacity: develop skills of health care, addictions and community workers.

The expected immediate, intermediate and final outcomes for the MHA Cluster are:

Immediate Outcomes

• Increased and improved collaboration and networking.
• Improved continuum of programs and services in First Nations and Inuit communities.
• Increased participation of First Nations and Inuit individuals, families and communities in programs and services.
• Increased awareness of healthy behaviours.

Intermediate Outcomes

• Increased practice of healthy behaviours.
• Increased First Nations and Inuit community ownership and capacity to combat substance abuse, suicide and other mental health issues.
• Improved access to quality, well-coordinated programs and services for First Nations and Inuit individuals, families and communities.

Final Outcome

• Contribution of the MHA Cluster to improved health status of First Nations and Inuit individuals, families and communities through a strengthened continuum of mental health and addictions programs and services.

The connection between these activity areas and the expected outcomes is depicted in the logic model below (Figure 2-1).

2.3  Program Authority and Resources

The MHA Cluster falls under Program Authority 3.1 Primary Health Care, 3.1.1 Health Promotion and Disease Prevention, 3.1.1.2 Mental Wellness (name of cluster area changed in 2011) of the Health Canada Program Alignment Architecture (PAA).

The overall financial data for the years 2005/06 through 2009/10 are presented below (Table 2-1).  It should be noted that this information includes corporate costs (i.e., as well as program delivery costs.

The above table illustrates total investments of $894.3M over the evaluation period. Between 2005/06 and 2009/10, FNIHB Headquarters primarily funded FNIHB regional offices that, in turn, funded First Nations and Inuit communities, regional organizations and Territories using various funding models.^{Footnote 6} In addition, some FNIHB funds were used to support targeted projects with a national scope designed to examine innovations for possible application to national programming.

3.0 Evaluation Description

3.1 Evaluation Scope

As per the 2007 Results-based Management and Accountability Framework (RMAF) for the MHA Cluster, the scope of this evaluation was to assess the relevance and performance of MHA Cluster programs and services. All of the MHA Cluster programs listed in Section 2.1 were included in the scope of this evaluation for the period of 2005/06 - 2009/10.

In total, more than 165 people in First Nations and Inuit communities and organizations, across Canada, including the three territories, participated in the evaluation through interviews, surveys and focus groups with community leaders, Elders, clients, health directors, community workers, territorial employees and treatment centre directors. Community visits took place with fifteen First Nations and two Inuit communities.

Five core issues are outlined in Treasury Board of Canada's Policy on Evaluation (2009). The specific evaluation questions used in this evaluation were based on these core issues) see Chart 3-1).

3.2 Evaluation Approach

A goal-based evaluation approach was used for the conduct of the MHA evaluation^{Footnote 7} to assess the progress the MHA Cluster has made towards the achievement of the expected outcomes, whether there were any unintended consequences, what the learnings were and how to improve.

The evaluation also included a participatory approach by actively involving internal and external stakeholders in the development of the draft evaluation tools. Key stakeholders who contributed to the evaluation were:

• Assembly of First Nations;
• Inuit Tapiriit Kanatami; and
• FNIHB Regions.

Their participation was facilitated through their membership in an Evaluation Advisory Committee and/or an Evaluation Working Group.

3.3 Evaluation Design

The Treasury Board's Policy on Evaluation (2009) guided the evaluation design and data collection methods. A non-experimental design was used, which means that there was neither random assignment of sample groups for inclusion in the evaluation nor a control group to compare the sample to. As a non-experimental design, the evaluation relies on correlation to demonstrate effect, and does not imply causation. As such, the evaluation was designed to demonstrate the likely contributions of MHA Cluster programs to the expected outcomes, rather than demonstrate direct causal links between MHA programs and outcomes.

3.3.1 Community Sample

Twenty-seven communities south of the 60th parallel were initially selected for community visits as part of the evaluation, which included 25 First Nations communities and two Inuit communities. The sampling strategy took into account seven major criteria which were recommended by the Evaluation Working Group and approved by the Evaluation Advisory Committee:

• region;
• community population size;
• isolation factor;
• funding type;
• number of programs delivered in community;
• amount of funding received by community; and
• how long communities have been receiving MHA funding.

Although the MHA programs are national in scope, there are differences between programming in the three territories compared to the rest of Canada. For the most part, Territorial Governments and Self Governing First Nations in Yukon have responsibility for primary health care delivery with the support of funding from FNIHB and other sources. In terms of health promotion and prevention initiatives (e.g., MHA programs and services), the FNIHB northern region works closely with territorial governments in Nunavut and the North West Territories who deliver all eligible FNIHB community-based programs. The territorial governments would complete their own program level evaluations. Because of the mixed funding sources at the community level and the responsibility for program management resting with territorial governments, different types of key informants were sampled to speak about programs north of the 60th parallel and community visits were not completed. For additional detail about the sampling strategy used for this evaluation, refer to Appendix A.

3.3.2 Evaluation Participants

Health Canada regional staff, community leaders and community health staff from each site south of the 60th parallel identified potential interviewees (see 3.5.3) at the community level. Interviewees were chosen on the basis of their knowledge of and experience with the MHA Cluster programs and services delivered in the community. There was no requirement that workers had to have been working in the community for a specific length of time. Information was collected on the length of time the respondent had been working in their position and this information was considered during data analysis.

Health Canada staff and Territorial Assistant Deputy Ministers identified 42 people in 25 organizations north of the 60th parallel as potential key informants, representing both territorial and community level perspectives. Key informants were selected who would be able to provide informed feedback about MHA programming and would likely be available during the data collection time-frame.

Interviewees at the national and regional level of FNIHB were recommended by the Evaluation Working Group and the Regions, based on program and other experience, such as policy experience in MHA programs. National Aboriginal Organizations nominated their representatives for key informant interviews.

3.4 Data Collection Methods

External consultants collected and analyzed data from multiple sources.

Sources of information used in this evaluation include:

• key internal documents and data including:
  • working group minutes, special studies, performance reports, RMAF, annual reports, strategic plans, relevant databases, and program files;
• key informant interviews with the following groups of people:
  • FNIHB staff including regional managers, national MHA managers and FNIHB evaluation staff;
  • National Aboriginal Organizations (i.e., the Assembly of First Nations and Inuit Tapiriit Kanatami);
  • treatment centre directors;
  • Territorial representatives;
  • Indian Residential School service providers;
  • Tribal Councils;
  • community leaders including Chiefs, Elders and/or Council Members;
  • community workers and health staff including health directors
• surveys from program participants;
• focus groups in Inuit communities; and
• literature review.

For additional information about the data collection methods used for this evaluation refer to Appendix B.

3.5 Limitations for the Conduct of the Evaluation and Mitigation Strategies

As with any evaluation, challenges were encountered. As such, mitigation strategies were used to ensure that the data collected produced a credible evaluation report with evidence-based conclusions and recommendations.

• Inability to determine the extent of change due to lack of baseline data - Many of the anticipated outcomes imply an incremental change from the baseline (e.g., increased, improved). However, there were no baseline measures for these outcomes established in 2005 when the Cluster was initially developed and limited performance data was available for the evaluation to quantify the extent of the increase or improvement.

  Mitigation Strategy

  • Retrospective perceptions of key informant interviewees were collected using qualitative data collection methods.
• Inability to have communities validate the evaluation findings - Due to the evaluation timelines and funding limitations, the evaluation findings were not validated with the communities visited.  Communities south of the 60th parallel were provided with two to three page summaries of the information collected at the community visit; however, the project timelines did not allow communities to respond to the summary prior to the writing of the final report.

  Mitigation Strategy

  • While it was not possible for communities to validate the preliminary findings, Health Canada appreciates the efforts of the Assembly of First Nations and the Inuit Tapiriit Kanatami staff that reviewed the preliminary findings and subsequently provided guidance concerning how the evaluation findings corresponded to their experience.
• Limited perspective from program participants - Despite efforts to include program participants in the evaluation, data from program participants is limited because of: (1) the lack of program participant data in First Nations communities, and (2) low participation in the program participant survey. Additional data from the perspective of the program participants regarding the awareness and practice of healthy behaviours as well as overall improvements to their health may help to fully capture progress on these outcomes in the future.

  Mitigation Strategy

  • Given the evaluation used multiple methods to collect data, data was collected on all outcomes from various audiences and triangulated supplying credible findings, conclusions and recommendations for the evaluation.
• Lack of response to community worker and community health director surveys - The community worker and community health director surveys were designed to complement the information collected through the key informant interviews and had different questions. The surveys were available for completion by telephone, online, and through paper and pencil (mail-back survey). Despite the assistance of community health staff and the Assembly of First Nations, there was no uptake on the community health director and community worker surveys.

  Mitigation Strategy

  • Community health directors and community workers were included in the community visits (south of the 60th parallel) and completed in-depth interviews supplying sufficient information to support evidence based conclusions and recommendations.

Many of these limitations can be addressed in the future through an improved and implemented Performance Measurement Strategy. For example, enhanced quantitative data sources are required to capture the links between program outputs and expected outcomes to provide additional evidence to support the conclusions about the contribution the MHA Cluster is making towards these outcomes. Consistent data collection across the HC regions may facilitate a roll up of information to provide a national view of participation in programs and range of programs. Overall, improved availability of and access to high quality data (specifically information on program participation, the awareness of and uptake of healthy behaviours by participants, changes to mental health, addictions and overall health) may help to further support evidence-based decision making in policy, expenditure management and program improvements for the MHA Cluster in the future.

4.0 Key Findings

4.1 Relevance

4.1.1 Core Issue #1: Continued Need for the Program

• Evaluation findings indicate a demonstrable need for MHA Cluster programs and services in First Nations and Inuit communities across Canada.  The prevalence of mental illnesses and substance abuse (i.e., suicide ideation and death by suicide, self-reported depression and major depressive episode, post-traumatic stress disorder and solvent abuse and binge drinking) are much greater in First Nations and Inuit communities, compared to the total Canadian population.  A multitude of factors exist in First Nations and Inuit communities that may increase the likelihood of experiencing mental health issues and addictions and also present challenges to thoroughly addressing these needs. Most notably, historical trauma, compulsory attendance at Indian Residential Schools and poor living conditions which have been linked to a greater prevalence of mental illness and addictions among First Nations and Inuit.

"Our problems can't be solved by 'programs'. Our people need jobs. They need to be able to meet their basic needs. The high cost of living here makes it very difficult to survive."
- Inuit Community Member

Demonstrable Need

It is often difficult to obtain accurate estimates of the prevalence of mental illnesses and addictions among Aboriginal individuals for the following reasons: (1) mental illness is highly likely to co-occur with physical ailments including cardiovascular disease, stroke, and diabetes (Bombay et al., 2009); (2) First Nation and Inuit individuals are more likely to live in rural, remote, and isolated areas than non-Aboriginal Canadians, and therefore have less access to primary health care services where initial screening, diagnosis, and treatment could take place (Statistics Canada, 2011); and (3) mental illnesses and addictions often co-occur. Despite these limitations, the literature and document review indicated that the prevalence of the following mental illnesses and substance abuse conditions are much greater in First Nations and Inuit communities, compared to the total Canadian population, demonstrating a need for the MHA Cluster:

• suicide ideation and death by suicide;
• self-reported depression and major depressive episode;
• post-traumatic stress disorder;
• solvent abuse and binge drinking; and
• concurrent mental health and addictions diagnoses.

Suicide

While suicide rates vary widely across First Nations and Inuit communities, suicide and self-injury are the leading causes of death in First Nations youth and adults between the ages of 10 and 44 years (Health Canada, 2006), and the second most frequent cause of death among residents of Inuit Nunangat (Inuit Tapiriit Kanatami, 2010).

Compared to the national average, First Nations youth (aged 15-24 years) are five to seven times more likely to die by suicide. First Nations women and men also die by suicide at a higher rate than the national average, but men are at a higher risk in general (Health Canada, 2007a). Inuit suicides occur at a rate approximately eleven times higher than the national average and they tend to do so at younger ages than do Canadians in general (Ajunniginiq Centre, 2006; Inuit Tapiriit Kanatami, 2010). While Aboriginal men, women, and youth are all at risk for suicide; it disproportionately affects adult men and youth.

Mental Health Needs

Certain mental health conditions appear to be more prevalent than others in Aboriginal populations. Recent evidence indicates that depression is highly prevalent among First Nations individuals living on and off reserve (Christian, 2009; Statistics Canada, 2006). Approximately 30% of on-reserve First Nations individuals report feeling sad, blue, or depressed for two weeks or more, an indicator of potentially clinical levels of depression (Christian, 2009). Post-traumatic stress disorder (PTSD) also may be more common among Aboriginals than non-Aboriginals, as it has been associated with Indian Residential School experiences (Corrado & Cohen, 2003).

Addictions Needs

Addictions to a variety of substances are more common among Aboriginal populations than non-Aboriginals in Canada in general. In particular, alcohol abuse is more prevalent within Aboriginal communities. Patterns of alcohol use differ between First Nations and the general population. On one hand, First Nations individuals are more likely to abstain from using alcohol than the general population (34% versus 21%; Bombay et al., 2009) and are less likely to drink alcohol regularly (18% of First Nations versus 44% of the general population drink alcohol on a weekly/daily basis; Dell & Lyons, 2007). On the other hand, First Nations individuals are more than twice as likely to binge drink once or more per week (16% versus 6%) (Bombay et al., 2009). Aboriginal individuals also start drinking alcohol earlier than do non-Aboriginal Canadians. In the Alberta Youth Experience Survey, 35% of Aboriginal youth reported signs of alcohol abuse, compared to 12% of non-Aboriginal youth (Christian, 2009). Reasons youth gave for drinking included: to have fun, to be sociable, to temporarily escape reality, to find extra courage, to relieve stress or take a break from school, "because I can," and/or because of difficult community situations and negative adult role models (Christian, 2009). As well, substance abuse issues have been linked to other social problems and traumatic events, such as sexual abuse, poverty, and HIV status (Mehrabadi et al., 2008).

Evidence suggests that abuse of other substances is also more common in Aboriginal populations than in the general population. Aboriginal individuals are more than twice as likely to use illicit drugs than the general population (7% compared to 3%) and more than twice as likely to die as a result of illicit drug use (7 per 100,000 compared to 3 per 100,000 deaths; Dell & Lyons, 2007).

Concurrent Mental Health and Addictions

The Centre for Addictions and Mental Health estimates that, depending on where the sample is gathered, 20% to 80% of people (in the general population) with a mental illness also have substance abuse issues (Christian, 2009). In Aboriginal populations, depression and adjustment disorders often co-occur with alcohol abuse. For example, 56% of Aboriginal people in the United States of America with alcohol addictions also were diagnosed with psychiatric disorders in one study (Robin et al., 2001). Further, approximately 34% of Aboriginal individuals with PTSD also have substance addictions (Christian, 2009).

Finally, First Nations youth who use alcohol and illicit drugs are believed to experience more frequent suicidal thoughts (Christian, 2009). The administrative data collected in the YSAC (Youth Solvent Abuse Committee) database confirms that suicidal ideation and action are common among Aboriginal youth receiving treatment in MHA-funded solvent addiction programs. Nearly half (46%) of youth receiving treatment have spoken or written about killing themselves, and a third (34%) have attempted it.

Thus, concurrent mental illness and addictions diagnoses are relatively common. They also tend to make treatment and recovery more difficult (National Institute of Mental Health, 2011). Services may be most effective if they address mental health and addictions together.

Addressing Mental Health and Addiction Needs

According to the literature review and most key informants, fully addressing the mental health and addiction needs in First Nations and Inuit communities is complex due to the following: (1) historical trauma including discrimination and marginalization (Bombay et al., 2009); (2) compulsory attendance at Indian Residential Schools for several generations of children (Aboriginal Healing Foundation, 2006; Barron, 2009; Canadian Institutes of Health Research, 2009; Health Canada, 2007b; Loppie-Reading & Wein, 2009; Mussell, Cardiff, &White, 2004; White & Jodin, 2007); and (3) poor living conditions including substandard housing, community isolation, and poverty (Barron, 2009; Bombay et al., 2009; First Nation Health Council, 2007; Minore & Katt, 2007; Ogborne et al., 2005; White & Jodoin, 2007).

Nonetheless, improvements have been made in the mental health and addictions of First Nations and Inuit individuals in recent years. The Regional Health Survey data showed that between 2002/03 and 2008/10 suicide attempts and contemplation decreased among First Nations youth, while for adults, there was a reduction in suicide contemplation only and the proportion who had attempted suicide remained the same (2005, 2010). Key informants both north and south of the 60th parallel agreed that MHA programs are addressing demonstrable mental health and addiction needs.

Responses from key informants (south of the 60th parallel) at the community level indicated they believe the MHA Cluster is more successful at meeting mental health needs than at meeting addictions needs. On the other hand, key informants at the national and regional level reported lower levels of agreement that the MHA programs were meeting needs, and were somewhat more likely to agree that addictions needs were being met, compared to mental health needs.

For the Northern Region, the wide range of programs that were offered suggests that they can be targeted toward individual needs, and can apply different approaches depending upon community needs. When asked whether these activities worked well to address mental health and addictions issues in their community/territory, the overall response was positive, with 87% (n=15) of the key informants north of the 60th parallel answering 'yes' or 'somewhat'.

To improve the Cluster's ability to address First Nations and Inuit mental health and addictions needs, interviewees suggested numerous ways to improve or increase mental health and addictions programs and services. These included: (1) customization of the programs based on the unique needs of the different communities, (2) the integration of First Nations' holistic healing approaches, and (3) more service availability in communities including detoxification services, promotion, prevention and early intervention programs for youth, and prescription drug abuse prevention. There were also a number of suggestions related to training and human resource management such as: (1) training for the community members and workers, with an emphasis on early intervention; (2) recruitment and retention of culturally competent health professionals from outside small communities to address confidentially issues which are aggravated by the stigma associated with mental illness and addictions; (3) more mental health and addictions workers (i.e., increased access to specialized services such as psychologists and psychiatrists); and (4) developing ways to better integrate new staff into communities so that program workers can build the trust and rapport required for community members to seek help.

All lines of evidence indicate a need for MHA programming to address complex mental health and addictions issues present in First Nations and Inuit communities. Fully addressing these needs is challenged due to the myriad contributing factors (i.e., social determinants of health, historical trauma and discrimination including compulsory attendance at Indian Residential Schools for several generations).

4.1.2 Core Issue #2: Alignment with Government Priorities

The MHA Cluster aligns with priorities and goals featured in recent Speeches from the Throne, the United Nations Declaration on the Rights of Indigenous Peoples (to which Canada is a signatory), the International Wharerata Declaration (which Canada supports) and the priorities set by the Government of Canada in Health Canada's Report on Plans and Priorities. Examples of shared priorities include prioritizing the health of Canadians, supporting communities to develop their own health solutions, strengthening First Nations and Inuit programming and improving First Nations and Inuit access to health services through integration of federal and provincial/territorial programs.

Prioritizing the health of Canadians and in particular the Aboriginal population has been highlighted as priorities for the federal government including:

• "Making Canada the best place for families" by prioritizing the health of Canadians and aiding communities to develop their own health solutions (Government of Canada, 2010);
• "Standing up for those who helped build Canada" by building on the promises made in the Indian Residential Schools Apology (Government of Canada, 2010);
• "In November of 2010, the Government of Canada became a signatory to the United Nations Declaration on the Rights of Indigenous Peoples, which professes the right to mental health for Indigenous peoples;^{Footnote 8}
• "Here for hard-working families" by ensuring Canadians have accessible health care (Government of Canada, 2011); and
• In 2010 a Private Members Bill was tabled that recommended developing a federal framework for suicide prevention for Canada in consultation with non-governmental organizations, provinces and territories, and relevant federal departments. At the time of writing this evaluation report, Bill C-300, an Act Respecting a Federal Framework for Suicide Prevention was at first reading in the Senate.

The MHA Cluster supports these priorities by collaborating with and building capacity in First Nations and Inuit organizations and communities to improve mental health and addictions outcomes and prevent mental health issues and substance abuse through effective services and programs; and, addressing First Nations and Inuit individuals' rights to control their health care services. In particular, the MHA Cluster builds on the federal government's apology for Indian Residential Schools and helps fulfill the commitment made by the federal government to provide mental and emotional support to individuals involved in the Residential School Settlement process through the Indian Residential Schools Resolution Health Support Program.

Over the evaluation period, ensuring that First Nations and Inuit individuals have better health outcomes and health equality with other Canadians was a strategic outcome of Health Canada. During this time, the activities conducted under the auspices of this strategic outcome included: providing and strengthening community programs; protecting the health of First Nations and Inuit individuals, families, and communities; and promoting healthy lifestyles, including suicide prevention and mental wellness enhancing activities (Health Canada, 2007, 2008, 2011 a, c). The priority status of mental health and addictions programming for the federal government is also reflected in MHA Cluster funding. Spending on program activities in this area increased by $35 million between 2008/09 and 2009/10 (Health Canada, 2011a). Furthermore, the funding allocated to each MHA Cluster program increased or remained the same every year between 2005/06 and 2009/10 (Health Canada, 2007).

In addition to Health Canada's priorities, the specific priorities of FNIHB are to ensure:

• First Nations and Inuit have access to the same quality and availability of services as the rest of the population living in similar geographic areas;
• stronger capacity for promotion of good health and prevention of illness;
• First Nations and Inuit have access to health services in a seamless way through better integration of federal and provincial/territorial programs;
• First Nations and Inuit have an effective role in the planning and delivery of their health services;
• availability of a public health system in First Nations and Inuit communities; and
• transparent accountabilities and responsibilities for the administration, delivery, measurement, and reporting of health programs.

To support the direction of the above priorities, Health Canada, the Assembly of First Nations and Inuit Tapiriit Kanatami have recently developed two plans (the Mental Wellness Strategic Action Plan and the Alianait), which focus directly on improving mental health and addictions within First Nations and Inuit communities through extensive community involvement and input in establishing and administering mental health and addictions programs.

4.1.3 Core Issue #3: Alignment with Federal Roles and Responsibilities

MHA Cluster programs are aligned with federal roles and responsibilities. Recognizing the policy to improve First Nations and Inuit health, efforts continue to support collaboration between First Nations and Inuit communities and the federal government. Through the creation of the Indian Health Policy (1979) and the establishment of the First Nations and Inuit Health Branch within Health Canada in 1962 (then called the Medical Services Branch), the federal government renewed and redefined its role in supporting improved health and wellness outcomes for First Nations and Inuit.

The Government of Canada recongizes it's role to improve the health and well-being of First Nations communities and individuals. In 1962, in order to provide adequate health services for Aboriginal communities, Health Canada created the Medical Services Branch, which was a merging of Indian Health and Northern Health Services^{Footnote 9} into other federal field services. In 1988, the first Final Transfer Agreement between the Government of Canada and a territory government (Northwest Territories) was signed (Pierre, Pollack, & Fafard). Currently, all territories have signed Final Transfer Agreements (Pierre, Pollack, & Fafard). These agreements devolve responsibility for First Nations and Inuit health to territorial governments.

In 2000, the Medical Services Branch was renamed the First Nations and Inuit Health Branch (FNIHB) (Health Canada, 2011a). This Branch provides health services to First Nations and Inuit populations across Canada through conducting research, and providing funding, staff support, and health care services to First Nations and Inuit communities across Canada.

The MHA Cluster supplies communities with the funds necessary to address needs as identified by the communities, which is consistent with the Government of Canada's policy recognizing the inherent right of self-government under Section 35 of the Constitution Act, 1982, and the benefits of having both individuals and communities actively engaged in developing and delivering programs and services which serve their needs.

All National Aboriginal Organization key informants (N=5) agreed or strongly agreed that MHA programs and services are aligned with the federal roles and responsibilities. While they agreed with this, they also stressed a need to focus on the broader social determinants of health (e.g., poverty, housing) within First Nations and Inuit communities.

The majority (89%, n=45) of national and regional key informant interviewees who answered the question, agreed or strongly agreed that MHA programs are relevant to the federal government's roles and responsibilities related to First Nations and Inuit. Interviewees noted that MHA program activities allow First Nations and Inuit communities to build upon internal leadership and resilience, and apply these strengths in building healthier, stronger communities.

According to the literature review and key informants, delivering the MHA Cluster of programs is an appropriate role for the federal government and the FNIHB.

4.2 Performance

4.2.1 Core Issue #4: Achievement of Immediate Outcomes

Immediate Outcome 1: Increased and Improved Collaboration and Networking

Collaboration and networking have increased over the evaluation period and are critical to planning, developing and delivering MHA programs and services. Key informants reported that collaboration and networking must continue to evolve to support greater service delivery integration and harmonization with provincial and territorial services.

Importance of Collaboration and Networking

All key informants, whether community workers, community health directors, treatment centre directors, NAOs, or national and regional key informants emphasized the importance of  continued collaboration and networking to improve the health outcomes of First Nations and Inuit through increased and enhanced service delivery. Interviewees from north of the 60th parallel noted that partnerships and collaborations play a critical role in the implementation of MHA programs, especially in small, remote communities and, that while communication with various levels of government existed (hamlet, regionally, territorially, federally), these need continued focus and attention.

The collaborative relationships identified by key informants varied and were developed to support specific aspects of program planning and implementation (i.e., program delivery, information sharing, access to training, cross-referral between services and policy development).

Program Delivery

Some communities work with local service-providing agencies (primarily local social service or family service agencies) to help with program delivery. For example, one of the communities visited developed a partnership with a regional family and health services agency which serves First Nations and delivers programming in the First Nations community through a formalized funding agreement. The health director of this community reported that the agency provides in-kind support and staff resources to deliver NNADAP, Brighter Futures, and Building Healthy Communities programming. Community health directors indicated that key partnerships with other First Nations communities are critical in planning and developing community-based and community-led strategies for addressing mental health and addictions issues.

The Inuit communities visited reported having several partnerships with other agencies serving children such as: the province, the local child and family services authority and other community-based agencies. These partners worked together to deliver mental health and addictions services.

Data from NAYSPS provided further context on the role of partnerships and collaborations in strengthening MHA programming. Most projects in the 2007/08 NAYSPS Inventory identified NNADAP workers/counsellors (69%), Elders (67%), mental health workers (63%), youth groups (57%), and youth workers (51%) as internal project partners (i.e., services offered by community health programming). Further, external partnerships (i.e., services offered by external agencies) included mental health workers (64%), school programming (62%), youth groups (55%), counsellors (54%), and the RCMP (49%). Representatives from NAYSPS projects identified that partnerships and collaborations at all levels were a key part of their program's successes. Through analysis of Northern Region reporting templates, it was found that NAYSPS programming demonstrated consistent collaboration throughout the time period of the evaluation, which benefited programming. The following are some examples:

• When NAYSPS was introduced in 2005, the Government of Nunavut organized a 2 day conference in Rankin Inlet to: engage in strategic consultations regarding the purpose of NAYSPS funding, introduce the Embrace Life Council: and notably, discuss the benefits of interagency communication regarding healthy communities. Representatives from all Nunavut communities were in attendance. The next year, 2006/07, there were 15 NAYSPS projects in 10 communities, with approximately 365 participants.
• In 2005, the Tl'oondih Healing Society organized a youth suicide prevention planning weekend, with NAYSPS funding. In attendance were 24 representatives from 8 Beaufort Delta communities. Participants gained a basic understanding of youth suicide prevention measures and they worked together to identify the assets and resource in their communities for combating youth suicide. Likewise, in 2005, an Inuvik Youth Conference was organized which focused on relationship building; and, the Tlicho community government organized a meeting in which elders, adults and families gathered to support and listen to youth.

Information Sharing

Key to positive collaborative relationships and networking is the sharing of information among all partners. Key informants noted an increase in information sharing opportunities. Most national and regional key informants (72%, n=39) who answered the question indicated that information sharing related to MHA programs and services had increased. Similarly, in key informant interviews, 100% (n=14) of community health directors and 90% (n=10) of treatment centre directors in First Nations communities south of the 60th parallel also agreed that opportunities for sharing information (e.g., conferences, meetings, workshops, conference calls) related to mental health and addictions programs and services had increased during the evaluation period. Overall, the community health directors in Inuit communities south of the 60th parallel felt that opportunities to share information related to mental health and addictions programming had stayed the same or somewhat increased between 2005 and 2010. National Aboriginal Organizations reported partnering with regional health authorities and other NAOs to support the delivery of the MHA Cluster. Key informants from the National Aboriginal Organizations indicated that these partnerships, in addition to having helped increase attention and awareness of mental health issues, have helped improve understanding of the role of culture in mental health and addictions programming.

Key informants from First Nations and Inuit communities south of the 60th parallel provided examples where opportunities to share information have increased:

• use of technology to share information (e.g., text messaging, social media, email, teleconferences) with community members and with professionals in other communities;
• availability of research, publications and brochures; and
• networking and information sharing opportunities (e.g., conferences) for workers.

Access to Training

Treatment centre directors (80%, n=10) and community health directors in First Nations communities south of the 60th parallel (36%, n=14) stressed that, because of partnerships and collaboration, access to training and referred services has improved, and costs have been reduced because of resource sharing with local First Nations communities and other levels (i.e., provincial) of health services.

Cross-referral between Services

Many regional interviewees (75%, n=12) stressed the importance of formalized partnerships with local health centres within their provincial jurisdictions to help link community members to off-reserve services, such as counselling and friendship centres in urban centres as well as formalized partnerships with post-secondary education institutions to help provide training and certification for program staff in communities.

Policy Development

National interviewees (34%, n=35) indicated that their most important partnerships involved formalized high-level planning and policy development partnerships with NAOs, including the Assembly of First Nations and Inuit Tapiriit Kanatami. Examples of key initiatives that resulted from these partnerships included: the NAYSPS; Mental Wellness Teams; Honouring our Strengths: the NNADAP Renewal Framework; and British Columbia's Transformative Change Accord: Tripartite First Nations Health Plan.

The NNADAP Renewal process is regarded by NAOs as a best practice model for policy development and program renewal specific to First Nations (Polson-Lahache, 2012), and elements of this process will help to inform future efforts to strengthen and renew Inuit addiction and mental wellness services. In a transparent and collaborative manner, the NNADAP Renewal Process successfully engaged over 2,000 First Nations people in the development of Honouring Our Strengths: A Renewed Framework to Address Substance Use Issues Among First Nations People in Canada. Health Canada is working in close partnership with First Nations at all levels to implement the framework. First Nations Chiefs unanimously endorsed a draft of Honouring our Strengths by passing a resolution of support at the December 2010 AFN Special Chiefs Assembly. It is expected that key learnings and successes from the NNADAP Renewal process will inform the development of an Inuit-specific process to develop an Inuit mental wellness continuum, inclusive of mental health and addictions.

In addition, the First Nations and Inuit Mental Wellness Advisory Committee's Strategic Action Plan (MWAC SAP) and Alianait, the Inuit Mental Wellness Action Plan were developed in partnership with the Assembly of First Nations and Inuit Tapiriit Kanatami as referenced earlier in the report (section 4.1.2).

Continued Need for Collaboration and Networking

Key informants (21%, n=47) highlighted that partnerships and collaborative relationships must continue to evolve to support greater service delivery integration. They indicated that while collaborative relationships and partnerships exist with provincial governments, these relationships and partnerships are still relatively new and require continued focus and attention. Partnerships at the regional and community level have the opportunity to increase provincial investment in First Nations and Inuit programming, potentially leading to greater access to provincial services for First Nations and Inuit communities addressing currents gaps in programming.

Key informants in the Northern Region agreed that partnering and sharing information was most effective at the community level, and with other MHA providers, but that the MHA Cluster was not as successful in increasing or improving collaboration and networking between MHA providers outside of their communities, or with other levels of government. Barriers to improved collaboration and networking north of the 60th parallel included high staff turnover, issues of privacy and confidentiality, and high northern travel costs.

Findings indicate that collaboration and networking opportunities have increased over the evaluation period and that these opportunities are valued by all key informants. Furthermore, key informants stressed the need for both more partnerships and continued strengthening of existing partnerships. For example, while collaborative relationships and partnerships exist with provincial and territorial governments, these relationships and partnerships are still relatively new and require continued focus and attention.

Immediate Outcome 2: Improved Continuum of Programs and Services in First Nations and Inuit Communities

There has been some improvement in the continuum of MHA programs and services delivered in First Nations and Inuit communities. In particular there has been an increase in the range of programs and services available to First Nations and Inuit communities. There has been very little program overlap and continued integration of mental health and addiction programming in the communities. Nonetheless, gaps in programs and services - particularly relating to aftercare, as well as accessing detoxification and other provincial services - remain critical issues. Further, while progress appears to have been made in integrating MHA programs at the community level, findings suggest that improving the integration of programs and services continues to be a challenge and, as such, should remain a priority.

To have a continuum of services, there must be access to a range of services; and, to have continuity of care, this range of services should be integrated. Integrated service delivery is 'the management and delivery of health services so that clients receive a continuum of preventive and curative services, according to their needs over time and across different levels of the health system' (WHO, 2008). For the client, an integrated continuum means that their care is easy to navigate, that health workers are aware of their health as a whole, and that health workers from different systems communicate well (WHO, 2008).

Continuum of MHA Services

Key informants in the Northern Region agreed (67%, n=15) that the continuum of programs delivered in First Nations and Inuit communities has somewhat improved between 2005 and 2010. The addition of National Aboriginal Youth Suicide Prevention Strategy and Indian Residential School programming during this period further expanded the continuum of programs available to Northerners.

Key informants at the community level south of the 60th parallel, and at the regional and national levels agreed that there was an increased range of MHA programs and services. The following findings relate to south of the 60th parallel communities:

• 100% (n=15) of community workers, leaders and health directors in the Inuit communities visited agreed that the range of MHA programs and services had either somewhat increased or significantly increased.
• The majority of First Nations community workers (66%, n=58) who responded agreed that the range of MHA programs and services had increased.
• Most (64%, n=14) First Nations community health directors indicated that the range of available programs in their community had increased during the evaluation period.
• Almost all treatment centre directors (90%, n=10) indicated that, during the evaluation period, the centre had expanded the range of programming. Specifically, many centres introduced more traditional and culturally appropriate programming, including land-based programming, and incorporated more traditional cultural practices into day-to-day programming. Other treatment centre directors indicated that they had changed programming to: adapt to new and changing needs, including more gender-specific care (i.e., care for women), to deal with Indian Residential School related issues, deal with specific drugs (e.g., crystal meth), and provide and facilitate aftercare.
• The vast majority (80%, n=39) of those who were able to respond to this question) agreed that the range of MHA programs and services had either somewhat increased or significantly increased over the evaluation period.

Program Overlap

While continued efforts remain to further improve the continuum of services available to First Nations communities, program overlap was not identified as a critical issue by the key informants.

• The majority (57%, n=49) of the First Nations community workers and a higher percentage of Inuit community workers, leaders and health directors (70%, n=15) who could answer this question disagreed that there were overlaps between MHA and other programs. In their view, the scarcity of staff and resources to administer programs largely renders duplication or overlap a non-issue. Several community workers indicated that there is some overlap and/or duplication with available provincial services, but did not identify this overlap as a critical issue, given the difficulties that community members already experience attempting to access provincial services.
• In Nunatsiavut, a community health director reported that a protocol had been established to prevent overlap of services by tracking community programs and assets.
• Among national and regional key informant interviewees, overlap between MHA and other programs was not believed to be an issue. Most suggested that within First Nations and Inuit communities there are limited services with which to overlap.
• A minority of national and regional key informants suggested that the Brighter Futures and Building Healthy Communities programs overlap with FNIHB's Children and Youth Cluster programming.

Service Integration

According to key informants, while some progress has been made in integrating MHA programs at the community level, the continued integration of programs and services particularly with provincial services remains a challenge and a priority across the MHA Cluster. Of those who responded, only 21% (n=43) of national and regional key informants, 56% (n=39) of community workers in First Nations communities south of the 60th parallel and 80% (n=15) of community workers north of the 60th parallel reported that MHA programs and services were well integrated at the community level. Nine of the sixteen key informants (56%) north of the 60th parallel somewhat or strongly disagreed with the following statement: MHA-related programming between 2005 and 2010 was well coordinated with other health and social services, in my community in order to avoid duplication and gaps in service provision.

• One positive aspect of integration in First Nations and Inuit communities was highlighted in the levels of cross-referrals between the programs. In the Inuit communities visited, 90% (n=10) of the community workers, leaders and health directors who answered the question agreed that mental health workers refer clients to addictions workers and 100% (n=8) of those who answered the question agreed that addictions workers refer clients to mental health programs. Almost all of the community workers in First Nations agreed that community mental health and addictions workers refer clients to each others' programs. Ninety percent (n=37) of community workers who answered the question agreed that mental health workers made cross referrals to addictions workers and 92% (n=36) agreed that addictions workers made cross referrals to mental health workers. Although referrals are being made between services, challenges toward greater service integration remain.
• Key informants south of the 60th parallel spoke of ongoing, significant challenges relating to integration:
  • Unavailability of aftercare services within the community is the most pressing issue affecting service integration in First Nations and Inuit communities, particularly in the context of severe addictions (e.g., methadone).
  • Silos can result in a lack of information sharing and coordination between community workers. In interviews, key informants spoke of lack of communication between mental health and addictions workers and between community workers and treatment centres.
  • Community workers, in key informant interviews, identified a lack of psychiatric care and counselling in communities.
  • Community workers largely attributed the lack of integration of services within the community to short-term planning which results from short-term funding.

Gaps in Service

Program and service gaps, particularly in the areas of detoxification services, access to provincial services (e.g., psychiatry and acute care) and continuity of care (i.e., aftercare services) were evident south of the 60th parallel:

• The vast majority, 83% (n=41), of national and regional interviewees felt there were gaps in MHA programs or services. Some could not, or chose not to answer the question (n=6).
• In the Inuit communities visited, 93% (n=15) of Inuit community workers, leaders, health directors and treatment centre directors, disagreed when asked to indicate whether they felt there were no gaps in MHA programs or services (i.e., agreeing that there were gaps in service), while only one person agreed. Only one respondent could not, or chose not to, answer the question.
• A similar proportion (82%, n=48) of First Nations community workers disagreed when asked to indicate whether they felt there were no gaps in MHA programs or services, while only 14% agreed. Twenty respondents could not, or chose not to answer the question because they had worked in the community for less than five years.

When asked for more detail about gaps in MHA programs or services, community workers, community health directors, regional and national key informants indicated that the greatest gaps involved:

• Detoxification services. Interviewees indicated that jurisdictional issues between provincial governments and the federal government often inhibit access to detoxification services, given that these services are only provided by provincial health systems and not by the federal government. This is consistent with the literature (First Nations Addictions Advisory Panel, 2010).
• Access to provincial services. In general, access to provincial services (e.g., psychiatry, acute care, etc.) remains a challenge that was identified by all groups in key informant interviews. Waiting lists for community members to access such services are too long and access to health professionals and health treatment was seen as a critical service gap in isolated communities.
• Lack of continuity of care. Continuity of care is where the service providers are sharing information and coordinating/integrating services, so that the client experiences continuity from the beginning to the end of treatment. Presently there is insufficient aftercareand follow-up once an individual returns to his or her home community after treatment centre care. Eighty percent of both treatment centre directors and community health directors (n=24) in First Nations communities stressed that aftercare and continuity of care when clients return to their home communities remains a critical issue. The importance of aftercare is supported by the literature. When addictions clients (from the general population) who have completed a rehabilitation program do not receive quality aftercare, they have an increased risk of relapse (Christian, 2009).
• Health human resources. Eighty percent of treatment centre directors (n=10) reported that the number of trained staff available was insufficient to meet their needs, and 36% (n=14) of community health directors in First Nations communities reported that they require more qualified staff and funding to meet their communities' needs.

  "We also need other services such as psychiatrists and psychologists. We need the whole spectrum of care that individuals need [from] traditional healers to psychologists."
  Community worker

According to these findings, much work is required on improving the continuum of programs and services in First Nations and Inuit communities. While almost no overlap between services has been reported, there are major gaps in services and progress can be made to improve the integration of mental health and addictions services (e.g., using a case management approach).

Immediate Outcome 3: Increased Participation of First Nations and Inuit Individuals, Families and Communities in Programs and Services

Participation in MHA Cluster programs and services has increased over the years covered by this evaluation. The following barriers to participating in MHA programs were reported: concerns about confidentiality and privacy, difficulty travelling to services or lack of child care, continued stigma about having a mental illness or addiction and lack of culturally-safe or culturally-competent programming particularly for women.

Increase in Participation in MHA Programs and Services

All key informants perceived that the level of participation in MHA programs and services had increased between 2005/06 and 2009/10. The positive response of key informants regarding increasing participation rates is further corroborated by evidence obtained through an analysis of reporting from contribution agreements.

Data provided by FNIHB from the Indian Residential School Mental Health Services Tracking System (MHSTS) and the Management of Contracts and Contributions System (MCCS) indicate that First Nations and Inuit participation in the Indian Residential Schools Resolution Health Support Program, has dramatically increased from 2005/06 to 2009/10, suggesting a significant uptake of the newly available service.

The number of unique individuals^{Footnote 10} accessing Indian Residential School counseling services increased from 356 in 2005/06 to 4,441 in 2009/10 as the program became fully implemented. As well, the number of counseling sessions provided for individual clients increased from 1,837 in 2005/06 to 26,748 in 2009/10. In addition, there were significant increases in individuals accessing transportation services and the number of transportation trips that these individuals took.

The number of former students and family members accessing resolution health support workers services has increased as the program became fully implemented across the country. The number of family members accessing resolution health support workers increased almost four times from 2007/08 to 2008/09 (though declined somewhat in 2009/10 compared to 2008/09), while the number of former students accessing resolution health support workers more than doubled from 2007/08 to 2008/09 (though also declined somewhat in 2009/10 compared to 2008/09). The decline in participation rates in 2009/10 may indicate that most family members of Indian Residential School survivors and/or former students wishing to access these services had done so as the services became available between 2007/08 and 2008/09.

The number of former Indian Residential School students and family members accessing cultural support providers also significantly increased. The number of family members accessing cultural support providers increased each year in the period data was available, increased more than twelve times overall during that period (824 in 2007/08 compared to 10,290 in 2009/10). The number of individual Indian Residential School former students accessing cultural support provider services increased from 1,743 in 2007/08 to 24,864 the following year, though declined in 2009/10 compared to 2008/09.

Factors Impacting Participation Rates

Many factors affecting the participation of First Nation and Inuit community members were noted during the interviews south of the 60th parallel. Participation can be influenced by access or barriers to accessing program. Barriers for First Nations and Inuit community members identified in the evaluation include:

• Issues of confidentiality and privacy. Community members often prefer to access services outside the community. However, these services may not be culturally competent or culturally safe and, moreover, may be more difficult to access, even in non-isolated communities.
• Difficulty travelling to access MHA services.
• Stigma about mental health and addictions issues. While stigma is declining, it continues to  impact community members' willingness to access programs and services.
• Women who face particular difficulties accessing services. Women, who have experienced abuse from men, may be intimidated or afraid to open up to male therapists, nurses, or doctors and would prefer that female mental health workers be available.
• Lack of available child care.
• A lack of culturally-safe and culturally-competent programming in some communities and the limited flexibility in programming to deal with the multiple and changing mental health and addictions needs of community members.
• Staff turnover among health care providers and poorly-trained providers were also reported as impacting program participation. Key informants commented that high staff turnover affects the ability of program participants to experience a continuity of care with a specific health professional where confidence and trust may have developed.

Several of these factors affecting participation were also mentioned by key informants north of the 60th parallel, including: stigma, which was identified as the main barrier to increasing participation, issues of confidentiality and privacy, and high staff turnover.

Although the evaluation revealed barriers to participating in MHA programs and services, not all communities share the exact same issues or require the same solutions. Communities need to continue to be supported, through a variety of means to identify and address their unique barriers to program participation.

"The people have been living here for years. These people are finding it difficult to trust the people, workers involved with the delivery of the services. People are scared of other people in the community finding out about their problems."
- Elder (non-isolated community)

Immediate Outcome 4: Increased Awareness of Healthy Behaviours

The awareness of healthy behaviors in First Nations and Inuit communities had increased and the increase was often attributed to attendance at MHA programming. Youth who received treatment for substance abuse were provided with information about the impacts of their unhealthy behaviours and the evaluation found that 82% of First Nations youth in solvent addiction treatment programs and 90% of their families were aware of the effects of substance abuse.

Community workers, treatment centre directors, regional and national key informants were asked whether the awareness of healthy behaviours (e.g., coping skills, healthy relationships, eating well and being physically active) had increased in First Nations and Inuit communities.

• Of those who could answer the question, 30% (n=39) of national and regional key informants agreed that awareness of healthy behaviours had increased in communities.
• In First Nations communities south of the 60th parallel, most community health directors agreed (67%, n=12), and of those community workers who could answer the question 67% (n=51) agreed that awareness of healthy behaviours had increased in their communities.
• Increases in awareness of healthy behaviours were often attributed to attendance at MHA programming. Among the programs offered, community workers identified prevention programs for youth (e.g., presentations on drugs that expose youth to the negative side of drug use), counseling, and emotional health programs as particularly effective. Social groups/circles were also noted to increase awareness of healthy behaviours among community members in general.
• The majority (76%, n=34) of national and regional key informants, who felt able to respond to the question, indicated that, in their view, the amount of information available to communities on healthy behaviours has increased over the evaluation period.
• The key informants for northern programming were asked if the available information helped increase the level of awareness among program participants about the differences between healthy and unhealthy behaviours related to mental health and addictions. The majority (56%, n=15) identified that 'yes', the information did help increase awareness, while another 25% (n=15) indicated that the information available was 'somewhat' helpful.
• Focus group respondents in the two Inuit communities said that there has been increased awareness of healthy behaviours in the community as a result of MHA programs and services.
• Youth who received treatment for substance abuse were provided with information about the impacts of their unhealthy behaviours. The YSAC database indicates that 82% of First Nations youth in solvent addiction treatment programs and 90% of their families were aware of the effects of substance abuse.

There is a perceived increase in the awareness of healthy behaviours; however, additional information from program participants would help to further assess progress being made on this outcome.

4.2.2 Core Issue #4: Achievement of Intermediate Outcomes

Intermediate Outcome 1: Increased Practice of Healthy Behaviours

The evaluation found that members of the First Nations and Inuit communities were engaging in more healthy behaviours due to MHA programming. Community members are engaging in more positive social interactions, becoming involved in traditional practices, and seeking help when needed. A few interviewees also commented on reduced alcohol and drug use in their community. Community workers did stress however, that there is still room for improvement in increasing the practice of healthy behaviours.

Over half of the community worker key informants (n=68) indicated that there has been an increase in coping skills, self-esteem, and positive relationships in their community. The healthy behaviours most commonly practiced, as reported by First Nations community workers south of the 60th parallel, included:

• making healthier food choices;
• engaging in more positive social interactions (e.g., volunteering, socializing with community members);
• exercising more;
• engaging in traditional practices (such as visiting Elders, being on the land, etc.); and
• seeking help when needed (including physical/mental health services and addictions services).

A few interviewees had also observed a reduction in alcohol and drug use and an increase in positive parenting practices among community members.

"I see less drinking, more talking with Elders, and more willingness to talk about mental health and drugs."
  - Elder

In the two Inuit communities visited (south of the 60th parallel), the majority (72%, n=18) of participant survey respondents who answered the questions reported that socializing between community members increased over the evaluation period. While only 40% (n=15) of the same survey respondents reported that volunteering among community members had increased, 53% (n=15) noted that the participation of community leaders in healing activities had increased over the evaluation period.

Three-quarters (75%, n=51) of community workers in First Nations communities south of the 60th parallel who could answer the questions reported that there had been an increase in healthy behaviours such as coping skills, self-esteem, and positive relationships.

Data from the National Native Alcohol and Drug Abuse Program Treatment Centre Outcome Study are consistent with the reported observations of community workers south of the 60th parallel. Findings from the study suggest that, after completing treatment, clients engaged in more healthy behaviours. Most clients who participated in the Outcome Study indicated that they had taken part in more addiction support activities (50%), cultural and social activities (57%), spiritual practices (62%) and healthy and physical activities (61%) after treatment. Slightly more than four-in-ten (45%) engaged in more mental health supports after treatment.

There is conflicting evidence on whether the MHA cluster in the north of the 60th parallel has helped increase the practice of healthy behaviours among First Nation and Inuit populations across the three territories. Representatives from the Territorial Governments and Yukon First Nations reported, in their annual Community Based Reporting Templates, many behaviours which had improved due to MHA programming. The improved behaviours included: coping skills, self confidence and positive relationships, among others. However, these positive findings were not supported by key informants. Key informants north of the 60th parallel, including territorial government representatives, community level representatives and Indian Residential School service providers, reported that each type of healthy behaviour listed in the interview (positive coping skills, good self-esteem, ability to 'bounce-back', cultural knowledge, participation in traditional activities, stress management, and living free of substance abuse) had worsened during the 2005/10 time period. This may mean that the programs are helping to increase the practice of healthy behaviours among those that the programs reach, while not contributing to an overall increase in healthy behaviours at the population level.

"We're going to go back to the healthy ways, living off the land and eating healthier."
  -Community Worker

Findings from the Outcome Study suggested that, after treatment, clients agreed that they seek support (90%), that their overall health has improved (84%), and that they are more ready to improve their education and/or employment status if opportunities are available.

National Aboriginal Youth Suicide Prevention Strategy case studies indicated that:

• The Taiga Adventure Camp (NWT) increased campers' confidence in themselves as individuals and as leaders. Campers were also reported to have improved their ability to appropriately deal with failure. Therefore, Taiga may improve self-esteem and coping skills among its campers.
• The Hobbema Cadets (Alberta) increased participants' reported number of friends, confidence, self-esteem, and hope.
• The Journey to Wellness (Saskatchewan) found that boys increased their knowledge of coping and problem-solving skills.
• A review of the Daughter Spirit in Action project (operated in Manitoba) found participants reported increased self confidence, and strengthened abilities to communicate their ideas and opinions. Many of the team members, Elders and youth felt that the suicide prevention training provided them with the skills necessary to understand, discuss, and provide suicide prevention support in their communities.

There has been an increase in healthy behaviours among community members such as more positive social interactions, more involvement in traditional practices, and seeking help when needed. Additional information from program participants will help to further assess progress being made toward this outcome.

Intermediate Outcome 2: Increased First Nations and Inuit Community Ownership and Capacity to Combat Substance Abuse, Suicide and Other Mental Health Issues 

First Nations and Inuit ownership and capacity to combat substance abuse, suicide and other mental health issues have increased over the evaluation period. There have been increases in trained staff, local control, flexibility, and increased cultural expression in programs. As community ownership to combat mental health and addictions issues increases, communities are able to tailor programming to meet their unique needs. While training opportunities have increased, the need for training remains high in part due to high rates of staff turnover and the need to address gaps in training as identified by key informants (i.e., management training at the community level, cultural awareness training of non-Aboriginal service providers and current issues in the mental health and addiction field).

Increases in Trained/Skilled Workers

The majority of community health directors in First Nations communities (n=14) south of the 60th parallel indicated that between 2005 and 2010 there had been additional training of staff to support them in effectively working with mental health and addictions issues. In recent years, there has been more funding allocated to the training and hiring of skilled workers, which has allowed health centres to provide more services to community members. All of the health directors in Inuit communities visited south of the 60th parallel (N=3) reported that the number of trained/skilled workers (i.e., workers who have the appropriate training to fulfill their job responsibilities) increased over the evaluation period.

Most treatment centre directors (n=10) indicated that they have a formal process to identify the type of training the treatment centre staff need. They also agreed that the training they were given was useful, and that they had the proper training to develop the skills to do their jobs however, fewer than half of these respondents agreed that it was easy to access available training.

The document review found evidence, in reports received from the Territorial Governments, that funding was used to train community workers, community members and youth. In many cases, Brighter Futures and Building Healthy Communities funding was used at the community level to provide training in the following areas: anti-bullying initiatives for schools, creative cooking classes, and coaching athletics, among others. Training generated from National Aboriginal Youth Suicide Prevention Strategy and National Native Alcohol and Drug Abuse Program funding was generally strategically planned by the Territorial Governments and offered to representatives from a diversity of communities. Fifty percent (n=16) of the key informants north of the 60th parallel also agreed that MHA-related funding provided to Territorial Governments and First Nations communities between 2005 and 2010 was used for training purposes to help build capacity in MHA programming, while 43% (n=16) said 'no' or were unsure.  Eight of the nine respondents (89%), who could comment indicated that 'yes' the training did improve capacity to deliver MHA funded programming.

While key informants reported that training had increased, there continues to be a large need for training for community workers. Most treatment centre directors agreed that the need for certified training (programs at least one academic year in length), continuing education (short-term courses that upgrade or maintain skills), and short course training (between one week and three months that are not recognized in a certified program) have increased between 2005 and 2010. The majority of treatment centre directors indicated that the number of trained workers in addictions services in their centre was not enough to meet current demands. The respondents believed that training opportunities should be improved through increased funding dedicated to staff development. In addition, a number of treatment centre directors felt that the process of continuous intake impacts the centres' ability to plan training for staff ahead of time. Finally, the retention issues of high staff turnover and consequent higher staff training costs, impact the resources needed to dedicate to training.

Some of the training that key informants called for included:

• training on cultural awareness and traditional healing for non-aboriginal service providers;
• training on clinical supervision and human resources (e.g., management training); and
• training on opiates addictions and withdrawal.

Local Control in Planning and Delivering MHA Programs and Services

The importance of local control in planning and delivering MHA programs and services is consistent with findings from the literature review. Communities with more control over their programs, facilities, and institutions tend to have lower incidences of suicide and substance addictions. Specific self-determination factors related to these trends include land titles, self-government, high involvement of women in band governance, and control of education, police, cultural, health, and child welfare policies and services (Lalonde, 1998, 2008). The control over these services allows communities to protect their traditions and control their collective future (Lalonde, 2006). Although evidence specific to control of program design is not available, the literature does suggest that self-determination can contribute to community resilience, and the capacity of a community to adjust to change by reorganizing while retaining its identity (Hopkins & Dumont, 2010).

Key informants in First Nations and Inuit communities, and at the regional and national levels, reported that local control in planning and delivery of programs and services has increased compared to five years ago. Evidence of this is the number of unique communities in flexible or flexible transfer funding agreements that has increased over the evaluation period. In 2005, only one community and one Tribal Council responsible for 13 First Nations were in flexible transfer agreements. In 2009, 23 communities were in flexible agreements, while 51 communities were in flexible transfer agreements.

The majority of key informants believe that the communities now exercise greater control in the planning and delivery of programs and services.

• Many more community workers, leaders and health directors in Inuit communities south of the 60th parallel who answered this question agreed (78%, n=14), than disagreed (7%) that community control over programming had improved over the years in scope for this evaluation. Almost all of those interviewed responded to this question.
• More community workers in First Nations communities south of the 60th parallel who answered this question agreed (38%, n=68) than disagreed (6%) that control over programming had improved. A high percentage of those interviewed (38%) could not respond to this question and 18% neither agreed nor disagreed.
• Fifty-nine percent (n=47) of national and regional key informant interviewees who answered the question either strongly agreed or agreed that communities have more control over delivering mental health and addictions programs and services.

Despite the noted improvement in the amount of control divested to communities, interviewees were also adamant that the communities' abilities to control local community programming must continue to increase. Key informants made the following recommendations to increase the divestment of control to communities:

• build equitable partnerships between communities and FNIHB to improve understanding of community needs; and
• use tripartite partnerships (i.e., between their community, the province/territory, and FNIHB) to increase community control over the programs they offer.

These recommendations from the key informants are consistent with the literature, which suggests that tripartite agreements can play a pivotal role in increasing community control over health services. Tripartite agreements, memoranda of understanding, and joint committees like those that currently exist in British Columbia, Saskatchewan and Ontario, respectively, are designed to place more control of First Nations in the planning, design, management, and delivery of First Nations health programs to communities (Nursing Leadership, 2012). Further, MacKinnon (2005) stresses that tripartite approaches can better integrate, and thus, reduce gaps in care between First Nations and provincial health systems. Lavoie et al. (2010) even suggest that tripartite agreements, by increasing local autonomy and control, may also contribute to improved health outcomes.

There was some disagreement, however, on the appropriateness of MHA Cluster funding arrangements given the impact on local control. National and regional representatives and community workers stated that the current funding arrangements still inhibit local control and ownership over MHA programs and services. Community health directors south of the 60th parallel noted that current funding arrangements permit communities to choose those programs that are appropriate for the community, but do not sufficiently promote community design of programming.

Improvements were noted in the Northern Region with regards to the local control of MHA programming. From 2005 to 2007, the Government of Nunavut and the Government of the Northwest Territories had agreements that offered very little flexibility in which each program was delivered individually at the community level. During this time, Health Canada had integrated funding arrangements with 14 Yukon First Nations for some of the MHA funding. Between 2007 and 2010, 11 of the 14 First Nations in Yukon became self-governing and had the ability to draw down and take ownership of many of the programs included in the MHA cluster, through a Programs and Services Transfer Agreement. After FNIHB's introduction of a new Contribution Funding Framework in 2007, the Governments of Nunavut and the Northwest Territories and non-self-governing Yukon First Nations entered into Transitional Agreements, through which recipients have increased flexibility to design health programs and allocate funds accordingly.

Flexibility of Programming to Address Local Needs

The ability of communities to identify and address local needs is enhanced with greater local control of programming. Key informant stressed that flexibility, adaptability, and responsiveness to local needs are critical components of MHA programs and services. Most interviewees agreed that MHA programs and services are flexible and responsive to address local needs as described below:

• 65% (n=43) of national and regional key informants, who felt able to answer the question, agreed that services are flexible and responsive to local needs.
• 64% (n=45) of First Nations community workers south of the 60th parallel agreed, (twenty-three community respondents chose 'don't know' or 'not applicable').
• 93% (n=15) of Inuit community workers, leaders and health directors south of the 60th parallel agreed, (only one respondent chose 'don't know' or 'not applicable').
• In the North, 56% (n=16) of key informants at the community and territorial level agreed that MHA programs were somewhat flexible, while the remaining 44% said they were somewhat, or highly, inflexible.
• The majority of treatment centre directors and all community health directors in First Nations communities agreed that MHA programs are flexible enough to respond to local needs.

Cultural Expression in Programs

All national, regional and territorial representatives, treatment centre directors, community workers and health directors in both Inuit and First Nations communities, agreed that it is important for MHA programs to draw on community cultural practices.

Incorporating cultural practices into mental health and addictions programming is well documented in the literature as critical to the success of mental health and addictions programming in First Nations populations. Communities and individuals with higher levels of cultural identity and traditional language fluency show lower rates of suicide and substance abuse (Canadian Institute for Health Information, 2009; FNAAP, 2010; Hopkins & Dumont, 2010; Lalonde, 1998, 2006). In addition, engaging in spiritual practices is believed to be an important aspect of preventing health issues and encouraging healing among Aboriginal youth (FNHC, 2007). One aspect of Aboriginal culture and spirituality which has received particular attention is the belief in interconnectedness between people and the land. Aboriginal peoples traditionally believe that they were created of and placed on specific territories, and the land was necessary for identity as well as for food and shelter (Hopkins & Dumont, 2010).

The document review highlighted the incorporation of cultural practices in two MHA program areas:

• A recent study of the National Native Alcohol and Drug Abuse Program found that program completion rates are higher in programs that include cultural components than those that do not (Health Canada, 2011). This suggests that cultural practices can aid retention in addictions programming.
• Activities that encourage a connection with the land (e.g., the cultural and traditional hunting activities funded through Brighter Futures) have been found to offer protection against mental health issues, particularly for men (Health Canada, 2006).

The majority of interviewees also supported the finding that MHA programs and services do, in fact, draw on the cultural practices of communities. This included agreement from 84% (n=47) of national and regional representatives, 80% (n=68) of community workers in First Nations communities south of the 60th parallel, 88% (n=15) of community workers, leaders and health directors in Inuit communities south of the 60th parallel who responded to the question and 88% (n=16) of community and territorial respondents north of the 60th parallel.

Interviewees believed that cultural practices were key to individual, family, and community cohesion. Interviewees indicated that culture was "part of everything [they] do". Examples of how MHA programs draw on community cultural practices provided by community workers and health directors included:

• sweats/sweat lodges;
• sharing circles;
• smudging;
• prayer;
• community feasts;
• delivering programs in traditional languages;
• using traditional values and conceptualizations of healing (e.g., the medicine wheel, healing lodges, traditional healers);
• traditional crafts (e.g., drum-making);
• drawing on the knowledge and leadership of Elders;
• teaching the history of the community/culture to youth (e.g., giving children and youth spirit names); and
• spending time on the land or in sacred/historical sites.

Community members from the two Inuit communities visited commented that mental health and addictions programs and activities (particularly those for youth) generally try to incorporate the cultural practices (as noted above) of the community.

"[Cultural programming] helps us stitch together members of the community who might not otherwise come in [to program activities]"
- Elder

The importance of integrating First Nations and Inuit culture and practices into the treatment of clients was stressed by key informants at all levels (NAOs, community level, and HC national and regional representatives) as critical for encouraging communities to actively participate in program delivery. Although it should not be assumed that any intervention has 'pan-Aboriginal' relevance, culture is a key protective factor that protects community members from suicide, addictions, and mental health issues (Kirmayer, 2009). Thus, treatment in Inuit communities cannot be based on a First Nations model and treatment models developed in one First Nations or Inuit community cannot be applied universally. Regional and national key informants reported that there are challenges in fitting cultural practices into the MHA Cluster program parameters set out by FNIHB. Some reported that excellent cultural activities were available, but they were not taken advantage of because they did not relate directly to program objectives/purposes.

According to key informants there have been increases in trained staff, local control, flexibility, and increased cultural expression in programs. Despite these increases, the findings indicate the need for continued training and divestment of control to First Nations and Inuit for the design, development and delivery of programming targeted to the unique needs of each community or region.

Intermediate Outcome 3: Improved Access to Quality Well-Coordinated Programs and Services for First Nations and Inuit Individuals, Families and Communities

Access to quality, well-coordinated programs has improved and the MHA programs are of high quality. Issues of staff retention and training, availability to specialized and culturally sensitive services (e.g., psychiatric services) within the community, and gaps in the existing continuum of services (i.e., aftercare) were reported to be ongoing  and may  impact the quality of MHA programs.

Access to MHA Programs and Services

The majority (71%, n=38) of national and regional interviewees who answered the question agreed that access to programs and services has somewhat or significantly increased. When First Nations community workers south of the 60th parallel were interviewed, 14 people, or nearly 20%, either did not know whether access had increased, or felt unable to answer the question because they had worked in the community for less than five years. For the remaining workers who felt able to answer the question, 66% (n=54) agreed that access to programs and services had somewhat or significantly increased.

Almost the same percentage of community workers, leaders and health directors in the two Inuit communities (south of the 60th parallel) visited (67%, n=15), who felt able to answer the question, agreed that access to programs and services has somewhat or significantly increased. Only one Inuit respondent felt unable to respond.

Quality^{Footnote 11} of MHA Programs and Services

To comment on the quality of MHA programs and services the evaluation reviewed the availability of accredited/certified professionals, the extent to which best practices were incorporated into programming including, traditional or culturally relevant approaches, and gathered the perspective of key informants.

Of the community workers south of the 60th parallel who could comment, almost all, 81% (n=33) agreed that programs were of high quality. Fifty-one percent did not know or could not comment. The majority of community health directors in First Nations communities south of the 60th parallel agreed that MHA programs and services were of high quality and that the quality of services had improved between 2005 and 2010. All of the community health directors, leaders and community workers (N=11) in the two Inuit communities visited (south of the 60th parallel) who answered the question agreed that the quality of MHA programs, services and supports offered in the community had improved over the evaluation period. Most (75% of those who responded, n=12) further agreed that MHA programs, services and supports are of high quality. The vast majority of national and regional key informants who could answer the question (82%, n=40) agreed that MHA programs and services improved between 2005 and 2010.

Nonetheless, community workers (n=68) and community health directors (n= 13) in First Nations communities south of the 60th parallel stressed that quality could be further improved by:

• having highly trained long-term staff to improve client confidence in program treatments, whether through recruitment of qualified staff or more staff training;
• providing community level staff with additional training on cultural sensitivity to heighten the sense of trust between the workers and the clients;
• improving communication and providing more resources and funding to increase services and access to more specialized services within the community (e.g., psychiatric services); and
• increasing aftercare for many severe addictions issues, such as methadone.

In addition, community workers (n=10) and health directors (n=3) in the two Inuit communities visited (south of the 60th parallel) stressed the need to:

• create a more integrated case management process;
• develop more outreach services;
• increase client program attendance by mitigating concerns over privacy and confidentiality;
• reduce staff turnover; and
• introduce proactive suicide prevention programming in the communities.

  "We need to have a counsellor for people to speak to. We can't just put it all on the Elders and Health Centre staff. They don't have the training."
  -Community Worker

Treatment centre directors were asked to comment on the access to accredited services as a measure of quality. According to the directors (n=10), the availability of certified/accredited professionals varied between centres as found below:

• All treatment centres have access to treatment and alcohol/substance abuse counselors in the centre.
• Most have access to psychologists, psychiatrists, and clinical social workers outside of their centre.
• Most have access to a psychiatrist or a dietician.
• One-half of the centres have in-centre crisis workers.
• A minority of the centres have in-centre nurses, clinical social workers, a psychologist, or a nutritionist.
• None of the centres have an on-site psychiatrist.

The extent to which key informants believe that best practices are incorporated into programming, whereby creating quality programs, varies:

• The majority of treatment centre directors (60%, n=6) agreed or strongly agreed that MHA programs and services are based on evidence including best or promising practices.
• Most community health directors (n=14) in First Nations communities south of the 60th parallel indicated that they did not know or could not speak to whether MHA programs and services were based on evidence-based practices.
• Many community workers, health directors and leaders (n=15) in the Inuit communities visited (south of the 60th parallel) also could not speak to whether MHA programs and services were based on evidence. Of 15 workers in total, only eight answered the question. Of those, 75% agreed while 25% neither agreed nor disagreed.
• More than half (57%, n=39) of national and regional key informants who responded to the question stated that MHA programs were based on best practices for providing appropriate mental health and addictions treatment options for First Nations and Inuit individuals and communities.
• Several regional interviewees (n=10), while agreeing that programs are indeed evidence-based, stressed that implementing proper evidence-based programs and best practices within some communities was more challenging because of limited resources, knowledge and training.

According to the document review, best practices have been identified and applied in MHA programs. For example, the Indian Residential Schools Resolution Health Support Program applies the best practice of integrating traditional Aboriginal and Western approaches to healing. It does so by providing former students of Indian Residential Schools with access to both Western counseling and traditional wellness supports such as cultural support providers and resolution health support workers, who are provided by First Nations and Inuit organizations located in communities.

Several regions have provided evidence that they use best practices and evidence-based approaches in their programming. The Nimkee NupiGawagan Healing Centre in Ontario partnered with the Canadian Centre on Substance Abuse and Carleton University to conduct research on treatment practices for youth with solvent abuse issues. Based on the results of this research, they then increased the length of their residential program to four months (Bobet, 2010). Additionally, the Saskatchewan region has identified multiple evidence-based treatment models that were in active use in the National Native Alcohol and Drug Abuse Program services in 2009, including bio-psychosocial, resiliency, stages of change, and holistic models (PRA Inc., 2009).

Interviewees from all groups noted that, while access to quality mental health and addictions services may be available in communities, the large number of individuals needing access to these services appears to out number available services, thus many individuals go without the required help.

4.2.3 Core Issue #4: Achievement of Long-Term Outcomes

Long-term Outcome: Contributes to the Improved Health Status of First Nations and Inuit Individuals, Families and Communities Through a Strengthened Continuum of Mental Health and Addictions Programs and Services.

The MHA Cluster programming has contributed to improved mental health status, reduced addictions issues, and overall improved health status in target communities. 

Data from the NNADAP Treatment Centre Outcome Study suggests that treatment centres have had success bringing about positive changes in sobriety and abstinence among clients. However, data from the Outcome Study is from a convenience sample of 10 of the 55 centres, thus findings may not be applicable to all treatment centres.

Based on the responses of over one hundred NNADAP clients who completed treatment, the percentage of clients who reduced substance use following treatment ranged from 76% (opioids) to 100% (inhalants), depending on the substance used.

Further, many clients indicated they terminated substance use altogether after completing treatment. These rates are lower for alcohol (47%) and cannabis (48%) than for other substances.

The Regional Health Survey (RHS) has reported positive changes in First Nations individuals' mental health and addictions issues between the RHS Phase 1 (2002/03) and Phase 2 (2008/10), including:

• The number of youth who attempted suicide declined from 9.6% to 5.9% (RHS 2002/03, 2008/10), and the number of youth considering suicide dropped from 21.1% to 16.5% between 2002 and 2010 (RHS 2002/03, 2008/10). These data may suggest that youth are receiving assistance or resources to enable them to decide not to attempt suicide, or resort to suicide as a means of dealing with stress or issues they are faced with.
• The proportion of First Nations adults who had thought about suicide decreased from 30.9% in 2002/03 to 22.0% in 2008/10.  However, the proportion who reported having attempted suicide remained relatively the same (15.8% in 2002/03 and 13.1% in 2008/10).

Qualitative data from key informant interviews at the national, regional, and community level suggest that MHA programs and services contribute to improved health status:

• Most (80%, n=14) First Nations community health directors south of the 60th  parallel  tended to be neutral or agree that members of their community had seen improvements in their mental health status, addictions issues, or overall health. Only one community health director disagreed that members of their community had seen improvements.
• Most First Nations community workers south of the 60th parallel who could answer the question (67%, n=45) indicated they had seen improvements in community members' overall health status. Some of those community workers who elaborated (in comments during their key informant interviews) on the changes indicated that they had seen improvements in community members' coping skills and willingness to seek help or treatment for mental health and addictions issues. However, community workers stressed that considerable mental health challenges remain in their communities, including stigma and issues related to Indian Residential School experiences and resulting inter-generational trauma.
• Most Inuit program participants (80%, n=15) in the two south of the 60th  parallel communities visited, who answered the question, agreed that mental health and addictions programs have helped them to improve their health.
• Overall, community workers, leaders and health directors (n=15) in Inuit communities south of the 60th  parallel felt that MHA programs and services contributed to improved health in the areas of alcohol abstinence, reduced stigma towards accessing mental health services and improved life skills.
• Almost half (49%, n=38) of those national and regional key informants who felt able to respond to the question, felt that MHA programs and services have resulted in improvements to the mental health of First Nations and Inuit.
• More than half (53%, n=39) of national and regional key informants who were able to answer the question felt that MHA programs and services have resulted in improvements to the overall health status of First Nations and Inuit.

  "[Community members] have started doing things without needing alcohol or drugs, and spend more time with families."
  - Elder

This data demonstrates a measurable improvement in health status in areas related to MHA programming, specifically suicide and substance use. The extent to which these improvements can be attributed to MHA programs and services, and other factors such as social determinants of health and other related programming is unclear. However, findings from the NNADAP Outcome Study and Regional Health Survey do provide some evidence that MHA programs and services contribute to measurable improvement in health status, specifically in the areas of substance use and suicide.

4.2.4 Core Issue #5: Assessment of Economy and Efficiency

Some improvements in efficiency have been made (e.g., in the processing of funding proposals), but there is room for improvement in the efficiency with which MHA programs are delivered.  Performance measures based on program outcomes would better support the assessment of the programs' economy and efficiency in future evaluations.

The Treasury Board's Policy on Evaluation (2009) summarizes the demonstration of efficiency and economy as an assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes. This requires an expenditure breakdown by program streams or key activities within the MHA Cluster. While overall financial information for the MHA Cluster was available, data on the number of programs operating at the community level and records of the reallocation of funds from within the MHA Cluster, and between the MHA Cluster and other FNIHB programs were not available. This limited financial data presented a challenge to do a robust assessment of economy and efficiency of the MHA Cluster programs. Therefore, the following alternative approaches were used to assess economy and efficiency of the MHA Cluster in this evaluation:

• Addressing questions about economy at the level of program implementation and delivery stage (activities, outputs) through an assessment of available financial data (resource utilization review) and key informant responses.
• Obtaining and reviewing qualitative data from key stakeholder interviews (community and management-level) regarding the efficiency of MHA programming i.e., the use of best/promising practices in the design and delivery of the MHA programs, the level of satisfaction with the efficiency in the planning processes (i.e., program renewal, funding and accountability processes), and the perception of flexibility of MHA programs to respond to local needs.

Economy

Although not unique to the MHA Cluster, there is a lack of consistent financial performance data tracking as it relates to activities and outputs (known as object costing). Financial performance data was not collected for specific component-related program activities (streams) by the program area. These limitations presented challenges to conducting a robust assessment of economy, however, the evaluation did attempt to provide a general sense of resource utilization by reviewing the funding expenditures and cost drivers as well as seeking the perspectives of key stakeholders to provide additional information on economy.

The financial data currently available for the years 2005/06 through 2009/10 is presented in Table 4-5. It should be noted that this data on expenditures includes corporate costs, as well as program delivery costs. A separate breakdown of the program administrative costs was not available for the evaluation.

During the period covered by this evaluation, there were changes in the way that funding to communities with Transfer Agreements was recorded. In 2008/09, the MHA Transfer Program Activity (PA) code was created to capture costs associated with programs under the MHA program authority included in multi-year Health Service Transfer Agreements between First Nations and Inuit recipients (i.e., communities) and Health Canada. This code did not exist prior to 2008/09, when a more general FNIHB transfer code existed. Therefore, it was not possible to separate MHA specific expenditures from these overall FNIHB transfer amounts (for 2005/06 to 2007/08) at this time. In addition, the NAYSPS program began in 2005 but was not reaching communities until 2006/07 and 2007/08.

According to the Health Canada Departmental Performance Report (2006/07), the funding initially allocated to each MHA Cluster program increased or remained the same every year between 2005/06 and 2009/10 (Health Canada, 2007). Spending on program activities in MHA increased by $82 million between 2007/08 and 2008/09 as programming for Indian Residential School program ramped up and the MHA transfer amount became coded and reported.

Regional and national interviewees were divided regarding how satisfied they are with the economy of the MHA Cluster (i.e., the extent to which the key informants thought that the MHA Cluster demonstrated minimal resource utilization to optimize the program outputs needed to achieve immediate outcomes), with approximately equal numbers satisfied and unsatisfied. A high proportion of interviewees (44%, n=21) were unable to speak to economy, either because they were uncomfortable doing so or because they felt they did not have the knowledge or background to speak to the economy of the MHA Cluster.

A lack of financial data (expenditure allocations by activity-type) presented a challenge to demonstrate overall economy of the MHA Cluster. Without this information, the resource analysis conducted was only descriptive (qualitative in nature) and not able to determine how the Cluster balanced its resources with output maximization or achievement of expected results.

Efficiency

The majority of key informants indicated that they were satisfied with the efficiency with which the MHA Cluster is meeting its immediate outcomes. At the same time these informants commented on how aspects of program delivery and the delivery environment can impact efficiency.

Interviewees spoke to efficiency in terms of community control and resources. Treatment centre directors (80%, n=10) and community health directors in First Nations communities south of the 60th parallel (36%, n=14) stressed that access to training and referred services has improved, and costs have been reduced because of resource sharing with local First Nations communities and other levels (i.e., provincial) of health services through partnerships.

Many interviewees felt strongly about adjusting services to meet the specific needs of communities, suggesting that increased community control over the services offered will positively affect efficiency by targeting the needs and strengths of communities. Many national and regional respondents also mentioned that increased integration across MHA programs and increased collaboration would improve efficiency through decreasing "silo-ing" between MHA programs and working with partners to efficiently offer a continuum of care.

"I feel that we have some flexibility with the programs that enable us to integrate the services a little better."
- Key Informant

The majority of regional and national interviewees (n=33) agreed that there is room for improvement in the efficiency with which MHA programs are delivered. Several human resource issues were discussed, including staff training and retention. It was noted by national and regional respondents that having trained staff will improve the effectiveness of delivery and that bringing down staff turnover rates would improve efficiency. The availability of community resources, including medical services as well as the willingness of the community to participate in program activities were noted by national and regional interviewees as factors that could influence the efficiency of service delivery.

Most national and regional key informants who commented on their satisfaction with the efficiency of planning processes (i.e., funding allocation and disbursement as well as annual reporting linked with specific funding agreements) either did not agree (33%) that the funding processes allowed for efficient results achievement or were neutral (31%) on this matter. Although, in their comments during key informant interviews, several national and regional respondents noted that they had seen improvement in the efficiency with which funding proposals were processed over the years in scope of this evaluation. While the funding proposal process had become more efficient, 44% of the key informants did not agree that the efficiency supports or ensures appropriate accountability. That is, the overall process to complete and submit funding proposals and reporting may be improving but may not be capturing the information required to ensure accountability for the funding (e.g., community health directors noted that the focus of the reporting was on how the money was spent rather than linking the funding to the achievement of expected outcomes to determine how effective the program has been).

"The reports usually take up 40% of my work time."
- Community Health Director

Community health directors in First Nations communities south of the 60th parallel, on the other hand, were divided on the efficiency of funding and accountability processes. Half of them indicated that the funding and accountability processes are efficient, while the others disagreed, suggesting that these processes can be improved. Some community health directors were concerned that the accountability processes were time-consuming and focused on outputs (i.e., products or services provided) rather than outcomes (i.e., program effectiveness). They felt that reporting requirements should focus more on how effective programs are at achieving valuable outcomes.

Addition performance measurement for this Cluster is required to fully assess economy and efficiency and identify opportunities to enhance these for the MHA Cluster.

5.0 Conclusions

5.1 Relevance

Continued Need for MHA Programs and Services

All sources of information used in this evaluation confirmed that there is a demonstrable need for mental health and addictions programs in First Nations and Inuit communities. Furthermore, it can be concluded that the MHA Cluster is a mix of programs designed to addresses these mental health and addictions needs, however the magnitude and complexity of these needs presents challenges to thoroughly addressing them. There is a greater prevalence of mental health issues and substance abuse in First Nations and Inuit communities, compared to non-Aboriginal communities in Canada, and addressing these needs is complex, involving mental health and addictions programs, as well as improvements to other determinants of health.

Alignment with Government Priorities

The MHA Cluster is aligned with Government of Canada priorities. Specifically, the Cluster aligns with priorities and goals featured in recent Speeches from the Throne, the United Nations Declaration on the Rights of Indigenous Peoples (to which Canada is a signatory), the International Wharerata Declaration (which Canada supports) and the priorities set by the Government of Canada in Health Canada's Report on Plans and Priorities. These policy directions include prioritizing the health of Canadians and supporting communities to develop their own health solutions, strengthening health related programming in First Nations and Inuit communities and improving First Nations and Inuit access to health services through integration of federal and provincial/territorial programs.

Alignment with Federal Roles and Responsibilities

The MHA Cluster programs are aligned with the federal role and responsibilities to First Nations and Inuit. Consistent with the Government of Canada's policy to improve First Nations and Inuit health,the MHA Cluster is designed to encourage both individuals and communities to become actively engaged in developing and delivering programs and services which reflect the needs of each unique community.

5.2 Performance

Achievement of Expected Outcomes

Overall, the evaluation found that MHA has demonstrated progress towards its stated outcomes and has identified areas that could assist with further addressing the mental health and addiction needs in First Nations and Inuit communities across Canada.

Immediate Outcome - Increased and Improved Collaboration and Networking

The evaluation concluded that collaboration and networking among MHA programs at the community level and with provincial and territorial mental health and addictions services have increased over the evaluation period, and enhanced the planning and delivery of MHA programs and services. However, the relationships and partnerships with provincial and territorial governments are still relatively new and require continued focus and attention to promote greater harmonization between federal and provincial/territorial services.

Immediate Outcome - Improved Continuum of MHA Programs and Services in First Nations and Inuit Communities

There has been some improvement in the continuum (i.e., improvement in access and range of services) of programs and services delivered in First Nations and Inuit communities with no significant program overlap, but gaps in MHA programs and services exist specifically in the following areas: access to aftercare, detoxification and provincial services (e.g., (psychiatry, acute care).

Immediate Outcome - Increased Participation of First Nations and Inuit Individuals, Families and Communities in MHA Programs and Services

Participation in MHA Cluster programs and services has increased over the years covered by this evaluation. This may be in part due to an increased range of MHA programs and servicesfundedover the evaluation period (i.e., over the evaluation period more programs, covering a broader range of services, were funded and delivered including the Indian Residential Schools Residential Health Support Program and the National Aboriginal Youth Suicide Prevention Strategy). The evaluation identified opportunities to help increase participation in programs by addressing the continued stigma attached with seeking mental health and addiction support and ensuring the confidentiality of services particularly in smaller communities. Although the evaluation revealed barriers to participation in programs, not all communities share the same issues or require the same solutions. The evaluation found that any solutions to address barriers to program participant need to use a gender based analysis and address the need for appropriate programming for women who face particular difficulties accessing services (e.g., who have experienced abuse), as well as ensure access to culturally appropriate programs.

Immediate Outcome - Increased Awareness of Healthy Behaviours

While data on the awareness of healthy behaviours is limited, the evaluation found that awareness of healthy behaviours among program participants had increased, and this increase was often attributed to attendance at MHA programming. Additional data regarding the awareness of healthy behaviours may help to fully capture progress on this outcome in the future.

Intermediate Outcome - Increased Practice of Healthy Behaviours

While data on the practice of healthy behaviours is limited, the evaluation found that the practice of healthy behaviours by program participants had increased, and this increase was often attributed to attendance at MHA programming. Additional data about the uptake of healthy behaviours by participants is required to capture progress on this outcome in a comprehensive way that includes input from the program participants themselves.

Intermediate Outcome - Increased First Nations and Inuit Ownership and Capacity to Combat Substance Abuse, Suicide and Other Mental Health Issues

First Nations and Inuit ownership and capacity to combat substance abuse, suicide and other mental health issues has increased over the evaluation period. As community ownership to combat mental health and addictions issues increases, communities are able to tailor programming to meet their unique needs and strengths. While the amount of control divested to communities has increased, the evaluation found that communities would like to see continued attention on the development of equitable partnerships between FNIHB and communities to enhance the federal government's understanding of community needs. As well, all parties would like to increase the use of tripartite partnerships to further increase the role of First Nations and Inuit communities in program design, development and delivery. The evaluation noted that First Nations and Inuit communities with more control over their programs, facilities, and institutions tend to have lower incidences of suicide and substance addictions. The evaluation found that MHA programs and services draw on the cultural practices of communities.The incorporation of cultural practices critical to the success of mental health and addictions programming in First Nations and Inuit communities and is evidence of an increase in community ownership of MHA program delivery.

While training has increased and contributes to increased capacity to deliver programs at the community level, there is a need for continued training to address the identified gaps (e.g., cultural awareness training for non-Aboriginal service providers and management training) and address the high rate of staff turnover.

Intermediate Outcome - Improved Access to Quality Well-coordinated Programs and Services for First Nations and Inuit Individuals, Families and Communities

The evaluation found that access to MHA programs had improved and that MHA programs are of high quality. Best practices and evidence-based approaches are followed in the design of MHA programs and examples were found showing how best practices were incorporated into programming delivery. However, the evaluation found that there remains an ongoing need to fully address: (1) staff retention and training; (2) improved access to specialized and culturally sensitive services within the community (e.g., psychiatric services); and (3) the existing gaps (i.e., aftercare services) in the continuum of services available to First Nations and Inuit communities.

Final Outcome - MHA Contributes to the Improved Health Status of First Nations and Inuit Individuals, Families and Communities through a Strengthened Continuum of Mental Health and Addiction Program and Services

The evaluation revealed evidence to suggest that MHA programs and services are contributing to improved health among First Nations and Inuit. However, there was limited data from follow-up or pre-post studies to fully assess the extent of the contribution MHA programs are making to these improvements in health outcomes.

Improving the Cluster's ability to address First Nations and Inuit mental health and addictions needs requires access to a full continuum of provincial, territorial and federal mental wellness programs and services as well as actions to increase the effectiveness of community-based programming, such as strengthened recruitment and retention of culturally competent health professionals and training for community workers.

Assessment of Efficiency and Economy

The evaluation found some improvement in the efficiency of the MHA programs and services over the evaluation period; however due to the limited financial data (i.e., not having expenditure allocations by activity type or object costing or expenditure records by program components and/or sub-components), a robust assessment of the economy and efficiency of the MHA Cluster was not possible. Improved financial data on program expenditures linked to program activities and outcomes would support a more complete assessment of the program's economy and efficiency in the future.

Performance Measurement in Support of Planning and Reporting

Many of the challenges encountered in the design and conduct of this evaluation can be addressed through an improved and implemented Performance Measurement Strategy. Furthermore, improved availability of and access to high quality data (specifically information on program participation, the awareness of and uptake of healthy behaviours by participants, changes to mental health, addictions and overall health) may help to further support evidence-based decision-making in policy, expenditure management and program improvements for the MHA Cluster in the future.

6.0 Recommendations

Recommendation 1 - Strengthen the continuum of mental health and addictions services available to First Nations and Inuit individuals, families and communities.

There are gaps in the continuum of mental health and addictions care. To address these gaps it is recommended that the Program^{Footnote 12} identify and prioritise, with First Nations and Inuit, gaps in mental health and addictions program and service delivery, and develop, implement and monitor a strategy that uses a phased approach to ensure gaps are addressed in the short-, medium-, and long-term. To strengthen the continuum, the program area would need to:

• ensure that the continuum incorporates and provides access to the best practices of traditional, cultural and mainstream treatment approaches;^{Footnote 13}
• identify and work towards closing the gaps in aftercare;
• identify and work towards closing the gaps in access to detoxification and other provincially funded services; and
• develop, implement and monitor a strategy to improve the integration of Mental Wellness programs and services at the community level.

Addressing gaps in the services available and improving access requires extensive partnership building between jurisdictions (i.e., provinces, communities and FNIHB), and sectors (e.g., housing, income support, education), and is therefore, also supported by the following recommendation on partnership development.

Recommendation 2 - Build and maintain partnerships between FNIHB, First Nations and Inuit communities, National Aboriginal Organizations, and provincial/territorial health services.

There is a need for continued focus and attention to be placed on partnership development and collaborative relationships particularly with provincial and territorial governments. The Program should promote and foster partnership development with relevant First Nations and Inuit stakeholders in initiatives like the Mental Wellness Advisory Committee Strategic Action Plan, Alianait, tripartite agreements, memoranda of understanding, and joint committees like those that currently exist in British Columbia, Saskatchewan and Ontario, respectively.

Recommendation 3 - Address the barriers to accessing services.

The Program should address the key barriers to accessing MHA programs and services as identified in the evaluation. This includes addressing issues of privacy and confidentiality through the development of guidelines and training for community staff; addressing the stigma associated with mental illness and addictions; and ensuring culturally-safe and culturally competent programming by recognizing that the solutions to increasing the access to MHA programs need to be community driven. The MHA Cluster can then provide resources and support to improve access, so that the communities are able to identify their unique barriers and strengths and address these through appropriate community-based solutions.

Recommendation 4 - Increase training available to mental health and addictions workers.

The Program should, with First Nations and Inuit partners, offer training for mental health and addictions workers. Increased training may help to reduce staff turnover. In addition, the uptake of best practices and evidence-based approaches in the delivery of Mental Wellness programs can be improved through training, as workers become exposed to new and evidence-based approaches through the training that they receive. The Program should offer options for training that reflect community needs.

Recommendation 5 - Implement an improved performance measurement strategy.

The Program should work more closely with internal Health Canada partners, community partners and National Aboriginal Organizations to:

• improve the quality and consistency of data collected by the Program in all regions;
• collect information on program participation (numbers participating and frequency of participation), the uptake of healthy behaviours by participants, and changes to mental health, addictions, and overall health; and
• increase timeliness of data availability to have national level aggregated data available within two years of its collection.

In addition, the Program should work with internal Health Canada partners to collect better financial data on program expenditures, which includes the planned and actual expenditures by program and the tracking of reallocations between programs.

An improved and implemented Performance Measurement Strategy would assist with assessing levels of participation in programs and the impact of programming. This information would also be valuable for planning and reporting purposes and for assisting in decision making at all levels: community; regional; and national.

Appendix A -Community Sampling Strategy

The total number of communities visited for this evaluation was 27. Selection of the 27 communities was done taking into account the available budget, and a sampling strategy consisting of seven criteria reflective of community characteristics. These included:

• regional distribution;
• community population size;
• isolation factor;
• funding type;
• number of programs delivered in the community;
• amount of funding received by community; and
• how long communities have been receiving MHA funding.

Initially the sampling strategy was used to randomly select 36 out of 631 potential communities across Canada. Health Canada staff then considered the need to make sure the evaluation did not add strain to communities in crisis^{Footnote 14}, and reduced the list to 27 on the basis of institutional knowledge about the circumstances of these communities. Communities in crisis were, however, given the opportunity to participate in the evaluation community visits at a later date (visits took place between fall 2011 and spring 2012) and the data collection timelines were adjusted to meet the needs of the communities. In a few instances when it was not deemed prudent to include the community in the evaluation, another community with similar traits (using the seven sampling criteria, listed above) was selected.

Additionally, two Inuit communities south of the 60th parallel were added to the original sample of two, bringing the total to four (thus Inuit communities were oversampled in comparison to their proportion of all First Nations and Inuit communities south of the 60th parallel). This oversampling was designed to allow for greater than proportional representation of Inuit communities. However, the final number of communities visited included only two Inuit communities, due to difficulties scheduling site visits.

Delays in approving, arranging, and completing community visits resulted in time pressures to complete all evaluation activities, including community visits, by March 2012. FNIHB Head Quarters worked closely with Regions to follow up with communities to take part in the evaluation and, where necessary, to find suitable replacement communities that had expressed a willingness to participate. In total 17 community visits were completed instead of the proposed 27.

Appendix B - Technical annex

The use of multiple lines of evidence facilitated the triangulation of data to support evidence-based findings and conclusions. The lines of evidence included:

Non-Community-based Stakeholder Survey and Key Informant Interviews

Survey questions and interview guides were developed and written in plain language and were also translated into French.

Telephone or in-person interviews were completed with representatives from FNIHB Headquarters and Regions (N=48), National Aboriginal Organizations (N=5), and treatment centre directors (N=10).

Interviews were also completed with 16 key informants across the Northern Territories, including: territorial representatives (e.g., departments of health and social services), community level service organizations, First Nations community representatives, Indian Residential School service providers, and tribal councils.

Community Visits - Interviews, Focus Groups, Surveys

In the First Nations communities visited (south of the 60th parallel), the following data collection methods were used:

• key informant interviews with community leaders, including Chiefs, Elders and/or Council Members (N=23);
• key informant interviews with community workers and health staff (N=68);
• key informant interviews with the community health directors (N=14); and
• distribution of 30 to 40 participant surveys in each community with 19 surveys completed.

In the Inuit communities visited (south of the 60th parallel), the following data collection methods were used:

• key informant interviews with community leaders (N=2);
• key informant interviews with community workers and health staff (N=10);
• key informant interview with the community health directors (N=3);
• key informant interview with the treatment centre director (N=1);
• in-person surveys completed with MHA Cluster program participants (N=23);
• one focus group with community workers and community leaders, stakeholders (N=8)^{Footnote 15}; and
• two focus groups with program participants (16 participants in total).

In addition, the Assembly of First Nations distributed the community health directors' survey to all community health directors in attendance at the Assembly of First Nations Health Forum in November 2011. No responses were received (more detail on the mitigation strategy is provided in section 3.6).

Database Review

The document and literature review included administrative data, performance measurement data and documentation, relevant databases and program files, and published literature. The database review was intended to obtain information on the background of the MHA Cluster programs and services. It provided insight into their delivery, including any trends in their implementation by year and by province. Several databases were reviewed and the evidence that was obtained from each is detailed below.

• NAYSPS Database provided information on the number of projects, the amount of funding received and type of funding, and the activities of suicide prevention projects by year and by region.
• Financial Data provided detail on the amounts spent per program by year.
• Youth Solvent Abuse Committee (YSAC) Database - provided data on the NYSAP clients, including their substance abuse, psychological and educational history, as well as their family and community supports.^{Footnote 16}
• Indian Residential School (IRS) tables provided information about the number of clients (former students or family members) who accessed resolution health support workers (RHSW), cultural support providers (CSP), counseling and transportation services by year.
• National Native Alcohol and Drug Abuse Program Treatment Centre Outcome Study provided data from an audit of all clients who entered a treatment program and a post-treatment survey for a sample of those that completed the treatment program. The report shed light on the differences between those who completed the treatment program and those who did not, as well as some self-reported measures of outcomes from the treatment program such as reduced substance abuse.

The feasibility of using data available in the reporting tool introduced in 2008, the Community Based Reporting Template (CBRT), was assessed for the fiscal years 2008/09 and 2009/10 in order to prepare a database of key mental health outputs over time and across sites. However, the National Summary Report on data from the CBRT was not developed for the first reporting year (2008/09) due to factors such as resources and data quality issues, and the NationalSummary Report on Data from the CBRT for the second reporting year (2009/10) was not finalized at the time of this evaluation.

Data Analysis Plan

In accordance with best practices in evaluation, multiple lines of evidence were used including primary and secondary, qualitative and quantitative data. The data collected was analyzed using the following procedures:

• Systematic review of data (qualitative and quantitative) extracted from the documents, in which summaries were created and conclusions were drawn based on the summaries.
• Statistical analysis where counts and percentages were generated to analyze quantitative survey data and appropriate charts created.
• Thematic analysis in which qualitative data from open text survey questions, focus groups and interviews was analyzed and responses were systematically reviewed and emergent themes were identified and categorized.
• Triangulation of data gathered from the literature review, document reviews, database review, stakeholder surveys, interviews and focus groups was conducted to synthesize data from the disparate sources and validate trends as part of the findings of this evaluation.

References

Aboriginal Healing Foundation. (2006). Final report of the Aboriginal Healing Foundation: Volume 1: A healing journey, reclaiming wellness. Ottawa, ON: Aboriginal Healing Foundation.

Barron, M. (2009). International approaches to Indigenous communities in crisis: Lessons learned. A literature review prepared for Health Canada, First Nations and Inuit Health Branch. In the Wake of a Storm: The resilience of First Nations Communities. First Nations Communities in Crisis Think Tank.

Bobet, E. (2010). Renewing NNADAP: Common themes from the regional needs assessments reports and the January 2010 NNADAP national renewal forum. Discussion paper prepared for NNADAP Regional Partners and members of the First Nations Advisory Panel. Available from http://www.nnadaprenewal.ca/sites/www.nnadaprenewal.ca/files/en/2010/07/common-themes-regional-needs-assessment-reports-and-january-2010-nnadap-renewal-national-forum.pdf

Bombay, A., Matheson, K., & Anisman, H. (2009). Intergenerational trauma: Convergence of multiple processes among First Nations peoples in Canada. Journal of Aboriginal Health, 5, 6-47.

Canadian Institute for Health Information. (2009). Mentally healthy communities: Aboriginal perspectives. Ottawa, ON: Author.

Christian, T.-M. (2009). Alberta regional needs assessment: Final report. Available from http://www.nnadaprenewal.ca/sites/www.nnadaprenewal.ca/files/10_03_08/en/Alberta_RNA_Final%20Report.pdf

First Nations Addictions Advisory Panel. (2010). Honouring our strengths: A renewed framework to address substance abuse issues among First Nations people in Canada. Available from http://www.nnadaprenewal.ca/en/honouring-our-strengths-renewed-framework-address-substance-use-issues-among-first-nations-people-ca

First Nations Health Council. (2007). Provincial forum on First Nations youth suicide: Courage, strength and identity. Available from http://www.fnhc.ca/index.php/initiatives/mental_wellness/

First Nations Information Governance Centre. (2005). First Nations regional longitudinal health survey 2002/03. Ottawa, ON: Author.

First Nations Information Governance Centre. (2011). First Nations regional health survey: RHS phase 2 (2008/10) preliminary results. Ottawa, ON: Author.

Government of Canada. (2010). Speech from the throne: A stronger Canada, a stronger economy, now and for the future. Ottawa, ON: Author.

Government of Canada. (2011). Here for all Canadians: Stability. Prosperity. Security. Ottawa, ON: Author.

Health Canada. (2006). Evaluation of First Nations and Inuit Health Branch's Brighter Futures and Building Healthy Communities Programs: Final Report. Ottawa, ON: Author.

Health Canada. (2007). Departmental performance report 2006/07. Ottawa, ON: Author.

Health Canada. (2007b). What is working, what is hopeful: Supporting community-based suicide prevention strategies within Indigenous communities. Ottawa, ON: Author.

Health Canada. (2008). 2008/09 report on plans and priorities. Ottawa, ON: Author.

Health Canada. (2009). Performance report 2005/06 to 2007/08: Mental Health and Addictions Cluster. Ottawa, ON: First Nations and Inuit Health Branch.

Health Canada. (2010a). Canadian Alcohol and Drug Use Monitoring Survey: Summary of results for 2009. Available from http://hc-sc.gc.ca/hc-ps/drugs-drogues/stat/_2009/summary-sommaire-eng.php#salvia

Health Canada. (2010b). National Native Alcohol and Drug Abuse Program. Available from http://www.hc-sc.gc.ca/fniahspnia/substan/ads/nnadap-pnlaada-eng.php

Health Canada. (2011a). Departmental performance report 2009/10. Ottawa, ON: Author.

Health Canada. (2011b). First Nations, Inuit, and Aboriginal health. Available from http://www.hc-sc.gc.ca/fniah-spnia/index-eng.php

Health Canada. (2011c). 2011/12 estimates: Part III - report on plans and priorities. Ottawa, ON: Author.

Health Canada. (2011d). First Nations, Inuit, and Aboriginal health: Indian Residential Schools Resolution Health Support Program. Available from http://www.hc-sc.gc.ca/fniah-spnia/services/indiresident/irs-pi-eng.php

Health Canada. (n.d.). Departmental performance report. Available from http://www.hc-sc.gc.ca/ahc-asc/performance/estim-previs/dpr-rmr/index-eng.php

Health Canada. (2012). Northern Region Mental Health and Addictions - Summary of Program Delivery and Key Findings 2005/10. Ottawa, ON: Author.

Hopkins, C., & Dumont, J. (2010). Cultural healing practice within National Native Alcohol and Drug Abuse Program/Youth Solvent Addiction Program Services. Mental Health and Addictions Division, Community Programs Directorate, First Nations and Inuit Health Branch, Health Canada. Available from http://www.nnadaprenewal.ca/sites/www.nnadaprenewal.ca/files/en/2010/08/cultural-healing-practicemedicine-within-nnadapNYSAP.pdf
http://www.ccsa.ca/2005%20CCSA%20Documents/ccsa-004028-2005.pdf

Lalonde, C. (2006). Place and health in First Nation communities: Is there a connection between governance and youth suicide? Presentation made at Health Canada Policy Forum, Ottawa.

Lavoie JG, Forget EL, Prakash T, Dahl M, Martens P, O'Neil JD (2010). Have investments in on-reserve health services and initiatives promoting community control improved First Nations' health in Manitoba? Soc Sci Med. 2010 Aug;71(4):717-24.

Leatt, Peggy. (2002). Sharing the Learning: Health Transition Fund Synthesis Series: Integrated Service Delivery. University of Toronto, Toronto.

Loppie-Reading, C., & Wein, F. (2009). Health inequalities and social determinants of Aboriginal peoples' health. Victoria, BC: National Collaborating Centre for Aboriginal Health.

McCormick, R., & Quantz, D. (2010). Improving mental health services and support in the National Native Alcohol and Drug Abuse Program. Available from the National Native Alcohol and Drug Abuse Program http://www.nnadaprenewal.ca/sites/www.nnadaprenewal.ca/files/en/2010/09/improving-mental-health-services-and-supports-national-native-alcohol-and-drug-abuse-program.pdf

Minore, B., & Katt, M. (2007). Aboriginal health care in Northern Ontario: Impacts of self-determination and culture. Choices, 13, 1-22.

Ogborne, A. C., Paglia-Boak, A. & Graves, G. (2005). Interventions supported by scientific evidence for substance abuse treatment, harm reduction and prevention. Available from the Addictions and Mental Health Community Programs Directorate, and the First Nations and Inuit Health Branch of Health Canada

Mussell, B., Cardiff, K., & White, J. (2004). The mental health and wellbeing of Aboriginal children and youth: Guidance for new approaches and services. Chilliwack, BC: Sal'i'shan Institute.

National Institute of Mental Health. (2011). Depressed teens with history of abuse less likely to respond to combination treatment. Available from http://www.nimh.nih.gov/science-news/2011/depressed-teens-with-history-of-abuse-less-likely-to-respond-to-combination-treatment.shtml

Polson-Lahache, W. (2012). National Native Alcohol and Drug Abuse Program (NNADAP) Renewal Engagement Review: Final Report. Ottawa, ON: Health Canada

PRA Inc. (2009). First Nations and Inuit health, Saskatchewan region, mental health and addictions needs assessment. First Nations and Inuit Health, Saskatchewan Region.

R.A. Malatest and Associates Ltd. (2012) Mental Health and Addictions Cluster Evaluation Report. Edmonton, AB.

Statistics Canada. (2011). Labour force activity (8), Aboriginal identity (8), highest certificate, diploma or degree (14), area of residence (6), age groups (12A) and sex (3) for the population 15 years and over of Canada, provinces and territories, 2006 Census - 20% sample data. Available from http://www12.statcan.gc.ca/census-recensement/2006/dp-pd/tbt

Waldram, J. B., Herring, D. A., & Young, T. K. (2006). Aboriginal health in Canada (2nd ed.). Toronto, ON: University of Toronto Press.

White, J., & Jodoin, N. (2007). Aboriginal youth: A manual of promising suicide prevention strategies. Calgary, AB: Centre for Suicide Prevention.

World Health Organization (WHO) (2008). Integrated Health Services - What and Why? Technical Brief No.1. Geneva.