Health Planning, Consultation and Research Projects - Synthesis Evaluation 2005-2006 to 2009-2010

Final Report

March 15, 2012

Acronyms

AFN: Assembly of First Nations
AHI: Aboriginal Health Institute
AHRCP: Aboriginal Health Research and Coordination Projects
CA: Contribution Agreement
CBRT: Community-based Reporting Template
CDC: Communicable Disease Control
CIHR: Canadian Institutes for Health Research
FN: First Nations
FN/I: First Nation and Inuit
FNIHB: First Nations and Inuit Health Branch
FTEs: Full Time Employee
GoC: Government of Canada
HQ: Headquarters
HCL: Health Consultation and Liaison
HGIS: Health Governance and Infrastructure Support
HPM: Health Planning and Management
ITK: Inuit Tapiriit Kanatami
NAHO: National Aboriginal Health Organization
NGO: Non-governmental Organization
O&M: Operations and Maintenance
SFT: Speech from the Throne

Executive Summary

This synthesis evaluation of selected Health Governance and Infrastructure Support (HGIS) components assesses program relevance and performance (effectiveness, efficiency, and economy) between 2005/06 and 2009/10. Findings highlight outcomes, lessons learned best practices and challenges to program objectives for: 1) Health Planning Management (HPM); 2) Health Consultation and Liaison (HCL) and 3) Aboriginal Health Research and Coordination Projects (AHRCP).

The HPM component supported communities in the planning and development of health services and program delivery models as well as health infrastructure. Activities aimed to: 1) enable recipients to design health programs, develop health plans, establish services and/or allocate funds according to health priorities; 2) ensure that recipients have an optimized flexibility for health programming and services, and 3) strengthen and enhance the accountability of recipients regarding the management and the delivery of health programs and services.

The HCL component increased consultations and partnerships of Aboriginal organizations and governments and worked to: 1) establish and maintain productive lines of communication and exchanges on policy, research, evaluation and program delivery issues; 2) ensure substantive involvement of Aboriginal leaders and community representatives in decisions relating to health care policy and delivery; 3) develop and maintain health consultation processes; and 4) develop Aboriginal awareness and build capacity to undertake consultation on health matters.

The AHRCP component enhanced knowledge through: 1) capacity-building, 2) information dissemination, 3) knowledge translation, 4) original research, 5) data gathering and analysis and 6) funding to the National Aboriginal Health Organization and the Canadian Institutes for Health Research, for various research activities, including consultations, conferences and health surveys.

The HPM and HCL components supported the HGIS cluster Objective 1: to support First Nations and Inuit control over Health Programs and Services. The AHRCP component supported Objective 2: to improve First Nations and Inuit capacity to generate and access information and build health knowledge.

Since these components are designed to support the implementation of other First Nations and Inuit Health Branch (FNIHB) programs and activities, the current synthesis draws from previous evaluations of FNIHB clusters, which included document reviews, interviews, surveys, case studies and site visits. The evidence used in the synthesis evaluation was also drawn from a literature review focussed on programs related to the objectives and strategies of these HGIS components. Overall, the synthesis evaluation approach was to synthesize the range of available evidence to address the core issues with respect to the components required under the Policy on Evaluation.

Relevance

HPM, HCL and AHRCP components have been consistent with GoC priorities in the areas of health services, and Aboriginal people, as defined by Speeches from the Throne (SFT). They are also consistent with Departmental strategic outcomes for 2005/06, 2006/07, 2007/08, 2008/09 and 2009/10: Better Health Outcomes and Reduction of Health Inequalities between First Nations and Inuit and Other Canadians. Activities supported First Nations and Inuit Health - Strengthened community programs. Additional activity areas linked to Strategic Outcomes for 2009/10 included First Nations and Inuit Health Programming and Services. The components are also central to one of the three responsibility areas set out in the departmental Report on Plans and Priorities for 2005/06, 2006/07, 2007/08, 2008/09 and 2009/10.

The role of the federal government in First Nation and Inuit (FN/I) health has been clearly supported through the Canada Health Act, which acknowledges that maintaining and improving health requires continued access to quality health care, as well as the Federal Indian Health Policy (1979), which aims to achieve an increasing level of health in Indian communities, generated and maintained by Indian communities themselves.

There is support in the broader research literature for the relevance of self-determined First Nations health services for meeting Government of Canada priorities of a sustainable Canadian health care system. The literature also supports the approach to increase flexibility in health funding among FN/I as one way to improve health outcomes, suggesting that increased levels of local control can improve the relevance of these services in meeting local needs.

Performance - Effectiveness

The design and activities of the HPM and HCL components support FN/I planning and management of health programs and services aim to support increased levels of local control. The theory that funding models supporting increased local control should improve services is sound. It is also clear from program descriptions that the activities under these components are overall aligned with the approaches to local control proposed by the literature. However, the performance measurement data that is available tends to indicate that the increased transfer of control has been minimal.

Evidence from other FNIHB program clusters (Children and Youth, Communicable Disease Control, Chronic Disease and Injury Prevention, Environmental Health and Environmental Research) indicates that capacity for these services and programs have been supported by HPM, and HCL activities. Some evidence from these evaluations have also indicated that flexibility for FN/I to identify their own needs and priorities, and to make allocations of expenditures accordingly, was positive with respect to these programs.

Performance - Efficiency and Economy

The synthesis evaluation assessed, where possible, whether the HGIS components had utilized resources in a way that minimized the resources required to achieve the expected results, and whether there were ways to improve the efficiency and economy of program delivery. A lack of activity-based accounting broken down by agreement type (Contribution Agreements, Branch FTEs, and operations and maintenance), as well as the unavailability of data showing the links between activities and results, has made it difficult to assess the efficiency and economy of the components overall.

Conclusions and recommendations

Conclusions

Overall, the synthesis evaluation found that these HGIS components are relevant with respect to Government of Canada priorities in the area of FN/I health, including those set by subsequent Speeches from the Throne, as well as those identified by the office of the Prime Minister. This includes the priority of providing universally accessible and sustainable health care system for all Canadians, and for improving the lives and health of Aboriginal people. The objective of increasing self-determination is further in line with Government of Canada priorities set through Speech from the Throne commitments, as well as its commitments as a signatory of the UN declaration on the Rights of Indigenous Peoples. In addition, the literature on Aboriginal and indigenous health supports the relevance of the program activities, and validates the approach of local control as a means to improve health outcomes.

The evidence presented illustrates that these HGIS components have been effective in addressing their main objectives, with the exception of transitioning more FN/I communities to Transfer/Flexible funding agreements. While the literature tends to support the approach of local control taken by the program, there is a lack of performance information to clearly assess whether the program activities have led to effective forms of local control. This is supported by evaluation findings of other program clusters, which indicates mixed results about whether these programs could be characterized as giving FN/I higher levels of ownership or control, and whether self-determination has been a significant factor in improving health outcomes. Although one study in the literature review confirmed that the Flexible/Transfer funding model had led to improved rates of avoidable hospitalization in Manitoba.

The synthesis evaluation also confirms that the FNIHB approach to funding models and self-determination is likely sound, given the available performance measurement data and other relevant evidence. While it may have been too early to see the impact of funding models in this synthesis evaluation, evidence in the literature supports the theory that more flexible funding models is linked with positive health outcomes.

A number of potential barriers to implementing funding models were identified in the synthesis evaluation (including the willingness of participants to transition to funding models that offer higher levels of local control, the capacity of communities and human resource infrastructures to make the transition, and community leadership), and future evaluations of community programs should include specific assessments of the reasons for slow transitions to Flexible/Transfer funding models. Better performance measurement data (on activities, outcomes, and financials) would help support these assessments, and would also help leaders and decision-makers identify which types of program activities/supports are most effective in supporting this transition. These will be addressed in future evaluations.

Recommendations

This synthesis evaluation report has made no specific recommendations since this synthesis evaluation drew on other program evaluations, each of which had its own recommendations that should not be duplicated here.

Background and Context

This synthesis evaluation of selected Health Governance and Infrastructure Support (HGIS) components assesses program relevance and performance (effectiveness, efficiency, and economy) between 2005/06 and 2009/10 in accordance with the 2009 Government of Canada Policy on Evaluation. The synthesis evaluation highlights the attainment of outcomes, lessons learned, best practices and challenges to program objectives. It also meets the requirement to evaluate all Health Canada direct program spending every 5 years under the Financial Administration Act.

As illustrated in the figure below, the synthesis evaluation covers the following three components of HGIS for 2005/06-2009/10: 1) Health Planning Management (HPM); 2) Health Consultation and Liaison (HCL) and 3) Aboriginal Health Research and Coordination Projects (AHRCP). Other components of the HGIS structure have been evaluated separately. Since these components are designed to support the implementation of other First Nations and Inuit Health Branch programs and activities, the current synthesis evaluation draws from previous evaluations to address the performance of these HGIS components. The relevance issues draw from key documents that indicate the priorities and mandates of Health Canada and the Government of Canada.

Health Planning Management and Health Consultation and Liaison components supported the HGIS cluster Objective 1: to support First Nations and Inuit control over Health Programs and Services. The Aboriginal Health Research and Coordination Projects component supported Objective 2: to improve First Nations and Inuit Capacity to Generate and Access Information and Build Health Knowledge.

Program Description

The First Nations and Inuit Health Branch (FNIHB) worked with First Nation and Inuit (FN/I) communities and national organisations to improve health outcomes and reduce health inequalities between FN/I and other Canadians. For the 2005/06 - 2009/10 timeframe, Health programs and services were designed and delivered to respond to recognized challenges, such as higher than average rates of injury, higher overall disease burden, and population concentrations in difficult-to-serve small, rural and remote communities. This makes the coordination among levels of government and non-governmental organizations (NGOs) essential to improve service efficiencies and increase responsiveness to FN/I needs.

Activities under these HGIS components for this period aimed to 1) increase FN/I control over health programs and services, and 2) improve FN/I capacity to generate and access information and build health knowledge. As such, the HGIS components HPM, HCL and AHRCP provided planning and other support to management, depending on the funding model in place for each community, for programs and services delivered in FN/I communities across Canada. This included those offered under Community Programs (such as Children and Youth, Mental Health and Addictions, and Chronic Disease and Injury Prevention), Health Protection (such as Communicable Disease Control and Environment Health and Environmental Research) and Infrastructure Support (such as Health Facilities and Capital Program).

Program Areas

Health Planning and Management

Health Planning and Management (HPM) supported communities in the planning and development of health services and program delivery models as well as health infrastructure (Health Director or Health Committee) established at the community, regional, or national levels. The activities were created to 1) enable recipients to design health programs, develop health plans, establish services and/or allocate funds according to health priorities; 2) ensure that recipients have an optimized flexibility for health programming and services, and 3) strengthen and enhance the accountability of recipients regarding the management and the delivery of health programs and services.

As per the logic model in Appendix A, HPM worked to produce the immediate outcomes of: 1) program delivery based on health priorities, and 2) FN/I community ownership, effective control and capacity to manage and administer health programs and services. These immediate outcomes were designed to contribute to the final outcome of strengthened and sustainable health planning, administration and delivery system at the local, regional and national levels. These intended outcomes aimed to contribute to the overall Branch's objective of increased FN/I control over health programs and services, which links to the FNIHB strategic outcome of: Better outcomes and reduction of health inequalities between First Nations and Inuit and other Canadians.

FNIHB formal strategies for meeting this strategic outcome for the 2005/06 - 2009/10 period are also consistent with the design of HPM, including: 1) transferring existing health resources to First Nations and Inuit control within the time frame to be determined with them; 2) supporting actions to address health status inequalities affecting First Nations and Inuit communities; and 3) establishing a renewed relationship with First Nations and Inuit.

Health Consultation and Liaison

Health Consultation and Liaison (HCL) supported communities by increasing consultations and partnerships of Aboriginal organizations and governments to improve health outcomes. Organizations such as the Assembly of First Nations (AFN) and Inuit Tapiriit Kanatami (ITK) are provided funding in order to: 1) establish and maintain productive lines of communication and exchanges on policy, research, evaluation and program delivery issues and concerns between Aboriginal people, health care delivery agencies, and other levels of government; 2) ensure substantive involvement of Aboriginal leaders and community representatives in decisions relating to health care policy and delivery; 3) develop and maintain health consultation processes; and 4) develop Aboriginal awareness and expertise in the field of health care and build capacity to undertake consultation on health matters.

As stated in the logic model in Appendix B, HCL has the intended immediate outcome of: increased opportunities for Aboriginal people to participate and influence policy and program development, which is designed to support the final outcome of greater involvement and effective role of Aboriginal people in the governance of the health care system and service delivery. These are in line with the Branch strategies listed above related to transferring control of health resources and addressing priorities identified by FN/I.

Aboriginal Health Research and Coordination Projects

Aboriginal Health Research and Coordination Projects aimed to enhance knowledge in the area of Aboriginal health. Funding supports Aboriginal health research projects including community, regional, national, and international initiatives. Activities include 1) capacity-building, 2) information dissemination, 3) knowledge translation, 4) original research, and 5) data gathering and analysis. One example of the type of activities supported is the funding to the National Aboriginal Health Organization (NAHO) and the Canadian Institutes for Health Research (CIHR), for various research activities, including consultations, conferences and health surveys.

As per the logic model in Appendix C, the intended immediate outcome of AHRCP is improved dissemination of knowledge and evidence based information on policies and programs, which supports the final outcome of access to health information through research and knowledge. These outcomes are in keeping with the Branch strategy to support decisions of FN/I through identified priorities.

Funding Models

Before the funding models that were put in place in 2005, FNIHB had three types of Consolidated Contribution Agreements − General, Integrated, and Transfer contribution agreements. With these agreements, FNIHB and each recipient identified a funding model that determined the financial arrangements a recipient had to follow across all of the programs and services it delivered under that agreement. Since these funding models could not be adapted to respond to recipient's needs, in most cases a given recipient had two or more contribution agreements (CA) (sometimes using more than one type of funding model) to ensure that funding was provided based on recipient's capacity and desire to deliver programs and services.

With Treasury Board approval of FNIHB program authorities in 2005, FNIHB introduced four new funding models − Set, Transitional, Flexible and Flexible Transfer. One or more of these funding models could be accommodated in a single funding agreement with each recipient. This feature allowed funding arrangements structured to accommodate the financial and reporting requirements of different program clusters and gave built-in flexibility to address the unique levels of managerial capacity of a recipient, as well as changes in that capacity during the period of an agreement. The funding model used to support the delivery of a specific program was assessed separately for each recipient. With the implementation of these funding models, programs were grouped into a single CA, and some recipients entered in a health planning process.

The four funding models varied in: 1) the type of planning required; 2) the flexibility to move funding within and among program authorities; 3) the ability to use a surplus and/or to carry forward unspent funds from one fiscal year to the next; 4) the duration of agreement; and 5) financial and activity reporting requirements (FNIHB Funding Models Framework, 2007).

Recipients were given the flexibility to adapt program delivery to community needsincluding different levels of control, flexibility, reporting requirements, and accountability. Regions were able to negotiate contribution agreements through discussions and consultations between FNIHB and the recipient. Access to more flexible funding models was intended to be phased in over time through a capacity-building approach based on increasing levels of responsibility. Figure Two illustrates the continuum of control available to a recipient through the various levels and mechanisms of funding.

**Figure 2 - Recipients' Continuum of Control**

In the Set funding model, the recipient can only use the funds for the purpose described in the contribution agreement. There was limited flexibility over re-directing funds. The Transitional model allowed re-allocation of funding component/clusters within the same Authority. Recipients with the written approval of the Minister were able to carry forward program funding for reinvestment in the following fiscal year within the same Authority. The Flexible funding model allowed recipients the ability to re-design new programs and redirect funding to the other programs with the condition that the mandatory program components set by the contribution agreement have been delivered. The Flexible Transfer model gave complete flexibility to the recipient over program delivery decisions towards health priorities.

The recipient was responsible for ensuring that the programs were delivered in accordance with the terms and conditions of the contribution agreement. FNIHB headquarters and regional offices, however, shared the responsibility with the FN/I communities to design, monitor, and report on community-based cluster programs.

The funding arrangements for the contribution agreements were determined on an individual basis, and on criteria related to demonstrated capacity in program planning, management and delivery, financial management, and administration. The funding provided for implementation of the health plan included both program funding and management funding. For the program funding, a funding arrangement could have utilized more than one funding model depending on the health programs included and the recipient's past performance in managing each one. For example, the Flexible funding model could have been applied to most of the programs but the Set model might have been used for a specific program in which the recipient had experienced challenges in meeting program requirements or the program terms and conditions do not allow flexibility. If the recipient had not demonstrated a major improvement in managing that program during the health planning process, then the Set model would apply for a certain period, giving the recipient time to build their capacity to manage the program successfully.

As a result, the recipient would have limited flexibility in the allocation of funds provided for that program. The Transitional, Flexible and FlexibleTransfer funding models provided greater flexibility to allocate funds. This flexibility came with increased responsibility for the management and delivery of programs and services. Through carrying out the planning process, the recipient should have had an idea of what will be required to manage programs and services effectively, and how much additional responsibility it is capable of taking on. The regional office would then discuss the funding models with the recipient and recommend the most appropriate funding arrangement. Regardless of the level of flexibility and control provided in the agreement, the recipient must meet various requirements for program activities and reporting to remain accountable to its members and to funders. Starting in 2008-2009 fiscal year, recipients signing a new agreement or renewing their agreement were required to complete, as part of their reporting requirements, the Community-based Reporting Template (CBRT) and to prepare financial reports. Recipients were expected to examine and report on how successful they were in managing health services and to determine any changes in health status.

All recipients working with Flexible and Flexible Transfer funding models were also required to carry out an evaluation every five years. The recipient was expected to regularly review and update its health plan in discussion with its members and the region. Updates were based on findings of the recipient's annual reviews and 5-year evaluations and reflected changes in priority health issues, program activities, budget, and management and administration. Also, recipients had to update other aspects of their health plan, including the training and evaluation plans, as well as certain plans and policies to reflect any changes in federal or provincial legislation and recipient circumstances. Recipients with multi-year program plans or multi-year work plans also had to update their plans at least annually as part of their annual review. Set agreements follow a prescribed program plan.

Funding

Table one includes the actual expenditures for 2005/06 to 2009/10 broken down by Component (HPM, HCL, and AHRCP). Each line includes Health Canada salaries and O&M expenditures. The Aboriginal Health Research and Coordination Projects includes $5M annual funding to the National Aboriginal Health Organization, as well as annual contributions for specific research projects. The higher overall funding for Health Planning and Management between 2005/06 and 2007/08 are due to funds for Health Services Transfer, Management and Support, and planning activities which were provided for these years to support the recipients' continuum of control.

Table 1 - Actual Expenditures by Component 2005/06 - 2009/10 ($)
Component	2005/06	2006/07	2007/08	2008/09	2009/10	Total
Health Planning and Management	218,601,288	217,079,506	222,897,222	89,586,073	97,323,812	845,487,901
Health Consultation and Liaison	10,776,243	11,164,509	17,162,484	19,563,457	20,469,957	79,136,650
Aboriginal Health Research and Coordination Projects	5,399,072	7,267,260	8,779,157	10,915,198	8,818,727	41,179,414
Total	234,776,603	235,511,275	248,838,863	120,064,728	126,612,496	965,803,965

Source: Chief Financial Officer Branch - Financial Records.

Synthesis Evaluation Context

Synthesis Evaluation Objectives

The synthesis evaluation was conducted to meet the Treasury Board requirement to evaluate selected HGIS components according to its activities between 2005/06 and 2009/10. The synthesis evaluation also supports the effective management of these HGIS components at the program and branch levels, and provides a synthesis of evidence to demonstrate its contribution to branch and departmental objectives.

Synthesis Evaluation Approach

This report presents an overview of findings, conclusions and recommendations from a range of reviews and evaluations of FNIHB programs. In addition, background literature related to the relevance and performance of strategies in the area of indigenous self-determination in health was reviewed to draw implications for the approach taken. While the contexts of these groups differ from those of FN/I in Canada, the lessons and evidence drawn from research into their experiences is relevant to the current synthesis evaluation.

The synthesis evaluation assesses the relevance of selected HGIS components according to federal government objectives and departmental strategic outcomes. It assesses relevance according to program alignment with the roles and responsibilities of the federal government. It also assesses whether the design of these components has been oriented to address the needs of FN/I, and to be responsive to the different and changing needs of FN/I.

Performance is defined as effectiveness as well as efficiency and economy. Effectiveness questions assess progress towards expected outcomes, as defined by the program logic models (Appendices A, B and C). Efficiency and economy refers to the assessment of resource utilization in relation to production of outputs and progress toward expected outcomes. Evaluation questions are specified in Table 2.

Table 2 - Core Issues and Synthesis Evaluation Questions
Core issues	Description	Synthesis Evaluation Questions
Relevance	Linkages between program objectives and (i) federal government priorities and (ii) departmental strategic outcomes	Are these HGIS components aligned with Government of Canada (GoC) priorities? Are the components aligned with departmental priorities and strategic outcomes?
	Roles and responsibilities of the federal government in delivering the program	Are the components consistent with federal roles and responsibilities?
	Assessment of the extent to which the program continues to address a demonstrable need and is responsive to the needs of Canadians	Is there a need for increased FN/I control over health programs and services? Is there a demonstrable need for improved health outcomes for FN/I?
Performance: Effectiveness	Support for FN/I Control over Health Programs and Services	Have the HPM and HCL components supported FN/I control over health programs and services? Are there any barriers or challenges to implementing increased FN/I control over planning?
	Effectiveness of Increased FN/I control over Health Programs and Services	Does increased control over programs and services improve outcomes?
	Improved Health Information and Knowledge	Has the AHRCP component improved FN/I Capacity to Generate and Access Information and Build Knowledge? Are there any barriers or challenges to improved FNI capacity to generate and access information and build knowledge?
Performance: efficiency/ economy	Assessment of resource utilization in relation to production of outputs and progress toward expected outcomes	Have the resources for these HGIS components been used for delivery approximated the minimum required to achieve the expected results? Are there ways to improve the efficiency and economy of the delivery of these components?

Synthesis Evaluation Methodology

The synthesis evaluation findings were based on a range of evidence from a number of reviews and program evaluations, which included document reviews, interviews, surveys, case studies and site visits. The evidence used in the synthesis evaluation was also drawn from a literature review focussed on programs related to the objectives and strategies of these HGIS components. Overall, the synthesis evaluation approach was to synthesize the range of available evidence to address the core issues with respect to the components required under the Policy on Evaluation.

Data Collection Methods

Program Evaluations and Reviews

Since one aim of these HGIS components was to support the delivery of other programs, the approach and successes of increased FN/I control over health delivery has implications outside of HGIS activities. As such, the current synthesis evaluation draws from other program evaluations and reviews, and draws higher-level implications for effectiveness. These previous evaluations included: Children and Youth cluster, Communicable Disease Control cluster, Chronic Disease and Injury Prevention cluster, Environmental Health and Health Research cluster, and the evaluation of the National Aboriginal Health Organization. Each of these evaluations included a range of methods, including: background literature review, document review, key informant interviews, surveys, financial data review and case studies.

Literature Review

A review of the existing scholarly and grey literature related to Aboriginal and indigenous control and self-determination in health programming was used as a line of evidence to demonstrate the relevance of the approach of increased flexibility of funding. The approach of improving health delivery and outcomes through increased local control and involvement in the areas of planning, administration and delivery has been covered in these sources from a Canadian and international perspective. Most notably two scholarly sources (Lavoie et al, 2007; Lavoie et al, 2010c) assessed the impacts of FNIHB funding arrangements, and provide a strong source of evidence on effectiveness.

Limitations

A main challenge to the synthesis evaluation was the complexity of the range of program activities that make up these HGIS components. While some implications could be drawn, and theoretical assumptions could be made, the synthesis evaluation was limited with respect to conclusions about how the components interact, and how some of the more disparate activities contribute to overall objectives. This was to be expected, especially since the component objectives imply support for other branch programs not included under these HGIS components.

Secondly, the synthesis approach involves no primary research into program effectiveness and relies mainly on secondary sources of information. This is limited with respect to, for instance, assumptions about the degree to which the community control over the planning and delivery of programs and services supports the effectiveness of these programs and services in improving health outcomes. While the links between health planning and program planning are theoretically relevant, the program cluster evaluations included in this synthesis evaluation did not address these links directly.

To partly mitigate these limitations, the synthesis evaluation included external scholarly sources that support an assessment of outcomes of FNIHB funding model approaches, as well as a range of international studies that support the theoretical approach of increased local control over planning and implementation of health services and programs as a means to improve delivery and health outcomes. These external sources provide additional support for the relevance and effectiveness of the overall program approach to FN/I local control over health programs.

Relevance

The synthesis evaluation assessed the questions of relevance as defined by the 2009 Policy on Evaluation, including whether these HGIS components are aligned with Government of Canada priorities and departmental strategic outcomes, as well as whether they are consistent with federal roles and responsibilities. Relevance issues also included whether there is a need for increased FN/I control over health programs and services, and whether there is an ongoing need to improve health outcomes through this increased control.

Are these HGIS components aligned with Government of Canada (GoC) priorities?

The components have been consistent with Government of Canada priorities in the areas of health services, and Aboriginal people, as defined by subsequent Speeches from the Throne (SFT). The 2006 SFT set a priority for a timely and sustainable health care system through innovative delivery systems for a universally accessible and equitable system as per the Canada Health Act. The 2007 SFT committed to improving the lives of Aboriginal people and the health of Canadians, and the 2009 SFT was based on key consultations with Aboriginal leaders to direct economic stimulus towards infrastructure and support for First Nations.

The 2010 SFT acknowledged support for the UN declaration on the Rights of Indigenous peoples in a way that is consistent with Canadian Constitution and laws. In 2010, Canada became a full signatory to the declaration, indicating a renewed support for self-determination of First Nations. The 2011 Speech from the Throne commitment to improve the social participation of First Nations is further supported by the objective to increase First Nations control over health programs and services.

There is support in the broader research literature for the relevance of self-determined First Nations health services for meeting Government of Canada priorities of a sustainable Canadian health care system, for instance through the burden of avoidable hospitalization due to decreased access to local services (Lavoie et al, 2007; Lavoie et al 2010b; Lavoie et al 2010c).

In line with the principles of the Canada Health Act, these HGIS components also contribute to one of the five priorities of the Government of Canada to deliver the health care services that Canadians need, as set out by the office of the Prime Minister.

Are these HGIS components aligned with a departmental priorities and strategic outcomes?

The objectives of these HGIS components have been closely aligned with a departmental strategic outcome and the expected results associated with them.

Departmental strategic outcomes for 2005/06, 2006/07, 2007/08, 2008/09 and 2009/10 included: Better Health Outcomes and Reduction of Health Inequalities between First Nations and Inuit and Other Canadians. Activities supported First Nations and Inuit Health - Strengthened community programs. Additional activity areas linked to Strategic Outcomes for (2009/10) included First Nations and Inuit Health Programming and Services.

These components are central to one of the three responsibility areas set out in the departmental Report on Plans and Priorities for 2005/06, 2006/07, 2007/08, 2008/09 and 2009/10. Health Canada supports basic primary health care services in First Nations, including: support for remote and isolated First Nations communities, home and community care in First Nations and Inuit communities, community-based health programs for First Nations and Inuit - focussing on children and youth, mental health and addictions, and chronic disease and injury prevention, environmental health and public health programs for First Nations. The need to support these programs through effective planning and management is addressed through these HGIS components.

Are these HGIS components consistent with federal roles and responsibilities?

The role of the federal government in FN/I health has been clearly supported through the Canada Health Act, which acknowledges that maintaining and improving health requires continued access to quality health care. The role of the federal government is also captured in the Federal Indian Health Policy (1979). The goal of the Policy is to achieve an increasing level of health in Indian communities, generated and maintained by Indian communities themselves. The improvement of access to health care for FN/I communities remains a mandate of the federal government, and is consistent with the roles and responsibilities as set out in the Canada Health Act and the Indian Health Policy.

Is there a need for increased FN/I control over health programs and services?

The literature review conducted focussed on both the indicators of effectiveness as per scholarly research that assessed the links between FN/I levels of self-determination, self-government and local control of health policy and implementation. It also considered any relevant research into the overall effectiveness of increased local control among indigenous groups internationally and improved health outcomes. While these indicators are not directly applicable to the FN/I situation in Canada, they do provide indications of the relevance and effectiveness of increased local control, and help to validate the theory behind these HGIS components.

The 2006 Evaluation of the First Nations and Inuit Health Transfer Policy, as well as a review and analysis of other documentation found that FN/I communities need the following to make sure that program design and delivery is optimal:

increased flexibility in program planning and design;
a greater role in planning and managing health programs and services;
to base their programs and services on community needs, not strictly on policies, funding formulas and contribution agreements;
to build more capacity to assume these roles;
a high level of community involvement and ownership to ensure relevant and flexible programs and services; and
flexible planning and funding processes to allow FN/I to respond to their needs.

In addition, the background literature on control and self-determination in health supports these strategies as a way to improve both the local relevance of programs and their health objectives. These sources tend to focus on the effectiveness of self-determination strategies, and indicate potential barriers or challenges to the longer-term outcome of improved health. Mills et al (1990) provide a general definition of decentralization of health: "the transfer of authority, or the dispersal of power, in public planning, management and decision-making from the national level to sub-national levels of government, or more generally, from higher to lower levels of government". For example, Wamai (2009) concluded that decentralization has been pursued by both developing and developed nations as a cost-saving measure, and has been promoted by the WHO as a way to encourage local development. As such, the most general definition of decentralization concurs with the principles of self-determination.

In the Canadian context, Saltman et al (2007) point out that the creation of Regional Health Authorities is often described as a decentralization of control from the federal and provincial levels. However, since this also involved taking control from individual hospitals and health centres, it also involved a measure of centralization. The move towards self-determination and increased control over health services in First Nations through BC tripartite agreements may also involve some measure of centralization, since many of the programs delivered through FNIHB have historically involved direct funding transfers to local communities, and the new arrangement includes the creation of First Nation health authorities to work alongside the Regional Health Authorities in BC. Lavoie et al (2010a) focus on the processes of contracting agreements between central governments and indigenous groups, and similarly conclude that there is a spectrum of contractual arrangements at work in First Nations and government relations, and that there is not a clear distinction between classical and relational contracting arrangements.

It is in this context that sources caution against assumptions that decentralization is a common concept that can be applied to different settings, or can be assumed to have the same implications and outcomes across these settings (e.g., Lavoie et al, 2010a; Mills et al, 1990; Schwartz et al, 2002; Smith, 1997). In line with the observation that self-determination means different things and has different implications across different contexts, the literature also indicates a series of areas that require attention if increased control is to be effective. These include the need for effective performance monitoring, coordination between agencies and partners relevant to health outcomes, local capacity to identify priorities and engage planning, and the capacity of central authorities to provide sanctions and incentives for meeting performance standards.

Smith (1997) concludes that, while there are several options over the types of governmental structures to support decentralization, the decisions are bound to be politically charged. Government planners and community leaders did not always share the same goals, and there is a strong need to establish coordination across departments, since health objectives are shared by a number of different agencies and program centres.

The World Bank (2011) discusses the impacts of decentralization of health authority, for which they argue little evidence exists. They contend that the theoretical impact on improving health services depends on whether the existing centralized authority is not adequately informed about local health needs. They also warn that the decentralization of health authority within developing nations should be approached with caution, since the local capacities to take this authority are often weak, and may lead to poor outcomes. Similarly, Mills et al (1990) caution that decentralization can have many different side-effects, including the deconcentration of services, gaps in human resources as local authorities become hiring authorities, and an overall lowering of service integration. Schwartz et al (2002) also found that decentralization led to decreases in allocative efficiencies in the Phillipines, as well as lower levels of spending on public health and overall reduced health outcomes in the area of immunizations.

Bossert (1998) studied Columbia as a case of health decentralization, and concluded that, while decentralization can support innovation in decision-making, given that local authorities are inclined to make decisions that are different from central authorities, there is a need to establish 'decision-spaces' for these local authorities to make decisions that are oriented to better performance in health. This includes incentives oriented to local conditions and needs, which needs to be supported by information systems that can effectively monitor performance and allow central authorities to apply sanctions and incentives effectively. Hutchinson and Lafond (2004) also noted that, while developing countries internationally have moved to decentralize health care as a means to increase outcomes, the process has more often resulted in reductions in available evidence to support performance measurement, including a reduction in comparable data and lower capacities of local authorities to produce performance measurement overall.

Leiberman et al (2002) notes that arguments for decentralization internationally have tended to focus on the expenditure benefits, while avoiding the need to reallocate roles in health to different levels of government. He concludes that the effectiveness of decentralization depends on: 1) the substantial role of externalities, 2) the high degree of specialization required, 3) the critical role of quality and timeliness, and 4) the high level of knowledge required to participate in the health care system at all levels (p.155). In his study of decentralization of health in Indonesia, the Philippines and Vietnam, he concluded that the stewardship functions of the central authorities needed key support through local financial data, policy-setting and spending allocation authority

Putnam (1993) found that, in Italy, decentralization was more likely to be effective in localities that had higher levels of social capital as indicated by a higher concentration of voluntary civic organizations that created higher levels of performance expectations, and established higher trust exchanges between individuals. Wamai (2009) found that the role of non-governmental organizations (NGO) was critical in reaching the health and development goals of decentralization. He also found that the supportive role of NGOs requires a high level of capacity within NGOs themselves, as well as high levels of trust and engagement between governments and NGOs.

Again in the Canadian context, Lavoie et al (2010b) describe changes in the First Nations health care system as a shift from assimilation to participatory processes that have involved a measure of political autonomy. This has been advocated at the policy level as a means to improve health outcomes and reduce health inequalities. They describe the historical context of the Constitution Act that divided the jurisdiction for health care as a provincial jurisdiction and First Nations health as a federal jurisdiction. This established an on-reserve health system that was designed to complement the broader provincial system, which varied according to levels of isolation, and the higher levels of primary care services in remote communities. Lavoie et al (2010a) also found support for the FNIHB funding model approach. They concluded that continuous service delivery in First Nations is not supported by classical contractual environments and that relational contracts are more flexible and better accommodate the needs of community-based groups.

They note that the Indian Health Policy of 1979 specifically aimed to increase levels of health in First Nation communities as "generated and maintained by the communities themselves" (p.95). However, they identify pressures on First Nations communities that present challenges to achieving health, such as the increased populations of reserve residents who receive services, but are not registered as First Nations people, and the growth of health areas such as health promotion - both of which increase the burden of health service provision without concurrent increases in funding.

Overall, the approach to increase flexibility in health funding among FN/I as one way to improve health outcomes is supported in the literature. The activities of the HGIS components aimed at increasing local involvement in planning and implementation of health services tend to be validated by scholarly and grey literature, which suggests overall that increased levels of local control can improve the relevance of these services in meeting local needs.

Is there a demonstrable need for improved health outcomes for FN/I?

The need for improved health outcomes is indicated by higher levels of illness in FN/I communities, the differential access of FN/I to health services, and rates of hospitalization. Evidence of this need can be drawn from national survey data that measures rates of chronic and other health conditions, as well as studies of preventable hospitalization.

The First Nations Regional Health survey and Canada Community Health survey show that the First Nations On-Reserve population has an age-standardized rate of disability of 28.5%, compared to a rate of 25.8% among the general Canadian population.

The following table shows the prevalence of various health conditions for First Nations people on reserve and the general Canadian population. All of these conditions are more prevalent among First Nations, with the exception of emphysema. The rate of diabetes is almost four times higher among the First Nations population.

Table 3 - Age-standardized Prevalence of Selected Health Conditions First Nations
On-reserve (2002-03) and General Canadian Population (2003), Adults
Health Condition	First Nations On Reserve	General Canadian Population
High blood pressure	20.4	16.4
Diabetes	19.7	5.2
Asthma	10.6	7.8
Heart disease	7.6	5.6
Tuberculosis	3.9	n/a
Chronic bronchitis	3.7	2.8
Cancer	2.4	1.9
Effects of stroke (brain haemorrhage)	2.1	1.2
Hepatitis	1.2	n/a
Emphysema	1.0	1.4
The Canadian Community Health Survey excludes individuals living on Indian Reserves. Source: First Nations Information Governance Committee (FNIGC). First Nations Regional Longitudinal Health Survey (RHS) 2002-03; Results for Adults, Youth and Children Living in First Nations Communities. Assembly of First Nations (AFN), November 2005; Statistics Canada, Canadian Community Health Survey (CCHS) 2003.

A recent study conducted by Statistics Canada (Carrière 2011) using hospitalization rates as an indicator of access to care services used aggregate data to compensate for the lack of Aboriginal identity measures for hospital patients at the individual level. It included on-reserve residents who used hospitals off-reserve, and identified residence through postal codes. While the study was limited by this aggregation, it does provide a useful proxy indicator for the health outcomes across First Nations (FN) and non-FN populations. The study found that residents of communities with higher percentages of Aboriginal populations were considerably more likely to be hospitalized for acute conditions - 1,764 per 10,000 in populations with high rates, compared to 925 per 10,000 in areas with low rate Aboriginal populations. This pattern was found in every Province and Territory. They found that areas with high proportions of Aboriginal populations tended to have hospitalizations at younger ages.

The study concluded that: 1) residents of areas with a relatively high percentage of Aboriginal people had significantly higher hospitalization rates, compared to residents of areas where the percentage of Aboriginal people was low; 2) The highest hospitalization rates were among residents of areas where the predominant Aboriginal identity was First Nations; and 3) urban location and housing conditions had the strongest associations with differences in hospitalization rates between residents of high and low Aboriginal areas.

Overall, there is a demonstrated need for improved health outcomes for FN/I people as evidenced by the higher disease rates, and the higher hospitalization rates.

Performance -- Effectiveness

The synthesis evaluation addressed the main issues of performance as defined in the 2009 Policy on Evaluation by assessing whether the HGIS components have advanced the two main objectives of the program area, and whether the HGIS components achieved their intended outcomes (as outlined in their logic models). Firstly, the synthesis evaluation assessed whether the components have supported FN/I control over health programs and services through its support of other program clusters. It also assessed whether increased control over health planning through supports such as those provided by the components improves health outcomes for FN/I. Finally, it assessed whether there are barriers or challenges to implementing this increased control.

Secondly, the synthesis evaluation assessed whether the HGIS components have improved FN/I capacity to generate and access information and build knowledge through Aboriginal Health Research and Coordination projects, including efforts through the National Aboriginal Health Organization. It also assessed whether there are barriers or challenges to improving FN/I capacity to generate and access information and build knowledge.

Have the Health Planning and Management (HPM) and the Health Consultation and Liaison (HCL) components supported FN/I control over health programs and services?

It is clear in a general sense that the design and activities of the HPM and HCL components support FN/I planning and management of health programs and services aim to support increased levels of local control. The theory that funding models supporting increased local control should improve services is sound. It is also clear from program descriptions that the activities under these components are overall aligned with the approaches to local control proposed by the literature. However, the performance measurement that is available tends to indicate that the increased transfer of control has been minimal.

Number of FN/I Communities by Funding Model Type, 2007/08 to 2010/11
Funding Model Type	2007/08
Funding Model Type	Number of Communities	Percent	Number of Communities	Percent
Set	153	24.5	152	23.8
Transitional	204	32.6	203	31.8
Flexible/Transfer	268	42.9	283	44.4
Total Communities	625		638

The proportion of communities that were under a Flexible/Transfer agreement in 2010/11 was not significantly higher compared to 2007/08. However, the numbers of flexible funding arrangements is not necessarily a clear indicator of local control, since self-determination is also the power to decide the level of flexibility for each community. In other words, if a community decides to have a set agreement that gives them no flexibility, they are demonstrating local control. More information is needed to determine the reasons for slow transition to Flexible/Transfer agreements. In addition, there is a lack of qualitative information to indicate whether the activities have practically supported a transition to Flexible/Transfer agreements, or whether the differences between Set or Transitional and Flexible/Transfer at the community levels reflect effective conditions of local control.

As described in the logic model in Appendix B, the HCL component aims to increase opportunities for Aboriginal people to participate and influence policy and program development, which will then lead to greater involvement and effective role of Aboriginal people in the governance of the health care system and service delivery. The 2008 HCL performance report describes some of the main activities that were carried out to support these logic model outcomes:

Blueprint on Aboriginal Health, a collaborative effort by federal, provincial, territorial, FN/I and Métis representatives throughout 2005 resulted in First Nations, Inuit and Métis distinct frameworks to address gaps in health status between Aboriginal peoples and Canadians;
2006 National Summit on Aboriginal Health in British Columbia and the 2008 National Aboriginal Health Working Summit; these forums for dialogue, relationship building, collaboration and teamwork were attended by ministers and senior officials of federal, provincial and territorial governments and the leaders of five NAOs (AFN, MNC, ITK, NWAC, CAP);
five provincial/territorial/Aboriginal summits on Aboriginal health were held to address issues of importance to Aboriginal peoples;
creation of a Health Canada /Assembly of First Nations (AFN) Task Group in December 2006 as a forum to explore and develop approaches for improving First Nations' health;
creation of a Health Canada /Inuit Tapiriit Kanatami Task Group in April 2007 to explore and develop approaches for advancing Inuit health;
establishment of an Office of Inuit Health in the First Nations and Inuit Health Branch that was fully staffed and operational by Spring 2008;
Tripartite First Nations Health Plan for British Columbia was signed June 11, 2007 between the federal government, the province of British Columbia and the BC First Nations Leadership Council, to guide the interaction and collaboration.
MOU between the Canadian Institutes of Health Research-Institute of Aboriginal Peoples Health (CIHR-IAPH) and the AFN in November 2006 for the secondary analysis of survey data from the First Nations Regional Longitudinal Health Survey;
National Diabetes Surveillance System Collaborative Agreement, signed in December 2006 between FNIHB and the Canadian Institutes of Health Research to support two projects related to the National Diabetes Surveillance System in Alberta and Nova Scotia; and
Road Map to a Patient Wait Times Guarantee for First Nations including a paper developed by the AFN and pilot projects to test Patient Wait Times Guarantees for prenatal and diabetes care in select FN communities.

In addition, the effectiveness of the approach of the components to increase FN/I control over health planning and implementation can be addressed through existing program evaluations that have assessed the performance of programs with respect to FN/I control over planning and implementation. These evaluations of community-based health services and programs indicated that capacity for these services and programs have been supported by their activities.

Moreover, the program design to orient local planning and management of health programs and services to local needs and priorities can be inferred as positive support for FN/I to develop and implement them. Some evidence from other evaluations of programs and services at the community levels have also indicated that flexibility for FN/I to identify their own needs and priorities, and to make allocations of expenditures accordingly, was positive with respect to these programs.

Children and Youth

Part of the 2005/06 - 2009/10 Children and Youth cluster evaluation set out to examine the extent of FN involvement in decisions for the Children and Youth cluster programming and services, as well as the factors affecting the ability of FN to enter into Flexible Transfer agreements. A survey of FNIHB staff indicated that FNs were more involved in Children and Youth programs, as well as in setting priorities to meet the needs of some communities over the span of the program. In some cases FNs were directly involved in the planning stages to develop community health plans, as well as in the program delivery. Forty percent of community staff surveyed also agreed that there was more community involvement in decisions about the programs than there was 5 years ago. The evaluation found that the scope for First Nations decision-making was constrained by the amount of funding available and the type of funding model applied in the community. However, budget constraints are found to be a common barrier to program impacts across all programs.

Communicable Disease Control

Part of the CDC Program Cluster evaluation set out to examine the extent to which the program had increased community ownership and capacity to detect, report, and combat communicable diseases. It also examined the manner and extent to which the transfer payment impacted expected results. CDC intended to contribute to building community capacity by encouraging communities to take greater ownership of their health issues and by facilitating development of relevant skills by providing training and information to Community Health Nurses and other health providers.

The CDC supported FNs in the planning and development of health plans and programming through: 1) access to information and resources (ranging from research reports and specialized program/epidemiological expertise to information on standards and guidelines); and 2) advice, direction and assistance related to budgets, work plans, reporting, policy development, and evaluations. According to results from a survey, 29% of the Regional Coordinators were very satisfied with the assistance/support provided by the national office, 55% were somewhat satisfied, and 16% were not at all satisfied. In turn, the Health Directors and Community Health Nurses were generally satisfied with the support provided to them by the regional offices of FNIHB in that 48 % stated they were very satisfied, 34% were somewhat satisfied and 18% were not satisfied at all. Opportunities identified by Regional Coordinators to improve the support provided by headquarters included increasing the number of face-to-face meetings with regional CDC representatives, more clearly defining and communicating roles and responsibilities and reducing the reporting requirements.

The evaluation highlighted some factors that could affect the level of community ownership: 1) the skills of the local health care staff (including the interpersonal skills of the nurses and health director), 2) the emphasis placed on community involvement, 3) the level of staff turnover, 4) the number and prevalence of other issues in the community, 5) the transfer payment model under which the community is funded, 6) the extent to which community leadership is supportive of the programs, and 7) the community's previous experience with government and with communicable disease.

Perceptions regarding the extent to which communities had taken ownership of CDC issues and programming were mixed among those surveyed. 51% of respondents 'somewhat' or 'strongly' agreed, while 33% disagreed that the community had taken ownership of these issues and types of programs. Other indicators of increased ownership in some communities included increasing participation levels in awareness programs, rising immunization rates, and the greater involvement of Band staff, Chief and Council in the promoting, participating and otherwise supporting awareness and education progress.

The evaluation found that the CDC had a significant impact on community capacity through increasing access to training and other resources for Community Health Nurses and others in the communities. Based on the field research and other information provided by the regions, it was estimated that there were 1,600 people (700 nurses and 900 others) involved at the community level across all regions. The data obtained from the performance reports indicated that over 200 skill development sessions were staged between 2005-06 and 2007-08, involving nearly 10,000 participants.

With respect to the impact of funding models on expected results, the evaluation was unable to demonstrate that some types of transfer payment models were more effective than others in producing positive outcomes. So while some differences in outcome achievement were noted across communities with different types of models, these differences were likely more attributable to other factors (e.g., differences in the characteristics of the communities such as location, remoteness, infrastructure, or range of services available) than to the type of model itself. It was noted that it may have been too early to demonstrate the funding models' real impact due to the length of some Contribution Agreements (3 years). Overall, the evaluation found that most Directors, Nurses and Regional Coordinators perceived the programs to be flexible, allowing them to be tailored to meet the needs of the community or communities.

Chronic Disease and Injury Prevention

The 2005/06 to 2009/10 evaluation of the Chronic Disease and Injury Prevention cluster evaluation assessed the involvement of FN/I in program design and planning, as well as the specific impact of funding models or transfer funding on program implementation and effectiveness. The evaluation confirmed that FN/I involvement in the design and delivery had improved the cultural relevancy of programming, for instance through the First Nations, Inuit and Métis version of Canada's Food Guide, which has been tailored for to reflect the values, traditions and food choices of Aboriginal populations.

With respect to the impact of funding models and the provision of transferred funding agreement, community health staff indicated mixed opinions on which of the models supported the delivery of quality supports and services of the Aboriginal Diabetes Initiative. The evaluation found that a majority of these respondents indicated that the delivery helped either "a lot" (6 of 19 communities) or "somewhat" (10 out 19 communities), and a few reported that the funding model did not help (3 of 19 communities)". The evaluation found that communities with Set models were the least likely to report that the model was helpful (i.e. "a lot" 0; "somewhat" 5; "not at all" 3) while those with a Flexible agreement were the most likely to report that the model was helpful (i.e. "a lot" 6; "somewhat" 5; "not at all" 0). Respondents indicated that main challenges presented by Set funding models included: not being able to transfer funds across fiscal years, lack of flexibility in types of activities that could be funded, and increased reporting burden.

Environmental Health and Health Research cluster

The Environmental Health and Health Research Cluster evaluation for 2005/06 to 2009/10 attempted to assess the impact of funding models on the performance of the program. With respect to the Environmental Health components of the cluster, the evaluation did not find that increased participation in planning and implementation improved the delivery or effectiveness of the environmental health programs provided through the cluster. Instead, the evaluation found that assessments of expenditures and activities were seriously limited by the flexibility offered by the transfer agreements, given that financial data on allocations and expenditures towards activities were unreliable and inconsistent. The evaluation attributed this to the fact that, "at regional level, funds transferred by HQ for EPH may end up distributed to different priority areas outside the Cluster; at the community level, CA funds may be allocated to activities outside the Cluster mandate to meet competing priorities".

On the other hand, the evaluation did confirm that the Environmental Research component of the cluster contributed significantly to the evidence base in support of planning and management of environmental health risks at the regional and community levels, by identifying priorities and ways to mitigate environmental health risks. The community-based approach of this cluster component was valued as a source of planning and management information, since community members sometimes did not trust information when it was generated from outside the community.

Does increased control over programs and services improve outcomes?

Some scholarly sources have assessed health outcomes such as hospitalization rates, and have found positive links between FNIHB funding models and the improved health outcomes indicated by these links.

Lavoie et al (2010c) note that 83% of First Nations in Canada are currently involved in managing their own health services, but analysis of health outcomes have been lacking. As a way to increase the evidence base for the link between FN control and health outcomes, they analyse the rates of avoidable hospitalization- for example amputations due to diabetes. They identify the links between isolation of communities and access to health services within Manitoba, reasoning that this is a main factor in the health inequalities between First Nations and other Canadians.

They operationalized community control specifically through the health services transfer approach taken by FNIHB, which defines self-determination according to the level of local administration of community-based and regional programs, including the transfer of knowledge, capacity and funds in order to support locally defined needs and priorities. By linking hospitalization data with funding models of the community, they were able to determine that: 1) better access to primary care through local health services had led to lower rates of avoidable hospitalization, 2) communities that had entered into Transfer agreements had led to better health outcomes "the longer in a transfer agreement, the better the health outcomes", and that 3) those communities that had entered into an integrated agreement had not achieved the same improvements (p.723). Importantly, they also noted that transferred communities did not have higher levels of health at the outset of the study, indicating that the improvements could be attributed to the transfer approach. They estimate a two to five year delay between the signing of a Transfer agreement and improvements in outcomes.

Overall, they conclude that the transfer policy implemented by FNIHB over the previous 20 years had led to improved health outcomes for First Nations, which supports both FNIHB policy, as well as policies of indigenous self-determination of the United Nations, the World Health Organization (2008), the BC Provincial Health Officer's Report (2009).

Lavoie et al (2007) also confirmed that First Nations control over the planning and implementation of services that are designed to reflect local needs and priorities "offer considerable potential for improving health outcomes" (p.93). However, they caution that funding formulas need to be specific to variations in local needs and changing circumstances such as population levels and costs of health services.

The Canadian Medical Association Journal (2009) describe the research by Lavoie to identify a basket of essential services on reserve in Manitoba, and report that the findings indicate that the FNIHB initiative to transfer control of health care to First Nations communities had yielded success, "despite concerns about financial constraints" (p.e249). The journal also concluded that the findings were important, since previous research had been hampered by a lack of performance data related to health outcomes.

Lavoie et al (2011) conducted another key study of the impact of FNIHB's First Nations and Inuit Home and Community Care Program on rates of hospitalization for ambulatory care sensitive conditions. They were able to advance their research into community control and on-reserve access to primary health care, and concluded that these expanded activities had a demonstrable positive impact, and reduced rates of hospitalization.

The 2006 Evaluation of the First Nations and Inuit Health Transfer Policy concluded that the policy had led to improvements in access to health care and health outcomes, and that there was no evidence to support a recentralization of management or delivery of on-reserve health services to Health Canada. The transfer policy objective to provide opportunities to engage planning, administration and delivery of on-reserve health services was confirmed by FN/I respondents from transferred communities, who identified the benefits of a transfer agreement for setting local priorities.

These respondents also reported that local decision-making had improved program responsiveness to changing needs. Respondents from communities without a Transfer agreement were much less likely to report that funding could be allocated to meet local needs. Respondents in the evaluation reported that improvements in health services had been achieved by the transfer policy, and that local control had improved awareness of 1) community health issues, 2) determinants of health and 3) the availability of services. A majority of community members reported that health status had improved, and a series of community-based evaluations found that improvements were attributed to the transfer process.

The 2010 synthesis of First Nations Community-Based Evaluation Reports that were received between 2003 and 2008 examined the effectiveness of the transfer funding model for the management and delivery of health programs and services, and stands as a small-scale assessment of the Health Transfer Policy. The assessment provided a synthesis of 70 community-based evaluations, and included an assessment of the linkages between 29 community Health Plans and these evaluations. Sixty-three percent of the evaluations addressed the impact of health transfer on the community, and these reported overall positive impacts through: improved or maintained service quality, ability to self-manage and greater flexibility in health programs, leading to improved participation. With respect to outcomes, the evaluations reported on balance that health status was overall unchanged by the transfer process. However, only 76% of the evaluations used for the review included indications of health status or outcome, and most of those that did include these measures relied on self-reported perceptions.

Are There any Barriers or Challenges to Implementing Increased FN/I Control over Planning?

Both program cluster evaluations and the literature review identified barriers and challenges to implementing increased FN/I control over health planning and management.

Results from a survey of FNIHB staff in the Children and Youth cluster evaluation indicated that 47% rated funding models as 'Very/Quite appropriate' and 47% said they were 'Somewhat' appropriate. FNIHB staff was also asked to identify factors that facilitated FN ability to enter Flexible Transfer agreements. Some of the factors identified included: 1) community capacity & staff with the ability to manage funding and trained staff to put the plan into action; 2) having community health plans developed and followed, and 3) leadership support with a strong government structure and band management.

Alternatively, not having these factors in place was viewed by FNIHB staff in the Children and Youth cluster as the main barriers limiting the ability of FNs to enter into Flexible Transfer agreements. Other factors identified by FNIHB staff that facilitated or impeded entry into a Flexible Transfer agreement were: 1) having strong cluster teams in the FNIHB regions to support communities and planning discussions between the communities and FNIHB regions, 2) coordination of provincial health services, 3) the size and location of communities, 4) lack of guidelines, 5) financial challenges, and 6) third party management.

The Communicable Disease Control cluster evaluation also indicated some needs for improvement for implementing increased control. The steps recommended by headquarters and regional staff to increase community ownership included 1) increasing the involvement of the community in the planning and delivery of CDC activities; 2) further expanding the education and awareness programs; 3) providing more timely and accurate feedback to individual communities on the progress made in immunization rates, infection rates, and awareness; and 4) better coordinating and integrating program activities with community activities.

The 2009 audit of the FNIHB recipient audit function confirmed a need to improve the audit function, which provides a key line of evidence on programs and financial performance of the health funding process, and supports all areas of planning. The audit found that, while the risk-based process for recipient audits was adequately designed, delays in scheduled audits, as well as guidance provided by the Branch impacted on the consistency of the planning, conduct and reporting of recipient audits. A number of recommended actions were agreed upon by the Branch to adjust these deficiencies.

Has the AHRCP component improved FN/I Capacity to Generate and Access Information and Build Knowledge?

The synthesis evaluation assessed whether Aboriginal Health Research and Coordination Projects have contributed to improved FN/I capacity to generate and access information and build knowledge. It also considered whether NAHO had contributed to this outcome through evidence provided by the 2005/06 - 2009/10 NAHO evaluation, since NAHO was the lead recipient of Aboriginal Health Research and Coordination Project funds during this period, and made the majority of contributions to this objective.

The 2008 HGIS Performance Report describes the main activities of the Aboriginal Health Research and Coordination Projects in the areas of health research and coordination projects as:

A Joint Working Group for the Improvement of FN/I and Metis Infant Mortality Data was established to support collaboration between the federal government, academia and Aboriginal organizations in improving the quality and coverage of these types of data. The group led the development of two research projects.
The first cycle of the First Nations Regional Longitudinal Health Survey was developed and administered through over $11.7M in FNIHB funding. An additional $12.5M as committed for the 2nd cycle in 2011.
The FNIHB Framework for Research was developed through input from the Canadian Institutes for Health Research Institute for Aboriginal Population Health, the Assembly of First Nations and the Inuit Tapiriit Kanatami.
The Joint Summer Institute on Indigenous Health Research was launched in July 2007.
The summer institute was followed up with a two-day grant writing workshop in Calgary.
The Inuit Health Survey was implemented.
The Statistics Canada, Indian and Northern Affairs Canada and HC Geozones project included the publication of Life Expectancy in the Inuit-inhabited areas of Canada.
The Symposium on First Nations, Inuit and Metis Health Data was held in 2007.

The 2005-2008 process and impact evaluation of the NAHO included multiple lines of evidence, including a survey of 227 First Nation respondents. While, the evaluation found that members openly indicated difficulties in balancing the interests of member FN/I and NAHO in the area of organizational priorities and work plans, they also indicated that the issues had been improving. This included positive impacts on traditional knowledge and healing practices, as well as raising awareness among FN/I of these practices and the barriers to accessing traditional knowledge. The survey indicated increasing levels of awareness of FN/I health issues among FN/I through the NAHO role model program, as well as web-based information, as indicated by a 67% increase in NAHO downloads, amounting to over 1.5 million between 2005 and 2008.

The evaluation also indicated progress towards increased culturally relevant health research, policy and legislations through direct research, as well by increasing community capacity to conduct research. Respondents confirmed that NAHO activities had increased this research in the areas of youth, maternal and child health and disease prevention. Partner representatives indicated that NAHO had supported the on-going use of culturally relevant research by communities, and the organization had increased the capacity for this use among FN/I. This included increased awareness of the need for culturally relevant research to support health programs.

One external source (Smith and Lavoie, 2008) included an assessment of the effectiveness of increased traditional First Nation knowledge in health. They conducted detailed interviews of six First Nations across Canada, focusing on the effects of increased control over health planning and implementation through FNIHB's health transfer process. They included measures of quality assurance and health outcomes, as well as strengths and challenges. The found that local innovations in planning and evaluation included a traditional story-telling format to facilitate input and participation from service clients, which was preferred as a way of "adapting our own way of knowing" and avoiding academic and provider-centric perspectives.

Are there any barriers to improved FN/I Capacity to Generate and Access Information and Build Knowledge?

The 2006 evaluation of the Health Transfer Policy found that efforts to create a comprehensive and practical health information system were limited by the administrative resources required to participate in reporting. This in turn limited the capacity to track progress in health status and set priorities for planning and implementation of local services. However, the evaluation identified the progress of some communities in linking outcome data to their health information systems.

Similar gaps in evidence exist to show whether the improved FN/I capacity to generate and access information and build knowledge through the Aboriginal Health Research and Coordination Projects and NAHO. While the activities are likely valid approaches to increasing knowledge and improving health consultations and liaisons, there is little evidence to support an assessment of whether these have led to improved health outcomes.

Performance - Efficiency and Economy

Have the resources used for delivery approximate the minimum required to achieve the expected results?

A lack of activity-based accounting broken down by agreement type (Contribution Agreements, Branch FTEs and operations and maintenance), as well as the unavailability of data showing the links between activities and results, has made it difficult to assess the efficiency of the components overall. While the evaluations of programs supported by health planning, management and health information tended to show that these programs are delivered efficiently, they did not assess specifically whether the planning and management support for these programs through these HGIS components had produced efficiencies of their own. For its part, the 2005-2008 NAHO evaluation benchmarked the expenditures and results of the organization against similar knowledge-based organizations and concluded that NAHO was equally efficient in expending its funds. Budget variances of NAHO centres ranged from zero to 15 percent.

Are there ways to improve the efficiency and economy of the program delivery?

Again, a lack of financial data to support evidence of ways to improve efficiency and economy makes it difficult to identify potential areas of improvement. For example, the level of funding to different activity types such as direct Contribution Agreements to FN/I communities vs. Branch FTEs to support planning and management activities were not available for the synthesis evaluation.

Conclusions

Recommendations

Appendix A

Health Planning and Management Logic Model
Inputs	FTEs, O&M and Contribution Funds
Activities	Development of health plans	Build health management capacity	Implement and manage a funding agreement based on health plan
Outputs	Assessments Health plans	Management structure established Infrastructure established	Agreements Reports
Immediate outcomes	Program delivery based on health priorities FN/I community ownership, effective control and capacity to manage and administer health programs and services
Final outcome	Strengthened , effective and sustainable health planning, administration and delivery system at the Local, Regional and National levels

Appendix B

Health Consultation and Liaison Logic Model
Inputs	FTEs, O&M and Contribution Funds
Activities	Support Aboriginal national and regional consultation and engagement processes	Support and maintain Aboriginal liaison capacity
Outputs	Consultation initiatives and liaison Consensus Models of engagement Policy positions Program design and tools	Agreements Aboriginal participation in health governance process Reports Projects Representation of National Aboriginal Organizations/P/T Organizations
Immediate outcomes	Increased opportunities for Aboriginal to participate and influence policy and program development
Final outcome	Greater involvement and effective role of Aboriginal in the governance of health care system and service delivery

Appendix C - Aboriginal Health Research and Coordination Projects Logic Model

Aboriginal Health Research and Coordination Projects - e-Health Solutions
Inputs	FTEs, O&M and Contribution Funds
Activities	Select, deploy and support e-health infrastructure	Build FN/I knowledge and information management capacity	Support Aboriginal health research and coordination projects/initiatives
Outputs	e-Health, systems and tools Connectivity: e-health sites deployed	Knowledge and information management workers Support Information collection and analysis	Projects/initiatives (including international)
Immediate outcomes	Improved e-health access in FN/I communities and in regions	Increased information management capacity	Enhanced knowledge and data quality on Aboriginal health
Intermediate outcomes	Improved dissemination of knowledge and evidence based policies and programs
Final outcome	Access to health information through research and knowledge contributes to the improved health status of First Nations and Inuit individuals, families and communities

Appendix D - References

Bossert, Thomas (1998) Analysing the Decentralization of Health Systems in Developing Countries in Developing Countries: Decision, Space, Innovation and Performance, Social Science and Medicine, 47(10):1513-1527.

Canadian Medical Association Journal (2009) Local Control over Aboriginal Health Care Improves Outcome, Study Indicates, Canadian Medical Association Journal, 181(11):e249-250.

Carrière, Gisèle, Rochelle Garner, Claudia Sanmartin, and LHAD Research Team (2011) Acute-care hospitalizations and Aboriginal identity in Canada, 2001/2002. Statistics Canada: Ottawa.

Hutchinson, Paul L. and Anne K. LaFond (2004) Monitoring and Evaluation of Decentralization Reforms in Developing Country Health Sectors. U.S. Agency for International Development

Lavoie, Josee, Evelyn L. Forget, Matt Dahl, Patricia Martens, John O'Neil (2011) Is it Worthwhile to Invest in Home Care? Healthcare Policy, 6(4):35-48.

Lavoie, Josee, Evelyn L. Forget, Matt Dahl, Patricia Martens, John O'Neil (2011) Is Community Control Over Health Services Good for First Nations' Health? Assembly of First Nations Manitoba Chiefs Task Force.

Lavoie, Josee, Amohia Boulton and Judith Dwyer (2010a) Analysing Contractual Environments: Lessons Learned from Indigenous Health in Canada, Australia and New Zealand. Public Administration, 88(3):665-679.

Lavoie, Josee, Evelyn L. Forget and Annette J. Browne (2010b) Caught in the Crossroad: First Nations, Health Care, and the Legacy of the Indian Act. Pimatisiwin: A Journal of Aboriginal and Indigenous Community Health, 8(1):83-100.

Lavoie, Josee, Evelyn L. Forget, Tara Pakash, Matt Dahl, Patricia Martens, John D. O'Neil (2010c) Have Investments in On-Reserve Health Services and Inititaives promoting Community Control Improved First Nations' Health in Manitoba? Social Science and Medicine, 71:717-724.

Lavoie, Josee, Evelyn L. Forget, John D. O'Neil (2007) Why Equity in Financing First Nations On-Reserve Health Services Matters: Findings from the 2005 National Evaluation of the Health
Transfer Policy. Healthcare Policy, 2(4):80-96.

Mills, Anne, J. Patrick Vaughn, Duane L Smith, Iraj Tabibzadeh (1990) Health System Decentralization: Concepts, Issues and Country Experiences. World Health Organization: Geneva.

Putnam, R. D. (1993) Making Democracy Work: Civic Traditions in Modern Italy. Princeton University Press, Princeton.

Saltman, Richard B., Vaida Bankauskaite, Carsten Vrangbaek (eds.) (2007) Decentralization in Health Care: Strategies and Outcomes. European Observatory on Health Care.

Schwartz, J. Brad, David K. Guilkey and Rachel Racelis (2002) Decentralization, Allocative Efficiency and Health Service Outcomes in the Philippines. Carolina Population Center,
University of North Carolina: Chapel Hill.

Smith, BC (1997) The decentralization of health care in developing countries: Organizational options. Public Administration and Development, 17:399-412.

Smith, Ross and Josee Lavoie (2008) First Nations Health Networks: A Collaborative System Approach to Health Transfer. Healthcare Policy, 4(2):101-112.

Wamai, Richard G. (2009) Reforming health systems: the role of NGOs in Decentralization - lessons from Kenya and Ethiopia.

World Bank (2011) Decentralization: Health. World Bank website.

World Health Organization (2008) Closing the Gap in a Generation: Health Equality Through Action on the Social Determinants of Health. Final Report of the Commission on the Social Determinants of Health. World Health Organization: Geneva.

Page details

Date modified:: 2020-09-01

Health Planning, Consultation and Research Projects - Synthesis Evaluation 2005-2006 to 2009-2010

Table of Contents

Acronyms

Executive Summary

Relevance

Performance - Effectiveness

Performance - Efficiency and Economy

Conclusions and recommendations

Conclusions

Recommendations

Background and Context

Program Description

Program Areas

Health Planning and Management

Health Consultation and Liaison

Aboriginal Health Research and Coordination Projects

Funding Models

Funding

Synthesis Evaluation Context

Synthesis Evaluation Objectives

Synthesis Evaluation Approach

Synthesis Evaluation Methodology

Data Collection Methods

Program Evaluations and Reviews

Literature Review

Limitations

Relevance

Performance -- Effectiveness

Children and Youth

Communicable Disease Control

Chronic Disease and Injury Prevention

Environmental Health and Health Research cluster

Performance - Efficiency and Economy

Conclusions

Recommendations

Appendix A

Appendix B

Appendix C - Aboriginal Health Research and Coordination Projects Logic Model

Appendix D - References

Page details