Office of Nutrition Policy and Promotion - Evaluation

Help on accessing alternative formats, such as Portable Document Format (PDF), Microsoft Word and PowerPoint (PPT) files, can be obtained in the alternate format help section.

Final Report

March 2012

Management Action Plan - Office of the Nutrition Policy and Promotion - Evaluation

Recommendation	Response	Key Activities	Deliverables	Responsible Manager	Timeframe
Plan to increase analysis and reporting on surveillance and monitoring data. It is recommended that HC improve its tracking and analysis of the eating patterns of Canadians and their needs for nutrition information.	Agree	The ONPP is developing a data analysis plan to assess Canadians' diets according to the CFG using CCHS 2.2 data. The ONPP will develop an indicator framework to inform future data collection, analysis and communication.	Data analysis plan approved by ONPP management. Indicator framework approved by ONPP management.	Director, Research, Monitoring and Evaluation, Office of Nutrition Policy and Promotion (ONPP), Health Products and Food Branch (HPFB), Health Canada (HC)	May 2012 October 2012
Improve planning and project management for the next revision of CFG. It is recommended that HC establish a review cycle to determine if revisions to CFG are required and the scope of revisions.	Agree	The Program is developing a proposal for an ongoing CFG assessment cycle.	Assessment cycle document approved by ONPP management.	Director, Policy and Standard Setting, ONPP, HPFB, HC	June 2012
Improve the existing performance measurement framework so that expected outputs and outcomes are systematically monitored, adjusted and progress reported. It is recommended that the ONPP's performance measurement framework be updated to meet operational planning, reporting and accountability requirements, including reporting on outcomes and strategic reviews. This process should begin by reviewing the logic model to ensure that it is still relevant in the current context.	Agree	The Program will establish a working group (and consult with departmental and branch experts) to revise the performance measurement framework, including updating the logic model, and identifying data sources and a data collection strategy.	Performance measurement framework approved by ONPP management.	Director, Planning, Dissemination and Outreach, ONPP, HPFB, HC	March 2013

Office of Nutrition Policy and Promotion - Evaluation

Final Report

February 13, 2012

Table of Contents

• Executive Summary
• 1.0 Introduction
  • 1.1 Background
  • 1.2 Objectives of the Evaluation
  • 1.3 Approach
    • 1.3.1 Case Study
    • 1.3.2 Document and Literature Review
    • 1.3.3 Key Informant Interviews
    • 1.3.4 Survey
    • 1.3.5 Analysis Strategy
    • 1.3.6 Challenges, Limitations and Mitigation Strategies from Data Collection Activities
• 2.0 Findings, Conclusions and Recommendations
  • 2.1 Relevance
  • 2.2 Performance - Efficiency (Design and Delivery)
  • 2.3 Performance - Effectiveness (Success)
  • 2.4 Performance - Economy
• Appendix A - Office of Nutrition Policy and Promotion - Logic Model
• Appendix B - References

List of Acronyms

AAFC

Agriculture and Agri-Food Canada

BC

British Columbia

BEC

Branch (HPFB) Executive Committee

CCHS

Canadian Community Health Survey

CFG

Canada's Food Guide

CFGHE

Canada's Food Guide to Healthy Eating

CIHR

Canadian Institutes of Health Research

DPR

Departmental Performance Report

DRI

Dietary Reference Intakes

EAC

Expert Advisory Committee

FBDG

Food Based Dietary Guide

FG

Focus Group

FNSS

Food and Nutrition Surveillance System

FPTGN

Federal/Provincial/Territorial Group on Nutrition

GoC

Government of Canada

HC

Health Canada

HPFB

Health Products and Food Branch (HC)

ILA

Intradepartmental Letter of Agreement

INAC

Indian and Northern Affairs Canada now renamed Aboriginal Affairs and Northern Development Canada

JCSH

Joint Consortium for School Health

MRRS

Management, Resources and Results Structure

MoU

Memorandum of Understanding

NIH

National Institutes of Health (US)

OCAPI

Office of Consumer and Public Involvement (HPFB, HC)

OGD

Other Government Departments

ONPP

Office of Nutrition Policy and Promotion

PAA

Program Activity Architecture

PHAC

Public Health Agency of Canada

PMDP

Performance Measurement Development Project

PMF

Performance Measurement Framework

POR

Public Opinion Research

P/T

Province/Territory

RPP

Report on Plans and Priorities

SES

Socio-Economic Status

TB

Treasury Board

WHO

World Health Organization

Executive Summary

Objectives and Approach

This report presents the findings of an evaluation of the Office of Nutrition Policy and Promotion (ONPP) at Health Canada (HC). The objective of the evaluation is to assess the relevance and performance (efficiency, effectiveness and economy) of the ONPP between April 1999 and March 2009. Findings are based on the results of a document and literature review, a case study of Canada's Food Guide (CFG) revision and dissemination process, key informant interviews (n=12) with program staff and external stakeholders, and a survey of internal staff and stakeholders (n=55).

Background

In order to ensure that Health Canada is working to achieve tangible results for Canadians, it has established three strategic outcomes and key areas of program activities (its Program Activity Architecture) (Health Canada, 2011c). The ONPP falls under the Access to Safe and Effective Health Products and Food and Information on Healthy Choices program strategic outcome (Health Canada, 2011c). The ONPP contributes to Strategic Outcome #2: Canadians are informed of and protected from health risks associated with food products, substances and environments, and are informed of the benefits of healthy eating (Health Canada, 2011c). The ONPP supports the nutritional health and well-being of Canadians by collaboratively defining, promoting and implementing evidence-based nutrition policies (Health Canada, 2011b).

Relevance and Continued Need

Overall, the evidence indicates that the ONPP is consistent with federal roles and responsibilities in the areas of health protection and health promotion as well as with federal priorities as indicated in some Speeches from the Throne.

There is evidence showing that nutrition is a key determinant of good health. Unfortunately, many Canadians have poor eating habits that lead to increased risk of nutrition-related chronic diseases and obesity (Shields, 2005; Tjepkema, 2005). Hence there is a continued need for the federal government to remain active on nutrition issues and to build and maintain partnerships with other jurisdictions within and outside Canada. Further, an international comparison shows that other comparable countries have implemented organizations similar to the ONPP in terms of mandate, activities and key products.

While there is potential for duplication with other organizations, there is evidence that duplication is avoided between the ONPP's activities and those of other groups/departments as a result of partnerships and working groups/collaborations.

Performance - Efficiency (Design and Delivery)

Recommendation # 1 - Plan to increase analysis and reporting on surveillance and monitoring data

It is recommended that HC improve its tracking and analysis of the eating patterns of Canadians and their needs for nutrition information.

According to the evaluation findings the ONPP's processes and products are science- and evidence-based as they reflect ongoing research and consultations with experts. The ONPP uses science-based resources, such as Dietary Reference Intakes (DRIs) and surveillance data. However, there is a need for more timely analyses of surveillance data to maximize its use. This in turn would improve the ONPP's ability to provide more up-to-date information and products to stakeholders and Canadians.

There is also evidence that the ONPP assesses and considers various risks in its planning and product development work, especially through consultation on an ongoing basis with experts and other organizations. However, it is felt that there could be more consultations with the general public, especially about dissemination of products and information.

Recommendation # 2 - Improve planning and project management for the next revision of CFG

It is recommended that HC establish a review cycle to determine if revisions to CFG are required and the scope of revisions.

In the area of planning, there is evidence that the ONPP has an adequate short-term planning process. However, the evaluation found that more systematic planning for revising key products through multi-year activities, such as CFG, would benefit the Program. Nevertheless, based on available evidence, overall, the ONPP's main activities seem to have been implemented according to TB commitments and obligations

Recommendation # 3 - Improve the existing performance measurement framework so that expected outputs and outcomes are systematically monitored, adjusted and progress reported

It is recommended that the ONPP's performance measurement framework be updated to meet operational planning, reporting and accountability requirements, including reporting on outcomes and strategic reviews. This process should begin by reviewing the logic model to ensure that it is still relevant in the current context.

The ONPP has developed a logic model and performance measurement framework for reporting purposes. However, there is an opportunity to update the logic model as well as the performance measurement framework.

A review of program documentation suggests that numerous activities contributed to most commitments to Central Agencies. However, the evaluation was unable to conclude on the extent to which commitments were met because there was no systematic collection of information to compare achievements stated to previous targets and objectives. Further, the ONPP may have met many of its commitments to TB but has not directly reported on these to Central Agencies as there is no regular reporting mechanism to the Department apart from the provision of information to meet Management, Resources and Results Structure (MRRS) reporting requirements.

Performance - Effectiveness

According to evidence the ONPP made numerous contributions to the evidence base related to healthy eating through papers, articles and other products. Many governmental and nongovernmental policies, initiatives and programs use one of the ONPP's key products, CFG, as a foundation. There is also evidence from documentation that some of the ONPP's products have contributed to increase the awareness and understanding of healthy eating and nutritional issues among Canadians. While evidence for the intermediate outcomes is more limited, there is some evidence that CFG has contributed to better-informed choices and improved healthy eating by Canadians (including those at risk), although it is acknowledged that products such as CFG should be part of broader health promotion strategies. The lack of quantitative evidence about the long-term impacts of the ONPP prevented the evaluation from assessing these aspects of performance.

Performance - Economy

Efficiency was assessed by looking at financial information and by evaluating the effectiveness of the ONPP to leverage resources from partnerships. The evaluation also addressed potential alternatives to current structures. The ONPP has engaged in partnerships, which in turn led to Dietary Reference Intakes (DRI) reports, publications, performance data and outreach products. As for alternate structures, while a few options have been explored in the past, there is no evidence that other structural arrangements would significantly improve the performance of the ONPP.

1.0 Introduction

This report presents the findings of an evaluation of the Office of Nutrition Policy and Promotion (ONPP) at Health Canada (HC). Findings are based on the results of a document and literature review, a case study of the revision and dissemination process for CFG, key informant interviews and a survey of stakeholders internal and external to the ONPP. Section 1 provides the background of the program and the objectives and approach of the evaluation. Section 2 provides the findings and conclusions from different lines of evidence.

1.1 Background

Mandate of the ONPP

The ONPP serves as a focal point and authoritative source for nutrition and healthy eating policy and promotion in Canada (Health Canada, 2006c). The ONPP supports the nutritional health and well-being of Canadians by collaboratively defining, promoting and implementing evidence-based nutrition policies. In order to do this, the ONPP integrates science, policy and intervention activities as well as promotion activities related to nutrition (Health Canada, 2006c).

The ONPP has a number of objectives: to promote the nutritional health and well-being of Canadians by collaboratively defining, promoting and implementing evidence-based nutrition policies and standards; it acts as the focal point and authoritative source for nutrition and healthy eating policy and promotion; it also disseminates timely, evidence-based and authoritative information to Canadians and stakeholders to enable them to make informed decisions and healthy choices (Health Canada, 2011d).

In order to ensure that Health Canada is working to achieve tangible results for Canadians, it has established three strategic outcomes and key areas of program activities (its Program Activity Architecture) (Health Canada, 2011c). The ONPP falls under the Strategic Outcome #2: Canadians are informed of and protected from health risks associated with food, products, substances and environments, and are informed of the benefits of healthy eating (Health Canada, 2011c).

The ONPP is part of the Health Products and Food Branch and is organized by three pillars: 1) Research, Monitoring, and Evaluation; 2) Policy and Standard Setting; and 3) Planning, Dissemination and Outreach. The ONPP has approximately 35 full-time staff including a director general, a director and manager for each of the three pillars.

The ONPP works collaboratively with internal and external partners and stakeholders. Within Health Canada it partners with the Bureau of Nutritional Sciences and the Bureau of Food Policy and Science Integration of the Food Directorate, Health Products and Food Branch, the First Nations and Inuit Health Branch and the Marketing and Communications Services Directorate of the Public Affairs, Consultation and Communications Branch. Across the Health Portfolio it maintains relationships with the Public Health Agency of Canada and the Canadian Institutes of Health Research. It also works with other government departments (OGDs) such as Statistics Canada and Agriculture and Agri-Food Canada. In addition, it consults with provincial and territorial (P/T) governments and non-governmental organizations (NGOs). There are several examples of these relationships including the important partnership with the Federal/Provincial/ Territorial Group on Nutrition (FPTGN), which provides leadership in stimulating and accelerating actions towards achieving nutritional well-being for all Canadians. An example of partnerships with NGOs (e.g. nutrition organizations, health organizations, consumer groups and the food industry) is the Network on Healthy Eating, which enhances collaboration, cooperation and coordination of efforts to support healthy eating and nutrition in Canada.

Expected Outcomes and Logic Model

In 2001-02, the ONPP participated in a Performance Measurement Development Project (PMDP) led by the Food Directorate. One of the products of this effort was a draft logic model (dated 2003-2004) to guide performance measurement within the ONPP. This model was validated in 2009 during the process of preparing this evaluation and is based on a six-step performance pathway. It includes inputs, activities, outputs, reach/coverage, immediate impacts/outcomes and long-term impacts/outcomes. The ONPP logic model is attached as Appendix A.

The immediate outcomes are:

• Increased knowledge of information related to healthy eating;
• Improved policies, programs and initiatives to support healthy eating; and,
• Increased awareness and understanding of healthy eating among Canadians.

The intermediate outcomes are:

• Increased evidence-based decision-making nationally and internationally; and,
• Improved healthy eating by Canadians.

The long-term outcome is:

• Reduced level of nutrition-related concerns, conditions and chronic diseases.

The ultimate outcome is:

• Improved health and well-being of Canadians.

1.2 Objectives of the Evaluation

The objective of the evaluation is to assess the relevance and performance (efficiency, effectiveness and economy) of the activities of the ONPP. The timeframe of the evaluation is from April 1999 to March 2009 (although some later materials were used as they were deemed relevant for the forward-looking aspects of the evaluation). The evaluation addresses the evaluation commitments for both A-base and Treasury Board (TB) submission funding. This information is provided to senior management at the Health Products and Food Branch (HPFB) of Health Canada (HC) to guide and influence decisions on the implementation of the ONPP's present and future initiatives. The evaluation was guided by the following questions:

Relevance

• 1. Is there a continued relevance and need for the ONPP's activities as they are defined?
  
  
  1. How do the ONPP's activities align with the Government's priorities?
  2. Are the ONPP's activities consistent with federal roles and responsibilities?
  3. Are the ONPP's activities meeting the needs of Canadians?

Performance - Efficiency (Design and Delivery)

• 2. Were the ONPP's activities appropriately designed?
• 3. To what extent have the ONPP's activities been implemented as planned?
• 4. What were the key achievements, lessons learned and challenges experienced in delivering the activities of the ONPP?
• 5. To what extent have the requirements/commitments to Central Agencies, departmental and other requirements been addressed?

Performance - Effectiveness (Success)

• 6. To what extent have the ONPP's activities contributed to the achievement of the immediate, intermediate and long-term outcomes?

Performance - Economy (Cost-Effectiveness)

• 7. Are the current structure and activity delivery methods of the ONPP the most cost-effective means of achieving the intended outcomes?
  
  
  1. In view of the current ONPP activity delivery methods, are there any alternate delivery methods that could be considered and in what areas?
  2. In view of the current structure of the ONPP, are there any alternate structures that could be considered and in which areas?

1.3 Approach

The evaluation went through various phases, including a design phase that was completed in May 2010, a data gathering phase between June 2010 and May 2011, and an analysis and reporting phase between June 2011 and December 2011. The data collection was divided in two steps. First, the evaluation team conducted a case study of CFG (that involved a document review and interviews). After approval of the case study document, the second step involved the completion of the documentation and literature review, interviews and survey for the purposes of the overall evaluation. The evaluation employed multiple lines of evidence as described below.

1.3.1 Case Study

A case study of the latest version of CFG (2007) was completed in 2010 with the aim of providing an in-depth understanding of one of the ONPP's key products. The case study was guided by the evaluation issues and questions outlined in section 1.2. A total of 357 documents provided by HC were reviewed to understand the activities, history, etc. Documents included: meeting minutes, planning documents, annual reports, research summaries, and other CFG related articles. Some of these documents were also used for the purpose of assessing the performance of the ONPP in general. These documents were supplemented with searches of scholarly electronic bibliographic databases and Google Scholar for peer-reviewed articles that could address the evaluation questions. A wide variety of websites were reviewed, including federal health and provincial health and education websites, and health-related non-governmental organizations such as the Heart & Stroke Foundation, the Canadian Cancer Society, Dietitians of Canada, and the Canadian Diabetes Association. The documents are listed in Appendix B.

Forty-three individuals were also interviewed either individually or in a group. The table below presents the number of interviews by respondent group as well as information on the focus groups. The findings were rolled-up into a technical report.

Table 1: Profile of Case Study Respondent/Focus Groups

Interviewee Group	Key informant	Approximate Time to Complete Interview	Total Interviewed and Method
Interviews internal to the Government of Canada (HC, PHAC, AAFC and INAC)	Staff and above	1 hour-1.5 hours	11 in-person interviews and 3 phone interviews. 7 of these interviews were with the ONPP's staff
External stakeholders (health, industry and experts)	Staff and above	30-45 minutes	1 in-person interview and 14 phone interviews.
			8 of these interviews were with respondents in the field of health (including academics and experts in nutrition).
Phone focus groups (Mostly external stakeholders and a mix of internal and external stakeholders)			14 representatives from industry, the Food Guide Advisory Committee and the Expert Advisory Committee on DRIs.
			In total, three focus groups were conducted by telephone due to the diverse locations in which stakeholders resided. One focus group contained 6 individuals from industry; one focus group consisted of 4 individuals from the Expert Advisory Committee on DRIs; and one focus group contained 4 individuals from the Food Guide Advisory Committee.

The following is a list of the different types of respondents who were interviewed:

Internal to GoC Interviewees (organizations in the Government of Canada that assisted in the development of various ONPP tools and products). They included:

• Health Products and Food Branch - Food Directorate;
• Office of Nutrition Policy and Promotion;
• HC Regional Offices;
• First Nations and Inuit Health Branch, Community Programs Directorate;
• Communications, Marketing and Consultation Directorate, Marketing and Corporate Services;
• Strategic Policy Branch - Applied Research and Analysis Directorate; and
• Strategic Policy Branch - Bureau of Women's Health and Gender Analysis.

Other government departments and agencies such as:

• Public Health Agency of Canada, Centre for Chronic Disease Prevention and Control and the Centre for Health Promotion;
• Agriculture and Agri-Food Canada - Market and Industry Services Branch and Food Bureau;
• Indian and Northern Affairs Canada - Strategic Policy and Devolution Branch and Northern Food Security Section;
• Canadian Institutes of Health Research - Institute of Nutrition, Metabolism and Diabetes; and
• Canadian Heritage, Multiculturalism and Human Rights Branch.

Other groups, such as Health Canada advisory committees:

• CFG Advisory Committee.

Stakeholder Organizations:

• Researchers and academics, such as University of Ottawa Heart Institute, Minto Prevention and Rehabilitation Centre;
• Non-governmental organizations including associations at the national, P/T and regional levels, e.g. Dietitians of Canada, Heart and Stroke Foundation of Canada, Canadian Diabetes Association, Canadian Obesity Network, Chronic Disease Prevention Alliance of Canada, Osteoporosis Canada, and Canadian Public Health Association;
• Intermediaries, such as Health professionals, healthcare facilities, and educators;
• Food industry, such as manufacturers and distributors;
• Canadian public (recipient, users, purchasers) including consumer associations and patient groups; and
• Media.

1.3.2 Document and Literature Review

Two separate document and literature reviews were completed. The first was done for the CFG case study. The second was done to extend the focus of the evaluation to other non-CFG activities of the ONPP. The second review, by the contractor, built on a preliminary review conducted by HC staff. The material collected for the document and literature review was used to answer the relevance questions and provide information on the program context, history, activities and outputs. The findings were rolled-up into a technical report.

For the CFG case study, a total of 357 documents provided by the ONPP were reviewed. Documents included: meeting minutes, planning documents, annual reports, research summaries, and other CFG related articles and documents. All of these documents were reviewed and are listed in Appendix B.

Both broad and more focused searches of scholarly databases were conducted to find appropriate literature. Focused searching was done using terms relevant to each evaluation question. Overall, over 2,000 article titles and abstracts were scanned for relevance and helped focus the search.

A wide variety of websites were visited and reviewed including federal health and provincial health and education websites, and health-related non-governmental organizations such as the Heart & Stroke Foundation, the Canadian Cancer Society, Dietitians of Canada, and the Canadian Diabetes Association.

For the second document and literature review, 571 documents were consulted by HC staff and Goss Gilroy Inc. A partial list of documents and literature had been summarized by HC in tables by evaluation issue and indicator. The consultant evaluation team then used the summaries of this work in addition to information contained in the case study of the Canada's Food Guide Report (Goss Gilroy, 2011), produced for the purpose of this evaluation. Other sources were also added to complete the information.

1.3.3 Key Informant Interviews

In addition to the key informant interviews conducted during the case study, key informant interviews were also conducted to fill in data gaps found in the document and literature review for the evaluation. Fourteen key informants from the Program, HC partners and P/T stakeholders were interviewed in person or by telephone. The following respondents were interviewed:

• Current and former ONPP management/staff (all three directors, the DG, some managers and some employees who had been with the ONPP more than five years) (9 in total);
• Two current Food Directorate management level respondents;
• Three representatives from P/T working groups and other key partnerships.

The interview guides were available in both official languages. The findings from all lines of evidence were combined in the final report.

1.3.4 Survey

An online survey of internal and external ONPP stakeholders was conducted between April and May, 2011. The survey collected information to triangulate data from the three other methods. The respondents in this sample were chosen because of their knowledge of the Program's activities and outputs. The sampling frame for stakeholders was provided by HC and included representatives from:

• Health Canada's ONPP;
• Health Canada's Health Products and Food Branch;
• Health Canada's Strategic Policy Branch;
• Health Canada's First Nations and Inuit Health Branch;
• Health Canada's Public Affairs, Consultation and Communications Branch;
• Health Canada's Healthy Environments and Consumer Safety Branch;
• Health Canada's Regions and Programs Branch;
• Health Portfolio Partners (i.e. CIHR and PHAC);
• Other Government of Canada Departments (Aboriginal Affairs and Northern Development Canada, Agriculture and Agri-Food Canada, Canadian Food Inspection Agency, Canadian Institutes for Health Information, Statistics Canada);
• Members of the ONPP's DRI committee;
• Members of the Federal/Provincial/Territorial Group on Nutrition;
• Members of the Network on Healthy Eating;
• Academics in the field of nutrition; and
• Representatives of Health Practitioner Associations.

In order to maximize the number of completed survey questionnaires, the evaluation team implemented a multi-phase and multiple mode approach. First, Health Canada sent an e-mail to potential respondents informing them of the upcoming survey invitation and encouraging them to participate. The survey was launched Tuesday May 31st, 2011. The consultants sent weekly e-mail reminders to persons who had not yet completed the survey questionnaire until the close date of June 27th, 2011. Finally, all invited respondents who had not yet completed the survey received a follow-up telephone call in the second week of the survey reminding them of the survey closing date and encouraging them to complete the survey. The final response rate for the online survey was 34%. The population targeted by the survey tended to be professionals, some of them reported having only been tangentially involved in consultations or being involved quite a few years ago. Given these facts, the response rate obtained for this survey is considered acceptable.

Table 2: Survey Outcomes: Survey Response Rate

Response	n	Percentage
Incorrect addresses	15	8.3%
Refused	7	3.9%
Ineligible (away during time of survey)	5	2.8%
Completed survey	55	30.6%
Total participants emailed	180	100.0%

Table 3: Response Rate Calculation

Category of Contacts	Total
Total contacts (total participants emailed minus incorrect addresses and ineligible)	160
Completed surveys	55
Response Rate (completed ÷ total contacts)	34.4%

Survey Participant Characteristics

Table 4: Survey Participant Characteristics

Organization	n	Percentage
Health related NGO (e.g. The Dietitians of Canada, Canadian Paediatric Society, etc.)	15	27.3%
Provincial and territorial government/partner	9	16.4%
University, college and other educational body	8	14.5%
Health Canada- Health Products and Food Branch	8	14.5%
Health portfolio organization	8	14.5%
Food industry (includes associations)	5	9.1%
Other	2	3.6%

The majority of survey respondents were from health related NGOs, followed by provincial/territorial partners and university, college and other educational bodies; Health portfolio organizations; and Health Canada, Health Products and Food Branch. More than 81% of the survey respondents had been involved with the ONPP for more than five years. In terms of the nature of their involvement with the ONPP, the majority (66%) of participants in the survey had used the ONPP's tools and products^{Footnote 1}. The next most common type of involvement (51%) was providing feedback or comments on tools and products, followed by being involved in the development of these (38%) as well as dissemination activities (35%) (percentages add up to more than 100 as some respondents were involved in multiple ways).

1.3.5 Analysis Strategy

A triangulated approach was used to collect and analyze the data. This entailed using different types of measures and/or data collection techniques to examine the Program and answer the evaluation questions. Using this approach improves the validity and reliability of the findings. The findings, based upon triangulating the data, were organized and anlayzed by the evaluation questions.

1.3.6 Challenges, Limitations and Mitigation Strategies from Data Collection Activities

The following challenges and limitations were encountered in the evaluation:

Program documents

The evaluation framework was developed with the assumption that high quality and systematically collected financial and non-financial program data would be available for all outputs and outcomes as identified in the Program logic model and for the time period under review. However, it became clear that the available data was produced sporadically and for purposes other than performance measurement. The data often lacked the level of detail that is necessary in an evaluation. Thus, the documents that were available contained gaps in the written account of the Program's activities, outputs and outcomes for the period under review.

Another limitation encountered in the document review was the usefulness of some of the documents. Some internal documents were in draft form and contained tracked changes. This led to uncertainties regarding the accuracy or validity of some internal documents. Other documents did not identify an author or a date, making it very difficult to reference in APA style. Additionally, some of the documents described in the HC initial document review were not referenced.

Survey and key informant interviews

These respondents provided useful insights on the ONPP activities and, to some extent, its outputs. However, their views on the achievement of outcomes, while important, could not be used as the sole criteria for determining whether the program was successful at the outcomes level. In cases where the only source of information about an outcome is the survey or key informant interviews, findings could not be used since the information could not be verified by other types of evidence.

Use of key informant interviews

Key informant interviews were used for the case study and later for the overall evaluation. While the methodology was extremely useful to gain in-depth perspectives on a range of issues, only a limited number of interviews were conducted for the overall aspects of the ONPP.

Mitigation Strategy

Triangulation of various lines of evidence was used to address these limitations and to ensure that the findings were valid and reliable. Interview questionnaires were designed to build on and fill in gaps in the document review or explain ambiguities in the data. Evidence from the literature review was used where interview data were not able to answer some evaluation questions. In addition to triangulating evidence, the Program was provided with each technical report (line of evidence) in order to correct for factual errors and omissions in the findings.

2.0 Findings, Conclusions and Recommendations

This section summarizes the findings, conclusions and recommendations by evaluation issue.

2.1 Relevance

Evaluation Question 1: Is there a continued relevance and need for the ONPP's activities as they are defined?

This evaluation question was addressed by examining three aspects of continued need and relevance:

• Extent of alignment between the ONPP's and the Government of Canada's priorities;
• Extent of consistency between the ONPP's activities and federal roles and responsibilities; and
• Extent to which there is a continuing need for the ONPP's activities.

Alignment with the Government of Canada's priorities

According to the findings, there is alignment between the ONPP's priorities and the Government of Canada's priorities in the area of health protection and promotion. Two Throne Speeches state that the health of Canadians is a priority for the Government of Canada. In 2004 it was stated that government would work with partners on preventive measures to help reduce the incidence of avoidable disease and improve the overall health of Canadians. In 2010, it was stated that "Protecting the health and safety of Canadians and their families is a priority of our Government." The program's alignment with the Government of Canada's priorities can be seen in its vision and mission statements. According to its vision statement, the ONPP is an "evidence-based organization focused on working with partners to improve the nutritional health and well-being of Canadians" (ONPP, 2007a, pg. 3). Its mission statement echoes this by stating that the ONPP promote(s) "the nutritional health and well-being of Canadians by collaboratively defining, promoting and implementing evidence-based nutrition policies and standards" (Health Canada, 2011b).

Consistency with federal roles and responsibilities

The ONPP's priorities and activities related to health are also consistent with federal roles and responsibilities. The legislative framework for nutrition is found within the Department of Health Act, whichstates that the duties of Health Canada (the minister) include the promotion and preservation of the physical, mental and social well-being of the people of Canada (Minister of Justice, 2010a). According to Health Canada's mission statement, "Health Canada is committed to improving the lives of all of Canada's people and to making this country's population among the healthiest in the world as measured by longevity, lifestyle and effective use of the public health care system" (Health Canada, 2011e). The ONPP's key activities are: 1) conducting surveillance and monitoring, knowledge development and exchange; 2) developing, revising and contributing to policies, standards and strategies; and 3) dissemination and outreach. These nutrition activities are consistent with the federal roles and responsibilities outlined above.

Continuing need for the ONPP's activities

There is extensive evidence from literature supporting the need for government-led activities to conduct research on and promote healthy eating. First, literature confirms that nutrition is a key determinant of good health. Obesity due to poor dietary practices is a contributing factor to poor health for many Canadians (Shields, 2005; Tjepkema, 2005). Studies also indicate that Canadians are not consuming enough of certain types of food (e.g., fruits and vegetables) in order to meet their nutritional needs (Garriguet, 2004). According to the 2004 Canadian Community Health Survey (CCHS), 23% of Canadians aged 18 or older, an estimated 5.5 million adults, had a body mass index (BMI) of 30 or more, indicating that they were obese (Tjepkema, 2005). An additional 36.1% (8.6 million) were rated as being overweight. Literature also confirms that poor nutrition is associated with a number of negative economic impacts such as rising costs for the healthcare system. The economic burden of a poor diet in Canada was estimated to be $6.6 billion in 1998. Direct costs amounted to $1.3 billion and indirect costs (e.g. healthcare and lost productivity) contributed an additional $5.3 billion.^{Footnote 2} An unhealthy diet is a preventable risk factor in cardiovascular disease, some forms of cancer, type-2 diabetes as well as osteoporosis and contributes to significant economic costs.

An international benchmarking review of nutrition programs and initiatives in Canada and similar countries such as the United States, Australia, the United Kingdom and Japan was undertaken by Health Canada. All these countries, including Canada, engage in similar nutrition science, education and health promotion activities. According to all key interview respondents (internal and external to ONPP), there is also a need to continue the ONPP's activities in Canada such as the provision of national-level guidance on healthy eating (for the general population and specific sub-groups such as pregnant women). A review of the history and progression of the structure of the ONPP (and its predecessors) within the federal government and HC provides evidence of the need for activities focusing on nutrition issues and on building partnerships and collaborations with other jurisdictions, within and outside Canada (Health Canada, 2006c).

Conclusions

Overall, the evidence indicates that the ONPP activities are consistent with federal priorities in the areas of health protection and health promotion as well as with federal roles and responsibilities. The ONPP's activities are aligned with the Government of Canada's priorities in general, as demonstrated in two Speeches from the Throne.

There is also extensive evidence that nutrition is a key determinant of good health and that, unfortunately, many Canadians have poor eating habits which could lead to increased risk for nutrition-related chronic disease and obesity that contribute to higher health care costs. This is not unique to Canada and many other similar countries consider good nutrition and healthy eating as a priority. There is also a need for activities within the federal government to focus on nutrition issues and to build and maintain partnerships with other jurisdictions within and outside Canada (Health Canada, 2006c).

2.2 Performance - Efficiency (Design and Delivery)

Evaluation Question 2: Were the ONPP's activities appropriately designed?

One of the purposes of this evaluation is to assess the design, structure and activities of the Program. An analysis was conducted of the ONPP's activities and structure, including the performance measurement framework. In particular, the ONPP's activities were assessed to determine the extent to which they 1) are science and evidence-based; 2) utilize risk-based strategies; 3) make use of consultations with partners and stakeholders; and 4) are monitored by an appropriate performance measurement framework. The ONPP was also assessed by comparing it with other countries that are similar to Canada.

Extent to which the activities of the ONPP are science and evidence-based

The evaluation assessed the extent to which the ONPP's activities and products are science and evidence-based through key informant interviews and the document review. Overall, evidence indicates that the ONPP develops and utilizes scientific evidence, including the Dietary Reference Intakes (DRIs), which in turn are used to develop nutrition-related information products. There was general agreement among the key informant interview respondents and survey respondents that the ONPP products are based on the most current scientific information. More than 89% of the survey respondents agreed that the 2007 version of CFG and the food and nutrition surveillance products produced by the ONPP were based on the latest and most reliable scientific information.

Many mentioned that CFG and other products are based on research and scientific information, including the Dietary Reference Intakes (DRI). The DRIs are a comprehensive set of nutrient reference values that can be used for assessing and planning diets. They are established by Canadian and American scientists through a review process overseen by the U.S. National Academies, which is a non-governmental body. The DRIs reflect the current state of scientific knowledge with respect to nutrient requirements and are published as a series of reports. The ONPP contributes to the development of the DRIs and uses the information to develop its own products, including CFG. CFG made use of statistical models to establish dietary patterns based on DRIs (Katamay et al. 2007). The DRIs and other scientific sources were also used to develop the revised 2004 Infant Guidelines and the 2009 Prenatal Guidelines. The ONPP also utilizes evidence from surveillance data and authoritative sources, such as the World Health Organization (WHO) and the U.S. National Institute of Health (NIH), to develop products that are accessible to the general public, according to interviewees.

Three nutrition indicators are now included in Health Indicators, which is a series of annual reports published by CIHI and Statistics Canada. They measure health status, non-medical determinants of health, health system performance and community and health system characteristics. The three nutrition indicators are based on the work of the F/P/T Group on Nutrition and the Food Nutrition Surveillance System Working Group between 2003 and 2005.

However, while respondents generally agreed that products are based on the best science available at the time, there remain limitations, including a lack of analytical information from surveillance data to meet the information needs of the ONPP, according to some respondents.

Extent to which the activities of the ONPP utilize risk-based strategies

The evaluation assessed the extent to which the design of the ONPP's activities utilizes comprehensive risk-based strategies and approaches to nutrition policy and promotion. The key informant interview respondents and survey respondents generally agreed that this was the case. The ONPP's personnel reported that some risk-based strategies were employed in its priority-setting and planning processes. Four out of the six respondents who could answer the question agreed that the ONPP's planning process is risk-based. According to respondents and documents consulted for the document review, the development of CFG, for example, utilized risk-based strategies. The design of CFG considered various factors that could affect the use and effectiveness of CFG, such as socio-economic and socio-cultural factors (e.g., literacy), and users with chronic disease. There was evidence that the ONPP hired experts to review the CFG publications for literacy levels. Health Canada documents, including issue sheets on topics such as mercury and trans-fat, provide some evidence that certain nutrition risks to human health were considered during the CFG revision process. For example, when developing CFG, the ONPP considered the risk of consuming fish containing mercury, before making recommendations about the amount of fish to consume. However, although obesity rates were an important consideration during the development of CFG, some observers stated that the risk of obesity was not given enough attention (e.g. not enough guidance on caloric intake, insufficient warnings about the consequences of eating foods high in calories/fat, etc.) (Merrifield, 2006).

According to the HC website, the Food Directorate and the ONPP work with federal, provincial and territorial partners on a variety of food and nutrition surveillance activities including the collection and analysis of data on what Canadians are eating, and the provision of guidance on interpreting surveillance data (HC, 2007d). This informs programs and policies by identifying risk areas. However, as previously discussed, some internal key informants mentioned that there is a lack of capacity at the ONPP to analyse the surveillance data. Some external stakeholders also felt that HC could do more of the analytical work around surveillance data. They felt that some organizations which need this information to inform their own programming did not have the resources (time or skill) to interpret complex surveillance data sets. They felt only the federal government had a sufficient level of resources and expertise to do this kind or work, the results of which are then useful for other organizations (provincially, regionally and NGOs).

Moreover, external stakeholders were divided on the issue of whether the ONPP's tools and products were addressing the right priorities. All agreed that the nutritional information met the general needs of the population. External stakeholders also felt that the ONPP should continue its efforts to explain DRIs, reviewing and translating evidence and conducting research. The majority of external stakeholders commented on the difficulty of ensuring that the tools and products reflected the most recent evidence and felt that some of the tools and products should be reviewed more often (e.g., an organization had started the process of developing a guide similar to CFG before the 2007 revision was announced because the older version did not reflect recent advances in nutrition science, others felt that the infant nutrition guidelines were not reflecting recent evidence).

Extent to which the activities of the ONPP make use of consultations with partners and stakeholders

The extent to which the ONPP's activities involve and make use of consultations with partners and stakeholders was assessed. Consultations with partners and stakeholders are a way of ensuring that the ONPP research and policy tools are useful and relevant to Canadians. According to key informant interview respondents and survey respondents, there is extensive evidence that the ONPP consults with various groups, from Canadian citizens to recognized experts, when setting priorities, planning and developing products.

The ONPP regularly consults with partners and stakeholders on its policies, standards and strategies. It co-chairs and provides secretariat support for the Federal/Provincial/Territorial Group on Nutrition. The FPTGN members include representatives from each provincial or territorial health department with responsibility for nutrition planning, programs and policies, and Health Canada representatives from HPFB and FNIHB (Health Canada, 2004d). The FPTGN is co-chaired by a provincial/territorial representative and a federal representative and provides leadership in stimulating and accelerating actions towards achieving nutritional well-being for all Canadians by:

• Promoting and supporting the development and review of public policies in food and nutrition with F/P/T significance;
• Providing technical input into the development of programs, position papers and resource materials and participating on national committees;
• Identifying emerging issues and priorities for federal/provincial/territorial collaboration;
• Providing advice to federal/provincial/territorial and international committees and subcommittees on food and nutrition issues, programs and policies;
• Sharing information about technical food and nutrition issues, programs and policies; community/public health nutrition personnel issues; and issues that impact on the nutritional health of Canadians. (Health Canada, 2000).

The ONPP co-chairs the Network on Healthy Eating and provides secretariat support for the organization. In addition to Health Canada, the Network on Healthy Eating includes representation from national organizations and associations, advocacy groups, industry, marketing boards, health charities with a focus on nutrition and healthy eating (Health Canada, 2004d). The network, which is co-facilitated by a federal government representative and a Network member, provides an opportunity to share information, foster partnerships and collaboration and enhance communication to improve the integration of messaging on key issues in healthy eating and nutrition.

On the subject of the nutritional health of Aboriginal people, the intradepartmental letter of agreement (ILA) with the Community Programs Directorate of FNIHB has been successful in supporting the inclusion of considerations specific to Aboriginal people into the ONPP's national nutrition policy and standards, and the broader work of the ONPP.

This ILA was in place from Jan 2003 to March 2009 and was an important horizontal approach to advancing the nutritional health of Aboriginal populations. Products and activities which came about through this partnership include the Nutrition Labelling Toolkit for Educators - First Nations and Inuit version, Eating Well with Canada's Food Guide - First Nations, Inuit and Metis, and the oversampling of Aboriginal people in the 2004 Canadian Community Health Survey.

For CFG, many consultation methods were used, including special meetings, focus groups and advisory groups. Some concrete examples of consultations include those conducted for the CFG revisions, as evidenced from the document ONPP Accomplishments 2006-2007 (ONPP, 2007a). It indicates that more than 7,500 individuals were consulted (including respondents to surveys) concerning the revision and dissemination of the new CFG. According to Martineau and Pronk (2008), science-based input was provided by the following advisory bodies:

The Expert Advisory Committee on Dietary Reference Intakes was established to advise Health Canada on how best to apply Dietary Reference Intakes (DRIs) to promote the nutritional health of Canadians.

TheFood Guide Advisory Committeeincluded individuals who were chosen for the varied perspectives they brought from public health, health policy, nutrition education, disease prevention, industry and communication.

The Interdepartmental Working Group was made up of representatives from a number of federal departments including Health Canada, the Public Health Agency of Canada, Agriculture and Agri-Food Canada, Indian and Northern Affairs Canada, the Canadian Institutes of Health Research, and Canadian Heritage. This working group provided a broad federal government perspective to the development of CFG.

Some concrete examples of the results of this collaboration and consultation include the creation of an Expert Advisory Committee to review Vitamin D supplementation in breastfed infants and the duration of exclusive breastfeeding.

Despite the above accomplishments and activities, survey respondents indicated that the level of consultation is lower among some groups, especially the Canadian public. According to survey results:

• 93% of survey respondents agreed that experts in the field of nutrition were well represented during past consultations;
• 89% reported that intermediaries (e.g., health professionals using the tools) were well represented during past consultations;
• 79% said other relevant departments/agencies were well represented during past consultations;
• 73% said other levels of government were well represented; and
• 57% said that the general public was well represented during consultations.

When prompted why they thought the general public was not well represented, some survey respondents indicated that not enough had been done to reach out to the general public (this was also the opinion of some key informant interviewees). This issue is a challenge to address because of the limited ability of any government department to conduct public opinion research. Some mentioned that dissemination issues, in particular, are especially important discussion items with representatives from the Canadian public. It was explained by key informant interviewees that the Internet has become an increasingly important dissemination medium (vs. other means such as through health professionals and educators) and that it is vital to discuss this means of dissemination directly with potential users. Other survey respondents indicated that there was an over-representation of government and industry representatives at the CFG consultations.

Extent to which the activities of the ONPP are monitored by an appropriate performance measurement framework

The ONPP has begun to develop a performance measurement framework to report annually to the Treasury Board on its commitments outlined in a Treasury Board submission. However, the ONPP's PMF is still in its development stages and has not been implemented. In 2001-02, the ONPP participated in a Performance Measurement Development Project (PMDP) led by the Food Directorate. One of the products of this effort was a draft logic model (dated 2003-2004). An untitled document dated January 2006 indicated that there is a need to refine the logic model in order to align it with Branch and Department performance measurement activities. The logic model contained in this document was reviewed (but not changed) in 2009, in preparation for this evaluation.

A review of the Management, Resources and Results Structure (MRRS) also shows that data has not yet been collected to provide information for some of the indicators in the ONPP's PMF. In some cases, the performance indicators are still in flux because some do not fit a yearly data collection schedule (e.g., monitoring levels of nutrients consumed by Canadians, because data on this topic is only available through the CCHS every eight to ten years). Another challenge is the limited ability to conduct public opinion research (which is an issue for all of Health Canada, not only the ONPP). Because of this limitation, some previously used/identified indicators are no longer being regularly collected to inform the Program including:

• Percentage of consumers who are aware of nutritional issues included in communication campaigns;
• Percentage of stakeholders who are knowledgeable about nutrition and information to make healthy food choices;
• Percentage of stakeholders who are aware of information to make healthy food choices; and
• Percentage of Consumers who have used HC information to make healthy food choices.

The creation of the ONPP's Accomplishments document provides an annual roll-up of program data (starting in FY 2003-04 and ending in 2007-08). These documents are descriptive in nature, nevertheless, they have been used to help inform operational planning activities, to reduce staff burden for providing input into corporate commitments and to assist in reporting activities to internal stakeholders. Web enquiry data (log statistics) has also been tabulated and analyzed to determine how the usage of the ONPP's online products helps enhance and inform decision-making around web activities. Overall, the ONPP has not systematically or consistently collected information to report on all performance indicators for the period that this evaluation covers.

Interviews with ONPP managers also indicated that, while in some areas they had enough information for regular reporting on operations (e.g., outputs, finances, etc.), there was a lack of data about the impacts of the tools (e.g., CFG, nutrition labelling campaign) on the general public's attitudes and behaviours. As mentioned earlier, there is also a lag in the analysis of some information about the food intake of Canadians. According to a few interviewees, some of the CCHS data from 2004 still has not been analysed. Obtaining and analyzing these data were seen as "fundamental" activities for the ONPP. However, two issues limited the timeliness of the analysis of food intake data from the CCHS Nutrition Focus Survey (2004). An unexpected issue under the Statistics Act arose, affecting dissemination of food data from Statistics Canada to Health Canada and other data share partners. The second issue was related to the Canadian Nutrient File (CNF), a key methodological tool in assessing dietary intake relative to the Food Guide. Changes involving multiple internal partners were required to the food classification system in the CNF to align foods with the Food Guide. That process is expected to be finalized in spring 2012. Another challenge with respect to the timeliness of data is that there have been significant gaps in time in the collection of comprehensive data on dietary intakes of Canadians. Prior to 2004, the last time comprehensive food consumption data were collected was in 1970-72. The next national nutrition survey is being planned for 2015.

Nevertheless, the ONPP did undertake a number of key surveillance activities during the time of this report:

• The ONPP undertook and/or worked with others to support analysis of CCHS 2.2 (2004) data in a timely fashion. For example, it worked with the Food Directorate and Statistics Canada to develop a series of data tables (national and provincial level) on nutrient intakes from food, comparing the results to Dietary Reference Intakes, which provides essential information on nutrient intakes of Canadians not available since the early 1970s.
• The ONPP released a series of articles on nutrition issues of Canadians based on the CCHS 2.2 data, interpreting nutrient intake data tables for use by a wider audience.
• The ONPP released a report on household food insecurity based on CCHS 2.2 data and continues to analyze and release data on key nutrition indicators (frequency of fruit and vegetable consumption, household food insecurity, breastfeeding practices).
• The ONPP has played an instrumental role in building capacity among nutrition data users (e.g., released a data users' guide to assist a large community of nutrition researchers and other data users, unaccustomed to using and interpreting such complex data, to make the best use of CCHS 2.2; facilitated dialogues among users at conferences and by forming a CCHS 2.2 Users' Group; partnered with Food Directorate, Statistics Canada and CIHR on a focused Request for Proposals to stimulate analysis of CCHS 2.2 data on priority public health nutrition issues).
• The ONPP has influenced the collection of nutrition data on other surveys (e.g., CCHS Focus Survey on Healthy Aging; Survey of Household Spending -Redesign).
• The ONPP, through work with the FNSS WG, influences the collection of nutrition biomeasures (Canadian Health Measures Survey).
• Surveillance data have been used to inform the policies of the ONPP and others.

The ONPP's activities in comparison to those of other comparable jurisdictions

Finally, the evaluation compared various aspects of the ONPP's programming with those of similar organizations in other countries. While this comparison is not, in itself, a direct indicator of effective design, it does provide a sense about the extent to which the ONPP's mandate, activities and products are similar to those of other jurisdictions. The source of evidence is mainly the international benchmarking review document produced for this evaluation. Overall, the review suggests that the ONPP has very similar mandates and activities when compared to other countries.

The review examined other developed countries in order to compare their nutrition-focused programs (the US, the UK, Japan and Australia). The programs and organizations compared with the ONPP include:

• Center for Nutrition Policy and Promotion (US);
• Nutrition Section, Healthy Living Branch (Australia);
• CCDHD Division of the UK Food Safety Agency (United Kingdom); and
• National Institute of Health (Japan).

The following highlights the results of the comparison:

• When comparing mission and mandate statements, the findings indicated that these organizations have a mandate to conduct (or fund) research and to disseminate information on/promote healthy eating;
• The US and Australian organizations have a mandate to develop policies related to nutrition;
• All have guides about healthy eating (MyPyramid Food Guide (US); Australian Guide to Healthy Eating; The Eatwell Plate (UK); Food Guide Spinning Top (Japan);
• Periodic surveys are used by most organizations for surveillance and monitoring; and
• In the area of partnerships, some types of domestic partnerships (e.g., with other government departments, local government, academia, associations and media) were common to each country while other domestic partnerships were unique to each country's specific health organization or system.

Table 5: Summary of International Benchmarking Comparison Exercise

Country	Organization Title	Mandate	Food Guides	Policies, Standards, Strategies and Education Initiatives	Surveillance and Monitoring	Knowledge Development & Exchange	Partnerships & Stakeholders
Canada	Office of Nutrition Policy and Program	Promote the nutritional health and well being of Canadians by collaboratively defining, promoting and implementing evidence based nutrition policies and standards	Canada's Food Guide	Eating Well With Canada's Food Guide- A Resource for Educators and Communicators My Food Guide Examples for meals for different families Translated Food Guides Nutrition Labelling education resources: Interactive nutrition label and quiz, nutrition labelling toolkit, posters including first Nations and Inuit consumer posters etc.) Infant feeding guidelines - - vitamin D supplementation for Breast fed infants, nutrition for Healthy term infants Prenatal nutrition guidelines Canadian Guidelines for Body Weight classification in Adults Body Mass Index Nomogram Contribution to the development of WHO School Policy Framework Sodium Working Group Report National Nutritious Food Basket	Engaging in analysis, and interpretation of data to support the development of evidence based healthy eating policies and promotion activities and inform the assessment of their impact Defining and developing indicators and tools to monitor nutrition and healthy eating issues Secretariat for the Food and Nutrition Surveillance System Working Group Releasing surveillance related products such as the Guide to Accessing and Interpreting CCHS 2.2 Data and Canada's Nutrition and Health Atlas	Plans, conducts coordinates and synthesizes knowledge development and exchange to support national nutrition policies and programs Co-sponsor Dietary Reference Intakes published by the IOM CJPH supplement:" Understanding the Forces that Influence our Eating Habits- What we Know and what we Need to Know"	Health Canada Public Health Agency of Canada Agriculture and Agri-food Canada Statistics Canada Aboriginal Affairs and Northern Development Canadian Institutes of Health Research Federal, Provincial, Territorial Group on Nutrition Network on Healthy Eating Academia and universities Health Professional Organizations NGO's Industry Media US Institute of Medicine for the DRIs US Dep't of Agriculture Organization for Economic Co- Operation and Development Pan American Health Organization WHO
USA CNPP	Center for Nutrition Policy and Promotion	The mission of the USDA Center for Nutrition Policy and Promotion is to improve the health of Americans by developing and promoting dietary guidance that links scientific research to the nutrition needs of consumers. Develop nutrition policies	MyPyramid Food Guide	My Foodapedia Healthy Eating MyPyramid Menu Planner MyPyramid Tracker My Pyramid for Kids and Pre-schoolers MyPyramid for Pregnancy and Breastfeeding Co-sponsor Dietary Reference Intakes published by the IOM Nutrition Insights USDA Food Plans US Food Supply Series Nutrient Content of US Food	Devises dietary methodology for the National Health and Nutrition Examination Survey Collects and processes dietary data obtained from the Survey	Conducts research to inform the dietary guidance Leads scientific review to inform policy and serve promotion and education	Cooperative Research, Education, Extension State Service Academic Institutions (e.g. Tufts University Friedman School of Nutrition Science and Policy) Dietary Guidelines Alliance Hispanic Communications Network Professional Organizations Naturally Nutrient Rich Coalition (NNRC) Media Industry Schools
UK. CCHD	CCDHD Division of the UK Food Safety Agency	Improve the health of the population by encouraging individuals to eat balanced diets as described in the Eatwell Plate Help reduce the prevalence diet-related illnesses by raising public awareness about balanced diets.	The Eatwell Plate	Eatwell- 8Tips for Making Healthier Choices Balanced Diet How to eat well on a budget UK Dietary Reference Values UK Dietary Survey Findings Guidance for Caterers Let's Cook What's Cooking Guide Advertising to Children Campaign	Commission The National Nutrition and Diet Survey, Low Income Diet and Nutrition Survey Undertakes Surveillance projects	Funds social research	British Education Departments British Health Department Local Authorities Scientific Advisory Committee on Nutrition Nutrition Strategy Steering Group Food and Drink Advertising Forum Consumers Groups Public Health Interest Groups Industry Schools European Union (EU) EU member states WHO FAO
Japan NIHN	National Institute of Health	Promote the health of the population by identifying and compiling scientific evidence to inform health and nutrition policies. Contribute to improving the health and well being of the population by conducting research to inform nutrition policy development.	Food Guide Spinning Top	Dietary Reference Intakes for Japanese Nutrition education programs and materials for consumers participating in food assistance programs	Tabulates and analyzes data for the National Health and Nutrition Survey	Conducts research using epidemiological tools to determine nutrition intake and health status	Universities Japanese Society of Nutrition and Dietetics Japan Dietetic Association Japan International Cooperation Agency Private companies Asian Countries (e.g. India, Korea, Malaysia) WHO FAO
Australia NS/HLB	Nutrition Section, Healthy Living Branch	Nationally coordinate health promotion activities to....improving the health and well being of the population and reducing nutrition related chronic illnesses and conditions. Develop nutrition policies	Australian Guide to Healthy Eating	Dietary Guideline for all Australians Australian Guide to Healthy Eating - Background Information for Nutritional Educators Dietary Guidelines for Australian Adults Food for Health (Poster) Nutrient Reference Values for Australia and New Zealand Results from the National Nutrition Surveys	Funds Australian National Children's Nutrition and Physical Activity Survey (2007) Evaluates policies resources and initiatives on nutrition and healthy eating	Conducts research e.g. Nutrition outcomes on vulnerable groups	Council of Australian Governments National Health and Medical Research Council

Conclusion

Many efforts have been made by HC to ensure that the ONPP's tools and products reflect the most robust scientific evidence and that a wide range of stakeholders are consulted. The evaluation found limited evidence that surveillance data were used to inform the priorities and the development of ONPP's tools and products. This information gap is a key issue which needs to be remedied.

Recommendation #1 - Plan to increase analysis and reporting on surveillance and monitoring data

It is recommended that HC improve its tracking and analysis of the eating patterns of Canadians and their needs for nutrition information.

Evaluation Question 3: To what extent have the ONPP's activities been implemented as planned?

Evaluation Question 4: What were the key achievements, lessons learned and challenges experienced in delivering the ONPP's activities?

Evaluation Question 5: To what extent have the requirements/commitments to Central Agencies, departmental and other requirements been addressed?

Evaluation questions 3-5 were combined in the answers below as similar data were used to provide evidence for all three questions.

These evaluation questions were addressed through the document review and key informant interviews.

Planning Process at the ONPP

According to key informant interviews and the 2006 document "Office of Nutrition Policy & Promotion Planning Framework", the ONPP has implemented a three-tier planning process to ensure it can deliver on its commitments. The process is first guided by an overall strategic plan focused on managing for results (tier 1). This strategic plan focuses on the medium to long-term activities and results to ensure the ONPP achieves its mission and meets its short, medium and long-term objectives. It is also meant to ensure that the ONPP can communicate internal activities to external audiences. The third tier consists of operational planning. This is an annual exercise which makes the link between operational and strategic plans. During this annual exercise the ONPP and Branch planning representatives examine the ONPP's plans to accord these with strategic directions and outcomes at various levels (ONPP, branch, department and government). The focus of operational planning is on short-term objectives (12-18 months). This level of planning includes work planning, budgetary processes, work plan tracking and reporting. A mid-level planning process was added to the two existing processes more recently, according to key informants. This second tier planning process takes into account the medium-term objectives and serves as a link between strategic and operational planning. In late 2009, the ONPP engaged in a series of work planning exercises that resulted in a number of work plan documents which identified goals and objectives for fiscal years 2009-2010 through 2011-2012.

Extent to which planned activities have been realized/accomplished

The main source of evidence that captures the extent to which activities have been implemented as planned are the ONPP's Accomplishment documents (2003-04, 2004-05 and 2006-07). The documents include the following accomplishments (organized by activity area):

1. Improved evidence base related to healthy eating issues

According to documentation, keyaccomplishments in this area include:

• Completion of five synthesis papers that summarized what is known about the determinants of healthy eating, including both individual and collective determinants, and their interactions (ONPP, 2004). This led to the development of a supplement to the 2005 Canadian Journal of Public Health.
• Contribution to the advancement of the Dietary Reference Intakes (DRI) with the US Institute of Medicine. (ONPP, 2007).
• Commission and release of English and French versions of The Dietary Reference Intakes: the Essential Guide to Nutrient Requirements (ONPP, 2007).
• Release of the CCHS 2.2 Nutrition (2004): A Guide to Accessing and Interpreting the Data and accompanying resources (ONPP, 2007).
• Release of a report on the Canadian Community Health Survey Cycle 2.2, Nutrition (2004): Income-Related Household Food Security in Canada (ONPP, 2007)
• Working with Statistics Canada and other partners, ONPP participated in the analyses of nutrient intakes from foods. The result of this collaboration was the development of a collection of CDs entitled: Canadian Community Health Survey (CCHS) Cycle 2.2, Nutrition (2004) Nutrient Intakes from Foods, Summary Data Tables Volume 1, 2 and 3.

According to interview respondents, the ONPP thus contributed to the science of healthy eating and advanced knowledge through these and other products.

2. Improved policies, programs, and initiatives to support healthy eating

Accomplishments, according to documentation, include input into various strategies and frameworks:

• Provided input about nutrition for the development of the 2005 Integrated Pan-Canadian Healthy Living Strategy (ONPP, 2004).
• Provided input into the development of the WHO Global Strategy on Diet, Physical Activity and Health (ONPP, 2004).
• Provided input on recommendations for priorities in research, policies and programs for the Joint Consortium on School Health (ONPP, 2005a).
• Provided input to the development of a strategy framework and scan of activities related to nutrition strategies for the Chronic Disease Prevention Alliance of Canada (CDPAC) Pan-Canadian Nutrition Strategy (ONPP, 2005a).
• Organized and participated on a panel titled "Population Approach to Healthy Weights: Enhancing our Understanding of the Issue" at the International Heart Health Conference in Milan, Italy (ONPP, 2005a).

Interview respondents indicated that the ONPP brought forward issues of healthy eating with health portfolio partners to advance the agenda of nutrition at the federal level.

Dietary Guidance:

Key accomplishments included:

• Released revised infant feeding recommendations to health professionals and other interested groups: Vitamin D Supplementation in Breastfed Infants, Duration of Exclusive Breastfeeding, and an accompanying fact sheet and Q&A document (ONPP, 2005a)
• Eating Well with Canada's Food Guide: the revision and launch of the new CFG and its associated web and paper resources, including a web-based resource with an interactive tool (My Food Guide) (ONPP, 2007a). The guide includes specific guidance for different ages and stages: food intake patterns were presented for nine different age and sex groups and advice is provided for different age and sex groups (Bush et al., 2007).
• Revised "Exclusive Breastfeeding Duration - 2004 Health Canada Recommendation" providing guidance on the duration of exclusive breastfeeding.

Standards:

Accomplishments include:

• Development of National Nutritious Food Basket.
• Contribution to Canadian Guidelines for Body Weight Classification
• Contribution to DRI development.

3. Increased awareness and understanding of healthy eating among Canadians

Accomplishments include :

• Published and disseminated 40,000 copies of two new infant feeding recommendations to health professionals (40,000 copies) (Health Canada, 2005c).
• Widespread dissemination of new information on nutrition labels on pre-packaged food products. Distributed 300,000 copies of the Nutrition Labelling tear sheet for consumers and over 30,000 Nutrition labelling posters. This included distinct resources for First Nations and Inuit (Health Canada, 2005c).
• Launch of the Interactive Nutrition Label and Quiz, an online tool that helps consumers make more informed choices about the food they eat (ONPP, 2007).
• Revision and launch of the new Canada's Food Guide and the following accompanying resources: A consumer print resource, a print resource for educators and communicators, and web-based resources including an interactive tool (My Food Guide) (ONPP, 2007a). Translated Canada's Food Guide into 10 languages (in addition to English and French) (ONPP, 2008).

These do not directly measure awareness or changes in awareness and the program has little to no data which measures directly awareness and understanding of healthy eating among Canadians.

Identification and description of lessons learned during program delivery

The following lessons were learned about partnerships and communications, according to documentation and interview evidence:

• In the area of partnerships, it was said that it is essential to ensure that the expertise of the ONPP is complementary to those of its partners and that activities are well coordinated between stakeholders to reduce duplication.
• The ONPP's internal communications are a key success factor and it is important to keep lines of communication open within the ONPP to ensure that quality deliverables are achieved according to plan, without duplication of efforts. This also means ensuring that staff are aware of the priorities and why these activities are priorities while remaining open to change.
• In the area of nutrition labelling, it was learned that, while the regulations mandating the provision of standardised Nutrition Facts tables on food labels of most pre-packaged foods are important, education of the general public about labelling can increase the potential use of these tables (ONPP, 2007c). This suggests that ongoing efforts to educate the public about the Nutrition Facts tables are important and should continue.

Issues and challenges affecting the ONPP

There are an increasing number of issues and challenges that are impacting the ONPP, including demographic, socio-economic and globalization issues; rising rates of chronic disease and obesity; changes in the food supply; and shifting social and physical environments. An increasingly complex food environment is making it more challenging for Canadians to make healthy food choices. These issues have a direct impact on the activities of the ONPP, including their design and effectiveness. Some of these issues and associated challenges include the following:

Demographics

Changing demographics in Canada related to an aging population, immigration and changing family structures all have implications for the ONPP. Canada's population is aging and it is predicted that "by 2026, one in five Canadians will have reached age 65" (ONPP, n.d.). The enthno-cultural diversity of the population will continue to increase significantly over the next two decades (Statistics Canada, 2010). A variety of customs or cultural beliefs among ethno-cultural communities necessitates a varied approach to promoting the nutritional health and well-being of Canadians. The challenge for the ONPP is to ensure products continue to be relevant and reflect changing demographics in a timely and effective manner.

Socio-Economic Conditions

Across Canada, there are vulnerable populations for which regular access to food is a challenge. Low socio-economic status, i.e., limited income, has a major influence on food access. In 2007-08, almost one million Canadian households (7.7%) had limited or uncertain ability to acquire enough food to meet their nutrition needs (ONPP, 2010). The challenge for the ONPP in its efforts to support and promote healthy eating among Canadians is to consider these and other factors/conditions that influence the ability of Canadians to engage in healthy eating.

Nutrition-related chronic diseases and obesity

The rising rates of overweight and obesity particularly in children, as well as rates of nutrition-related chronic diseases is a major public health concern (OECD, 2010). The challenge for ONPP will be to continue to consider these issues in national dietary guidance and ensure healthy eating issues are appropriately integrated into broader healthy living strategies.

The Food Supply

As suggested above,Canada's food supply is also changing. New products are being introduced in the marketplace (Stiefelmeyer, Martin & Klimes, 2008). The market for functional foods, novel foods and nutraceuticals is rapidly growing resulting in an increasingly complex food environment. The challenge for the ONPP is to consider the evolving food supply in national dietary guidance is developed and in communicating healthy eating messages in an increasingly complex environment.

Gaps in Information/Activities

Achallenge for the ONPP has been to provide analytical information/data about Canadian eating behaviours to inform development and evaluation of policies and programs. For example, no national data existed on the eating behaviours of Canadians when CFG was revised (Katamay et al., 2007).

Extent to which requirements/commitments to Central Agencies, departmental and other requirements been addressed

Although extensive business planning processes were documented, beginning at the end of 2009, the evaluators were unable to find detailed medium term (3-5 years) program planning documentation for the evaluation timeframe. As a result, Treasury Board submissions, performance measurement frameworks and departmental strategic plans were used to identify planning objectives and goals that could be used to answer this question.

A review of documentation indicates that there are multiple documents that outline what was expected from the ONPP. The ONPP has not been the subject of a specific submission to TB. Rather, the funding it has received has been the result of multiple submissions for the Health Products and Food Branch (HC), to which the ONPP belongs. Documents reviewed include multiple Treasury Board submissions, strategic plans for the HPFB in general, and as mentioned earlier, a performance measurement framework that was developed in 2004 and later reviewed (including a logic model). Nevertheless, the following activities and outputs were derived from commitments made at the Branch level:

Food Safety and Nutrition (beginning in 1999-2000 and ongoing):

• a) Nutrition Research
  1. Knowledge development through data collection, analysis and interpretation, and through trend surveillance, monitoring;
  2. Dissemination and reporting of research and surveillance findings;
  3. Coordination of knowledge development planning for nutrition and healthy eating.
• b) Nutrition Network of Partners and Stakeholders
  1. Identification of process steps and participants for collaborative review and revision of nutrition and healthy eating guidelines;
  2. Collaborative review and revision of existing nutrition and healthy eating policies and guidelines;
  3. Promotion of updates to revised policies and guidelines.
• c) Public Education
  1. Development and implementation of public education framework;
  2. Development of initiatives to increase nutrition-related awareness, knowledge and skills (e.g., public education on nutrition label);
  3. Development and dissemination of community-based models and best practices for nutrition interventions.

Integrated Strategy on Healthy Living and Chronic Disease (beginning in 2005-06 and ongoing):

• d) Enhanced Multi-sectoral Leadership and Collaboration on Healthy Eating
  1. Provision of technical expertise and leadership on nutrition within the Healthy Living Strategy, including participating in the ongoing development and implementation of the Strategy and to ensure the appropriate integration of nutrition into the Strategy;
  2. Examination of policy and intervention approaches to address unhealthy eating as a risk factor for cardiovascular disease;
  3. Exploration of opportunities for collaboration with health sector and other sector partners to advance action on healthy eating; and
  4. Identification of policy responses in various settings, populations and at various levels of governments to improve the nutritional health of Canadians.
• e) Strengthening International Partnerships and Collaboration
  1. Collaboration with the US Institute of Medicine to support the advancement of the scientific underpinning (e.g., Dietary Reference Intakes) for the development of national dietary guidance;
  2. Collaborating in the implementation of the WHO Global Strategy on Diet, Physical Activity and Health; and
  3. Provision of technical expertise.
• f) Development of Nationally agreed upon Nutrition Indicators and Targets
  1. Review and selection of health and performance indicators related to nutrition;
  2. Identification of sources for monitoring of indicators; and
  3. Policy support for establishment of a set of comparable nutrition indicators.
• g) Healthy Eating Knowledge Development and Exchange
  1. Development of a solution-oriented, collaborative research agenda to identify knowledge needs and approaches;
  2. Synthesis of evidence to inform the development of healthy eating guidance and tools.
• h) Nutrition Surveillance
  1. Linkage and access to data sources, including needs analysis, inventory of nutrition data;
  2. Sources, purchase of food and nutrition data, expanded indicator development, development and/or revision of food and nutrition modules, and the development and implementation of new nutrition surveys;
  3. Data analysis and interpretation, analysis of the Canadian Community Health Survey 2.2 and other relevant data, comparative analyses, regional presentations of analysis, analysis capacity building efforts, support for development of P/T capacity and the development of regular reports and articles, including production of a food security report;
  4. Development/enhancement of surveillance-related methods, standards and tools, including improvement of data collection tools, validation, and implementation of new tools;
  5. Information dissemination, including website enhancements, interactive nutrition indicator and information mapping, and the publication of reports and articles; and
  6. Evaluation of, and periodic reporting on, nutrition surveillance activities.

Finally, since there is no direct reporting relationship between the ONPP and the Central Agencies (against commitments), there is no single source of information (report) matching the activities versus commitments. Moreover, there is no regular reporting mechanism to the Department apart from the provision of information to meet MRRS requirements, and the provision of various Accomplishment Documents, which do not refer to specific targets. Nevertheless, a review of documentation suggests that numerous activities contributed to most commitments to central agencies. The lack of specific targets, however, did not allow the evaluation to conclude on the extent to which commitments are truly met.

Conclusions

According to the evaluation findings the ONPP's processes and products are science and evidence-based as they reflect ongoing research and consultations with experts. The ONPP uses science-based resources, such as DRIs and surveillance data. However, there is a need for more timely analyses of surveillance data to maximize its use. This in turn could inform timely evidence based revisions of the ONPP's products.

The ONPP has also conducted consultations on an ongoing basis, especially with experts and other organizations, though it is felt that there could be more consultations with the general public, especially about dissemination of products and information.

The ONPP has implemented a three-tier planning process and has developed a logic model and performance indicators for reporting purposes. There is an opportunity to update the program logic model and implement its performance measurement framework.

Again in the area of planning, while there is no systematic reporting on the extent to which planned activities are realized as planned, there is evidence that, overall, the ONPP's main activities are aligned with its commitments to TB, including CFG, which was a key activity for the ONPP between 2004 and 2007 Nevertheless, CFG, could be better planned as no formal schedule exists to review it.

Key achievements and challenges were also assessed through the evaluation. According to the evidence, the ONPP has successfully completed a number of projects and activities, including the development of products and papers, and contributions to networks and policies related to healthy eating. Many factors affect the eating behaviours of Canadians, including various social-economic barriers, demographics and commercial factors, and the changing food environment. These present a number of challenges for the ONPP, including that of continuing to provide information products in a timely and effective manner.

As for its commitments to central agencies, documents indicate that the ONPP does not report directly to central agencies and there is no regular reporting mechanism to the department apart from the provision of information to meet MRRS requirements and achievement documents, which are not target-based. A review of documentation suggests that numerous activities contributed to most commitments to central agencies. The lack of specific targets, however, did not allow the evaluation to conclude about the extent to which commitments are truly met.

Recommendation #2 and #3

Recommendation # 2 - Improve planning and project management for the next revision of CFG

It is recommended that HC establish a review cycle to determine if revisions to CFG are required and the scope of revisions.

Recommendation # 3 - Improve the existing performance measurement framework so that expected outputs and outcomes are systematically monitored, adjusted and progress reported

It is recommended that the ONPP's performance measurement framework be updated to meet operational planning, reporting and accountability requirements, including reporting on outcomes and strategic reviews. This process should begin by reviewing the logic model to ensure that it is still relevant in the current context.

2.3 Performance - Effectiveness (Success)

Evaluation Question 6: To what extent have the ONPP's activities contributed to the achievement of the immediate, intermediate and long-term outcomes?

This evaluation question was addressed by assessing the extent to which the ONPP has achieved its expected immediate, intermediate and long-term outcomes, as stated in its program logic model. These include:

The expected immediate outcomes:

• Increased knowledge of information related to healthy eating;
• Improved policies, programs and initiatives to support healthy eating; and,
• Increased awareness and understanding of healthy eating among Canadians.

The expected intermediate outcomes are:

• Increased evidence-based decision-making nationally and internationally; and,
• Improved healthy eating by Canadians.

The expected long-term outcome is:

• Reduced level of nutrition-related concerns, conditions and chronic diseases.

The expected ultimate outcome is:

• Improved health and well-being of Canadians.

The following sub-sections assess these expected outcomes, based on the documentation, key informant interviews and survey evidence.

Immediate Outcomes: Increased knowledge of information related to healthy eating

According to this immediate outcome, the ONPP is expected to contribute to increased knowledge of information related to healthy eating. Evidence indicates that this was achieved through the production of papers and articles.

Papers and articles

Several articles, papers and conference presentations were prepared by the ONPP's staff since 2001. The ONPP's staff published in Nutrition Reviews an article entitled "Eating Well with CFG (2007): Development of the Food Intake Pattern". A number of reports resulted from the consultation activities undertaken by private research firms contracted by Health Canada. These reports describe the results of surveys, interviews, and focus groups undertaken with various groups of stakeholders throughout the CFG revision process (ONPP, 2007a).

Research

Publications quoting or using CFG provide additional information about how the ONPP contributed to the evidence base of healthy eating. A literature review indicates that at least 15 academic articles cited the 2007 version of CFG. Some of these studies used recommendations from the 2007 CFG to assess eating patterns of groups, such as different socio-economic groups (Tarasuk, Fitzpatrick & Ward, 2010), ethnic minorities (Quadir & Akhtar-Danesh, 2010), homeless people (Tse & Tarasuk, 2008), hospitalized pediatric patients (Obadia et al., 2010), and youth (Riediger, Shooshtari & Moghadasian., 2007; Li, Dachne & Tarasuk, 2009; Storey et al., 2009). One study used CFG to develop "normal sized meals" for patients (Tasca et al., 2009). Other studies simply mentioned the guide in passing (e.g., Berry et al., 2009; Maillot et al 2010; Merchant et al 2009; Visioli et al., 2007). These are clear examples of information uptake of CFG by health researchers and practitioners.

The ONPP contributed to the development of the DRIs ( a comprehensive set of nutrient reference values), and guides and reports based on the DRIs which are made available through the ONPP's web pages. These products can be used for assessing and planning diets.

Immediate Outcomes: Improved policies, programs and initiatives to support healthy eating

CFG has been used as a foundation for many policies, initiatives and programs including provincial school food guidelines and many health teaching and learning resources which refer to its healthy eating recommendations. For instance, many interviewees reported that various groups use CFG to develop their nutrition regulations, programs and policies. As one respondent explained: "We use these (products) all the time. They are fundamental for us. They guide the development of all our services, policies, publications. Anything they have we will use." Other examples provided by respondents included the Heart and Stroke Foundation's Health Check program as well as many programs and activities in public health (nutrition, pregnancy, infants, nutrition labeling) using the ONPP's tools and products. A significant majority (95%) of survey respondents also agreed that CFG and the other ONPP tools and products (83% agreed) are used by organizations to develop or improve policies and programs to support healthy eating.

The following are examples of how CFG has been used by various types of organizations:

• Most of the provincial and territorial governments use CFG to underpin their school food guidelines and policies (Health Canada, June 2011);
• School board policies on nutrition including the Hamilton-Wentworth District School Board (HWDSB, 2011); the city of Brantford municipal bylaws require lodging houses to provide meals which conform to CFG;
• Ontario's Healthy Food for Healthy Schools Act, 2008 (which is based on the 2007 version of CFG);
• The FPT Joint Consortium for School Health (JCSH) Healthy Schools Toolkit recommends that teachers and schools "use Canada's Food Guide materials to support the healthy eating curriculum" and "use Eating Well with Canada's Food Guide materials to support lesson plans related to healthy eating" (JCSH, 2008, p. 6). The JCSH document lists Eating Well with Canada's Food Guide - A Resource for Educators and Communicators as a resource; and
• The Food Mail Program (which is administered by Indian and Northern Affairs Canada, now called Aboriginal Affairs and Northern Development Canada, and which has been replaced by the Nutrition North Canada Program on April 1, 2011) supported healthy eating in isolated northern communities by providing a subsidy for healthy foods supported by CFG guidance.

Immediate Outcomes: Increased awareness and understanding of healthy eating among Canadians

Availability of Information

HC has developed many nutrition and health promotion products whose intended purpose is to increase awareness and understanding of healthy eating practices by Canadians. The following are key examples of such products:

• My Food Guide, the interactive Web-based tool where individuals can personalize CFG information based on age and sex and food and activity preferences; it can be completed in English or French; and is available to print in English, French, Arabic, Chinese (traditional or simplified), Korean, Farsi (Persian), Punjabi, Russian, Spanish, Tagalog, Tamil, and Urdu (Health Canada, 2008b).
• Eating Well with Canada's Food Guide - A Resource for Educators and Communicators and online power point presentations for intermediaries provide background information, tips and tools to complement each recommendation in CFG (Health Canada, 2008a).
• Eating Well with Canada's Food Guide, which has been translated into 10 different languages in addition to English and French (Health Canada, 2008c).
• Nutrition labelling information. The ONPP has produced and provides information on nutrition labeling, including interactive tools, factsheets, quizzes, etc.

The availability of these resources in print or electronic formats provides many entry points for consumers to access this information. External stakeholders interviewed for the evaluation agreed that CFG had increased the quantity/quality of information on what constitutes healthy eating. The availability and accessibility of tools are requirements for these to be used by intermediaries but there was no information about whether/how these tools were used by Canadians other than CFG as discussed below. This data limitation will be addressed in the recommendations.

Level of awareness of the ONPP's nutrition-related resources among Canadians

As stated before, there is limited information on the level of awareness of the ONPP's nutrition-related resources among Canadians. The exception is CFG. Most Canadians are aware of CFG as a nutrition-related resource. A study by Environics (2007) found that there is almost universal awareness of CFG (94%) and that half (48%) of parents have a copy in their home. Uptake of the new CFG was relatively strong; with close to half of these current users (or one-quarter of all parents) reporting that they had the new version of the CFG only two months after it was first introduced.

Nielsen's (2007) Health & Wellness Study found that more than one-third of Canadian households (35.2%) reported that they had seen the most recent edition of Canada's Food Guide. Thirty-seven percent of respondents identified Canada's Food Guide as a top source of information on the topic of healthy eating; this was up four percentage points from 2006. Focus groups with intermediaries who promote healthy eating among specific ethno-cultural communities found that awareness of the 2007 version of Canada's Food Guide varied: it was higher among nutritionists and nurses, but limited among ethno-cultural community workers and immigrant settlement workers(Health Canada, 2007b).

Respondents to the evaluation survey were asked if they believed key ONPP tools or products were easily accessible. Overall respondents agreed that the ONPP's tools and products were easily accessible. In particular,

• 93% agreed that the 2007 version of Canada's Food Guide was easily accessible;
• Only 64% felt that this was the case for food and nutrition surveillance products such as the Canadian Community Health Survey Cycle 2.2, Nutrition (2004): A Guide to Accessing and Interpreting the Data (2006), Canadian Community Health Survey Cycle 2.2, Nutrition (2004): Income-Related Household Food Security in Canada (2007); and
• Close to 71% of respondents agreed that the other ONPP tools and products were easily accessible.

Use of the ONPP's Products

There is limited evidence about the extent to which the ONPP's products are used by Canadians outside the above findings related to CFG. However, documentation does provide evidence about the extent to which products are distributed. According to a review of the ONPP's website (2008), the number of web hits on the Eating Well with Canada's Food Guide web pages indicates it was the most-requested of all nutrition pages on Health Canada's website. Of the publications requested from Health Canada, about 57% were for Eating Well with Canada's Food Guide, while 34% were for the French version of this guide.

In 2007/2008, Eating Well with Canada's Food Guide and Eating Well with Canada's Food Guide: A Resource for Educators and Communicators were two of the most-requested ONPP resources (ONPP, 2008). In 2007/2008, 9,411,200 copies of the English version of CFG and 3,778,000 French versions were distributed. The same year also saw 174,622 document requests for the Resource for Educators and Communicators in English and 87,746 requests for the French version. According to internal estimates drawn from the Ottawa Logistic Warehouse Publication Report, the ONPP distributes approximately 3.5 million copies of CFG (in both languages) every year. Between the launch of My Food Guide Online in February 2007 and April 23, 2008, a total of 924,272 individualized documents were created by individuals accessing this resource online (ONPP, 2008). According to Belzile & Logue (2009), the most frequent request for publications through the Healthy Eating e-mail account was for Eating Well with Canada's Food Guide. Between January and December 2010, more than 408,000 My Food Guide Servings Trackers were also downloaded^{Footnote 3}.

The ONPP's most recent Accomplishments document for 2007/2008 also detailed requests for resources other than CFG. Besides CFG, the most requested document was Make Informed Food Choices (labelling tear sheet) of which 219,867 copies were requested and distributed. The Canadian Guidelines for Body Weight Classification in Adults was another document in high demand, with 198,642 copies being distributed. Overall, the total number of web enquiries to healthy eating was 1,194 in 2007/08. The highest level of web enquiries was recorded in 2004/05 (2,190 enquiries).

Stakeholders surveyed for this evaluation indicated that the ONPP tools and products were used by targeted users. In particular, CFG was seen as most used, easy to understand and to use, while the more complex FNSS products were seen as widely used by the target audience but more difficult to understand and use (which is to be expected since these resources are not targeted at the general population).

• 84% of respondents agreed that the 2007 version of Canada's Food Guide was used by targeted users, 89% agreed it was easy to understand and 79% agreed it was easy to use;
• 86% of respondents agreed that FNSS products were used by targeted users, 63% agreed they were easy to understand and 57% agreed they were easy to use; and
• 71% of respondents indicated that the other ONPP tools and products were used by targeted users, 74% indicated that these were easy to understand and 66% stated these were easy to use.

The evaluation also assessed the extent to which external stakeholders (e.g., immigrant settlement services workers, ethno-cultural social workers, and public health nutritionists, dietitians and nurses, who promote healthy eating among specific ethno-cultural communities) use these products for their own clients and beneficiaries. Public health nutritionists, dietitians and nurses stated that CFG (either the 2007 or the 1992 versions) is among the resources they use and is considered an important tool in their work (Health Canada, 2007b). The other external stakeholders were less likely to report using CFG.

All external stakeholders interviewed for the evaluation agreed that surveillance products published by the ONPP have assisted them and others in the field of health to better understand the factors that influence eating behaviours and practices. Stakeholders interviewed for the evaluation all agreed that the ONPP's tools and products reached their targeted audiences well. However, some respondents felt that the ONPP could do more to understand who was not using the tools or products and the reasons why.

Impact on level of understanding of Canadians about healthy eating

Survey respondents were asked if they thought that CFG was likely to help Canadians better understand healthy eating and how to do so.

• 67% (35/52) agreed with this statement;
• 19% (10/52) disagreed with the statement;
• 14% (7/52) felt they could not comment.

Of those who agreed (35/52) that CFG helped Canadians better understand healthy eating:

• 71% (25/35) felt CFG helped people to understand healthy eating;
• 20% (7/35) remarked that CFG helped but that changing eating behaviours is complex and that CFG is only one component of changing eating behaviours; and
• 14% (5/35) indicated that although the guidance was helpful, they doubted it would have an effect on actual behaviour (especially as a standalone document).

Those respondents who felt that CFG did not assist Canadians in better understanding healthy eating and how to do so (10/52) stated the following:

• 60% (6/10) said that there are gaps in the information provided in CFG or the information provided is too complicated; and
• 40% (4/10) said that CFG is too broad and oversimplified.

Survey respondents were also asked to rate different aspects of key ONPP tools and products they were familiar with. This included CFG, food and nutrition surveillance products as well as the prenatal nutrition guidelines and nutrition labelling products. The ratings for the two most well known products are reported below:

• 80% (35/44 survey respondents who rated this activity) agreed that the 2007 version of CFG was likely to increase awareness and understanding of healthy eating among Canadians;
• 62% (13/21 survey respondents who rated this activity) agreed that this was the case for food and nutrition surveillance products.

Ratings for all other tools and products were combined to give an overall perspective. The ratings given to the tools, overall, were very positive:

• 75% (41/55 survey respondents) indicated that other ONPP tools and products were likely to increase awareness and understanding of healthy eating among Canadians (directly or indirectly).

Intermediate Outcomes: Improved healthy eating by Canadians

There is some evidence from public opinion research that CFG activities have contributed to better-informed choices and improved healthy eating by Canadians. For example, according to Nielsen's (2007) Health & Wellness Study, of those Canadian households who had seen the newest edition of CFG, more than half (52%) indicated that they had made at least one change to their household grocery shopping habits as a result of seeing the new Food Guide. The top three changes that Canadian households have made as a result of seeing the new CFG include buying more fruits (31.7%), buying more whole grains/products made with whole grains (28.0%), and buying more dark green vegetables (26.1%).

Some experts question the ability of a food based dietary guide (FBDG) like CFG to have a significant impact on consumer behaviours. For example, Smitasiri and Uauy (2007) indicate that the positive impacts of FBDGs may be counteracted by the strong influence of marketing and advertising in shaping food choices of individuals and communities. These authors state that the resources supporting the marketing of unhealthy diets are between 100 and 1,000 times greater than the funds available to promote consumption patterns based on FBDGs. Although HC has strategies in place to work with the industry on nutrition issues, these factors may limit the level of improvement in healthy eating among Canadians. While CFG is an important policy tool that defines healthy eating, it is only one element within a broader, comprehensive strategy needed to promote and support healthy eating.

Level of change in eating / dietary practices of selected at-risk populations^{Footnote 4}

There is evidence that the ONPP's products have had impacts on eating behaviours and dietary practices among at-risk populations:

Older Canadians

Nielsen's Panel Track Health & Wellness Study (2007) provides some evidence that older Canadians have made positive changes in their dietary practices as a result of seeing the new CFG. In general, older households reported making more changes than their younger counterparts as a result of seeing the new CFG (Nielsen, 2007, p. 13).

Lower income households

The same study also provides some evidence that lower income households have made positive changes in their dietary practices as a result of reading the new CFG.

Children

no evidence available.

Intermediate Outcomes: Increased evidence-based decision-making nationally and internationally

There was little available evidence to answer this question. The following findings indicate some of the ONPP's contributions at the national and international level.

The ONPP commissioned the development of a background paper on healthy eating in schools that described effective policy options in the school setting, the roles of different stakeholders in school policies, as well as monitoring and evaluation of school policy implementation (ONPP, 2007a). This paper supported the development of the WHO's 2008 international policy framework entitled, "School Policy Framework: Implementation of the WHO Global Strategy on Diet, Physical Activity and Health", in collaboration with PHAC and WHO.

Long-term Outcomes: Reduced level of nutrition-related conditions and chronic diseases

The program has not tracked or collected the necessary information to demonstrate the contribution of its activities to reducing the level of nutrition-related conditions and chronic diseases in the Canadian population. While theoretically the program is designed to contribute to these outcomes, the empirical evidence for program attribution or contribution does not exist to demonstrate effectiveness at this outcome level.

Conclusions (Success)

The ONPP made numerous contributions to the evidence base related to healthy eating through papers, articles and the ONPP's products. Many policies, initiatives, and programs use one of the ONPP's key products, CFG, as a foundation. There is also evidence that the ONPP's products have contributed to increase the awareness and understanding of healthy eating and nutritional issues among some Canadians. While evidence for the intermediate outcomes is more limited, there is some evidence that CFG has contributed to better-informed choices and improved healthy eating by Canadians (including those at risk), although it is acknowledged that products such as CFG should be part of broader health promotion strategies. The lack of quantitative evidence about the long-term impacts of the ONPP prevented the evaluation from assessing these aspects of performance.

2.4 Performance - Economy

Evaluation Question 7: Are the current structure and activity delivery methods of the ONPP the most cost-effective means of achieving the intended outcomes?

The lack of outcome information prevented this evaluation from truly assessing the cost-effectiveness of the ONPP (cost-effectiveness assumes some sort of measurement of outcomes, for which there is no documentation). To address this issue, the efficiency of the ONPP was assessed by reviewing the ONPP's financial information; by assessing the extent to which resources are leveraged through partnerships; and by identifying alternative structures that could improve the efficiency and effectiveness of the ONPP.

The ONPP's Financial Information

The ONPP's budget increased in 2005-06 according to official documents. The "reduction" in 2008-09 was due to additional branch overhead costs reflected in the modifications to A-base and the adjustments figure. The ONPP's expenditure patterns were mostly due to the project-based nature of the ONPP's work. Overall expenditures increased every year until the release and reprints of the Food Guide in 2007-08. The decrease after 2007-08 is largely due to a decline in reprints required a few years after the launch of CFG, and because a fair number of employees focused on other projects after the CFG process was completed.

ONPP Financial Information (2001-2009)

Fiscal Year	Year End Budget	Program Expenditures	Surplus
2000-01	$1,464.00	$ 731.64	$ 732.36
2001-02	$2,377.23	$ 1,505.42	$ 871.81
2002-03	$2,688.12	$ 2,668.04	$ 20.07
2003-04	$2,689.54	$ 2,382.50	$ 307.04
2004-05	$3,285.43	$ 2,690.08	$ 595.36
2005-06	$4,832.71	$ 3,766.61	$1,066.10
2006-07	$4,149.35	$ 4,010.29	$ 139.06
2007-08	$4,607.98	$ 4,489.98	$ 118.01
2008-09	$3,815.08	$ 3,555.72	$ 259.35

Above table includes budget and expenditure figures for Salary, Students and O&M. It does not include EBP or corporate expenditures (e.g. accommodations, IT Way Forward, etc...).

Leveraging Partnerships

The extent to which partnerships are used to leverage resources was also assessed. The ONPP entered into partnership with academic institutions, other government organizations and private organizations. These led to MOUs and other agreements, which in turn led to DRI reports, publications, performance data, and outreach products.

Academic Partnerships

The ONPP entered into a contract with the Institute of Medicine and the National Academies Press for a targeted DRI Summary Report for health professionals such as nutrition educators and students, community nutritionists, dietitians and other interested stakeholders (ONPP, 2005a).

Partnerships with Other Government Departments, Agencies or Crown Corporations

Partnerships with the PHAC are extensive and cover a range of nutrition, healthy eating and healthy living issues. The ONPP collaborates with the PHAC as part of the Healthy Living Strategy (ONPP, 2004). In 2007, an MOU was signed between the Health Portfolio (ONPP, Food Directorate and PHAC) and Statistics Canada for the inclusion of five nutrition biomarkers in the Canadian Health Measures Survey (CHMS), Cycle 1 (ONPP, 2007a). In addition, the ONPP partnered with AAFC to purchase AC Nielsen data to inform dietary guidance in 2007 (ONPP, 2007a). In 2005, the ONPP partnered with the Food Directorate, Statistics Canada and the Canadian Institutes of Health Research (Institute of Nutrition, Metabolism and Diabetes) to develop a Request for Applications (RFA). With research priorities identified with input from provincial and territorial nutrition partners, the RFA supported analysis of the Canadian Community Health Survey (CCHS) Cycle 2.2 (Nutrition) data to help improve the evidence-base to inform and guide policies and programs at federal and provincial levels. Special considerations of Aboriginal peoples were included in the national nutrition policy and standards, and within the broader work undertaken by the ONPP, as a result of the ILA between Community Programs Directorate (FNIHB) and the ONPP.

Private Partnerships

The ONPP partnered with the Museum of Science and Technology and the Canadian Agriculture Museum to develop a touring exhibit on "Food for Health" to highlight food production in Canada (ONPP, 2007). Kellogg Canada and the Dietitians of Canada created education materials for elementary school children and partnered with the ONPP to include messages on nutrition labelling (ONPP, 2007).

Potential Duplication

Key informants reported that there were some areas where potential for duplication exists with other organizations. For example, the PHAC has the chronic disease prevention portfolio, under which nutrition plays a key role. It is also clear that P/T and local governments undertake health promotion and nutrition information activities in the area of chronic disease prevention. Most (71%) survey respondents said that the ONPP did not duplicate services offered by other organizations (federal/provincial/ regional governments, NGOs, nutritionists' associations). Eight respondents alluded, however, that there was duplication in policy areas (e.g., sodium) or that other organizations may even "copy" the work of the ONPP. Five of these respondents specified that the overlap was with HC, the PHAC and provincial governments.

Generally, most key informants agreed that duplication was avoided between the ONPP's activities and those in other departments because of the partnerships and working groups/collaborations. Since 2009, work was done with the PHAC to clarify their respective roles and responsibilities with regard to nutrition. These lines of communication between different departments ensured that key players are aware of what each other is doing and supporting more collaborative approaches and thus decreasing potential duplication. A few interviewees reported that surveillance was also part of the role of other branches in HC and the PHAC. Here again, these interviewees reported duplication was avoided through communication and collaboration. A few respondents added that it would be useful to re-establish more clearly each department/branch's role and responsibilities to more formally avoid duplication. The roles and responsibilities as originally set out when the ONPP was created were deemed clear. It was stated that the ONPP would remain the centre for nutritional expertise. Re-establishing original roles was thought to clarify any potential issues of duplication in roles.

Alternate Structures

Health Canada and its predecessors have provided national leadership in nutrition since the 1930s under a variety of organizational structures (Health Canada, 2006c). Significant changes to the Nutrition Division took place between 1970 and 1999: first, the Nutrition Division was moved under a regulatory umbrella; after a 1970 reorganization the Nutrition Division was moved under the new Health Protection Branch (Health Canada, 2006c). In 1978, the nutrition science areas were separated from the policy and education functions (Health Canada, 2006c). HC formed the Health Services and Promotion Branch (becoming the Health Promotions and Programs Branch in the 1990s) which brought together programs related to health promotion and the provision of health services (Health Canada, 2006c). Key nutrition functions were then split with the science and policy related to food staying under the Bureau of Nutritional Sciences, Health Protection Branch, while the education and policy related to dietary guidance moved to the new Health Services and Promotion Branch (Health Canada, 2006c).

In 2000 HC undertook a realignment exercise which provided an opportunity to remediate weak linkages between the science and policy functions (Health Canada, 2006c). To integrate its science, policy, health promotion and related research activities, HC created the Office of Nutrition Policy and Promotion, going back to the original structure (Health Canada, 2006c). This structure had a history of success for the first forty years of the Nutrition Division (Health Canada, 2006c).

In 2010, a proposal for an alternate structure was put forward by the ONPP. The proposal was for the creation of a Nutrition Directorate within HPFB that would serve as "the focal point on all science, surveillance, policy, promotion, submission review/approval, regulatory and outreach functions related to nutrition and healthy eating - within the scope of the Health Canada and Portfolio's mandates" (ONPP, 2010, p.1). The Directorate would combine the ONPP, the Bureau of Nutritional Sciences (BNS), and the components of the Bureau of Food Policy and Science Integration concerned with biostatistics, analysis and epidemiology.

There were many perceived benefits of the proposed Nutrition Directorate. The new Directorate could be more efficient and could reduce duplication on those files that currently require approval from multiple organizations. Various options for alternative structures have been discussed with senior HC staff over time but there was limited support for such structural changes.

Conclusions

The evaluation was not able to find evidence that the Program was cost-effective using standard methods. Rather, cost-effectiveness was assessed by looking at financial results information; by evaluating the effectiveness of the ONPP to leverage resources from partnerships; looking for potential duplication of activities; and by identifying potential alternatives to current structures. The ONPP's financial results have varied in terms of deficits and surpluses over the last decade.

The ONPP has engaged in partnerships, which in turn led to DRI reports, publications, performance data, and outreach products. As for alternate structures, while options have been explored in the past, there was little support for other structural arrangements.

The findings in this section suggest that the Program was able to leverage resources through partnering, avoid duplication and consider alternatives to its current program structure. However, these are not sufficient to demonstrate that the program was cost-efficient in terms of inputs ($) to outcomes.

Appendix A

Office of Nutrition Policy and Promotion - Logic Model

Inputs	Funding Human Resources Facilities, Infrastructure	Acts, regulations, policies, priorities	Science and technology Research data
Activities	Conduct surveillance and monitoring, knowledge development and exchange	Develop, revise and contribute to policies, standards and strategies	Conduct dissemination and outreach
Outputs	Publications and reports, e.g. Summary of the Research on Dietary Reference Intakes (DRIs) Recommendation documents / position papers Conferences / symposia	Policies / Guidelines Standards Strategy documents Position papers Regulatory / policy recommendations	Information, tools and resources Dissemination, communication and marketing products Promotional / social marketing campaigns Meetings
Target Groups	Health portfolio, including HC regulators OGDs P/T Governments Researchers and academia NGOs / intermediaries International governments and organizations	Health portfolio, including HC regulators Other Government Departments (OGDs) Provincial/Territorial (P/T) governments Non-Governmental Organizations (NGOs) / intermediaries International governments and organizations	Health portfolio, including HC regulators Other Government Departments (OGDs) Provincial/Territorial (P/T) governments Non-Governmental Organizations (NGOs) / intermediaries International governments and organizations
Immediate Outcomes	Increased knowledge of information related to healthy eating	Improved policies, programs and initiatives to support healthy eating	Increased awareness and understanding of healthy eating among Canadians
Intermediate Outcomes	Increased evidence-based decision-making nationally and internationally Improved healthy eating of Canadians
Long-Term Outcomes	Reduced level of nutrition-related concerns, conditions and chronic diseases
Ultimate Outcome	Improved health and well-being of Canadians

Appendix B - References

BC Ministries of Health and Education. (2007). Guidelines for Food and Beverage Sales in BC Schools. ActNowBC. Retrieved April 27, 2010 from: http://www.bced.gov.bc.ca/health/guidelines_sales07.pdf

Belzile, K., & Logue, S. (2009). Web Enquiries Annual Statistical Report 2008-2009. ONPP. Health Canada.

Berry, T.R. et al. (2009). A mixed methods evaluation of televised health promotion advertisements targeted at older adults. Evaluation and Program Planning, 32, 278-288.

Beverly Webster & Associates. (2004). Office of Nutrition Policy and Promotion, Health Canada : Co-op Student Partnership Project Evaluation Report.

Bush, M.A.A, Martineau, C., Pronk J. A., Brulé, D. (2007). Eating Well with Canada's Food Guide: "A Tool for the Times". Canadian Journal of Dietetic Practice and Research, 68(2), 92-96.

Butter Issue Sheet. (n.d.). Unpublished internal document.

Buy Local Issue Sheet (n.d.). Unpublished internal document.

Conroy, S., et al. (2007). An Integrative Review of Canadian Childhood Obesity Prevention Programmes. Obesity Reviews, 8, 61-67.

Corporate Research Associates (CRA). (2007a). Focus group research with intermediaries who promote healthy eating among specific ethno-cultural communities. Health Canada, 1-39.

Dewolfe, J., & Millan, K. (2003). Dietary Intake of Older Adults in Kingston Area. Journal of Dietetic Practice & Research, 64, 16-24.

Downs, S.M., & Willows, N.D. (2008). Should Canadians Eat According to the Traditional Mediterranean Diet Pyramid or Canada's Food Guide? Applied Physiology, Nutrition & Metabolism, 33, 527-535.

Environics Research Group. (2007). Post-Campaign Healthy Eating / Physical Activity Evaluation. HC POR-06-110, 1-136.

Food Guide Advisory Committee & Interdepartmental Working Group. (2006). Draft Meeting Summary. Four Points Sheraton, Gatineau, Quebec.

Garriguet, D. (2004). Nutrition: Findings from the Canadian Community Health Survey. Overview of Canadians' Eating Habits. Statistics Canada. Retrieved July 14, 2010 from: http://dsp-psd.pwgsc.gc.ca/Collection/Statcan/82-620-M/82-620-MIE2006002.pdf

Garriguet, D. (2008). Obesity and the eating habits of the Aboriginal population. Health Reports, Vol. 19, No. 1. Statistics Canada. Retrieved April 26, 2010 from: http://www.statcan.gc.ca/pub/82-003-x/2008001/article/10487-eng.pdf

Government of Canada. (2010). Speech from the Throne. Retrieved April 7, 2010, http://www.speech.gc.ca/eng/media.asp?id=1388

Government of Canada (2007). Government Response to the Seventh Report of the Standing Committee on Health: Healthy Weights for Healthy Kids. Retrieved January 11, 2010 from: http://www2.parl.gc.ca/ HousePublications/Publication.aspx?DocId=3067091&Language=E&Mode=1&Parl=39&Ses=1

Government of Canada. (2004a). Speech from the Throne to Open the Third Session of the 37th Parliament of Canada. Retrieved April 7, 2010 from: http://www.pco-bcp.gc.ca/ index.asp?lang=eng&page=information&sub=publications&doc=sft-ddt/2004_1-eng.htm

Government of Canada. (2004b). Speech from the Throne to Open the First Session of the 38th Parliament of Canada. Retrieved April 7, 2010 from: http://www.pco-bcp.gc.ca/ index.asp?lang=eng&page=information&sub=publications&doc=sft-ddt/2004_2-eng.htm

Government of Canada. (2002). Speech from the Throne to Open the Second Session of the 37th Parliament of Canada. Retrieved January 11, 2011 from: http://www.pco-bcp.gc.ca/ index.asp?lang=eng&page=information&sub=publications&doc=sft-ddt/2002-eng.htm

Government of Canada. (2001). Speech from the Throne to Open the First Session of the 37th Parliament of Canada. Retrieved January 11, 2011 from: http://www.pco-bcp.gc.ca/i ndex.asp?lang=eng&page=information&sub=publications&doc=sft-ddt/2001-eng.htm

Hazel, J. (2007). Advertising Project Approval: Canada's Healthy Eating Campaign. Unpublished internal document.

Health Canada. (2000). A Strong Tradition of Collaboration: The Federal/Provincial/Territorial Group on Nutrition. Retrieved from: http://www.hc-sc.gc.ca/fn-an/alt_formats/hpfb-dgpsa/pdf/nutrition/fptgn-gfptn-eng.pdf

Health Canada. (2004a). Strategic Plan 2004-2007 for Health Canada's Health products and Food Branch.

Health Canada. (2004b). Rick Wilson Consulting Inc. Diabetes Evaluation.

Health Canada. (2004c). Review of Canada's Food Guide to Healthy Eating: Stakeholder Meeting: January 20, 2004, Report.

Health Canada. (2004d). ONPP Partnership Activities. http://www.hc-sc.gc.ca/fn-an/nutrition/part/index-eng.php

Health Canada. (2005a). About Health Canada: Office of Nutrition Policy and Promotion. Retrieved April 16, 2010 from: http://www.hc-sc.gc.ca/ahc-asc/branch-dirgen/hpfb-dgpsa/onpp-bppn/index-eng.php

Health Canada. (2005b). Revision of Canada's Food Guide to Healthy Eating: Communications Brainstorming Meeting.

Health Canada. (2005c). Health Products and Food Branch 2004-05 Performance Report Highlights.

Health Canada. (2006a). Briefing Note: Revisions to Canada's Food Guide to Healthy Eating. Unpublished internal document.

Health Canada. (2006b). Health Products and Food Branch 2007-08 Business Plan. Our key priorities and activities.

Health Canada. (2006c). Nutrition in the federal health portfolio: A review from the 1930s to today.

Health Canada. (2007a). History of the Food Guide. Retrieved April 30, 2010 from: http://www.hc-sc.gc.ca/fn-an/food-guide-aliment/context/hist-eng.php.

Health Canada. (2007b). Focus group research with intermediaries who promote healthy eating among specific ethno-cultural communities. Unpublished internal document.

Health Canada. (2007c). Eating Well with Canada's Food Guide: A Resource for Educators and Communicators. Retrieved January 5, 2011 from: http://www.hc-sc.gc.ca/fn-an/alt_formats/hpfb-dgpsa/pdf/pubs/res-educat-eng.pdf

Health Canada (2007d). Food and Nutrition Surveillance. (Website)

Health Canada. (2008a). Educators and Communicators. Retrieved April 13, 2010 from: http://www.hc-sc.gc.ca/fn-an/food-guide-aliment/educ-comm/index-eng.php.

Health Canada. (2008b). My Food Guide. Retrieved May 12, 2010 from: http://www.hc-sc.gc.ca/fn-an/food-guide-aliment/myguide-monguide/index-eng.php

Health Canada. (2008c). Translated Versions of the Guide. Retrieved April 13, 2010 from: http://www.hc-sc.gc.ca/fn-an/food-guide-aliment/order-commander/guide_trans-trad-eng.php.

Health Canada. (2011a). Office of Nutrition Policy and Promotion Evaluation: International benchmarking Report Review of Nutrition Programs/initiatives in Selected Countries (Canada, United States, Australia, United Kingdom and Japan).

Health Canada. (2011b). Food and Nutrition (October 25, 2011). Retrieved October 25, 2011 from: http://www.hc-sc.gc.ca/fn-an/index-eng.php

Health Canada. (2011c). About Health Canada (October 12, 2011). Retrieved October 25, 2011 from: http://www.hc-sc.gc.ca/ahc-asc/activit/about-apropos/index-eng.php

Health Canada. (2011d). Program Activity Architecture and the Performance Measurement Framework (effective April 1, 2011).

Health Canada. (June 2011). Provincial and Territorial Food Guidelines and Policy.

Health Canada. (2011). About Health Canada. Retrieved November 12, 2011 from: http://www.hc-sc.gc.ca/ahc-asc/activit/about-apropos/index-eng.php.

Joint Consortium for School Health (JCSH). (2008). Healthy Eating Assessment for Anyplace High School (November 7, 2008). Retrieved April 20, 2010 from: http://www.health.gov.on.ca/english/providers/program/pubhealth/oph_standards/ophs/progstds/workshops/
foundational_7a_shes_nutritional_feedback_report_manske_leatherdale.pdf

Jua Management Consulting Services. (2007). Measurement and Evaluation Framework Development: Summary of Interviews. Unpublished internal document.

Katamay, S.W., Esslinger, K.A., Vigneault, M., Johnston, J.L., Junkins, B.A., Robbins, L.G., Sirois, I.V., Jones-McLean, E.M., Kennedy, A.F., Bush, M.A.A., Brule, D., & Martineau, C. (2007). Eating Well with Canada's Food Guide. Nutrition Reviews, 65 (4), 155-166.

Kondro, W. (2006). Proposed Canada Food Guide called "obesogenic". CMAJ, 174 (5), 605-606.

Li, A., Dachner, N., Tarasuk, V. (2009). Food Intake Patterns of Homeless Youth in Toronto. Canadian Journal of Public Health, 100(1), 36-40.

Maillot, M., Vieux, F., Amiot, M.J. & Damon, N. (2010). Individual diet modeling translates nutrient recommendations into realistic and individual-specific food choices. The American Journal of Clinical Nutrition, 91, 421-30.

Martin, S. (2002). Surprise! Women eat healthier than men. CMAJ, 167, 913. Retrieved June 18, 2010 from: http://www.cmaj.ca/cgi/reprint/167/8/913-a.pdf.

Martineau, C. (2007). HPFB BEC-RM TOOLKIT: BEC-RM Item Summary. ONPP, October 1, 1-3.

Martineau, C., & Pronk, J. A. (2008). The importance of stakeholder input in the development of the Food Guide Package. Unpublished internal document.

Merchant, A.T., Vatanparast, H., Barlas, S., Dehghan, M,, Shah, SM., De Koning, L,, & Steck, SE. (2009). Carbohydrate Intake and Overweight and Obesity among Healthy Adults. Journal of the American Dietetic Association, 109, 1165-1172.

Mercury Advisory Issue Sheet. (n.d.). Unpublished internal document.

Merrifield, R. (2006). Standing Committee on Health. House of Commons, Canada.

Minister of Justice (2010a). Department of Health Act. Retrieved from: http://laws.justice.gc.ca/en/H-3.2/

Minister of Justice (2010b). Public Health Agency of Canada Act. Retrieved from: http://laws.justice.gc.ca/PDF/Statute/P/P-29.5.pdf

Ministry of Health Promotion. (2010). The Nutritious Food Basket Guidance Document: Draft. Retrieved April 20, 2010 from: http://www.health.gov.on.ca/english/providers/program/pubhealth/oph_standards/ ophs/progstds/pdfs/nutritious_food_basket_guidance_document.pdf

Moffat, T. (2008). Food Consumption Patterns in Elementary School Children. Canadian Journal of Dietetic Practice & Research, 69, 152-154.

Murphy, S.P. (2008). Using DRIs as the basis for dietary guidelines. Asia Pacific Journal of Clinical Nutrition, 17, 52-54.

Murphy, S.P., & Barr, S.I. (2007). Food Guides Reflect Similarities and Differences in Dietary Guidance in Three Countries (Japan, Canada, and the United States). Nutrition Reviews, 65(4), 141-148.

Nielsen. (2007). Canada Homescan PanelTrack Survey Results. October 2007, HC POR 104-07, 1-13.

Obadia, M., Rakhsani, N., Jeffery, A. S., Chahal, N., Sangha, G., McCrindel, B. W., O'Connor, D. L., Maccarone, T., & Hamilton, J. K. (2010). MEALTRAIN: What do inpatient hospitalized children choose to eat? The Journal of Pediatrics, 156, 685-686.

OECD. (2010). Obsity and the Economics of Prevention: Fit or Fat. ISBN: 978-92-64084865 (PDF).

ONPP. (n.d.). Office of Nutrition Policy and Promotion Strategic Framework 2007-2012.

ONPP (2003). ONPP Accomplishments 2002-2003. Unpublished internal document.

ONPP (2004). ONPP Accomplishments 2003-2004. Unpublished internal document.

ONPP (2005a). ONPP Accomplishments 2004-2005. Unpublished internal document.

ONPP (2005b, October) HPFB/TAS, Mid-Year Review (PowerPoint)

ONPP. (2006a). Early Warning Report. March 23, 1-2. Unpublished internal document.

ONPP. (2006b). Update: Addressing Challenges in the Revision of Canada's Food Guide. Presentation to HPFB BEC, May 18, 2006.

ONPP. (2006c). Office of Nutrition Policy and Promotion Performance Measurement Framework. Unpublished internal document.

ONPP. (2007a). ONPP Accomplishments 2006-2007. Unpublished internal document.

ONPP. (2007b). Revised Canada's Food Guide Dissemination Plan. Unpublished internal document.

ONPP. (2007c). Providing Leadership in Healthy Eating: A Turning Point.

ONPP (2008). ONPP Accomplishments 2007-08. Health Canada. Unpublished internal document.

ONPP. (2010). Proposed Nutrition Directorate, Health Canada: Preliminary Considerations. Unpublished internal document.

Phoenix Strategic Perspectives Inc. (2005). Input of Multicultural Intermediaries into the Revision of Canada's Food Guide to Healthy Eating (POR-05-31). Health Canada, 1-63.

Quadir, R., Akhtar-Danesh, N. (2010). Fruit and Vegetable Intake in Canadian Ethnic Populations. Canadian Journal of Dietetic Practices and Research, 71(1), 11-16.

Riediger, N.D., Shooshtari, S., & Moghadasian, M. H. (2007). The Influence of Sociodemographic Factors on Patterns of Fruit and Vegetable Consumption in Canadian Adolescents. Journal of the American Dietetic Association, 107, 1511-1518.

Service Canada. (2007). 1 800 O-Canada Service Food Guide Weekly Activity Report. Report for Week Ending November 30, 2007. Canada Enquiry Centre.

Shields, M. (2005). Overweight Canadian children and adolescents. Nutrition: Findings from the Canadian Community Health Survey, 1, 1-32. Statistics Canada. Retrieved May 5, 2010 from: http://www.statcan.gc.ca/pub/82-620-m/2005001/pdf/4193660-eng.pdf

Smitasiri, S., & Ricardo, U. (2007). Beyond recommendations: Implementing food-based dietary guidelines for healthier populations. Food and Nutrition Bulletin of the United Nations University, 28(1) S141-S153.

Statistics Canada, (2010), http://www.statcan.gc.ca/bsolc/olc-cel/olc-cel?catno=91-551-x&lang=eng

Stiefelmeyer, K., Martin, L & Klimes, M. (2008). Food Regulatory Systems: Canada's Performance in a Global Marketplace. George Morris Centre.

Storey, K. E., Forbes, L. E., Fraser, S. N., Spence, J. C., Plotnikoff, R. C., Raine, K. D., Hanning, R, M., & McCargar, L.J. (2009). Diet quality, nutrition and physical activity among adolescents: the Web-SPAN (Web-Survey of Physical Activity and Nutrition) project. Public Health Nutrition, 12, p. 2009-2017.

Strategic Counsel. (2008). Final Report: Final Testing of the 2007 Healthy Eating Ad Campaign (HC POR 07-50). March 2008, 1-71.

Tarasuk, V., Fitzpatrick, S., & Ward, H. (2010). Nutrition inequities in Canada. Applied Physiology and Nutrition, 35, 172-179.

Tasca, G.A., Illing, V., Balfour, L., Krysanski, V., Demidenko, N., Nowakowski, J., & Bissada, H. (2009). Psychometric properties of self-monitoring of eating disorder urges among treatment seeking women: Ecological momentary assessment using a daily diary method. Eating Behaviours,10, 59-61.

Tjepkema, M. (2005). Adult obesity in Canada: Measured height and weight. Nutrition: Findings from the Canadian Community Health Survey, 1, 1-32. Statistics Canada. Retrieved May 5, 2010 from: http://www.statcan.gc.ca/pub/82-620-m/2005001/pdf/4224906-eng.pdf

Trans Fat Issues Sheet. (n.d.) Unpublished internal document.

Tse, C., & Tarasuk, V. (2008). Nutritional assessment of charitable meal programmes service homeless people in Toronto. Public Health Nutrition, 11(2), 1296-1305.

Visioli, F. et al. (2007). Assessment of nutritional profiles: a novel system based on a comprehensive approach. British Journal of Nutrition, 98, 1101-1107.

Woodruff, S. J., & Manning, R. H. (2010). Development and implications of a revised Canadian Healthy Eating Index (HEIC-2009). Public Health Nutrition, 13(6), 820-825.