Evaluation of the Quit4Life Youth Cessation Demonstration Project

This content was archived on June 24, 2013.

July 2004

Help on accessing alternative formats, such as Portable Document Format (PDF), Microsoft Word and PowerPoint (PPT) files, can be obtained in the alternate format help section.

Table of Contents

• I Background
• II Evaluation Framework and Methods
  • Purpose
  • Evaluation Framework
  • Evaluation Questions
  • Method
    • Participatory approach
    • Data collection tools
    • Data collection and analysis
• III Overview of the Pilot Project
• IV Evaluation Findings
  • Was The Quit4Life Group Program Implemented As Anticipated?
    • Overall approach: group program or active community outreach program
    • Facilitators
    • Organization of the programs
    • Group coordination
    • Program materials
    • Program concept and flow
    • Program content
    • Program evaluation tasks
    • Contextual issues in program implementation
  • Did The Program Reach The Intended Target Group?
    • Demographic composition
    • Smoking Characteristics
    • Students' smoking environment
    • Conclusion: Was the intended target group reached?
  • How Satisfied Were Participants With The Program?
    • Organizational aspects of the program
    • Satisfaction with specific program aspects
    • What students remembered best about the Q4L program
    • Satisfaction with the program materials
    • Recommending Q4L to a friend
    • Conclusion: How Satisfied Were Participants With The Program?
  • How Effective Is Q4L In Promoting Youth Cessation?
    • Did youth who participated in Q4L cut down or quit?
    • Do youth who participate in Q4L increase their number of quit attempts?
    • Do youth who participate in Q4L increase their motivation to quit?
  • Suggestions: How Can The Program Be Improved?
    • Organizational aspects of the program
    • Facilitator role
    • Program concept
    • Program Content
    • Program materials

I. Background

Almost all persons who have ever smoked had their first cigarette sometime in their teens, and at least half of all ever-smokers have tried smoking by age 15. ¹ Smoking by youth in Canada is a critical health issue. In 2002, 22% of Canadian teens aged 15-19 were smokers, down from 28% in 1999. ² More over, adolescence presents a crucial window of opportunity to intervene with smoking cessation programs; 70% of current smokers aged 15-19 reported one or more attempts to quit in the previous 12 months. ³

Recognizing that it is critical to capitalize on adolescents' motivation to quit smoking while still in their teens, in 1993 Health Canada developed a youth cessation program in partnership with the Canadian Lung Association and Ciba-Geigy Ltd. Directed at 15-19 year olds, the Quit 4 Life/Vie 100 Fumer (Q4L) youth cessation program was developed as a minimal contact, self-help program for teens who smoke cigarettes on a daily basis.

During the late 1990's, Q4L was converted to website format. The program, containing the original information highlighted through the use of four youth's stories on their smoking behaviour and the challenges they faced in quitting smoking, was available exclusively online. Many youth simply stumbled across the website while Internet surfing and decided to follow the program through on their own. The popularity of the program and the emergence of new information about best practices on youth cessation from international researchers ⁴, including the effectiveness of cognitive behavioural approaches with adolescents, led Health Canada to update the Q4L program in 2002.

The updated program was re-designed by Dr. Paul McDonald of the University of Waterloo, and includes the following changes:

• Targeted towards a younger age group (13-18)
• Aimed at youth who smoke daily or occasionally, and those who want to reduce as well as quit
• Introduction of behavioural principles, including counter conditioning and stimulus control and creating environments supportive of quitting, in order to increase youth's chances of success through examining their smoking behaviour.
• Greater focus on building self-efficacy: "Care was also taken to create self-efficacy through encouraging users to "practice" quitting. ", to practice being a non-smoker and build their personal capacity to undertake a significant behavioural change.
• Greater focus on motivation: "Although there is, as of yet, insufficient evidence to make conclusive statements, approaches based on motivational interviewing seem to be particularly promising. Therefore, a considerable amount of material was included to build motivation based on the principles of Miller and Rollnick (1991)."
• Change in 'navigational' of program, so youth can either follow the whole program or 'pick and choose' those sections of the Q4L program/site in which they are most interested. Dr. McDonald again cites best practice information, "The review also suggests that intensive interventions are more likely to be effective. This is problematic because other research suggests that adolescents prefer to use simple, brief interventions. An attempt was made to balance these competing demands."
• Addition of a Facilitator's Guide, evaluation tools and a Guide to Evaluation to accompany the program when delivered in a group setting, based on best practice information suggesting that interventions that take place in school, in a voluntary but structured environment, facilitated by a teacher or other credible adult, may yield greater successful outcomes.
• The Quit 4 Life website remained available for use by youth who came across it, individually or as part of a group application.

In 2003/2004, Health Canada piloted and evaluated the revised Q4L group program in five sites across the country:

• Mission, British Columbia (Fraser House Society)
• Winnipeg, Manitoba ( Manitoba Lung Association)
• Garden Village , Ontario (Nipissing First Nations Health Centre)
• Fredericton, New Brunswick (Canadian Research Institute for Social Policy, UNB)
• New Glasgow, Nova Scotia (Pictou County Women's Centre)

Health Canada's objectives for the pilot were:

• To develop a procedure for overseeing evaluation of pilot sites
• To provide ongoing contacts and supports to pilot site coordinators regarding the evaluation component of the pilot project;
• To develop templates and other tools, in consultation with pilot site coordinators, that will enable and facilitate the collection of data by coordinators;
• To collect and analyse base-line, post-program data, as well as Facilitator and Peer Assistant feedback from the pilot sites;
• To produce a final evaluation report on the pilot sites detailing methodology, process, outcomes, impacts and providing recommendations for adapting the program based on pilot site findings.
• To develop evaluation tools to be used by Health Canada and pilot sites at 6 and 12 month intervals post-program delivery.
• To develop an evaluation tool to be used on the Quit 4 Life website to gauge the program's effectiveness as a self-help program.

This report describes the evaluation process; provides an overview of program implementation and participants; summarizes short-term outcomes; and presents recommendations for improving Q4L materials and process.

---

¹ Canadian Tobacco Use Monitoring Survey. Annual Results 2001, Health Canada, (2002).

² Canadian Tobacco Use Monitoring Survey. Youth Smoking in Canada Fact Sheet, Health Canada, (2001.)

³ Canadian Tobacco Use Monitoring Survey. Youth Smoking in Canada Fact Sheet, Health Canada, (2001).

⁴ Sussman. Steve. (2000). Effects of Sixty Six Adolescent Tobacco Use Cessation Trials and Seventeen Prospective Studies of Self-Initiated Quitting, Canadian Tobacco Control Research Initiative.

II. Evaluation Framework and Methods

1. Purpose

Gentium Consulting undertook the evaluation of the Q4L program. The purpose of the evaluation was:

• To assess the process for implementation of the recently-revised Q4L group program. By piloting the Q4L program in five different sites, with very diverse characteristics, process issues that might affect consistent implementation were identified. The process evaluation addressed issues such as: How diverse were the program implementation strategies across the sites? What kind of support was needed across different sites? How was the program changed to reflect different circumstances? What were the responses to program structure, delivery, materials? What unanticipated contextual factors arose? Please note that this type of evaluation strategy is not expected to generate statistically generalizable results; rather, its intent is to provide insights into the unfolding of the program, and is essentially a descriptive study documenting how the program actually operates, to ensure that what is eventually evaluated was actually implemented as intended.
• To test evaluation mechanisms and data collection tools across different sites. Piloting tools across sites and languages will contribute to revisions in evaluation mechanisms and tools, prior to full roll-out of the program nation-wide.
• To provide immediate (sort-term) outcome data for each of the pilot sites. Given that the Q4L program aims to make specific, measurable changes in the behaviour of participants, an evaluation of outcomes (effect) should be possible in future. Please note that a full outcomes-based evaluation can only be carried out once a program is implemented with a relatively high degree of consistency. At this pilot stage, a simple pre- and post-design, using a theoretical control group, provides immediate feedback to pilot sites. The use of a theoretical control group was also consistent with the ethical question arising from randomizing potentially beneficial treatment, i.e., that it may not be ethical to randomly offer the Q4L program to only a subset of those youth who would like to participate in smoking cessation.

2. Evaluation Framework

The evaluation of the Q4L Pilot Program is primarily formative, focusing on program design and implementation across different sites. An evaluation of program effects can not be undertaken until the program is fully and consistently implemented.

Since the group program was being delivered for the first time, detailed program logic models were prepared for each of the sites, based on review of proposals and additional documentation, to understand and document program variations from the outset. Site co-ordinators and facilitators periodically provided further input to ensure that components, activities, target groups, and expected outcomes continued to reflect each site's program as it evolved. At project end, data from co-ordinators, facilitators, and participants, again revealed differences and similarities in implementation at each site and over time within each site. Revised program logic models were prepared (see Appendix C). As part of the process evaluation, particular attention was also paid to assessing program structure, content and materials (Facilitator's Guide, Participant's Booklet, Evaluation Guide), which were developed for this Pilot.

The Pilot provided the opportunity to test a model and tools in anticipation of a fuller evaluation of the effect of a national program in future.

The pilot sites did not propose designs that permitted random assignment of young smokers to a 'treatment (Q4L intervention group)' or 'no treatment' group- both for practical and for ethical reasons. A quasi-experimental design, in which a comparison group would be selected to match the treatment group, was considered impractical at the level of the co-ordinators delivering the program, given the time constraints and the varying level of school/site involvement.

An alternative design would have been to randomize at the level of schools, rather than individual youth participants. Due to contextual and socio-demographic factors along which individual schools may differ, as well as the voluntary nature of participation in the Q4L program, this design would also be difficult to implement at the pilot stage.

The outcome evaluation design used for testing at this pilot stage was a combination of a simple one-group, pre-program/post-program comparison (a reflexive comparison group), with the addition of a 'theoretical comparison group', in which it is assumed that, in the absence of the Q4L program, youth would stop smoking and try to quit at the level that has been shown to occur in other research (specifically, the Canadian Tobacco Use Monitoring Survey).

In terms of outcomes, key evaluation question would be: How effective is Q4L in promoting youth cessation? Specifically, do youth who participate in Q4L:

• cut down or quit to a greater extent than similar youth who do not participate in Q4L ?
• increase their number of quit attempts?
• increase their motivation to quit ?
• increase skills and knowledge about smoking cessation?

3. Evaluation Questions

In summary: The Quit4Life Pilot evaluation sought to answer the following five questions:

• Was the Quit4Life group program implemented as anticipated?
• Did the program reach the intended target group?
• How satisfied were participants with the program?
• How effective is Q4L in promoting youth cessation?
• How can the program be improved?

(Please refer to Appendix A for a more complete description of the evaluation design)

4. Methods

4.1 Participatory approach

A participatory process included the Health Canada representative, Pilot Site Co-ordinators, and youth program participants. It proved to be a successful way to obtain fullest possible feedback on program implementation, to receive youths' perspectives on the program, and to generate ideas and suggestions. Involving the site co-ordinators in evaluation planning also contributed to obtaining comprehensive and accurate evaluation data in a timely manner. Their participation was essential in the development of a valid and clear logic model, the first step in any evaluation.

The pilot site co-ordinators were encouraged to buy into the evaluation process; they in return received support for implementing data collection procedures to achieve high return rates. This participatory process also laid the ground work for follow-up evaluations at 6 and 12 months at each pilot site, if Health Canada chooses to do an outcome evaluation in future.

The participatory process also facilitated periodic review of the workplan, and permitted adjustment of activities (within predetermined budgetary constraints) to respond to unanticipated events and emerging issues. For example, the extreme weather conditions in New Brunswick led to unprecedented closure of many schools for as long as two weeks; this affected the timing of program delivery, and required adjustment of data collection strategies and dates.

In summary, the following process was followed for contacts and communication with the pilot site co-ordinators:

• Four teleconferences were held involving pilot site coordinators, the Departmental Representative, and the evaluation team (September, October, December, and March).
• A day-long meeting among all pilot site co-ordinators in January provided a forum for each co-ordinator to present an up-date on the program, and exchange suggestions and ideas. The evaluation team shared their understanding of the program assumptions and logic, reviewed data collection, presented sample results from one site, and facilitated input into the design of participant focus group questions and formats.
• Each pilot site co-ordinator was interviewed several times throughout the project: in relation to the development and revision of site-specific logic models; to confirm recruitment and other program concerns; and to discuss program implementation, in sites where co-ordinators also facilitated activities.
• Through ongoing telephone and e-mail communication the evaluation team provided support to the implementation of the evaluation component of the project: e.g., addressed concerns about administering surveys, answered questions regarding use of various tools, responded to site-specific concerns and issues, obtained collaboration for focus groups and facilitator contacts, and kept the co-ordinators informed and engaged in the evaluation process as much as possible.
• The evaluation team and the Departmental Representative ensured that drafts of all materials of relevance to co-ordinators (e.g., surveys for six-month outcome data; project logic models; evaluation framework; draft findings for each site) were circulated to all co-ordinators for their review. The co-ordinators responded with very valuable and thoughtful suggestions and input, which helped to fine-tune both the overall evaluation strategy and particular tools.

4.2 Data collection tools

For implementation questions, primary data was collected through

• facilitator and peer-facilitator surveys
• guided observation (facilitator logs)
• interviews
• focus group(s)
• teleconferences
• meetings
• analysis of documents and reports prepared by each site

For outcome questions,

• primary data was collected through pre- and post- participant surveys;
• secondary data was analyzed to create theoretical control groups for each site, as per the attached table.

4.2.1 Tool development

Four surveys previously developed to evaluate the Q4L pilots were used for data collection during the pilots (see, Evaluation Guide for Facilitators, Quit 4 Life, Health Canada ). Additional tools were developed to collect six and twelve month post-program evaluation data.

Pre- and post- program surveys measured the number of program participants who have

• Quit
• Cut down
• Increased quit attempts
• Increased their motivation to quit
• Built skills and knowledge about smoking.

These steps were followed to develop survey tools:

• Review efficacy of pilot tools in collecting appropriate and useful data, identifying items and sections that worked best, and those that need revision
• Review recent literature for up-dates on questionnaire items related to maintenance of quit status, youth, and other tobacco control literature
• Develop adapted tools, keeping a set of 'core' items from piloted tools
• Circulate to HC representative and site co-ordinators; elicit feedback from testers re: ease of use, clarity
• Test selected tools (e.g., six-month follow-up) with youth at one site, and revise accordingly

Other tools were developed to guide telephone conferences and interviews; to assist site co-ordinators with data management; and to obtain process and implementation data from facilitators and youth participants in interviews and focus groups. The following process was generally used for these tools: drafts developed based on experience of evaluators with qualitative tools, keeping in mind findings from recent literature; review and revision by the HC representative; distribution, review and piloting by site co-ordinators; revisions after first use if required.

The following is a list of the data collection tools used in this evaluation. (See Appendix B for tools not currently included in the Evaluation Guide for Facilitators)

• Participant surveys (pre- and post-program)
• Facilitator survey
• Peer Assistant survey
• Facilitator interview schedule
• Facilitator logs
• Focus group interview guides for youth participants

4.3 Data collection and analysis

Ten facilitator interviews were taped, transcribed, printed, coded, analyzed for content, themes, examples, and extreme cases, and summarized. Eighteen facilitator logs were analyzed for content, themes, and examples, and summarized. Forty-four facilitator surveys were returned.

Four youth participant focus groups were taped, transcribed, coded, analyzed for content, themes, examples, and extreme cases, and summarized. 328 of participant surveys were received (291 pre-surveys; 257 post-surveys). For 222 student participants, pre and post surveys were matched.

Open-ended answers from surveys were entered as text; coded; analyzed for content and themes; and summarized.

Technical reports on qualitative data were reviewed by the Health Canada representative, whose input was incorporated into the analysis.

All qualitative data summaries were further searched and compared, and reviewed in light of project-end narrative reports from each pilot site, seeking multiple perspectives, confirmations, discrepancies in interpretations, and specific recommendations and suggestions to improve the program.

Numerical survey data were coded and entered into a number of SPSS data bases. Summary frequency tables and descriptive statistics (including cross-tabulations) were created. Note: the respondents to the survey do not comprise a randomly selected sample. Tests of statistical significance have not been performed consistently. Results are presented in detailed tables (Appendices) and in summary figures (in this report).

III. Overview of the Pilot Project

Table 1 below summarizes the overall logic for Health Canada's pilot project.

Table 1 Health Canada Pilot Project Logic Model

Components	Implementation of Q4L pilot program	Testing Q4L program materials
Activities	HC recruits 6 sites across Canada Site coordinators identify facilitators and peer assistants recruit participants (youth) implement program 1-2 times Collect evaluation data, including written baseline and post-program surveys	Provide support to co-ordinators in all 6 sites Facilitate youth and facilitator input into program implementation and evaluation Evaluate program implementation and effectiveness: Develop web survey Summarize and analyze information Make recommendations for program adjustment
Target groups	Organizations (partners, decision-makers) working with youth Site co-ordinators Facilitators & peer assistants Youth who smoke	Health Canada Facilitators/site co-ordinators Participating youth
Short-term outcomes	1. Six sites implement pilot program facilitators identified programs advertised youth recruited youth retained 9 sessions minimum provided Facilitators Guide used 2. Evaluation data collected   3. Facilitators motivated to continue facilitating program 4. Youth participants: Have cut down or quit Increased # quit attempts Increased motivation to quit Increased skills & knowledge want to tell others about program are satisfied with program 5. Partner organizations : know about program promoted program supported implementation program	1. Evaluation report provides HC with increased knowledge about: program effectiveness and program satisfaction implementation of Q4L as group format suggested improvements 2. Facilitators provide input into program implementation and evaluation 3. Youth provide input into program implementation and evaluation 4. HC collects continuous data from web survey

(Note: items in italics in above figure not implemented as originally planned/ or no data available)

Given the considerable variations across the pilot sites, individual logic models for each site were also developed, using the following steps:

• Review of Health Canada documents on Q4L program history
• Review of current Q4L program components
• Review of recent literature on youth cessation programs
• Review of original pilot site funding proposals and related documents
• Development of overall logic model for the pilot (draft format)
• Revision: Creation of individual logic models for each site, to reflect the considerable variations evident already at conceptualization of pilot
• Circulation of draft logic models to all pilot site co-ordinators and HC representative
• Individual telephone interviews with each site co-ordinator (in some sites, including facilitators) to review draft logic model
• Revisions to logic models reflecting comments from interviews and written feedback from pilot site co-ordinators
• Circulation of logic models to all co-ordinators and HC representative
• Review of logic models post-implementation in light of survey findings and final site reports from co-ordinators.

This final step permitted documentation of variations between the anticipated and implemented program components, activities, and target groups at each site, as reported by pilot site co-ordinators at project end. Co-ordinators reviewed earlier versions of the logic models for each of their sites, and were prompted through telephone interviews to reflect on questions such as:

• Are links between causes and effects plausible?
• Are the goals and effects at clear, specific?
• Are program activities related to anticipated outcomes at each site?

At project end, data from facilitators and participants, as well as the final site reports prepared by each co-ordinator, led to the revision of models for each site. Challenges and achievements, expected and unexpected contextual factors, and overall impressions about program success were also collected, and are included in narrative reports.

Pre-implementation and mid-point logic models, based on each site's proposals and interviews with co-ordinators, are included in Appendix C. Short-term outcomes are summarized in section IV.3 of this report, with full tables included in Appendix E. Section IV.1 provides a narrative description of variations in program implementation.

IV. Evaluation Findings

1. Was The Quit4Life Group Program Implemented As Anticipated?

Five Pilot Sites were selected by Health Canada from sites that expressed an interest in participating in the project.

The Pilot Sites were non-government organizations/agencies with an interest and/or a responsibility for youth smoking cessation. Each Pilot Site was co-ordinated by a designated individual, generally a person with previous experience in cessation and/or addictions and/or work with youth. The organizations ranged considerably in size, mandate, and structure, which permitted the program to be piloted in very diverse implementation environments. For instance, in New Brunswick the University of New Brunswick's Centre for Policy Research was the pilot site, with considerable experience in educational research and concurrent parallel research projects which led to attempts at implementation in a large number of schools throughout the province, relying on a network of teachers and public health nurses to volunteer to facilitate the program in their schools. In contrast, the Nipissing First Nation is a small reserve in Northern Ontario, in which the Addictions Counsellor co-ordinated a small program delivered by a contracted facilitator, which focusing on two specific on-reserve schools whose population of adolescent smokers included many previously known to the co-ordinator and facilitator from other activities.

Q4L Pilot Site Co-ordinators had responsibility for all aspects of program implementation, including selecting the partners that would be involved in recruitment and delivery. They were guided by the Q4L program materials and were to maintain the fundamental structure and plan of the Q4L program: a step-by-step group cessation program, with some web access/backup, delivered by an adult facilitator with optional peer assistant support, directed at adolescent smokers.

The actual timing, length, and location of the program; the number of times the program was offered; the recruitment and selection of facilitators; whether facilitators were paid or volunteers; the recruitment and selection of participants; and the determination of any specific targets within the adolescent smoking population (e.g., gender), were left up to each program site.

All program materials (posters, facilitator guide, participant booklets) were available in English and in French. They were provided to the pilot sites at no cost by Health Canada.

It is important to note that during the pilot stage, a number of supports and additional elements were available which might not continue once the program itself is fully rolled out nationally. These included:

• frequent individual contact with the HC representative, including visits from the HC representative to assist with program recruitment and information
• teleconference calls with a group of other site co-ordinators throughout the implementation
• consultation and support from evaluation team for data collection
• feedback from evaluation team regarding evolving project design and draft results during implementation
• resources to pay facilitators and/or for incentives to promote youth participation.

As well, the pilot stage made additional demands on co-ordinators (communication with HC and evolution team; input into tool development; recruitment of participants for interviews and focus groups; data collection; frequent reporting) which would not be required as part of the regular implementation of the program once it is fully rolled out.

1.1 Overall approach: group program or active community outreach program

As anticipated, there are similarities and differences across and within pilot sites in how the Q4L program was implemented.

While the majority of facilitators attempted to provide a relatively formal and structured group cessation program, one pilot site also used the Q4L materials for active community outreach with no expectation of regular attendance or formal registration. At that site, two facilitators provided a range of individual cessation support activities outside of the high school building and at times quite spontaneously (i.e., at skateboard parks), to encourage young people to cut down and/or quit. The community outreach activities led to recruitment of a few students into a more formal group program, while continuing with informal, flexible, off-site contacts with the rest. It is important to note that surveys were only distributed to youth participating in group sessions; consequently, facilitator reports (interviews) and the co-ordinator's report provide the only data about the implementation and success of the 'community outreach' approach to delivering Q4L.

1.2 Facilitators

Although the site co-ordinators had overall responsibility for implementing the program, evaluation results suggest that individual facilitators had considerable leeway in the way that they presented the material. This is especially evident from reports of those sites, which held multiple groups at several locations during the pilot period. The program is designed as a 'stand alone' program without requiring that facilitators receive specific training to implement sessions.

In most cases, the Q4L groups were delivered by a single adult facilitator, with no additional peer facilitator involved. Co-facilitation and rotation of several facilitators was not uncommon in some sites. Two of the pilot site co-ordinators also facilitated at least some of the group sessions.

The professional background and experiences of facilitators were diverse, and included addiction counsellors, public health nurses, executive director of a non-government organization, program coordinators, and teachers. Over half (57%) of the facilitators had previous experience in a counselling role with youth (e.g., social workers, addiction counsellors, family support workers). One quarter were involved in recreation and sports with youth (coach, recreational worker); one fifth (21%) were teachers; and 14% were public health nurses.

All had frequent contacts with youth, most on a daily basis. Just over one third (37%) had little previous experience with youth cessation programs, while 41% had a lot. At the time the pilot data were being collected, a few facilitators had already delivered the Q4L program more than three times, whereas for others, this was the first experience.

Just over half (61%) felt they had a lot of knowledge about tobacco issues; overall, they tended to rate their level of knowledge bout tobacco much higher than their experience with cessation programs. Two thirds of facilitators were ex-smokers themselves; one third had never smoked.

The majority of facilitators were female (76%) and implemented the program in English (80%).

1.3. Organization of the programs

1.3.1 Location of group programs

Most of the group programs were offered at high schools, to students currently enrolled in those schools, during the regular school year, and during the regular school day. Students interviewed in focus groups often talked about going to 'classes' for smoking cessation.

1.3.2 Program supports

As well as the support specific to this Pilot (described above), facilitators in most sites received considerable support from their organizations to implement the program, in terms of their own time (46%); someone else's time (43%); other resources (e.g., technical support, internet access, program incentives) (43%); and help with recruitment (39%).

1.3.3 Recruitment

Most facilitators used standard recruitment strategies, such as announcements, posters, and one-on-one outreach. Many of the facilitators already had contacts with the students prior to running the Q4L program, because of their role as teachers, school health nurses or addiction counsellors. Many reported assistance from the school, from a principal or guidance counsellor, to identify youth for the group program. Some used other channels (e.g., contacted cafeteria staff or youth workers to inform them about the up-coming program, and hoped that they would pass on the information to students. Others used incentives (gift certificates) to attract students. Many facilitators attempted to assess the whole smoking culture and environment of each school to know to whom it would be best to talk, and have a better idea of the current issues that were affecting the school in relation to tobacco use.

Most students heard about the Q4L program through school announcements, and also through 'word-of-mouth', from friends and other students. Focus group participants felt that these recruitment strategies worked for different reasons. Those who had been successfully recruited through an individual, one-on-one contact, emphasized the importance of having someone who already knew them and whom they could trust reach out to them. Those who had heard about the program through a more public, general recruitment strategy (PA, posters, pamphlets) liked the fact that they could go to 'check it out', and then decide whether they wanted to stay or not.

1.3.4 Program length

The number of sessions for each group varied considerably, ranging between three and 26, with an average of 10.5 sessions per group. Overall, the tendency was to conduct more, rather than fewer, sessions.

1.3.5 Program timing

Half (50%) of the Q4L programs were implemented during the Fall of 2003, a third (36%) during the winter month of 2004, and a smaller number (14%) during the Spring of 2003.

1.3.6 Scheduling of group sessions

Facilitators varied the scheduling of group sessions during the day and throughout the school week. Some groups were offered only at lunch time; others, only during class time; some facilitators offered a combination of both. Most groups met for one hour per week; other facilitators provided two half-hour sessions each week; a few met every two weeks.

1.4 Group coordination

1.4.1 Participant selection/registration into the group

In some locations, students had to express their interest in attending to someone other than the facilitator (guidance counsellor, for instance); fill out the baseline survey as a way to establish commitment; review readiness to quit with the facilitator; and/or sign up for the Q4L group ahead of time; others were able to enter the program simply by attending the first session. A few facilitators ran the program in a very open way, with no expectation of regular attendance at the group and ongoing or 'drop-in' registration.

1.4.2 Attendance/participation

Facilitators reported different ways of responding to inconsistent attendance. To ensure program continuity, some allowed students to miss not more than three classes; if they did, they were asked to leave the program. For some facilitators, inconsistent attendance indicated a lack of motivation to be ready to quit smoking; others understood it that some of the 'target group' might also be facing other issues, such as high absenteeism.

1.4.3 Group size

The number of participants in each group session varied between 2 and 16 with an average group size of 9 participants. On average, seven participants completed each program. Facilitators using the 'community outreach' approach report contact with up to 60 individual youth.

1.4.4 Age, language and gender composition of groups

Almost all the group participants were between 12 and 19 year olds. More than half of the students were between 16 and 17. One site offered the Q4L program to young adult students (aged 17 to 36).

Three-quarters of the groups were mixed; 21% were only for girls, and 6% only had boys. Thirty groups (75%) were held in English, and ten (25%) in French.

1.5 Program materials

Each facilitator received a Facilitator's Guide and an Evaluation Guide, and a number of participant booklets, directly from Health Canada or through the Pilot Site Co-ordinator. Additional information was available through the Quit4Life website.

1.5.1 Facilitator guide

The 29-page ring-bound Facilitator's Guide contains an introduction of the rationale, program framework and session plan for the Quit4Life program; some ides for recruitment of participants; a description of activities for nine sessions, and eight "Black Line Masters" to photocopy for each participant, with information, forms, and activity materials.

1.5.2 Evaluation guide

The 37-page ring-bound Facilitator's Guide to Evaluation contains an introduction, an overview of program evaluation methods, instructions on data collection, explanations for the different surveys being collected, and masters four surveys for the facilitator to copy and distribute to Q4L Pilot participants.

1.5.3 Participant's booklet

The Participant's Booklet is a 37-page booklet, small enough to fit easily into a bag (9 " by 5"). Entitled "Learn Some New Death-Defying Skills: Quit4Life. The proven four-step guide to quitting for smokers under 19 years old", the booklet describes four different steps to quitting: Get Psyched, Get Smart, Get Support, Get On With It. It contains a tear- out sheet to track smoking behaviour, a tear-out support pledge card, and a worksheet in which users can write down their own goals and values. The booklet is illustrated with full-page and half-page full-colour photographs of young people, and includes the web-site address.

1.5.4 Web-site

The Quit4Life web-site is available at www.Quit4Life.ca. At the time of this pilot, the site focused on accessibly communicating the four steps to quitting that are in the Participant's Booklet. The site contained some of the same activities that are found in the Participant's Booklet (e.g., values listing), but did not echo the nine-session plan of the Facilitator's Guide. The Q4L site included stories from four fictional characters representing youth who are quitting smoking. The site was designed to encourage users to move between different sections according to their own interests and needs, rather than in a structured program format. It displayed some interactive features (e.g., a cost calculator), and was very well-linked to other HC sites and information, (e.g., on second-hand smoke; on using nicotine replacement). [Please note: an on-line survey to assess use of, and satisfaction with, the Q4L web-site was created for this evaluation. Since the site itself is currently being re-developed, data from that survey is not available for this report].

1.6 Program concept and flow

All facilitators used the Facilitator's Guide and the Participant's Booklet to plan and implement the Q4L program. Most facilitators tried to keep certain elements common to each session, without becoming repetitive: for example, many started with a 10 to 15 minute go-around each time the group met; or had a routine 'farewell' activity, as suggested in the Guide.

Use of the Participant's Booklet varied. Some facilitators provided the booklet to potential participants at a recruitment or information session. Some of these potential participants kept the booklets but did not register in or attend the group session. Some facilitators handed out copies of the booklet to all participants, who might or might not bring them in to the group session. A few facilitators stored all participants' booklets between sessions themselves, to ensure that they would be available during the group session.

1.6.1 Homework assignments

"Homework" (tasks to be done by participants between group meetings) was assigned by the majority of facilitators, especially the tracking form task. In practice, participants' compliance with homework was quite uneven. Facilitators used completed 'homework' assignments from students to contribute to the discussion during the sessions; others asked students to do the 'homework' tasks during the group meeting time.

1.6.2 Program style

Few facilitators followed the Facilitator's Guide suggestion for smaller group work during the sessions: the majority kept participants together as a 'larger' group throughout the sessions, or had students complete tasks individually, while sharing results in the larger group.

1.6.3 Peer Assistant

Only five facilitators report having involved a Peer Assistant to help implement the group program, as suggested in the Facilitator's Guide.

1.6.4 Computer access

The Facilitator's Guide suggests networking onto the Q4L web site during the group sessions. Few groups had computer access during the group meeting time, and/or at the pilot sites, so this element was not generally implemented during the pilot sessions. Most facilitators, however, did access the Q4L web-site to download additional activities and information for later use with the group.

1.6.5 Videos

To vary the teaching medium, almost all facilitators introduced videos or a resource person. Two thirds of the facilitators used a video or film in the group program. More than one third invited resource persons with particular expertise to add to the program.

1.7 Program content

The core program elements (9 session components) were delivered consistently at all sites to group participants.

Two thirds of facilitators delivered the program content in the same order as suggested in the Facilitator Guide. Most used the content and handouts that were provided.

Most facilitators expanded the 'base' Q4L program beyond the originally conceived 9 sessions, incorporating materials and resources from other health and smoking cessation programs, and adding components and activities to enrich the program or respond to student needs. Additional topics were introduced most often because students asked for more information. Very few facilitators deleted any of the sessions or content provided in the Q4L program materials.

1.7.1 Q4L website use

About half of the students (52%) were aware that there was a Q4L website, but very few (19%) had visited it. Slightly more male students knew about the existence of the website, than female students. Both had visited the website to the same extent.

1.7.2 Post-program support

A few sites provided the opportunity for participants to continue to meet in formal or informal 'follow-up' sessions; others encouraged participants who had not yet quit to re-register for a subsequent group session.

1.8 Program evaluation tasks

The task of distributing and collecting evaluation surveys from facilitators was undertaken by site Co-ordinators, and was integrated into program activities by many of the facilitators. Facilitators were responsible for distributing and collecting both pre-program (baseline) surveys to all participants as they entered the group program, and post-program surveys to all participants as they left the group program (including tracking drop-outs). Both facilitators and co-ordinators were involved in recruiting participants for focus group interviews; and co-ordinators identified facilitators to participate in facilitator interviews.

1.9 Contextual issues in program implementation

In their final Pilot Site reports, Co-ordinators identified a number of issues that impacted on program implementation. Salient among these were delays in program start-up or ability to deliver the program at the originally scheduled time. For instance, some sites had expected to start up in January 03, but didn't actually begin until September '03 or even January '04. The majority of delays were due to administrative delays that may be unique to the Pilot Project, such as delays in receiving funding and difficulty in receiving sufficient amounts of printed Q4L materials from Health Canada at the anticipated times. Other issues that impacted on implementation, such as concurrent recruitment for other youth cessation programs; unexpected closures of schools due to extreme weather conditions; school re-organization; by-law changes; evolving provincial-level youth tobacco policies; and traumatic events (student deaths), are likely to occur again, and should be taken into account in any future evaluation of the program.

IV. Evaluation Findings

2. Did The Program Reach The Intended Target Group?

The intended target population for the group program were young people who use tobacco; specifically, male and female youth aged 13-19 (or in grades 9-12) who smoke and want to quit smoking. The first iterations of the pilot site models included young people who are out of school, as well as those enrolled in school; some sites also had very broad targets (e.g., all the youth who smoke in the province). As the project evolved, the target groups became more focused, with the Pilot Sites trying to reach students who smoked and were attending particular high schools, rather than the broader 'youth smoker' population. The target age group also shifted slightly in response to actual interest in participating in the program, to include both older and younger students at several sites. Some sites originally planned to target non-smoking students who wanted to help a friend quit; the revised logic models don't focus on this population (see Appendix C).

Data are available from 328 participants from 40 programs in the five Q4L Pilot Sites across Canada.

2.1 Demographic composition

2.1.1 Age and gender of participants

The majority of participants (95.6%) were in the intended target age group of 12 to 19 year olds. More than half were between 16 and 17 (55.9%) and 16.2% were 18 to 19 years old. The age group distribution reflects the age of smoking up-take in the Canadian teenage population. Although slightly more female (55.6%) than male (44.4%) students participated in the program, they were distributed equally across age groups.

Figure 1 Age

2.1.2 Mother tongue and place of birth of participants

The majority of participating students (82.7%) spoke English as their mother tongue, 11.1% (32) spoke French and 4.5% had both mother tongues. Only five students (1.7%) had a non-official mother tongue. This distribution differs considerably from the Canadian population (Canadian Census 2001 figures, the Anglophones 59%, Francophones 23%, and Allophones 18%). Almost all (97%) student participants were born in Canada. This also differs considerably from the Canadian population.

Figure 2 Mother tongue

The Francophone participants were reported from only two of the Pilot sites, and differ somewhat in terms of gender and age distribution (see Figure 3 below as an example). These differences as well are likely to account for some of the apparent differences between 'linguistic' groups of participants.

Figure 3 Mother tongue and gender

Taking into consideration the very small size of the sub-sample, mother tongue differences will not be reported here. For a fuller discussion, please see Appendix D - Technical Report.

Figure 4 Mother tongue and age

2.1.3 Grade level of participants

Almost two thirds of the students (62.9%) attended grades 11 or 12, one third were in grades 9 or 10, and a few (4.2%) in Grade 8 or less.

Figure 5 Grade attended

2.2 Smoking Characteristics

At baseline (when first entering the Q4L program) students were asked about their smoking habits and smoking environment.

2.2.1 Smoking habit

The majority (93%) were daily smokers.

Figure 6 Smoking habit

Older (20>) and younger (14<) students were less likely to be daily smokers, than 15-19 year olds.

Figure 7 Age and smoking habit

2.2.2 Number of cigarettes smoked

About half of the students smoked 10 cigarettes or more every day: Over one quarter smoked between 11 and 15 cigarettes daily. 12% were smoking between 16 and 20 cigarettes, 5% between 21 and 25, and 3.3% more than 26 cigarettes per day

One third of student participants smoked between 6 and 10 cigarettes daily, and one in five smoked five or fewer per day.

Figure 8 Cigarettes smoked daily

Consistent with CTUMS figures, male students tended to be heavier smokers (20 cigarettes and more) and females were more frequently in the light to moderate smoking categories (6 to 15 cigarettes).

Figure 9 Gender and cigarettes smoked daily

Younger smokers (14 and younger) smoked fewer cigarettes than older smokers. Those participants who were in the 20+ age group showed a different pattern of daily cigarette consumption: one third smoked only 2-5 daily at baseline.

Figure 10 Age and number of cigarettes smoked daily

2.2.3 Smoking history: Age of first cigarette

Almost half of the student participants (47%) had smoked their first cigarette before the age of 11, and 82% had done so by the age of 13. Only 2% smoked their first cigarette at the age of 16 and older.

Figure 11 Age of first cigarette

Questions to determine how long students had been occasional versus daily smokers were answered inconsistently, suggesting that the questions might be confusing and should be re-worded. Data is therefore not reported here. (Section IV 5 improvements to the evaluation surveys).

2.2.4 Quit attempts

A large majority of the participating students (86.4%) had previously tried to quit smoking. Females were slightly more likely to have made a quit attempt than male participants. The ratio of students who had tried to quit before to those who hadn't was about the same in all age groups (9:1), with the exception of the oldest group: All students who were 20 and older had tried to quit before.

Figure 12 Tried to quit in the last 12 months

Male and female students had tried to quit in the past 12 months to the same extent.

Figure 13 Gender and tried to quit in the past 12 months

A survey item asked about the number of successful quit attempts lasting at least 24 hours. Replies suggest that the question was not well understood by respondents, and caution should be exercised interpreting the tables below. Recommendations for rewording this item are included in the Appendix F.

Figure 14 Quit attempts (24 hours)

Students in all age groups had tried to quit 2 to 3 times in the past year for at least 24 hours. More than a third of the male students (41%), and one quarter (26%) of female students, could not remember how often they had gone without smoking for 24 hours. Among those who could remember, male students had made more quit attempts than female students.

Figure 15 Gender and quit attempts (24 hours)

Younger students had made the fewest 24-hour quit attempts.

Figure 16 Age and remembered quit attempts (24 hours)

Student participants were also asked how long they had been able to go without smoking in the past year. The majority of both male and female students had been able to stay quit for between 2 days and one week. Less than one fifth (18%) had been able to quit for longer than one week, and 14% had only managed to stay smoke-free for a few hours.

Figure 17 Longest time without smoking

Older students had been able to stay smoke-free for longer periods of time.

Figure 18 Age and longest time without smoking

2.3 Students' smoking environment

Students' smoking environment was assessed in terms of number of friends who smoked, whether or not they were living with a smoker, and how much smoke they were exposed to at home and in other places. The degree to which students were bothered by second hand smoke also was assessed.

2.3.1 Friends who smoke

Almost three quarters (71%) of the student participants indicated that either most or all of their friends smoked. This pattern was similar regardless of gender.

Figure 19 Friends smoke

Older students had more friends who were smokers.

Figure 20 Age and friends who smoke

2.3.2 Living with a smoker

Three quarters of both male and female student participants (76%) lived with a smoker.

Figure 21 Lives with smoker

Younger students were more likely to live with a smoker: 92% of those 14 and younger, but only 62% of the 18-19 year olds and 67% of the 20 and older students, reported living with a smoker.

Figure 22 Age and live with smoker

The majority of students who lived with smokers stated that it was their mother and/or father and/or a sibling who smoked. One quarter lived with a friend who smoked, and 17% reported living with someone else (another relative, tenant, landlord, etc.).

Table 2 Who smokes

Who smokes?	Frequency	Percent	% n=220
Mother smokes	131	44.7%	59.5%
Father	108	36.9%	49.1%
Brother, sister	116	39.6%	52.7%
Friend	54	18.4%	24.6%
Somebody else	39	13.3%	17.7%

2.3.3 Exposure to smoke at home

The student participants are also exposed to smoke while at home: just over one third live in a smoke-free home, whereas half indicate that people smoke in their home most of the time.

Figure 23 How often people smoke in home

Female students were more likely to be exposed to smoke at home than male students.

Figure 24 Gender and smoke exposure at home

Younger student participants (14 and younger, and 15-17 year olds) were more likely to be living in homes where people smoke most of the time.

Figure 25 Age and exposure to smoke at home

2.3.4 Second hand smoke

Student participants were asked how much they felt bothered by second hand smoke. Regardless of gender, one third (37%) of the students was not bothered at all by second hand smoke, about half (46%) was bothered a little, and 17% a lot. There is a slight tendency for a smaller percentage of the older age groups to report that they are not at all bothered by second-hand smoke (42% of the 14 and under, compared to 25% of those 20+ were not at all bothered by second-hand smoke).

Figure 26 Age and how much bother by second hand smoke

2.3.5 Exposure to second hand smoke in various locations

Overall, students were not bothered very much by second hand smoke in many places. Half were never bothered by second-hand smoke in any of the locations mentioned (home, work, friends' place, school, parties, etc); one third were bothered sometimes, and one in ten was bothered by second-hand smoke in some locations (see IV 5 for comments on the construction of this item and limitations in interpretation).

2.4 Conclusion: Was the intended target group reached?

Overall, the Q4L Pilot Program reached the intended target group of male and female smokers, ranging from 11 to 20 year olds with the majority being between 13 and 19 years of age. Almost all had tried their first cigarette before the age of 14 and were mostly smoking on a daily basis, with a large range of daily cigarette consumption (1-70). The majority had tried to quit at least once before, and most had tried many times. Most students' friends were smokers and many also were living with some one who smoked. Students generally were not bothered very much by second hand smoke. Anglophones were overrepresented in the pilot groups.

IV. Evaluation Findings

3. How Satisfied Were Participants With The Program?

Student participant and facilitator satisfaction with the Q4L program was measured and compared with survey items focusing on organizational aspects of the program (length, number of sessions, etc.), program materials (content, lay out, etc.). Students also were asked whether or not participants would recommend the program to a friend. In an open-ended question student participants also were asked what they remembered best about the program. Student focus groups and facilitator interviews also provided qualitative data on program satisfaction.

3.1 Organizational aspects of the program

3.1.1 Program length

Both students and facilitators were satisfied with the overall length of the program: About 60% felt that it was just right, a third that it was too short and less than 10% that it was too long.

Figure 27 Program length - students and facilitators

Students who attended all or most sessions (7 and more) felt that the program was just right or could have been even longer.

Figure 28 Number of sessions attended and satisfaction with program length

3.1.2 Length of program sessions

The majority of both student participants (80%) and facilitators (75%) felt that the length of each session was just right.

Figure 29 Session length - students and facilitators

3.1.3 Preferred program delivery method for Q4L

When asked which method students would use if they were to implement a Q4L program, half would use the group program (50.4%) and 45.2% would use both the group program and the website. Only six students (2.4%) suggested to use only the website. These suggestions should be cautiously interpreted, given that half of the students did not know the Q4L web site existed, and only one fifth had used it.

Figure 30 Preferred program delivery

Slightly more female students preferred a group program and slightly more males a combination of group and website program.

Figure 31 Gender and preferred program delivery methods

3.2 Satisfaction with specific program aspects

Both students and facilitators liked the overall program a lot, ranking it at 2.7/2.8 on a scale of 1-3 (1=not at all; 3= a lot). They liked the style of the program that included group discussions as well as individual tasks (2.67 students, 2.68 facilitators). Students also enjoyed the group and other participants (mean 2.64). Program materials (2.58) and the design or lay out of materials (2.51) also received high average rankings, although on average facilitators tended to be slightly more positive in their assessment of the group and program materials than the students. The average rankings for the design or lay out of materials (students 2.51, facilitators 2.6) and the homework assignments were very similar (students 2.27, facilitators 2.36). Students ranked the homework assignments the lowest (2.27).

Figure 32 Satisfaction with program aspects - students and facilitators

3.3 What students remembered best about the Q4L program

Students were asked in an open-ended question what parts of the program they remembered best. Three quarters of the students (76.6%, 197) provided an answer. High response rates to open-ended questions generally indicate a high overall interest in the program.

Many students liked the program overall, stating that they remembered "everything"; or that they liked "all of them". Some students also mentioned that they did not remember anything in particular, that nothing stood out for them, or that they attended only a few sessions.

3.3.1 Program style

Many students pointed to the style of the program, mentioning the discussions as most memorable: "having group discussions, everyone giving their input and being able to tell their stories"; "group discussions"; "talking about each person's situation"; "eating and talking"; or "having fun rather than just hanging out". They also pointed out specific activities as particularly positive, for example, "when everyone wrote out positive comments about each other"; "Quand on a été faire un jeu de jepordy"; or "when we played football, basketball, soccer, etc. rather than smoking". Others appreciated the fact that they had an opportunity to be with their friends: "being with friends"; "participation of friends"; or "my friends".

3.3.2 Most memorable program activities

The majority of students were able to describe very specific program activities, content, or sessions that they remembered best or that had helped them the most. They included information and activities related to money; support; self-awareness in relation to smoking; specific quit steps and strategies; relaxation techniques; and health facts and information. For detailed student comments, please refer to Appendix D

A great many students had positive memories about videos they had watched during the Q4L group programs. Many mentioned specific videos, often in combination with discussions that followed.

Many students commented positively about the incentives that were provided during the program (such as survival kits, water bottles, gum) and how these helped them in their quit process.

Eating together and food also were highlighted by many students as something that had been important for them.

Some students emphasized how important the facilitator had been to the program, and the support or help that he or she had provided.

3.4 Satisfaction with the program materials

During interviews, facilitators spoke about all the program materials: the Facilitator's Guide as well as the Student Booklet, "Step on It." Overall, they found the Facilitator's Guide very useful. They liked its simplicity and clarity, found it well organized, easy to follow, and very accessible. The evaluation guide also was described as easy to understand; however, many facilitators felt the current tools were too long for ongoing use and should be shortened after the pilot data was collected, especially the post-evaluation survey for participants. A number of students commented positively about the materials that were used during the program, especially the Participants' Booklet and additional pamphlets they received. Facilitator and student comments about the Participant's Booklet are reported below.

3.4.1 Facilitator's Guide

"It's fine, it's actually a very easy, user-friendly guide. The questionnaires that are there, the little worksheets, those are all pretty adequate for kids." (FI7)

Surveyed facilitators were very satisfied with the Q4L Facilitator's Guide (4.41 on average on a 5 point scale (5=Very satisfied; 1=not at all satisfied).) They were most satisfied with the overall level of information and content (4.52) and the least satisfied with the handouts (4.25).

Specific sections in the Facilitator Guide were also rated on a 5-point scale (5=very useful; 1=not at all useful). Facilitators found the sections on Rationale, Q4L Program framework and the program descriptions most useful, and the section on recruitment and retention to be lest useful.

Figure 33 Satisfaction with sections of the facilitator's guide

Facilitators also were very pleased with the degree to which the guide had helped them to plan and implement the Q4L program. On average, they rated their level of satisfaction as 4.25, on a scale from 1 to 5 (1=no, not at all; 5=yes, a lot).

3.4.2 Evaluation Guide for Facilitators

Surveyed facilitators rated the usefulness of the Evaluation Guide on a scale of 1 to 5 (1=not at all useful; 5=very useful). Overall, facilitators found the guide very useful (4.32) and the evaluation activities easy to implement (3.96). The evaluation guide as a whole was rated as very useful (4.45). The detail provided (4.35), and the evaluation tools (4.34) also were rated as very useful.

Figure 34 Usefulness of evaluation guide items

Collecting baseline data was found the easiest of the evaluation activities to implement, whereas finding the whereas finding the drop-outs to complete the post-program survey was assessed as the most difficult.

Figure 35 Ease to implement evaluation activities

3.5 Recommending Q4L to a friend

As a final measure of satisfaction, students were asked whether they would recommend the Q4L group program to a friend who wanted to quit smoking: A large majority said yes.

Figure 36 Reccomend Q4L to a friend

3.6 Conclusion: How Satisfied Were Participants With The Program?

Both student participants and facilitators involved in the Q4L Pilot Program were satisfied with the program style, approach to quitting, organizational aspects, and materials.

IV. Evaluation Findings

4. How Effective Is Q4L In Promoting Youth Cessation?

Specifically, do youth who participate in Q4L:

• Cut down or quit smoking to a greater extent than similar youth who did not participate in Q4L?
• Increase their number of quit attempts?
• Increase their motivation to quit?
• Increase skills and knowledge about smoking?

4.1 Did youth who participated in Q4L cut down or quit?

We adjusted the originally conceived theoretical control group (evaluation design) to reflect the newest available CTUMS (June 2003) figures as much as possible, and included only those items in the theoretical control group for which data from Q4L student surveys was collected.

4.1.1 Comparison of theoretical control groups and Q4L sample at baseline

As reported by CTUMS, 16% of 15-19 year old males in the general population smoke. Of these, 63% smoke every day, and 38% smoke less frequently. 2% can be considered to be former smokers. The male smokers smoke on average 13.7 cigarettes daily. 41% have not tried to quit in the last year, whereas 41% have tried 1 to 3 times, and 19% more than four. 15% have been able to quit, but only 8% are long term quitters (quit longer than a year ago) and likely will stay off smoking more permanently whereas 4% are short term quitters (have quit less than a year ago).

19% of 15-19 year old females are current smokers. Of the smokers, 63% of the smokers smoke daily and 37% less frequently. 3% can be considered to be former smokers. Female smokers smoke on average 10.8 cigarettes daily. 32% have not tried to quit in the last year, whereas half (50%) have tried 1 to 3 times, and 19% more than four times. 16% have been able to quit, but only 6% can be considered long term quitters (quit longer than a year), whereas 8% short term quitters (quit less than a year ago).

CTUMS uses three definitions to determine quit rates: former smokers (smoked 100 cigarettes in their lives and are not smoking currently), long term quitters (quit more than one year ago), and short term quitters (quit less than a year ago). For this evaluation, statistics of short term quitters will be used only for comparison with the Q4L population. Only follow-up data collection would allow to predict long term quit rates.

Daily and non-daily smokers

The Q4L participants differ from the general population (theoretical control group) in several ways. As seen in the figures below, although at baseline Q4L participants aged 15-19 smoked about the same number of cigarettes in a day as their counterparts in the general population (13 cigarettes daily), they were more likely to smoke every day.

Figure 37 Daily smokers (CTUMS and Q4L baseline)

Previous quit attempts

59% of male smokers 15-19 had tried to quit in the last year. 41% had tried 1 to 3 times, and 19% more than four times (CTUMS 2002). In the Q4L group at baseline, 81% of 15-19 year olds indicated that they had tried to quit in the past year. 39% had tried to quit 1 to 3 times and 21% more than four times. 41% could not remember how often they had tried to quit.

Figure 38 Males - quit attempts at baseline : control and Q4L

The difference between the Q4L group and the general population is similar among the females. 68% of the female smokers 15-19 in the general population have tried to quit smoking in the last year. Half 50% had tried 1 to 3 times, and 19% more than four times (CTUMS 2002). In the Q4L group at baseline, 96% of the females had tried to quit in the past. More than half (56%) of the females had tried to quit 1 to 3 times in the past year, and 18% four or more times. 26% could not remember how often they had tried to quit.

Figure 39 Females - quit attempts at baseline: control and Q4L

Average daily smoking rates

Provincial average daily cigarette consumption rates for 15 to 19 year olds were compared with those of Q4L students (15 to 19 year olds). Except for Ontario, the average number of cigarettes consumed at baseline by Q4L participants at each site is quite similar to the provincial averages. The Ontario pilot program was conducted with First Nation youth on a reservation. Participating students were on average much younger than the students in other programs. Both these factors may account for some of the difference between the Ontario pilot site smoking rates and the provincial averages in Ontario.

Figure 40 Pilot sites compared with Provincial CTUMS (June 2003) average daily smoking rates

In summary, the average consumption rates for 15-19 year olds at each pilot site at baseline was similar to provincial averages. However: although Q4L males (15-19 year olds) smoked about the same amount at baseline as their counterparts in the general smoking population (13 cigarettes daily), they were more likely to be daily smokers and had tried to quit more often. Similarly, the Q4L females were more likely to be daily smokers than female smokers of the same age in the general population, although they smoked the same number of cigarettes per day (11 on average). They had also tried to quit more frequently than their counterparts in the general smoking population. These are possible indicators for a higher level of nicotine addiction than the general smoking population in that age group. A higher level of nicotine addiction may lead to lower quit rates.

4.1.2 Quit rates: Comparison of theoretical control groups and Q4L sample at program end

To determine short-term effect of participation in the Q4L program, the quit rates of participating youth post-program were compared to quit rates of the general population.

As described above, 6% of 15-19 year olds are short term quitters, they have quit smoking in the past year (CTUMS, 2002). Immediately after participating in the Q4L program, 11% of the 15-19 year olds had quit smoking. 4% of the males in the general population of 15-19 year olds were short term quitters compared to 10% of the Q4L males. 8% of the females in the 15-19 year general population were short term quitters, compared to 12% of the Q4L females.

Figure 41 Short term quitters - post: control and Q4L

Only follow-up surveys with Q4L graduates at 12 months can confirm how stable these figures are, but they do suggest an important short-term effect of program participation. If more than 7% (CTUMS 2002 long term quitters) of the Q4L graduates remain smoke free at 12 months, Q4L youth will be shown to have succeeded at quitting to a greater extent than the average Canadian 15-19 year old.

4.1.3 Cut down rates: comparison of theoretical control groups and Q4L sample at program end

The Q4L program also aimed to decrease the total number of cigarettes smoked by participants (cut down rates).

The average daily smoking rates for the 15-19 year old age group at baseline were comparable to average Canadian figures for the age group (12.2 cigarettes smoked daily) (CTUMS, June 2003).

Immediately after participating in the Q4L program, the average number of cigarettes smoked daily dropped to 5.6 (6.2 for males, 5.1 for females). These numbers are well below the average daily consumption of cigarettes for this age group in the general population. Youth who participated in the Q4L program were able to cut down cigarette consumption by more than half.

As mentioned above, only follow-up surveys with Q4L graduates can confirm whether participants are able to maintain the lower consumption rates over time.

Figure 42 Average daily cigarettes: CTUMS, Q4L baseline, Q4L post-program

4.1.4 Smoking habits at baseline and at the end of the Q4L program (based on all Q4L students who completed a pre and post surveys)

Average daily cigarette consumption was calculated for Q4L participants at the beginning of the program (baseline) and again at the end. All figures are based on students' self-reports and were not verified in any other way. Weekly smoking figures provided by occasional smokers were converted into daily cigarettes smoked to be able to include occasional smokers in a calculation of average number of cigarettes smoked daily. The average number of cigarettes smoked at the beginning of the Q4L program was 12 per day (ranging between 1 and 70). After attending Q4L the average number of cigarettes smoked per day dropped to 5 cigarettes per day (ranging between 0 and 45).

Figure 43 Average daily cigarettes smoked

Average cigarettes smoked fell from 12.7 cigarettes to 5.9 cigarettes for male students and from 10.9 cigarettes to 4.9 cigarettes for female students.

Figure 44 Average cigarettes smoked: male and female students pre and post

4.1.5 Conclusion: Do youth who participate in Q4L cut down or quit to a greater extent than similar youth who do not participate in Q4L ?

Youth who participated in Q4L programs were able to cut down cigarette consumption in half. Immediately after the program, they were smoking fewer cigarettes daily than the Canadian average for this age group of smokers. The 11% quit rate was higher than the Canadian average of 6% short term quitters. This number also should be interpreted in light of the higher number of possibly more addicted youth at program start. A number of Q4L facilitators also reported that participants quit after formal program completion, some because they had set their quit date after the last session. As mentioned above, follow-up with participants at 12 months would provide a better indication of the lasting effects of the program on amount smoked and quit rates.

4.2. Do youth who participate in Q4L increase their number of quit attempts?

At baseline, Q4L participants reported how often they had tried to quit in the past 12 months, and the longest period they remained smoke-free. Quit attempts are described in an earlier section. The number of quit attempts that may have occurred during the program itself were not recorded. Post-program surveys were administered approximately three months after baseline surveys, at program end. To assess whether participants had increased their number of quit attempts, we compared the longest smoke-free period reported during the year prior to attending Q4L, with the longest smoke-free period while attending and immediately after Q4L, over an approximately 3-month time frame.

Any interpretation of these figures should take into consideration the much shorter time frame; the fact that participants may have set their quit date a short time prior to program end, limiting the amount of time they could be smoke-free prior to the post-program survey; many facilitators encouraging participants not to attempt to quit too early in the program, to ensure sufficient preparation time. Taking these limitations into account, there are some promising results. Taking out participants who report a 'quit period' of less than one day, and those who cannot remember, 80% of participants quit for at least 24 hours while attending Q4L, compared to 78% having been able to quit for at least one day during the entire year prior to Q4L.

Figure 45 Smoke free time period: pre and post

4.2.1 How long were students smoke-free: pre (12 month period) post (3-month period)

Smoke-free time periods were compared for each individual for which both pre and post surveys were available. One quarter of the Q4L participants (27%) were smoke-free for a shorter time than in the previous year; another quarter (28.8%) stayed smoke-free for about the same period of time. One third (36%) quit for a longer of time during the Q4L program than they had done in the twelve months previous.

Figure 46 Length of quit attempts: pre and post compared

4.2.2 Conclusion: Do youth who participate in Q4L increase their number of quit attempts?

Q4L participants increased the number and the length of their quit attempts. All Q4L participants stayed smoke-free for at least a few hours, whereas 14% had not tried at all to quit in year prior to attending Q4L. 79% had quit for at least one day in the twelve months before starting Q4L; 84% achieved the same length of quit period in the three months of the Q4L program.

4.3 Do youth who participate in Q4L increase their motivation to quit?

Increase in motivation can be measured in a number of ways:

• Reduction in smoking rates (see above)
• Overall quit rates (see above)
• Perceived motivation (see below).
• Comparison of the motivation to join the program with the help that Q4L program provided (see below)

4.3.1 Perceived increased motivation

Students were asked on two occasions how much the Q4L program had helped motivate them to quit and to stay smoke free (see figure 47 below). On average, they indicated that the program had helped them to increase their motivation a lot (2.34). 41.5% had been motivated them a lot to quit and stay smoke free, 49.0% were a little motivated and 9.5% did not feel that the program had motivated them

Facilitators also rated students' motivation to quit or stay smoke free as having increased a great deal during the program. On average, they rated the increase at 2.66 on a three point scale (1=not at all, 3= a lot). Two thirds (66%) thought that students motivation had increased a lot and one third (33%) a little. None of the facilitators thought that students' motivation had not increased at all.

4.3.2 Comparison of the motivation to join the program with the help that Q4L program provided

Students were asked in the baseline survey what had motivated them to join Q4L. There were asked to rate a number of factors on a scale of 1-3 (1=not important at all; 3=very important). The most important factor for joining was consideration for their health "I want to be healthier" rated at 2.87, as very important, followed by not wanting to be out of breath (2.79), and not wanting to spend so much money on smokes (2.78); cutting down and quitting or knowing how to quit in future were the next most important reasons that motivated youth to join Q4L. Friends joining and wanting to change tobacco laws were the least important reasons.

Students were also asked, after the program, how much help they had received from the Q4L program in various areas. On average, students rated "Know how to quit in future" was rated the highest (2.71), followed by getting help with cutting down (2.68), and reducing the money spent on cigarettes (2.51). Living a healthier life was next (2.5), followed by reducing concerns about getting lung cancer (2.36).

As illustrated in Figure 47 below, some of the items that had motivated students to participate were not the same as those for which they had received the most help. Interviews and focus groups suggest that some of these differences might be due to gradual realization of how hard it actually is to quit. In addition, not all the items presented had equal priority within the Q4L program design.

Overall, Q4L was able to meet or exceed some very important student expectations for participation, which had motivated students to get involved, such as cutting down on smoking, knowing g how to quit in future, reduce the amount of smell, reducing the concern to get lung cancer, and to smoke less with friends.

Figure 47 Motivation and Help Received

In a second step students were asked to circle the "one most important reason" for joining Q4L. Wanting to be more healthier remained the number one reason, one third (31.8%) circled this item as their most important reason for joining. One quarter circled wanting to quit (24.7%), and 11.6% not wanting to spend all that money on smokes.

Table 3 One most important reason for joining Q4L (Baseline survey)

	Frequency	Valid Percent
I want to be healthier	63	31.8
I would like to quit smoking	49	24.7
Not spent money on smokes	23	11.6
Not to be out of breath when active	18	9.0
Know someone with lung cancer	11	5.5
I would like to cut down	7	3.5
Prove that I can quit	7	3.5
Someone else wants me to participate	6	2.8
Not smelling of smoke	5	2.5
Like to know, so I can quit in future	3	1.7
Joining because of friends	2	1.1
Better informed	2	1.1
Smoke less with friends	1	.6
Other reason	1	.6
Total	198	100.0

4.3.3 Facilitators' perception of the degree to which students were helped by Q4L

Facilitators also were asked (in a final facilitator survey) to rate on a 3-point scale (a lot; a little; not at all), how much the Q4L program had helped youth on a number of different dimensions (see Figure 48 below). Overall, facilitators felt that the program had helped participants more than a little (2.35) on average. Most items were rated as either having helped a lot or a little, with the exception of changing something about tobacco laws (1.49).

On average, facilitators felt that Q4L had helped participants the most in terms of knowing what to do so they could quit in the future, 2.89; how to cut down (2.84); and be motivated to quit or reduce smoking (2.66). These items also were three of the explicit objectives of the program. Facilitators felt that the program had helped the least with changing something about tobacco laws (1.49).

Figure 48 Facilitator perception - help received

When comparing facilitators' and students' perception about the amount of help received on 13 different items the similarities are striking. Facilitators were more likely than students to think that students had joined because of their friends. They also were less likely to think that the program had helped students change something about the tobacco laws (not addressed directly in the program curriculum).

Figure 49 Students and Facilitators Comparison

4.3.4 Conclusion: Do youth who participate in Q4L increase their motivation to quit?

Participating youth felt that the Q4L program had increased their motivation to quit. Facilitators also perceived that students' motivation to quit had increased a lot. The program met students' expectations in terms of their original motivation to join the program in many areas and exceeded their expectation in some (e.g., know how to quit in future, reduce concern about lung cancer, smoke less with friends, and opportunity to join the program with friends).

4.4 Do youth who participate in Q4L increase skills and knowledge about smoking?

4.4.1 Skills learned during Q4L - student perception

Students were asked to rate on a scale of 1-3 (1=not at all, 3=a lot) how much the Q4L program had helped them to develop a number of skills. They rated the overall level of skills acquired during the program very highly. Having learned skills to quit was ranked the highest, at 2.61, followed by learning about what to expect when quitting (2.58), and getting about creating a positive environment for oneself (2.53). Learning how to postpone the next cigarette received an average ranking of (2.41), followed by being motivated to quit and stay smoke free (2.32) and help to relax and not think about smoking after each of the session (2.32)

Figure 50 Skills learned during Q4L

4.4.2 Skills learned during Q4L - facilitator perception

Behavioural change is an important component of the Q4L conceptual framework. Facilitators felt that participants' skill level had increased a lot, on average 2.47 rated on a 3-point scale (3=a lot; 2=a little; 1=not at all) as a result of the Q4L program. The highest ranked items were: skills to quit (2.84; 'informing them about what to expect when they quit (2.77); and providing them with ideas how to create a positive environment' (2.77). Facilitators felt that participants only were helped a little (1.93) with skills to 'relax and not think about smoking after each session'.

Figure 51 Skills learned - facilitator perception

4.4.3 Skills learned during Q4L - Comparing facilitators' and students' perceptions

Students and facilitators ranked the skills that were learned during the Q4L program in the same order. The overlap of facilitators' and students' ranking indicates that facilitators were able to convey the main areas of skill development correctly. In that sense, the program is working well. For example, teaching and learning quit skills is a key skill development area of the Q4L program. In contrast, acquiring relaxation skills was not originally a priority area for skill development in this program, although it emerged as an important dimension from focus groups and interviews.

Figure 52 Skills learned - Students' and Facilitators' perceptions

4.4.4 Perceived knowledge increase

Pre and post average scores indicate that students perceived that their knowledge had increased at the end of the program in all areas. On average, they thought that their knowledge had increased the most in relation to learning about quitting steps, followed by dealing with withdrawal and preparing to quit or reduce. Knowledge about the effects of smoking on women and men was next, followed by quit methods, getting support and barriers to quitting. Students felt that their knowledge had increased less in relation to addiction to tobacco, benefits of quitting, health effects and smoking as a habit. In the baseline survey, many students had indicated that they already knew a lot about these issues.

Pre and post knowledge tests with students were not conducted.

Figure 54 Knowledge pre and post

Figure 55 Perception of Knowledge Increase: Pre and Post Average Knowledge Scores

4.4.5 Knowledge increase comparing students and facilitator perceptions

Students and facilitators showed great consensus on their average scores of participants' increase in knowledge. Facilitators in general were a bit more optimistic than students about the amount of knowledge increase on most items.

Figure 56 Knowledge increase - students compared to facilitators

4.4.6 Change in smoking environment

Participation in the Q4L program may have affected smoking environment in which participants find themselves. Participants report that fewer of their friends smoke at post-program than did at baseline: At program start, 70.9% indicated that most or all of their friends were smokers; by program end, this percentage had dropped slightly to 63.8%.

Figure 57 Friends smoke: Pre and Post

4.4.7 Degree to which students are bothered by second hand smoke

More students were bothered by second hand smoke at the end of the program than at the beginning. A quarter (24%) were bothered a lot at the end compared to 17% at the beginning of the program. 52% were bothered a little at program end and only 46% at the beginning. More than a third (37%) had been not bothered at all at the beginning of the program while at the end only a quarter (24%) did not feel bothered at all.

Figure 58 Bothered by secondhand smoke: pre and post

Although changes in smoking environment (reducing the number of friends who smoke, or making new friends who don't), was not a criteria for program success, it could be considered in future. As well, increased intolerance to second hand smoke could also be considered an indicator for program success.

4.4.8 Conclusion: Do youth who participate in Q4L increase skills and knowledge about smoking?

Students who participated in Q4L were able to increase their skills and knowledge about smoking. Facilitators and student participants were consistent in their perception of the areas in which skills and knowledge had increased, and the amount of that increase.

IV. Evaluation Findings

5. Suggestions: How Can The Program Be Improved?

5.1 Organizational aspects of the program

5.1.1 Recruitment

Lessons learned about recruitment:

The literature on youth cessation suggests that teens are reluctant to participate in group programs, and that it is not their preferred choice for quitting. Those who decide to participate in a group program, though, generally are satisfied with the program and find it helpful. The Quit4Life Pilot Program was no exception.

Final reports from Site Co-ordinators confirm that recruitment was challenging in most settings. Flexible, site-specific interpretation of the program allowed different co-ordinators and facilitators to use a variety of outreach techniques, responding to specific conditions.

Although a number of sites initially attempted to recruit students outside of school settings, the majority of programs were implemented within schools. Some coordinators felt that the effort to recruit youth outside of schools and to advertise the program in the community had been, in retrospect, a waste of time. The site where Q4L was offered as an outreach program, with multiple activities offered flexibly to any number of available students outside the school setting, did not experience the same difficulties as those which relied exclusively on school-based recruitment. It is worth noting that the Q4L facilitators at that site were youth addiction workers with experience in community youth outreach.

Recruitment success cannot be measured exclusively by the number of participants who attend a program. Some school-based programs were described as very successful in recruiting, having reached those students who most needed and could benefit from the program, even though these might have been few in number. In some high schools, for instance, the actual number of smokers is relatively small: one focus group reports that all the school smokers are in the program - a total of four!

As described earlier, facilitators and co-ordinators used a range of standard recruitment techniques. Most students had heard about the program during school announcements, but the most successful sites also incorporated word-of-mouth recruitment. Previous positive relationships between interested adults (teachers, guidance counsellors, public health nurses, school social workers) and the target students contribute to the effectiveness of this strategy: if the students know and trust the adult, they will respond positively to individual approaches to encourage them to participate in a Q4L program.

Word-of-mouth also functions horizontally across the student population: many students had heard about the program from friends, and mentioned attending with one or several friends. Incorporating a group of students who are friends into a Q4L program provides participants with a ready-built support network, and may help reduce social group temptations when they are in the process of cutting down and quitting.

Innovative ways to encourage program participation were used at several sites, including a contest format ("bring a friend and win a prize"), handing out gum, promising small incentives for attending the first session, or offering lunch during sessions (this was very positively received by many students). Finding the places were the smokers hang out at lunch or after school to recruit them right then and there was another successful way of recruiting students into the program.

Other recruitment techniques included open information sessions during which students were provided with details about the program and/or could take away the booklet for self-directed cessation attempts (no data available about the relative success of this strategy to date). Flyers, posters, announcements, were all seen as useful reminders, especially if the date and time of the next Q4L group session were included.

A small number of facilitators used committed peer counsellors or peer facilitators, who had previously participated in a Q4L or other cessation program, to assist with recruitment.

Barriers to effective recruitment included:

• Posters and materials not available at the ideal times
• Posters without specific session information (time, place)
• Difficulty identifying volunteer facilitators in the schools with time to both do recruitment and deliver the program
• Student concerns that group sessions will involve too much 'personal' discussion
• Not wanting to miss certain classes to attend sessions
• Having to get a 'permission slip' to miss classes to attend sessions
• Sessions offered through guidance department, identification with 'problem' students
• Co-ordinator/facilitators not speaking the language of the school administration.

Suggestions to improve recruitment:

• Work inside the schools to reach the enrolled student population
• Enlist support from key individuals in the school (principal, guidance counselor, PHN, youth social workers) for assistance with recruitment.
• Use a combination of outreach activities that includes announcements, flyers, and information sessions.
• Use individual outreach to smokers as much as possible: go where the smokers are.
• Include committed peers (ideally former smokers) in outreach activities.
• Revise the section on recruitment in the Facilitator's Guide to include more detailed suggestions and experiences from pilot facilitators (see Technical Report)
• Redesign posters to include space to write in specific session information

5.1.2 Registration and intake

Lessons learned about registration and intake:

Different sites established different methods for registration. As described earlier, some sites were very open, with continuous 'registration' and drop-ins; others had fixed registration and start dates, and kept the group closed until the completion of the sessions. There are advantages to both approaches, but a more closed group appears to be essential for the Q4L program to work as a sequential group cessation program. Some sites addressed this by establishing a 'waiting list' of interested students who were encouraged to register for a second session of Q4L.

A few facilitators screened students prior to letting them attend the Q4L program, in an attempt to assess how 'serious' they were about trying to quit. Some facilitators identified students with other issues (e.g., drug addictions), and steered them towards dealing with these prior to attempting to quit. (We are not able to assess the usefulness or success of this approach.)

Suggestions to improve registration and intake:

• Maintain site-specific flexibility.
• Offering a series of Q4L sessions over a school year may help to incorporate 'late' registrations into a sequential program
• Provide the Participant Booklet and/or access to the Q4L web site to students who are interested in quitting but not ready to register for the group sessions.
• Offer one-shot information sessions, as well as the group program; participants who attend information may have an increased interest in registering for the next group session.

5.1.3 Group size and retention

Lessons learned about group size and retention:

The pilot program permitted considerable flexibility in the size of Q4L groups. Minimum and maximum group sizes were determined by facilitators and were set quite differently within and across sites, to reflect local conditions and facilitator preferences.

On average, groups started out with 12 participants, and retained 7 until program end, but there were large variations, again reflecting specific conditions in different sites (as mentioned above, in some schools the number of smokers is actually quite small).

Program attendance varied greatly from site to site, and from group to group. Some facilitators established attendance rules (e.g., missing several sessions led to expulsion from the group). Others cancelled program sessions, or even did not run a group at all, if attendance seemed low.

To enhance attendance and retention, some facilitators worked with attendance incentives (e.g., draws for a gift certificate at each session). Others tracked down students, who had missed one or two sessions, to find out what was happening and encourage them to continue.

Insufficient data are available from drop-outs to understand fully their reasons for leaving prior to program end. Several facilitators commented that the structure of the program itself, in which a quit date is set near the final sessions, may have led some students to drop out after their quit date, perhaps feeling that they had already accomplished their goals.

Suggestions to improve group size and retention:

• To reflect particular conditions and fluctuations in the population of smokers and quitters at any site, a minimum group size should not be set; however, if a facilitator is working with only one or two students, the program should not be considered or assessed as a group cessation program: rather, it should be recognized that in those cases Q4L is being delivered as a one-on-one, individual support program.
• Maximum group size should not, in general, be greater than 15, to permit full application of the participatory elements of the program (including group discussions), and to ensure that the facilitator has sufficient time to pay attention to and support individual behavioural change plans of each group participant.
• Once a group begins to meet, it may be best to hold regularly scheduled sessions even if the number of students who show up to a particular session is small. It is important to show commitment to those students who come to a session, and respond to their motivation to quit and cut down, by continuing the program.
• More emphasis could be placed on the importance of continued attendance and support after the quit date is reached, so students understand the benefits of continued attendance for the final sessions of the program.
• Facilitators should consider providing additional short 'catch-up' individual sessions for students who have to miss a group session, so they avoid having to repeat material unnecessarily.
• Facilitator's Guide could include more information about typical retention rates for teens in group programs, to help less experienced facilitators put their own group retention rates in perspective.

5.1.4 Group diversity

Lessons learned about group diversity

Most groups included mixed ages and mixed gender. Facilitators in general welcomed mixed age and gender groups. They felt that boys and girls could learn from and about each other better in a mixed setting. Most participants had friends from both genders, so having a mixed group prepared them better for dealing with their social environment when cutting down and quitting.

Weight concerns were more frequently raised by females, but in practice were of concern to many young men as well. Males also profited from information about nutrition and exercise which might not have been emphasized as much in a male-only group.

Older teens (15 and older) participated in the program to a greater extent than students who were 14 and younger, reflecting smoking uptake patterns. Facilitators noted that younger students might be less motivated to quit completely, and were less aware of the impact of nicotine dependency and the increasing difficulty quitting the longer one smoked. In these cases, having older students in the group was a very positive influence, since they provided living proof of how much harder it would become to quit over time, and how much more expensive it would become to smoke more.

In a small number of cases, students participated in groups which were linguistically mixed (e.g., a group held in French, in a French immersion school, included both Francophones and Anglophones). They expressed some frustration and difficulty working in the 'other' language, especially in relation to open discussions and understanding complex written material.

Suggestions to improve group diversity:

• Unless students make specific requests for gender or age specific groups for unique reasons, groups should be open for all students to participate.
• Since the program relies on open discussion among students, as well as written work, attention should be paid to the language in which students are most comfortable working (which may not be the 'official' language of the school they currently attend).

5.1.5 Scheduling Q4L program sessions

Lessons learned about scheduling Q4L program sessions:

The Pilot permitted complete flexibility in relation to scheduling, and facilitators used a wide range of different formats to make it as easy as possible for students to attend the program. Each option has advantages and disadvantages, summarized below.

Group meets during lunch hours: sessions are either 45 minutes to one hour long, with lunch provided; or sessions are less than one hour, to permit students to make their next class and/or to eat their lunch first.

Advantage: integrated during school time; students do not miss class time; a group of friends who have similar schedules can stay together; eating together is a bonding experience.

Disadvantage: students want to spend time with friends who are not attending the program; there is no time for a smoke break during the school day; students reluctant to give up their free time; unless lunch is provided, takes time to buy food and eat prior to the session; students often are late or miss these sessions.

Group meets during class time: students get permission to miss classes in order to attend the Q4L program; group meetings are held at the same time each week; or are held on a rotating schedule, to ensure that students do not miss the same class each week.

Advantage: Integrated into school day; students get the message that the Q4L program is considered important and 'serious' enough for them to miss class time; students do not have to give up free time or miss out time with their friends; does not interfere with eating lunch or other 'free time' activities.

Disadvantages: students miss class time and have to catch up; some teachers do not release students who are experiencing difficulties in the subject; some teachers will not postpone or schedule special tests or quizzes for participants; students do not want to miss certain classes will not attend Q4L regularly; other students resent that smokers can 'get out of class'; rotating schedule creates difficulties as students forget when the next session will take place; requires great flexibility in terms of facilitator's time and availability during different days and times each week.

Group meetings times alternate: one during lunch, one during class time to balance the advantages and disadvantages that are described above. Lunch sessions 30 minutes in length; class sessions 60 minutes.

Group program included as part of the health curriculum.

Advantages: integrated into the lesson plan; no extra bureaucratic procedures are needed (e.g., signing attendance forms, getting signatures from teacher to be permit leaving class to attend program).

Disadvantages: students might feel forced into the program and research has documented that mandatory quit programs have been unsuccessful in the past; students might miss out on important physical activity portion of the school day, which is potentially a contributor for continued smoking (less active teens tend to smoke more); timing of sessions may have to be adapted to other parts of the health curriculum, and/or compressed.

Group meets before or after school: None of the facilitators during the pilot used this option, although some discussed the possibility. After-school commitments, especially work, would interfere with attendance; in rural schools, reliance on buses limits all sorts of after-school activities; before- school option would also be limited by busing concerns, as well as requiring considerable motivation (youth might not have the motivation to get up earlier than usual on a regular basis to attend a smoking cessation class).

Suggestions to improve scheduling:

• Flexibility should be maintained to permit individual facilitators to respond to the specific situation, keeping in mind the needs of administrators, teachers, and students, while making every effort to integrate the program into the school day, rather than before or after school.
• Lunch hour groups should provide food for students, to permit longer sessions and increase regular attendance.
• Teacher commitment to support Q4L participants should be obtained ahead of time, so that arrangements can be made ahead of time for tests and assignments without putting additional pressure and stress on youth who are trying to quit.

5.1.6 Program and session length

Lessons learned about program and session length

The Q4L program includes nine sessions with a recommended length of 60 minutes each. The Facilitator's Guide recommends providing additional support to students beyond the nine sessions (whenever possible).

In practice, the number of times each group met varied considerably within and across pilot sites, from three to 26 sessions, with an average of 10.5 meetings per group over ten weeks. The length of each session varied between 30 and 60 minutes. Most students and facilitators were pleased with the length of the sessions.

More than half of facilitators and students felt that the length of the program (number of sessions) was 'just right'. Note that this assessment was based on an average of 10.5 sessions actually implemented, rather than the originally planned nine sessions only. Regardless of how many sessions they actually held, one third of facilitators and students actually felt that there had been 'too few' sessions, while very few considered that there had been 'too many' sessions.

At the same time, facilitators were aware that a program that is too long and includes too many sessions might appear intimidating and keep people from trying to implement it. Having many more sessions might make it difficult to negotiate free time from teachers/administrators; and the task of running the program might become too unmanageable for a volunteer teacher or guidance counsellor who is already overloaded with other work.

Suggestions to improve program and session length:

• Maintain some degree of flexibility, permitting facilitators/sites to take into account specific conditions when determining the length of any one group program.
• Keep a number of key core activities, grouped into ten sessions, to provide facilitators and students with a guideline for how many times the group is likely to meet.
• Offer the ten core sessions flexibly, responding to specific conditions in terms of timing, but maintain a minimum program length of at least six weeks, to permit students to learn and practice skills.
• Hour-long sessions should be the norm, with some flexibility built in to respond to specific conditions.
• Provide optional follow-up sessions, or ways of scheduling regular follow-up contacts with students, especially if the group program is shorter than ten weeks.
• Respond to student needs in scheduling number and length of follow-up sessions; take into consideration exam schedules, vacations, special school events.
• Include a program template for one or two possible follow-up sessions, and a list of possible activities, in the Facilitators' Guide.

5.2 Facilitator role

Lessons learned about the role of facilitators

Every profession brings a unique approach and perspective to youth cessation. The pilot program was implemented by facilitators from many professional backgrounds. Some facilitators were volunteers; others were hired specifically to implement the pilot program; yet others (such as teachers, public health nurses) added on the task of facilitating a group to their regular work activities. Some facilitators were 'embedded' within the schools in which the programs were held; others worked out of the pilot site agencies. Some facilitators also performed co-ordination tasks; most were relatively removed from the day-to-day responsibilities of co-ordinating the pilot project as a whole. Their experience with tobacco issues, cessation, and working with youth was also varied.

Overall, a set of skills and specific knowledge were identified by co-ordinators, students, and facilitators themselves as important for future facilitators. These include: a solid knowledge of tobacco issues and specifically teen cessation; group facilitation skills; comfort and experience working with this age group; ability to build relationships with youth that extend beyond implementing the curriculum; flexibility; and awareness of students' needs.

Student participants emphasized that facilitators should be non-judgmental, positive, fun, caring and understanding. Being an ex-smoker also increased facilitators' credibility among youth.

Suggestions to improve facilitation

• Develop a self-assessment quiz so that facilitators can test their knowledge on teen cessation. Provide this either in the Facilitators' guide and/or on the web-site.
• Develop additional material on facilitation and facilitator qualities that are helpful. Provide this in the Facilitators' Guide and/or on the web-site.
• Develop common scenarios in supporting youth cessation, to give new facilitators an opportunity to deepen their skills and practice responding to the issues that may emerge. Provide this in the Facilitators' Guide and/or on the web-site.
• Consider offering a short, optional training session for new facilitators.
• Provide "fast facts" about recent youth smoking statistics, cessation, working with this age group, and facilitation tips, at a "Q4L Facilitators Support Page" on the Q4L web site.

5.3 Program concept

Overall, the majority of participants, facilitators and students alike, were very satisfied with many different aspects of the program.

5.3.1 Program approach

Lessons learned about program approach

Facilitators welcomed the behavioural approach on which the program is based, and felt comfortable with delivery of a planned step-by-step approach to quitting. In general, facilitators understood the importance of encouraging students' development of self-efficacy and motivation and were able to set a positive climate for learning and student self-exploration, as documented in findings from youth focus groups and surveys (see Appendix D - Technical Report).

The four steps as presented in the Participants' Booklet ("Get Psyched; Get Smart; Get Support; and Get On With It"), however, were not always well-understood; nor were they clearly related to the program activities and sessions as laid out in the Facilitators' Guide. There were also a number of inconsistencies between the Facilitators' Guide and the website that made it difficult for many facilitators to use the material on the site effectively.

Suggestions to improve program approach:

• Apply the four program steps consistently across all Q4L-brand program activities and materials (Facilitators' Guide; Participants' Booklet; Website; promotional materials).
• Increase opportunities to demonstrate and use the behavioural approach, including more activities that encourage students to develop and practice realistic, small-step quit strategies, increasing the likelihood of students' experiencing success.
• Give greater emphasis to the importance of encouragement and keeping a positive outlook as a central concept of the program.

5.3.2 Program implementation

Lessons learned about program implementation

Facilitators were able consistently to implement all the core program elements which were outlined in the 9-session curriculum. Most presented the sessions in the order suggested in the Facilitators' Guide. The way the activities were presented in the Guide made it possible for facilitators to vary the order of sessions somewhat, and to shift activities from one session to another, to respond to students' needs and emerging issues.

Overall, facilitators maintained the flow of the program: that is, activities designed for the early, middle, or end point of the program were delivered roughly in that time period.

Many facilitators at most sites added content not currently included in the program. (e.g., information about the patch, second-hand smoke), or expanded activities to give them more weight within the entire program (e.g., stress reduction).

Incentives also were used with great variation during the pilot site programs.

Suggestions to improve program implementation

• Re-formulate the program into a ten-session program with a choice of varied activities for each session, clearly linked to each of the sequential four steps.
• Include suggestions for flexible implementation and ideas for sources for additional activities in the Facilitator's Guide.
• Encourage facilitators to consider the needs of each new group and be responsive to these, adding more activities or shifting activities from one session to another, while maintaining the fundamental sequence of the four-step program.

5.3.3 Program style

Lessons learned about style of group program

Most facilitators and students enjoyed the style of program, which included a mix of individual and group activities; group discussions; and support. The participatory approach generally worked well; students were generally very positive about working in a group and group support. Some facilitators would have liked a more active, hands-on style that included less paper, less reading and writing, and instead encouraged more activities that required 'doing', such as trying out activities that could be done instead of smoking.

Facilitators who were working with students with low literacy skills converted some activities into active role plays, in which students could experience situations, rather than writing them down.

The expectation that participants would do tasks outside of the group session is problematic in a school session, since any such task takes on the image of being "homework". Students tended to forget or avoid doing 'homework', especially when written responses were required. All participants felt that these types of assignments should be kept to a minimum, or eliminated all together.

Some groups reduced the assignments to a few key activities, such as filling out the tracking form or getting the support pledge forms signed. Different facilitators and students had different responses about how well these tasks were completed, and how useful they proved to be.

Some students and facilitators distinguished between 'homework' and other out-of-class activities, such as working on cutting down between sessions, practicing quit skills, and planning to reduce smoking using a step-by-step approach.

Suggestions to improve program style

• Maintain the participatory and interactive style of the program.
• Encourage activities that promote a sense of group support.
• Include alternative activities that could be used instead of a written activity in the Facilitators' guide (e.g. role plays).
• Reduce written tasks between sessions to a minimum.
• Avoid describing as 'homework' tasks between sessions that have to do with complying with a quit plan.

5.4 Program Content

Lessons learned about program content

Most facilitators generally covered the core program areas as suggested in the Facilitators' Guide (see Appendix 3 - Technical Report Section). Many groups spent additional time on practicing quitting.

Facilitators and students most frequently mentioned the need to increase program content around stress and stress management. Additional content was introduced by many facilitators, including the following topics:

• Stress reduction (yoga)
• Practicing stress reduction skills
• Alcohol and partying
• Healthy lifestyles
• Nutrition and weight
• Physical activity
• Second-hand smoke
• Dealing with smoking environment at home
• Boredom, what to do instead of smoking
• Choice theory
• Stages of change model
• Dealing with feelings when quitting
• Addiction
• Teenage pregnancies
• Having sex (not only gender issues) and smoking
• Health and smoking (e.g., lung capacity)

Almost all groups included at least one movie during the program, and/or had an invited resource person to present additional information or talk about their own experience with smoking, quitting, and the effect that smoking had on their life. Many students mentioned the movie or the resource person as one of the most memorable experiences of the program.

Suggestions to improve program content:

• Include an additional session on stress and relaxation, relatively early in the program
• Develop program content for additional topics that facilitators can choose to implement should the need arise. Priority areas are: alcohol, marijuana, nutrition and weight, physical activity, second-hand smoke, dealing with a smoking environment at home.
• Include a movie and discussion into the program (provide list of suggested movies in the Facilitators' Guide and/or the web site)
• Invite resource person/guest speaker earlier in the program, but maintain flexibility to be able to match available resource person, participants' interests, and program flow.

5.4.1 Incentives

Lessons learned about incentives

As a Pilot Program, most Quit4Life sites counted on additional resources to purchase a number of 'incentives'. A few facilitators implemented the program without any incentives; others obtained small budgets from school council or gift certificates from local businesses.

Most facilitators used at least one of the following as incentives:

• food, especially if sessions were run during lunch hours
• rewards, such as gift certificates or coupons
• coping kit items, including suckers, stress balls, gum and candy
• celebration at the end of the program, going bowling, going to a restaurant

Most facilitators felt that it would be difficult to implement the program without any incentives. On the other hand, some facilitators who had implemented the program three times concluded that incentives could be reduced considerably, and also suggested greater selectivity in devising individual incentives. For example, not all students use stress balls or like suckers.

Suggestions to improve incentives

• Include ideas for how to run the program with incentives but without a budget in the Facilitators' Guide
• Encourage creative ideas about the types of incentives that can be provided without financing (e.g., attendance certificates; public recognition)
• List possible ways of obtaining the minimum 'ingredients' for a cope kit (e.g., request donations from local businesses, parent-school associations, approach other community organizations)

5.4.2 Setting a quit date

Lessons learned about setting a quit date

Facilitators were not certain about the ideal time for a quit date. Most of them handled this flexibly, considering students' level of readiness and need to be convinced or motivated to pick a date. A few facilitators avoided setting a quit date all together, because their students were successfully cutting down. Some students considered their 'quit date' to be the end of the group, and did not return after that time.

Suggestions to improve setting a quit date

• Maintain setting a quit date as an important part of the program
• Emphasize the benefits of setting a quit date.
• Encourage students to set the quit date early enough in the program so that they will be able to receive continued support after they quit.

5.5 Program materials

Lessons learned about program materials

Since assessing the program materials was a specific focus for this evaluation, students and facilitators were asked, in some detail, what they would change to improve the Q4L program materials. Approximately two out of three students and facilitators would not change anything. More facilitators wanted changes to handouts and format, while more students than facilitators wanted changes made to the amount of information (not specified: more or less information should be included); the lay-out and design, and the writing style.

Figure 59 Suggested changes - students and facilitators

Facilitators and students liked the program material and did not want to see many changes made to it (see Technical Report, Section III). A third of the facilitators wanted changes to be made to the handouts, and a quarter of the students to the amount of information and a fifth to the lay out and design.

Facilitators pointed out that the guide, student booklets and information on the Q4L website were not organized in a consistent manner. This had created confusion when directing students to the booklet or to the website to introduce certain content areas, or to encourage them to review information.

Suggestions to improve program materials in general

• Ensure consistency between guide, booklet and website.
• Revise handouts, make them visually more attractive.
• Revise student booklets (see below).

5.5.1 Facilitator's Guide

Lessons learned about the Facilitators' Guide

Most facilitators were very satisfied with the Facilitators' Guide. They found it clear and user-friendly, and were pleased with its length. The handouts (masters included at the back of the guide) were not considered visually attractive. They also would have preferred to have all handouts available in the Facilitators' Guide, rather than having some on the website only, since some did not have access or capacity to download accurately from the Q4L site. Facilitators made a number of suggestions, which are summarized below.

Suggestions to improve the Facilitators' Guide:

• Consider the benefits of additional content against possibly jeopardizing the current level of user-friendliness: a guide that is too bulky, all-encompassing and detailed may not be seen as 'accessible'.
• Revise the recruitment section to include suggestions from pilot facilitators
• Include a 'frequently asked questions' section to explain organizational aspects of program implementation in more detail.
• Include a self-assessment quiz for facilitators on teen smoking cessation
• Provide a section with a few ideas on 'ice-breakers'
• Include a list of videos, their availability, and language, with a brief explanation of the content that is covered.
• In addition to the suggested additional activities and 'choice' sessions (see 3.3), include more interactive ideas, games, role plays, quizzes, and 'fun' stuff.
• Include all activities in the guide (currently, some are only available on the website)
• Provide alternatives to 'paper and pencil' activities, especially for students who are less literate and are more 'hands-on' oriented.
• Provide a PowerPoint presentation on health facts.
• Adjust French terminology to ensure consistent use across the country.

5.5.2 Participants' Booklet

Lessons learned about the Participants' Booklet

Students did not provide much detailed feedback on the Participants' Booklet, other than generally stating it was useful. In the focus group discussions, several stated that they liked the booklet, found it useful during the group program, but did not use it much outside of the program or refer to it since. Several facilitators and some students pointed out that the photographs did not reflect the students who are currently smokers and participating in the groups (e.g., too old; too healthy-looking; not 'grungy' enough). Some facilitators commented that writing level was too high, and that a simpler style could be used.

students had an easier time following content that was presented in short bulleted sentences rather than lengthy paragraphs.

Recommendations

• Make images more reflective of the current teenage smoking population.
• Re-design booklet to provide more blank space (doodle or journaling space) for students to write down their comments and 'make the booklets their own'
• Review and lower reading grade level.
• Review style, present content in short, bulleted sentences rather than long paragraphs.

5.4.3 Evaluation Guide

Lessons learned about Evaluation Guide and tools

Facilitators found the Evaluation Guide to be very useful and easy to follow. The evaluation surveys were too long, especially the post-program students survey.

At data analysis, we noted a small number of questions which had not been well understood and can be re-written (see appendix).

Demographic data was only collected in the baseline survey in anticipation that every student would be encouraged to fill out a baseline survey when joining the program. In practice, a number of students only provided a post-program survey. The ethnicity question included in the original pilot surveys was not included in the current version of the tools, limiting interpretation of some demographics.

Currently, the evaluation tools only assess student perceptions about their increase in knowledge and skills; they do not test actual knowledge pre- and post.

Suggestions to improve evaluation guide and tools

(if these tools are to be used again in a more comprehensive evaluation of the program)

• Review questions on smoking habits periodically for consistency with other assessment tools used by Health Canada (e.g., CTUMS, youth smoking survey, profile tools that are in the process of being developed for the website).
• Revise the evaluation surveys to incorporate changes to questions that yielded unclear or inconsistent replies (see appendix)
• Include demographic questions in baseline and post program surveys.
• Consider including a brief pre- and post- program test of student knowledge rather than relying on self-reported perceived knowledge increase.
• Shorten surveys

5.4.4 Website

Lessons learned abut using the Q4L website as part of the group program

The facilitator guide recommends the use of the website throughout the program. Most facilitators were unable to do so, because of lack of access to computers during program hours. At the end of the program only half of the students knew about the Q4L website and only one fifth had used it.

Facilitators who used the Q4L website found it very useful and reported that the students had enjoyed the session in which they accessed the Q4L website. Facilitators commented, however, that students did not find the stories of the four characters on the site to be realistic. Many facilitators also reported that they had downloaded additional background information that they used during the program (e.g., about the patch).

Suggestions to improve use of the Q4L website as part of the group program

• Ensure that the program can be implemented without having to access the website, since not all schools and/or facilitators are able to use a computer.
• Ensure that the website is consistent with the content in the Facilitators' Guide and the Participants' Booklet
• Continue to suggest use of the Q4L website as an option.
• Revise youth stories, and ideally use 'real' life examples.
• Suggest that facilitators inform students about the availability of the web-site.
• Include a section for facilitators on the website, that includes all information from the facilitator and evaluation guide, and additional resources, tips, and links.