Substance Use by Canadian Youth - A National Survey of Canadians' Use of Alcohol and Other Drugs - Canadian Addiction Survey (CAS)

Some of the following hyperlinks are to sites of organizations or other entities that are not subject to the  Official Languages Act. The material found there is therefore in the language(s) used by the sites in question.

Chapter 2: Survey Design, Methodology and Analysis

Sample Design

Specific details on the research design and methods can be found in the CAS detailed national report (Adlaf et al., 2005) and the Canadian Addiction Survey 2004: Microdata eGuide (CCSA, 2004) both of which are available online at the Canadian Centre for Substance Abuse Web site www.ccsa.ca. Presented here is a summary of the general methodology and details specific to the analyses contained in this report.

The CAS is a general population telephone survey based on a two-stage (telephone household, respondent) random sample stratified by 21 regional areas. The sampling frame was based on an electronic inventory (Statplus) of active telephone area codes and exchanges in Canada. Fieldwork for the CAS was conducted by the research firm Jolicoeur et associés. The survey used random-digit-dialling (RDD) methods via Computer Assisted Telephone Interviewing (CATI).

The final unweighted sample consisted of 13,909 interviews, representing an effective response rate of 47.0%. The base sample allocation was for 10,000 completions, 1,000 for each of the 10 provinces. Some provinces purchased additional sample (1,200 in Alberta, 2,000 in British Columbia and 500 in Manitoba). Of the 13,909 Canadians surveyed, 2,085 were youth between the ages of 15 and 24 years. Interviews were conducted between December 16 and December 23, 2003 and from January 9 to April 19, 2004. The median interview time was 23 minutes. The CAS sample represents 24,214,815 Canadians aged 15 or older.

The CAS consisted of over 400 questionnaire items pertaining to the use of alcohol and illicit drugs, patterns of use, harms associated with such use, as well as opinions and attitudes toward substance use and related programs and policies. Questionnaire items were typically drawn from existing national surveys and internationally recognized scales for comparability over time.

A three-panel design was implemented to manage the range of items of interest in the CAS without increasing response burden. Demographic items and questions on prevalence of use and harms were asked of the full sample. Other items such as those examining driving while impaired or reasons for using or not using particular substances were asked only of certain panels. Each respondent was randomly assigned to one of three panels, totalling approximately 4,600 respondents in each panel. Accordingly, the number of cases available for analysis with the "panelized" questions was reduced as were many possible further in-depth analyses of these questions. Table 2.1 presents the breakdown of the panels for youth by the main demographic characteristics examined in this report.

Weighting and Design Effect

The weighting adjustment ensures that weighted CAS distribution compares favourably to census data for sex, age and province. The weights for the CAS sample are based on 252 population classes based on 21 regional strata by six age groups and by sex. The CAS sample tends to under-represent respondents who were never married and had some post-secondary education and over-represent respondents who were married and had a university degree.

As mentioned, the CAS is a complex sampling design, which includes stratification, weighting and multi-stage selection. Such complex survey designs underestimate the variance and the confidence intervals of estimates if assumptions of simple random sampling assumptions are used. In the CAS design, the design effects are primarily influenced by the two-stage selection and the disproportional sampling fractions related to equal provincial allocations. The CAS generally has a design effect of about 3.4, which indicates that the sampling design results in national sampling errors are three times higher than they would be if a simple random sample had been used. All estimates of variances, confidence intervals and related statistical tests are based on Taylor series methods implemented in Stata (Korn and Graubard, 1999; StataCorp, 2003) in order to account for the sample design and design effects.

Precision and Stability

There are two aspects to the statistical quality of survey data: precision--typically measured by the 95% confidence interval (CI), and stability--typically measured by the coefficient of variation (CV). This report follows Statistics Canada guidelines for ensuring the presentation of statistically reliable data. Estimates are evaluated as follows:

CV range Estimate stability 0-16.5 Estimate stable and reportable 16.6-33.3 Estimate has moderate sampling variability and should be interpreted with caution 33.3+ Estimate unstable and is suppressed

In addition to the use of the CV, estimates were also suppressed when the underlying cell sizes were less than 30.

Key Independent Variables

The following variables are commonly used throughout the various chapters in examining predictors of different outcomes for youth. Outcome variables are described in the relevant chapters.

Measure	Categories
Sex	Men; women
Age	3 categories: 15 to 17; 18 to 19; 20 to 24 to examine different outcomes specifically in youth; in addition some comparisons were made between youth 15 to 24, and the total population aged 15+ or adults only aged 25+.
Region	5 regions: used instead of province to maintain sufficient cell/sample sizes for comparisons-- Atlantic, Ontario, Quebec, Prairie and British Columbia.
Income Adequacy	Income adequacy is based on the combination of household income and number of residents in household: lowest, <$20K with 1 to 4 people or <$30K with 5+ people; highest, $60K+ with 1 to 2 people or $80K+ with 3+ people; not reported, did not report income; middle, all other respondents.
Household Location	Rural versus non-rural. Rural is defined by the presence of a "0" in the second character of the respondent's postal code.

Because youth of all ages have not had the equal opportunity to have completed all educational levels, education was omitted from the analyses. In addition, where relevant, age of initiation was also included as an independent variable in the regressions; for instance when looking at alcohol outcomes, age of alcohol initiation was used, and when looking at cannabis outcomes, age of cannabis initiation was used. In cases where it was possible to control for age of initiation, two regressions were conducted: one with only the demographic variables and one also including age of initiation; however, discussion and interpretation of the results was done only for the more detailed regression, that including age of initiation.

In the CAS (2004), a total of 13,909 Canadians, 15 years of age or older, were surveyed; 2,085 (16.5% weighted) of these were youth. Table 2.2 presents the demographic distribution of youth across the main demographic variables of interest and those that were controlled for in many of the analyses included in the youth report. When including the region variable in the analyses for youth, there are some breakdowns that lead to suppressed estimates for Quebec or Ontario. Any significant main effects of region wherein one or more regions has been suppressed should therefore be interpreted with caution. In some cases, additional supplementary analyses have been conducted to deal with this limitation.

Table 2.3 examines how the sample of youth surveyed by the CAS compares with the general population of youth surveyed by Census 2001 in terms of sex, age, region and education. Regarding the breakdown of youth by sex and region, the CAS sample of youth was similar to that of the Canadian census. In terms of age, however, the CAS sample underestimated 15- to 17-year-olds and overestimated 20- to 24-year-olds; in terms of education, the CAS sample underestimated youth with less than a high school education and those with some post-secondary education, and overestimated youth who had completed high school or had a university degree.

Description of Analyses

Analysis of the questions entailed both univariate and multivariate tests. To describe general trends or prevalence issues, univariate tests, more specifically cross-tabulations, were used to examine the distribution of responses to various questions across the major demographic variables of interest (e.g. the proportion of females who drank versus the proportion of males who drank, or differences in the proportion of residents from the British Columbia region who drank compared with the proportion who drank from the Atlantic Region).

It is imperative to go beyond looking at independent variables in isolation (i.e. the cross-tabulation results) in assessing the association between a dependent variable and two or more independent variables (predictors). This is because independent variables are often interrelated to varying degrees. Since the variables of interest are categorical, the method of choice is logistic regression. To determine where differences in the demographic variables lie, multivariate analyses were conducted using logistic regression to examine any differences in the characteristics of respondents who drank, used cannabis or used illicit drugs versus those who did not, or between those who experienced harms from such use compared with those who did not. The independent variables of interest included sex, age, region, household location and income adequacy.

The term "logistic regression" comes from the use of "logit" or transformed "odds" as the dependent variable. If a predictor is significant, it can be interpreted in terms of the direction and size of its odds ratio. An odds ratio greater than 1.0 indicates a greater than average odds, while an odds ratio less than 1.0 indicates a smaller than average odds for the dependent variable. The strength of a significant contribution can be judged by the adjusted odds ratio for a predictor. For odds ratios greater than one, the higher the ratio, the stronger the contribution is, whereas the opposite holds for odds ratios that are smaller than one. When a given predictor is significant, it is interpreted using the adjusted odds. This indicates that the predictor is significant when taking into account (adjusting for) all other predictors.

In addition to general descriptive analyses, in some instances it is important to examine changes in prevalence rates over time, such as differences obtained between the National Alcohol and Other Drugs Survey (NADS, 1989), Canada's Alcohol and Other Drugs Survey (CADS, 1994), and the CAS (2004). Evaluations of trends (i.e. changes from the 1989 NADS, to the 1994 CADS, to the 2004 CAS) were based on differences between confidence intervals. Significance was evident by non-overlapping confidence intervals. This method is crude, but conservative.

Data Limitations

The limitations of the CAS are those common to large telephone-based surveys involving self-report measures (Adlaf et al., 2005). For example, such surveys tend to over-represent those with higher education and under-represent those with lower education.

Telephone surveys assume that everyone in the population lives in a conventional residence with telephone access. However, a small proportion of Canadian households do not have telephones while other groups would not be accessed this way because they are in hospitals, prisons, military establishments or homeless. Nevertheless, since one of the objectives of the CAS was to generate estimates of the prevalence of substance use and abuse for the general population of Canada, the relatively small size of these excluded populations should have minimal effect on the reliability of estimates for the broader population.

Some interviews could not be completed because respondents could not adequately converse in English or French or were too ill or unable to respond.

The CAS deals with a sensitive subject matter--asking people to report behaviours that may not be socially acceptable and possibly even illegal. As a result, it is expected that some under-reporting of such behaviours may occur. In addition, when examining such issues in youth, this tendency may be inflated due to the presence of parental figures nearby and the fact that among under-age youth, even the purchase of alcohol is illegal. However, there is no more efficient way to obtain such information from a sample large enough to be representative of the population of Canada and its 10 provinces (territories not included). Additionally, as noted by Adlaf et al. (2005), while this bias may influence estimates for a single point in time, it likely remains quite stable over time, thus having less of an impact on estimating trends as long as under-reporting remains consistent.

Table 2.1: Number of respondents by panel, youth aged 15-24

	Panel A	B	Panel C
Sex
Male	339	347	353
Female	352	348	346
Age
15-17	201	195	185
18-19	141	143	155
20-24	349	357	359
Region
Atlantic	176	165	190
Quebec	44	53	51
Ontario	48	41	57
Prairie	266	279	265
British Columbia	157	157	136
Household Location
Rural	122	130	112
Non-rural	569	565	587
Income Adequacy
Lowest	100	97	92
Middle	225	213	237
Highest	97	100	116
Not stated	269	285	254

Table 2.2: Demographic distribution of youth surveyed in the CAS (2004)

	Number	Unweighted %	Weighted %
Sex
Male	1,039	49.8	51.1
Female	1,046	20.2	48.9
Age
15-17	581	27.9	25.3
18-19	439	21.0	21.8
20-24	1,065	51.1	52.9
Region
Atlantic	531	25.5	7.6
Quebec	148	7.0	23.6
Ontario	146	7.0	37.5
Prairie	810	38.8	18.3
British Columbia	450	21.6	13.0
Household Location
Rural	364	17.5	12.7
Non-rural	1,721	82.5	87.3
Income Adequacy
Lowest	289	13.9	13.5
Middle	675	32.4	33.8
Highest	313	15.0	17.0
Not stated	808	38.8	35.7
Education
< High school	628	30.1	27.5
Completed high school	704	33.8	33.5
Some post-secondary	572	27.4	28.6
University	174	8.3	10.4
Education 15-17
< High school	475	82.0	78.5
Completed high school	94	16.2	19.0
Some post-secondary	10	1.7	2.5
University	0	0	0
Education 18-19
< High school	65	14.9	16.7
Completed high school	254	58.1	58.0
Some post-secondary	111	25.4	24.5
University	7	1.6	0.8
Education 20-24
< High school	88	8.3	7.6
Completed high school	356	33.5	30.3
Some post-secondary	451	42.5	42.7
University	167	15.7	19.4

 

Table 2.3: Selected demographic characteristics: weighted CAS 2004 versus 2001 Census data, Canada, aged 15-24

	CAS 2004 (N = 2,085)				2001 Canada Census (N = 4,043,877)
Sex
Male	47.3	51.1	54.9		51.1
Female	45.1	48.9	52.7		48.9
Age
15-17	22.2	25.3	28.6	*	31.1
18-19	18.8	21.8	25.2		20.1
20-24	49.1	52.9	56.7	*	48.8
Region
Atlantic	6.8	7.6	8.4		7.8
Quebec	20.7	23.6	26.8		24.0
Ontario	33.7	37.5	41.5		37.2
Prairie	16.8	18.3	20.0		18.2
British Columbia	11.7	13.0	14.3		12.8
Education
< High school	24.3	27.5	30.9	*	42.6
Completed high school	30.0	33.5	37.2	*	15.6
Some post-secondary	25.3	28.6	32.1	*	36.5
University degree	8.2	10.4	13.2	*	5.4

Notes: CAS data refer to lower limit of 95% confidence interval, percentage estimate, and upper limit of 95% confidence interval.

*: indicates census data are not within the bounds of the CAS CI.

Source: Statistics Canada [online]. Available: http://www12.statcan.ca/english/census01/release/index.cfm