Health Canada Research Ethics Board - Researcher Survey 2004 : A Follow-up Summary Report

Help on accessing alternative formats, such as PDF, MP3 and WAV files, can be obtained in the Alternate Format Help section.

Table of Contents

• 1.0 Purpose
• 2.0 Research Approach
  • 2.1 Survey Design
  • 2.2 Sample and Response Rate
  • 2.3 Survey Implementation
• 3.0 Results
  • 3.1 Background Information about Participants
  • 3.2 Preparing the Application -- Documentation and Process
  • 3.3 Preparing the Application -- The Research Ethics Board Secretariat
  • 3.4 The Research Ethics Board Review
  • 3.5 Activities carried out by the Secretariat/Orientation
  • 3.6 Perceived Value of the Ethics Review Process
  • 3.7 Final Thoughts About the Health Canada Research Ethics Review Process
• 4.0 Summary of Results
  • 4.1 High Satisfaction and Agreement Ratings
  • 4.2 Generally Satisfied or in Agreement
  • 4.3 Opportunities for Further Discussion or Action
• 5.0 Conclusion

---

1.0 Purpose

In September 2002, Health Canada established the Research Ethics Board (REB) Secretariat, an independent research ethics board responsible for reviewing all Health Canada research involving human subjects. In 2003, Praxis Research, an independent research and consulting firm, was retained to conduct an independent assessment of the efficiency and effectiveness of the Research Ethics Board and the approval process. Researchers and Board members were surveyed.

In 2004 Praxis Research replicated the survey to assess researchers' perspectives about the REB process during its second year of operation. This report presents the findings of this survey of researchers. Comparisons to 2003 results are also provided.

2.0 Research Approach

2.1 Survey Design

A survey was designed which asked researchers to report about their experiences in the following areas:

• background information;
• preparing the application -- documentation and process;
• preparing the application -- the REB Secretariat;
• review by the Research Ethics Board;
• orientation sessions;
• perceived value of ethics review; and
• overall satisfaction with the review process.

The survey included a combination of closed- and open-ended questions. French and English versions of the survey were prepared.

2.2 Sample and Response Rate

The Health Canada REB Secretariat provided Praxis Research with a contact list of 54 researchers. Five names were removed from the list: three opted out, one reported that she was included on the list in error, and one name had been included twice on the list. The final sample size was 49 researchers who had submitted applications for ethics approval in the past year. Thirty-eight of the 49 researchers completed the survey, resulting in a response rate of 78% (as compared with 80% in 2003).

2.3 Survey Implementation

The survey was administered online. The Health Canada REB Secretariat provided Praxis Research with a list of email addresses for the researchers. An introductory email was sent to them by the Secretariat, followed by an email from Praxis Research which provided a link to the survey and an individual password. The purpose of the password was to ensure the confidentiality of responses and to secure access to the responses. Participants who were not able to complete the survey in one session were able to re-enter the survey using their password and complete it at a later date. The survey was administered online from mid-December 2004 until mid-January 2005.

3.0 Results

Data were analyzed using SPSS (Statistical Package for the Social Sciences). Frequencies are provided for the closed-ended questions. Responses to open-ended questions are presented as themes that emerged or as lists of suggestions provided by the researchers. The results are presented according to the main sections of the survey.

3.1 Background Information about Participants

Participants were asked to provide the current review status of their project. Eighty-nine percent of researchers indicated that approval had been granted, 8% had approval pending, and 3% had their research re-approved for an additional year. None of the researchers indicated that they had completed their research and submitted a termination form or that they were awaiting annual re-approval.

Forty-nine percent of the respondents indicated that their initial application was approved as submitted; 51% were approved with conditions.

The main research classification was "research undertaken in collaboration or partnership with Health Canada," accounting for almost half of all research. As compared with the 2003 survey, a smaller proportion of the research was classified as research carried out on Health Canada premises (8% as compared with 25% in 2003); a greater proportion was research funded by Health Canada Grants and Contributions (16% as compared with 6% in 2003).

Research Classification

• 47% in collaboration with Health Canada
• 16% were contract research
• 16% were funded by Health Canada
• 10% were intramurals
• 8% on Health Canada premises
• 3% involved partnership between a provincial government and industry, with some Health Canada involvement.

Awareness of REB

The majority of the researchers (76% as compared with 58% in 2003) became aware of the REB through communication from senior management or other colleagues. None of the researchers (as compared with 7% in 2003) learned of the REB through Health Canada Broadcast News. 8% were from the university that the researcher attends (2), funding requirement (1), and previous work with IRBs/ECs (1).

3.2 Preparing the Application -- Documentation and Process

Researchers were asked to rate their satisfaction with the clarity of six aspects of preparing the application. The results on the satisfaction with the clarity of...

• Whether research qualifies for full or expedited review, 2.6% very dissatisfied, 10.5% dissatisfied, 18.4% neutral, 44.7% satisfied and 23.7% very satisfied.
• Steps in the process, 2.6% very dissatisfied, 10.5% dissatisfied, 10.5% neutral, 50% satisfied and 26.3% very satisfied
• Five main components of the application package, 0% very dissatisfied, 7.9% dissatisfied, 18.4% neutral, 47.4% satisfied and 26.3% very satisfied
• Which forms need to be completed, 2.6% very dissatisfied, 5.3% dissatisfied, 21.1% neutral, 47.4% satisfied and 23.7% very satisfied.
• Printed resources, 0% very dissatisfied, 13.2% dissatisfied, 13.2% neutral, 52.6% satisfied and 21.1% very satisfied.
• Electronic resources, 0% very dissatisfied, 10.5% dissatisfied, 23.7% neutral, 44.7% satisfied and 21.1% very satisfied.

A strong majority of respondents reported that they were "satisfied" or very "satisfied" with most of the steps in the process. On each of the following variables, satisfaction scores were higher than they were in 2003: "steps in the process" (76.3% as compared with 62.5%); "five main components" (73.7% as compared with 65.6%); "which forms need to be completed" (71.1% as compared with 50.0%), and "printed resources" (73.7% as compared with 68.7%). Respondents remained less satisfied with "clarity about the type of review required" (68.4% as compared with 65.7%) and with "electronic resources" (65.8%, question added in the 2004 survey).

Ten researchers elaborated on their responses about the forms. Almost all of the comments were favourable: Most people reported that the forms were easy to follow and that they encountered no difficulties in completing them. One respondent felt that the forms were less suitable for non-medical and social sciences research; another commented that the forms did not accommodate special ethical circumstances faced by Aboriginal peoples.

A series of questions were also asked about the time it took to obtain information and documents from the Secretariat. The results on the satisfaction with the time it took to obtain...

• Application forms once requested, 2.9% very dissatisfied, 9.9% dissatisfied, 8.8% neutral, 35.3% satisfied and 52.9% very satisfied;
• Reply to questions about application, 5.9% very dissatisfied, 2.9% dissatisfied, 5.9% neutral, 32.4% satisfied and 52.9% very satisfied;
• Notification of additional requirements, 0.0% very dissatisfied, 9.7% dissatisfied, 12.9% neutral, 45.2% satisfied and 32.3% very satisfied;
• Supporting documents requested, 3.0% very dissatisfied, 6.0% dissatisfied, 15.2% neutral, 39.4% satisfied and 36.4% very satisfied.

The results presented above show that three-quarters or more of the researchers were "satisfied" or "very satisfied" with the time it took to obtain application forms once requested, to receive a reply to questions about the application, to obtain notification of additional requirements, and to receive supporting documents. These satisfaction scores are comparable to those obtained in 2003, with one exception: Satisfaction with the time it took to obtain notification of additional documents required to complete the application declined from 86.9% in 2003 to 77.5% in 2004.

3.3 Preparing the Application -- The REB Secretariat

As in 2003, the researchers answered extremely favourably to three questions about the REB Secretariat. All of the respondents indicated that the REB contact person was accessible, all but one reported that the contact person was helpful with answering questions and, all but two indicated that the Secretariat accommodated requests for time sensitive reviews.

Researchers were asked to provide additional written feedback about their experiences with the REB Secretariat. Almost all of the comments were extremely positive: Respondents stated that the contact person was helpful, pleasant, friendly, professional, knowledgeable, and responded promptly to their questions. One person felt that the contact person was not sufficiently knowledgeable about non-medical research. Another commented that it was "annoying to be summoned" by the REB with only five days notice.

3.4 The Research Ethics Board Review

Several questions were asked about presenting in front of the REB. As in 2003, 81% of the researchers reported that they found the opportunity to appear in front of the REB in person or via teleconference helpful. All of the respondents indicated that they had adequate time to discuss their application at the meeting.

Twenty-four researchers offered additional comments. Fifteen people expressed a great deal of satisfaction with the presentation process, describing it as a helpful and positive experience. Several people noted that Board members were very knowledgeable and they appreciated that members had read their applications thoroughly. Although some respondents very much appreciated the option of appearing by teleconference, one person found the experience to be intimidating. Two people were not satisfied with the range of expertise offered by the board; a few respondents questioned the need for REB approval when their projects had already been reviewed and approved by other review bodies.

Researchers were asked to identify how long it took to obtain ethics approval from the REB from the time of the application to the time the decision was communicated by the Board. Thirty-three respondents answered this question. The results shown below indicate that the response time varied across researchers with some receiving approval very quickly (e.g., a few days or one to two weeks) while others waited over six weeks. As compared with 2003, a higher proportion of researchers obtained approval in two weeks or less, but a higher proportion waited more than six weeks.

Length of time it took to obtain ethics approval

• 3 said within a week
• 9 said within 1-2 weeks
• 5 said within 3-4 weeks
• 4 said within 5-6 weeks
• 7 said within 6 weeks
• 5 indicated they couldn't remember

All but two researchers reported that the REB had communicated its decision in a timely and clear manner and had accommodated time-sensitive reviews. Several people added accolades about the effective, timely, and comprehensive communications from the REB; one person added that the Board was "extremely progressive and sensitive to research conducted with Aboriginal peoples." One respondent, however, noted that his project had already been approved by his own institution, and speculated about how one would respond if contradictory opinions or demands were expressed by two review boards.

Researchers were asked to rate their satisfaction with the overall timing/length of the review process. The results on the satisfaction with overall timing/length of review process

• 3% very dissatisfied
• 10% dissatisfied
• 13% neutral
• 37% satisfied
• 37% very satisfied

Seventy-four percent of researchers (as compared with 69% in 2003) were "satisfied" or "very satisfied" with the timing/length of the review process. The 2004 rating was only slightly lower than all of the individual measures of timing which were assessed (see section 3.2 -- Satisfaction with the time it took to obtain..). This finding suggests that, for the most part, the researchers were satisfied with both the timing of specific stages of the process and the time entailed by the overall process (i.e., from start to finish) whereas, in 2003, the researchers were somewhat less satisfied with the speed of the overall process.

Nine respondents offered comments about the overall speed or timing of the review process. Only two people felt that there was a need to speed up the process; an additional four noted that the process timing was either fine or faster than they expected. Two respondents mentioned that it would be helpful not to have a reduced number of meetings over the summer, as many projects take place during these months. One researcher stated that projects which do not include human subjects and which are not ethically sensitive do not require such an extensive review.

3.5 Activities carried out by the Secretariat/Orientation

Only 13% of the respondents (as compared with 19% in 2003) said they had attended an REB orientation session and 16% (as compared with 22% in 2003) indicated they had attended a REB Secretariat short presentation. All of the researchers who had attended an orientation and all but one who had attended a short presentation reported that it was very helpful to them.

The following suggestions were provided regarding activities the Secretariat could undertake to assist Health Canada researchers with research ethics issues:

• Increase awareness about the program
• Offer workshops on conducting research with Aboriginal peoples. Additional training on particular issues in dealing with "Indigenous Knowledge" should be provided to Health Canada staff
• Provide clear definitions as to what constitutes an ethics issue; clarify when the ethics review process is required. Decisions about the level of ethics review required should be made before researchers are requested to complete extensive review forms
• Include a researcher from the professional discipline matching the project for which ethical approval is sought
• Allow the use of both English and French throughout the process

3.6 Perceived Value of the Ethics Review Process

The following shows the frequencies for six statements pertaining to the value of the Health Canada research ethics review process.

The REB approval process

• Is necessary to publish my research, 5.4% strongly disagree, 8.1% disagree, 18.9% neutral, 37.8% agree and 29.7% strongly agree;
• Provides credibility to my research, 5.4% strongly disagree, 13,5% disagree, 10.8% neutral, 37.8% agree and 32.4% strongly disagree;
• Provides protection to human subjects, 2.7% strongly disagree, 5.4% disagree, 24.3% neutral, 32.4% agree and 35.1% strongly agree;
• Provided an independent review, 2.7% strongly disagree, 5.4% disagree, 8.1% neutral, 51.4% agree and 32.4% strongly agree;
• Provides integrity to my research, 5.4% strongly disagree, 10.8% disagree, 24.3% neutral, 29.7% agree and 29.7% strongly agree;
• Raised my level of awareness about ethical issues, 5.4% strongly disagree, 18.9% disagree, 27.0% neutral, 29.7% agree and 18.9% strongly agree.

As compared with 2003, the perceived value of the review process increased on some dimensions and declined on others. Eighty-four percent (as compared with 75% in 2003) of the researchers "agreed" or "strongly agreed" that the REB "provided an independent review of the ethics of their research proposals," and 48.6% of the researchers (as compared with 41.9% in 2003) agreed that "the process raised their level of awareness about ethical issues." There was no significant change with respect to whether the process "provides credibility" (70.2% as compared with 71.9%) or "integrity" (59.4% as compared with 62.5%) to their research. The perceived value of the process declined in 2004 from 2003 on two variables: "the approval process is necessary to publish my research" and "the approval process provides protection to human subjects" (67.5% as compared with 75% for both variables).

Researchers were also asked to rate the overall value of the research ethics review process on their research. The results range from 1 'no value' to 5 'a great deal of value'.

Perceived overall value of ethics review process:

• 1 = 3%
• 2 = 16%
• 3 = 16%
• 4 = 41%
• 5 = 24%

As indicated above, 65% (as compared with 66% in 2003) of researchers indicated that the review process was of some or a great deal of value to them. Nineteen percent (as compared with 26% in 2003) indicated that the process had little or no overall value. Researchers who rated the value as "3" or less were asked to provide their thoughts about how the value of the review process could be improved. As in 2003, the majority of respondents commented that there was no added value in obtaining a second approval when the research had already been approved by another institution or board. Two respondents mentioned that their process had already been approved by a First Nations governing body or elders, and that appearing before the REB was merely a courtesy or a formality.

3.7 Final Thoughts About the HC Research Ethics Review Process

At the end of the survey, researchers were asked to indicate their overall satisfaction with the review process and to comment about opportunities for improvements.

Overall satisfaction with the review process

• 3% Very dissatisfied
• 5% dissatisfied
• 22% neutral
• 46% satisfied
• 24% very satisfied

As indicated above, 70% of the researchers (as compared with 66% in 2003) were "satisfied" or "very satisfied" with the review process. This rating is consistent with the results for some individual variables (e.g., clarity of documents, overall length/timing of the review process, overall perceived value variables), and somewhat higher than the ratings provided for some of the perceived value measures.

Researchers identified the following opportunities for improvements to the research ethics review process:

• Create two distinct processes for surveillance and research involving human subjects
• Offer an expedited review process for low-risk research
• Ensure that at least one REB member has expertise in the subject area of and general methods being used in the research
• Modify the approval process so that the REB is not reviewing research that has already been approved by other review bodies

4.0 Summary of Results

In this section, the results of the survey are summarized according to: 1) areas with extremely high satisfaction and agreement ratings, 2) areas with generally high satisfaction and agreement ratings, and 3) areas that may require further discussion or action.

4.1 High Satisfaction and Agreement Ratings

Seventy percent or more of the researchers were satisfied/very satisfied or agreed/strongly agreed with the following areas:

• in preparing the application, the clarity of the steps in the process, the five main components of the application package, which forms need to be completed, and printed resources
• the timing of stages involved in the process (i.e., to obtain a reply to questions, application forms, notification of additional requirements, supporting documents; time to present application at the Board meeting, and timely communication of review decision)
• the timing and length of the review process overall
• accommodation of time-sensitive reviews
• service from the Secretariat (i.e., knowledgeable, accessible, helpful, responsive and efficient)
• among those few who had attended, the value of orientation sessions and short presentations
• interaction with the Board (i.e., opportunity and experience of appearing in front of the Board, helpfulness of the Board, clear communication of review decision)
• the perceived value of the approval process in terms of providing an independent review
• overall satisfaction with the review process

4.2 Generally Satisfied or in Agreement

Between 50% and 69% of the researchers were satisfied/very satisfied or agreed/strongly agreed with the following areas:

• the clarity of electronic resources and the type of review required
• the approval process as necessary to publish research, providing credibility and integrity to research, and providing protection to human subjects
• overall value of the review process

4.3 Opportunities for Further Discussion or Action

Comments and dissatisfaction/disagreement ratings revealed the following opportunities for further discussion or action:

• consider the fact that fewer than half of all respondents agreed that the approval process raised their level of awareness about ethical issues
• examine whether appearing in front of the Board should be required for all projects
• assess the value of having an additional review for projects that have already received approval from a partner agency/organization
• explore ways of expediting the review process for some types of research projects
• ensure that Board members reviewing the research project include people with expertise in the relevant discipline and methodology

5.0 Conclusion

The purpose of this research was to assess the efficiency and effectiveness of Health Canada's research ethics review process. The results demonstrate that the review process is perceived by researchers to be very efficient and effective in most areas, including

1. the clarity of most aspects of the application process, although electronic communications could be improved;
2. the timing of the steps in the review process and the process overall;
3. the services provided by the REB Secretariat to researchers; and
4. for most researchers, interaction with the Board. Researchers' levels of satisfaction in these areas have remained high and, particularly with respect to the clarity of the process, have notably increased over the past year. In addition, the fact that over three-quarters of the researchers learned of the REB through communication with senior management or colleagues suggests that knowledge about and awareness of the REB within the research community has grown appreciably over the past year. The REB may wish to re-visit the value of communication via Health Canada Broadcast News, as none of the researchers learned of the REB from this information source.

Satisfaction ratings suggest that the effectiveness and efficiency of the ethics review process may be improved by providing additional clarification about whether research qualifies for an expedited review. Confusion about the type of review required was expressed by researchers in 2003, and this does not appear to have changed over the past year. Likewise, some researchers continue to question the value of receiving Health Canada REB approval, particularly if approval has already been obtained from another ethics review body. As noted, the perceived value of the approval process for publishing research and as providing protection to human subjects declined from 2003 to 2004.

Additional opportunities for discussion or action include

1. continuing to explore ways of increasing attendance figures for the orientation sessions and short presentations by the REB Secretariat, and
2. exploring the concerns expressed by a few researchers that the Board did not include members with expertise in the applicant's discipline and proposed research methodology.