Integrating Internet Technology into Physician Quality Improvement Initiatives

Health and the Information Highway Division, Health Canada 2003

Table of Contents

• Executive Summary
• 1. Introduction
  • 1.1 Rationale
  • 1.2 Previous Studies
  • 1.3 Project Overview
  • 1.4 Project Goals
  • 1.5 Project Objectives
    
    
• 2. Methodology
  • 2.1 Sample Population
  • 2.2 Sample Size
  • 2.3 Survey Item Development
  • 2.4 Survey Item Pilot
  • 2.5 Survey Design: Mail-in and Online Approaches
  • 2.6 Survey Administration Method
  • 2.7 Analysis
    
    
• 3. Results
  • 3.1 Response Rates, Ineligible Respondents, and Final Sample Size
  • 3.2 Respondent Demographics
  • 3.3 Survey Item Descriptive Statistics, Correlations, and Predictive Relationships
  • 3.4 Mail vs. Online Respondents - Differences in Use, Needs, Barriers, and Wants
  • 3.5 Non-Computer / Non-Internet Users - Barriers, Needs, and Wants
  • 3.6 Geographic Variation
  • 3.7 Comments
    
    
• 4. Conclusion
  • 4.1 Ontario Physicians are Internet Users
  • 4.2 Internet Use Varies According to Proficiency, Patient Volume, and In-Office Internet Access - Not Age
  • 4.3 Internet Users' Barriers and Needs - Time and Proficiency
  • 4.4 Desired Online Resources
  • 4.5 Use Leads to Confidence
    
    
• 5. Recommendations
  • 5.1 Increase Access to Increase Internet Use in Practice
  • 5.2 Confidence in Security and Privacy of Patient Information
  • 5.3 College of Physicians and Surgeons of Ontario (CPSO) and Online Quality Improvement Resources
  • 5.4 Dissemination Plan
  • 5.5 Next Steps
    
    
• 6. Contact Information
• 7. Project Evaluation
  • 7.1 Evaluation Outline
  • 7.2 Achievement of Objectives
  • 7.3 Non Respondent Analysis
  • 7.4 Response Rate
  • 7.5 Representative Sampling Frame - Provincial Picture
  • 7.6 Representative Results - National Picture
  • 7.7 Survey Method / Administration - Feedback from Respondents
  • 7.8 Timelines, Milestones, and Deliverables
  • 7.9 Budget / Costs - Estimate vs. Actual
    
    
• Reference List
• Appendix 1: Survey Copy
• Appendix 2: Project Milestones and Deliverables
• Appendix 3: Online Survey Application Documentation

Researchers

Principal Investigator:

Elizabeth F. Wenghofer, M.Sc.

• Senior Epidemiologist, Clinical Practice Research Manager, Quality Management Division, College of Physicians and Surgeons of Ontario
• Ph.D. Candidate, Institute of Medical Science, University of Toronto

Investigator:

Dan Faulkner, H.B.Sc., M.B.A.

• Associate Director, Manager of Operations, Quality Management Division, College of Physicians and Surgeons of Ontario

Partners

Dave Davis, M.D., F.C.F.P.

• Associate Dean, Continuing Education, Professor, Departments of Health Policy, Evaluation & Management and Family and Community Medicine, Faculty of Medicine, University of Toronto
• Chair, OMA-MOHLTC Guidelines Advisory Committee

Brian Gamble, M.D.

• Chief of Staff, Chatham-Kent Health Alliance, Chatham, Ontario.

Project Funding

This project was funded through a contribution from the Knowledge Development and Exchange Applied Research Initiative of the Office of Health and the Information Highway, Health Canada. The Quality Management Division of the College of Physicians and Surgeons of Ontario also provided financial support by absorbing all costs associated with administration, printing, dissemination, and project evaluation.

Acknowledgements

The study investigators and partners would like to sincerely thank the participating physicians for their time and interest.

The support and time contributed by Mr. Sidney Biondi, Ms. Raquel Shaw Moxam, and Mr. Jacob Schiff, all of the College of Physicians and Surgeons of Ontario, are thankfully acknowledged.

Many thanks for the assistance and cooperation of the Quality Assurance Committee, Quality Management Division, and Corporate Services Division including Information Technology, Technical Services, and Finance Departments of the College of Physicians and Surgeons of Ontario.

Executive Summary

1) Project overview

Enabling the medical profession to utilize Internet technology to its fullest potential--for quality improvement and other enhancement-related activities--requires the determining of baseline competencies and concerns associated with Internet usage. Internet-based learning tools and resources are easy enough to provide, but they are ineffective if not used. Determining the number of physicians who currently use the Internet, how they use it, why they do not use it, and what they require to help them become Internet users will establish the profession's current level of familiarity with Internet technology, the extent and type of training required, and how the profession feels educational and training issues might best be addressed.

This study consisted of a survey to assess physicians' Internet use and skills, and their needs for encouraging and improving both. Physicians provided their opinions on Internet-based technology for online quality improvement initiatives such as self-assessment, quality evaluation feedback, and patient surveys. The final component of the study examined the concerns that physicians have with security, patient confidentiality, and privacy when using online tools for any practice-related activities.

2) Objectives

i) Do physicians use the Internet? - The proportions and characteristics of physicians in Ontario who currently do / do not have access to and use Internet technology was determined through this objective. In addition, the characteristics of the physicians (e.g., solo practitioner, specialist, gender, age, etc.) who currently use / do not use the Internet were examined so as to determine whether there are patterns in technology usage. This is particularly important for ensuring that Internet education and training can be targeted at those groups with the greatest need.

ii) Why do physicians use / not use the Internet? - The reasons why physicians use the Internet (e.g., for e-mail, medical literature, billing, courses, clinical resources), how often they use it, and their general Internet confidence and knowledge were addressed through this objective. For those individuals who do not use the Internet, this objective focused on determining the barriers (e.g., cost, time, lack of knowledge, security)--perceived or otherwise--that physicians face in using Internet technology.

iii) What do physicians want from Internet resources? - This objective addressed the types of assessment and educational resources (courses, self-assessment tools, patient satisfaction information, and colleague clinical and professional support) that physicians would like to see improved or have available online. Physicians' opinions of their confidence in Internet security for exchange of practice and patient confidential data were also evaluated.

3) Survey population

The sample for this survey was drawn from the College of Physicians and Surgeons of Ontario (CPSO) registry. Inclusion criteria for the study were as follows: (i) practice address in Ontario, (ii) active registrant, (iii) currently practising in Ontario (i.e., not retired, on extended leave, etc.). No sampling restrictions were placed on geography, certifications, area of practice specializations, physician age or gender. A random sample was drawn from all individuals meeting inclusion criteria. A sample of 1 500 was drawn from the approximately 22 000 physicians in the CPSO registry who meet the inclusion criteria.

4) Survey design and administration

The survey items were designed through an iterative process and were to be applicable to both Internet users and non-users, so that barriers and needs in both groups could be examined. Questions were designed so as to allow physicians, including those who would indicate that the Internet has no practice value, does not add to patient care, and that they do not wish to incorporate Internet resources into practice to provide varying opinions of Internet use in practice. Some practice-related information was collected in order to determine if practice structure, patient volume, and hours worked would provide insight into patterns of use, needs, and barriers to Internet use in practice. In addition, perceived proficiencies and general types and volumes of Internet use were also addressed. The final two sections of the survey specifically addressed physician confidence in Internet security and use of the Internet for CPSO activities, including peer assessment.

The survey was administered using a two-streamed survey approach. All subject physicians were initially contacted by mail, at which time they all received a copy of the survey booklet. In the instructions, physicians were invited to participate online and provided with the Internet address and access information for the online version of the survey. Questions in the online version of the survey were identical to those in the mail-in version. In order to ensure that valuable data could be collected from non-computer and / or non-Internet users, it was essential that the survey method did not exclusively utilize an online version of the survey. Reminders were sent seven days, three weeks, and seven weeks after initial contact.

5) Project findings

a) Response Rate and Respondents

A total of 1 483 physicians were eligible for inclusion in the study. The response rate for the survey was 61.4% (911 / 1 483). Approximately 84% (769 / 911) of the responses were received by mail and 16% (142 / 911) were received online.

The age of respondents was normally distributed, with a mean age of 50.8 years. The respondent population comprised 75.3% males and 24.7% females. The demographics of the physicians responding online varied somewhat from those who responded by mail; online respondents were significantly younger by a mean value of 2.4 years and comprised significantly more males.

b) Physician Internet Use

The results of the study clearly show that Ontario physicians do use the Internet. Encouragingly, 77.9% of Ontario physicians indicated that they use the Internet for professional reasons. The majority of Ontario physicians use the Internet for 1 to 10 hours per week, with an average of 40% of that time being spent on professional activities. Although there were slight fluctuations across areas, the total Internet use and Internet time in professional activities did not vary significantly across geographic areas.

A small proportion (10.2%) of the physicians in the study were non-computer and / or non-Internet users. These individuals were significantly older and more isolated (i.e., solo practitioners) than the user population, which many indicate that they are near retirement and unlikely to begin a new approach to practice that includes the Internet at this point in their careers.

c) Types of Online Professional Activities

Physicians indicated that they primarily use the Internet to access medical literature and practice resources such as clinical practice guidelines. Almost 60% of respondents indicated that they use e-mail to communicate professionally with colleagues. A large number of respondents also specified that they use the Internet for a variety of educational activities, including personal continuing medical education (CME) and teaching. Two-thirds of the respondents indicated that they use the Internet for five or more different professional activities.

d) Computer Proficiency

In general, males and younger physicians indicated higher levels of computer proficiency and general use. However, age and gender were not significant factors in overall Internet use or Internet time in professional activities. Those physicians who felt more proficient and had in-office access were also more likely to use the Internet for general and professional activities, regardless of age or gender.

Computer proficiency was a significant predictor of the sum number of barriers and needs indicated by respondents. Age and gender were not predictive of perceived barriers or needs. In-office Internet access was a significant predictor of total and professional Internet activity. If the Internet is not available at the point of service, its application and utility to practice are minimal. The number of patient visits per week was a strong determinant of total Internet use and Internet time in professional activities. Physicians attending to higher patient volumes were less likely to feel proficient or use the Internet for any purpose, including professional activities. Solo practice set-up also appeared to limit Internet use in a similar fashion. Physicians who see high volumes of patients or are in solo practice settings are likely to be pressed for time.

e) Barrier to Professional Use of the Internet

The top three cited barriers to use of the Internet in practice were busy schedule, length of search times, and high volumes of information to sort through online. All three of these barriers are related to time and timely access to information. The needs specified by respondents echoed this theme. Physicians indicated that listings of reliable practice-related web sites would be helpful in gaining timely access to information, and time-effective tools for incorporating Internet into practice would be beneficial in helping them use Internet resources more in their practices.

Respondents who are non-computer / non-Internet users indicated barriers and needs related to a lack of familiarity with computer / Internet technology and to improving computer skills. This group also indicated that time (i.e., busy schedule) is another main barrier to use of the Internet in practice. Gaining Internet skills and experience is difficult to do when there is little time to devote to these activities.

f) Professional Internet Resources Wanted by Physicians

Respondents indicated that access to full-text articles, accredited CME, and prescription information, including online compendiums and drug bulletins are all online resources that they would like to see made available or improved. Access to important patient information such as diagnostic test results, hospital discharge summaries, and consulting / referring physician notes was listed by 60% of physicians as a desired online resource.

g) Online Access to Patient Information

Although 60% of respondents indicated that online access to patient information would be helpful in practice, approximately half of the respondents indicated that they feel that the potential benefits of a provincial electronic medical record (EMR) do not outweigh their concerns with security and privacy of patient information. Those individuals who were generally more confident with online security and the concept of provincial EMR indicated that they are proficient Internet and computer users. Over 30% of respondents indicated that they do not know if the system from which they access the Internet has a firewall, and 67% of respondents indicated that they are not sure / do not know what types of technical security items would increase their confidence in Internet security. The analysis indicated that respondents who knew more about the technical aspect of system security, whether or not their systems actually have that security, were more likely to have a higher level of confidence in the security of information on the Internet. Once again, it appears that computer proficiency, both in terms of actual user skills and technical knowledge, is a key element in promoting use of and confidence in online activities.

6) Conclusions

This study indicates that computer proficiency is a key element in professional Internet use. Knowledge and familiarity build confidence with regard to general computer / Internet proficiency, overall Internet use for professional activities, and exchange of confidential information online. It is clear from the results that the limiting factors associated with proficiency are related primarily to organizational and system-level problems, and less to individual practitioners. Although overall proficiency was also predicted by age and gender, these elements were not predictive of professional use or barriers.

It is recommended that programs and policies designed to increase Internet use in daily practice focus on issues associated with access. The more physicians can use the Internet in practice, the more they will use the Internet in practice. To adequately address these issues, all aspects of access must be considered, including: (i) physician time availability to access the Internet for practice, (ii) Internet access at the point-of-service provision, and (iii) timely access to information online, with regard to searches and information review.

1. Introduction

1.1 Rationale

One of the goals of the College of Physicians and Surgeons of Ontario (CPSO) is to protect the health of the public by assuring that medicine in the province is practised at a "state-of-the-art" level of quality. The focus of the CPSO is shifting, and education is beginning to overtake discipline as the backbone of a more enlightened approach to guiding the medical profession. This new approach is predicated on the position that assessment drives learning. Since the CPSO is pushing education and assessment to the fore, it is important to bring those processes to a state-of-the-art level. To further these objectives, the CPSO supports the integration of Internet technology into several crucial aspects of physician quality improvement, including education, performance evaluation, and patient input.

Enabling the medical profession to utilize Internet technology to its fullest potential--for quality improvement, enhancement or practice-related activities--requires determining baseline competencies and concerns associated with Internet usage. Internet-based learning tools and resources are easy enough to provide, but they are ineffective if not used. Determining the number of physicians who currently use the Internet, how they use it, why they do not use it, and what they require to help them become Internet users will establish the profession's current level of familiarity with Internet technology, the extent and type of training required, and how the profession feels that educational and training issues might best be addressed. Determining the specific ways in which physicians can be encouraged to use Internet-based technology (e.g., technical training, confidence in security and privacy, user-friendly formats) will enable training initiatives to be client-based and user-specific, in order to ensure efficient application of resources and maximal impact.

1.2 Previous Studies

Previous studies have indicated that physicians and medical educators have recognized the potential in Internet-based education and clinical resources.^1-6 Medical schools have started to emphasize the increasing importance and availability of technology as a practice and educational tool,⁷ and ^1;3 Internet usage on a general level among physicians has been steadily increasing.^8-14 There are very few studies which focus on barriers to Internet implementation into daily practice or on specific needs--training or otherwise--to improve integration of Internet resources.

Improving access and availability of physician evaluation and educational tools improves the ability of physicians to incorporate quality improvement into daily practice, where it can directly improve the care they provide to patients. The extensive literature in the area of physician quality improvement indicates that physicians and their patients benefit when physicians regularly participate in quality improvement, and that the largest barrier preventing physicians from participating in quality improvement initiatives is their lack of knowledge and access to quality improvement activities.¹⁵ Quality initiatives and policies to improve clinical performance should be influenced by the needs presented by both the individual practitioners and the system in which they work. The majority of practice evaluation tools are designed to measure a specific aspect of the care that is provided.¹⁶ As a result, no single tool is sufficient for evaluating physician clinical performance, and only a combination of methods will offer reliable evaluation results.^17-19 Comprehensive evaluations using multiple methodologies are both costly and time consuming. The Internet has the potential to maximize the efficiency of both cost and time. Tools can be developed and shared across jurisdictions, which would enable physicians from all provinces, even those in remote areas, to participate in quality improvement activities. Providing reliable, inexpensive, and highly accessible self-assessment, performance evaluation, and patient satisfaction feedback to physicians online will encourage regular use of quality improvement tools. It will also provide physicians with the direction they require for effective CME and continuing professional development (CPD) that translate into improved patient care. However, before any of this potential can be realized, it must first be determined if physicians have the resources, attitudes, and skills required to fully utilize all that the Internet can offer.

1.3 Project overviews

This study consisted of a survey to assess physicians' Internet use and skills, and their needs for encouraging and improving both. Physicians provided their opinions on Internet-based technology for online quality improvement initiatives such as self-assessment, quality evaluation feedback, and patient surveys. The final component of the study examined the concerns physicians have with security, patient confidentiality, and privacy when using online tools for practice-related activities.

1.4 Project goals

To move ahead with the development of online educational and quality improvement tools without ensuring that the profession has the resources to use them effectively could result in a negligible impact of these initiatives on physician quality and, ultimately, on patient care. There is an ever-present need to continually expand and improve the available quality improvement activities for physicians. Although this need will likely never be fulfilled, due to the continually changing landscape of medicine, the long-term goal of this project is the integration of Internet resources, in order to enhance current methods of physician education and quality improvement.

The immediate goals of this survey were to determine baseline physician computer / Internet skills; levels of current use; barriers to use; training or resources required to improve use; and desired online resources, thereby determining the appropriate needs assessment required to establish the most effective path to utilize the potential of the Internet for physician quality improvement initiatives. The Internet training needs of one group of physicians may vary significantly from those of another group. This study aimed to evaluate those differences, so as to ensure that the profession's level of Internet competency rises as a whole. In doing so, this project will have established the priorities for developing a full-scale online physician quality improvement program.

1.5 Project objectives

(1) Do physicians use the Internet? - The proportions and characteristics of physicians in Ontario who currently do / do not have access to and use Internet technology were determined through this objective. In addition, the characteristics of the physicians (e.g., solo practitioner, specialist, gender, age) who are currently using / not using the Internet were examined in order to determine if patterns in technology usage exist. This is particularly important for ensuring that Internet education and training can be targeted at those groups with the greatest need.

2) Why do physicians use / not use the Internet? - The reasons why physicians use the Internet (e.g., for e-mail, medical literature, billing, courses, clinical resources), how often they use it, and their general Internet confidence and knowledge were addressed through this objective. For those individuals who do not use the Internet, this objective focused on determining the barriers (e.g., cost, time, lack of knowledge, security)--perceived or otherwise--that physicians face in using Internet technology.

(3) What do physicians want from Internet resources? - This objective addressed the types of assessment and educational resources (e.g., courses, self-assessment tools, patient satisfaction information, and colleague clinical and professional support) that physicians would like to see improved or have available online. Physicians' opinions of their confidence in Internet security for exchange of practice and patient confidential data were also evaluated.

2. Methodology

2.1 Sample population

The sample for this survey was drawn from the CPSO registry. Inclusion criteria for the study were as follows: (i) practice address in Ontario, (ii) active registrant, and (iii) currently practising in Ontario (i.e., not retired, on extended leave, etc.). No sampling restrictions were placed on geography, certifications, area of practice specializations, physician age or gender. A random sample of 1 500 physicians was drawn from the approximately 22 000 physicians who met the inclusion criteria on the CPSO registry. The accuracy of the CPSO registry is dependent upon the membership reporting any address or practice changes. Therefore, it was expected that some of the individuals who initially met all the inclusion criteria would not be eligible once further information was received from them.

2.2 Samle size

A random sample of 1 124 physicians was required to ensure a +/- 3% margin of error with a 95% level of confidence. An over-sampling was required in order to account for non-responses. A 70% response rate was the target response level; therefore, a sample of 1 462 physicians was required in order to reach the desired level of statistical power. A total of 1 500 physicians were sampled, so as to account for the possibility of ineligibility due to limitations of the CPSO registry, as explained above in Section 2.1.

2.3 Survet Item Developement

The survey was developed to address each of the evaluation outline study objectives. The survey items were designed through an iterative process so as to be applicable to both Internet users and non-users, and so that barriers and needs could be examined in both groups. Questions were designed so as to allow physicians, including those who would indicate that the Internet has no practice value, does not add to patient care, and that they do not wish to incorporate Internet resources into practice to provide varying opinions of Internet use in practice.

Some practice-related information was collected to determine if practice structure, patient volume, and hours worked would provide insight into patterns of Internet use, needs, and barriers in practice. In addition, perceived proficiencies and general types and volumes of Internet use were also addressed.

The final two sections of the survey specifically addressed physician confidence in Internet security and use of Internet for CPSO activities, including peer assessment. All physician demographics, including age, gender, practice address, and certification were derived from the CPSO registry data. The format of the survey questions and layout followed those outlined in the Dillman Method.²⁰ A copy of the survey is provided in Appendix 1.

2.4 Survey item pilot

The survey was pilot-tested with a group of 10 physicians and 5 non-physicians. Project partners were also consulted for their suggestions / comments on the design of the survey items and proposed methodology. Comments obtained from the pilot study were incorporated into the final version of the survey. Specifically, the physicians participating in the pilot process felt that it was especially important to know where physicians access the Internet and how they currently use computers in their practices. Generally, they commented that they felt the survey format was user friendly, not too long or too cumbersome to fill out, and that this study was valuable and important.

2.5 Survey design: Mail-In and Online Approaches

The survey was administered via a two-streamed survey approach. All subject physicians were initially contacted by mail, at which time they all received a copy of the survey booklet. In the introductory letter and survey instructions, physicians were invited to participate online and provided with the Internet address and access information for the online version of the survey. Physicians were able to login to the secure site using their CPSO registration number as their login identification and their date of birth as their password. Date of birth information is not available on the physicians' public profile on the CPSO web site.

In both the survey booklet and the online survey, physicians were reassured that their responses would be kept completely confidential and available only in aggregate form, and that at no time would their individual answers be released.

The online version of the survey had questions identical to those of the mail-in version. In order to ensure that valuable data could be collected from non-computer and / or non-Internet users, it was essential that the survey method did not exclusively utilize an online version of the survey.

2.6 Survey Administration Method

The survey administration closely followed the method outlined by Dillman.²⁰ This method depends on a series of carefully formatted timed reminders and incentives. The survey was mailed to all 1 500 physicians on November 4, 2002. The incentive for responding was entry into a draw for a digital camera package. Physicians were reminded of their entry into the draw in each of the reminders. Seven days after the surveys were mailed, a reminder postcard was sent to the total sample of physicians. Three weeks after the initial mailing, a letter and another survey were sent to all physicians who had not responded to the study (approximately 1 100 physicians). Seven weeks after the initial mailing, another reminder letter was sent to all non-respondents (approximately 765).

2.7 Analysis

Descriptive analyses were performed on all survey items. The effects of physician and practice demographics, including age, gender, certification, and practice organization were evaluated. A series of comparative analyses were conducted using Chi-square and t-test methods. Correlations and predictive models were developed to evaluate the interrelationships among survey items and the relationship between survey items and physician and practice characteristics. Linear regression and logistic regression analyses were used to develop the predictive models that account for the effects of covariates. For ease of presentation, the details of the statistical tests performed are provided in the Results section of this report, in conjunction with the results of each test.

3. Results

3.1 Response rates, ineligible respondents, and final sample size

The sampling frame comprised 1 500 physicians. A total of 928 responses were received. As mentioned previously, the accuracy of the CPSO registry is dependent upon the information reported by its membership; of the 928 responses, 17 physicians indicated that although they still maintain an active licence, they are fully retired and are no longer seeing patients, or that they no longer maintain a practice address in Ontario. These 17 physicians were ineligible for the study and as a result, the total sample was 1 483 (N = 1 483) and a total of 911 responses were received. A response rate of 61.4% (911 / 1 483) was achieved. Mailed-in responses accounted for 84.4% (769 / 911) of the total responses and 15.6% (142 / 911) of responses were received online.

3.2 Respondent demographics

The age of respondents was normally distributed, with a mean age of 50.8 years. The minimum age was 27 years and the maximum was 86 years. The respondent population comprised 75.3% males and 24.7% females. Slightly over half (50.8%) of the respondents were RCPSC certified (i.e., specialists) and 49.2% of the remaining respondents practise as family / general practitioners. Of those family / general practitioners, 46.2% had College of Family Physicians of Canada certification (CFPC). Respondents practise in 147 cities across Ontario, with approximately 32.1% of respondents practising in Area 8--the Municipality of Metropolitan Toronto--and 5.9% practising in Area 9--Northern Ontario.

Practice Area

		Frequency	Percent Valid	Percent
Valid	1	36	4.0	4.0
	2	73	8.0	8.1
	3	52	5.7	5.8
	4	102	11.2	11.3
	5	125	13.7	13.8
	6	46	5.0	5.1
	7	121	13.3	13.4
	8	290	31.8	32.1
	9	59	6.5	6.5
	Total	904	99.2	100.0
Missing System		7	.8
Total		911	100.0

Demographics of online respondents varied somewhat from those who responded by mail. T-tests indicated that online respondents were significantly younger by a mean value of 2.4 years than mail-in respondents (p = .025). Chi-square tests indicated that there were significantly fewer female respondents (18.3% females) in the online group than in the mail-in cohort (25.9% females) (p = 0.033). No significant differences were found between mail-in and online respondents with regard to CFPC, Royal College of Physicians and Surgeons of Canada (RCPSC) certification, or solo practice set-up.

3.3 Survey item descriptive statistics, correlations, and predictive relationships

Response summaries, descriptive statistics, and frequency distributions for each section and question in the survey are presented below in tabular and graphical format. Group differences calculated through Chi-square tests and t-tests, and linear and logistic regression analyses to evaluate predictive relationships are all presented in this section.

3.3.1 Q-1 - Frequency Distribution: Which best describes your practice set-up?

Approximately 48% of respondents indicated that they practice in a solo setting. Administrative group practices where costs, overhead, staff, etc. are shared were found in 18.3% of respondents' practices. Practices where costs, overhead, staff, etc. are shared and physicians attend to and / or over each other's patients were found in approximately one-third of all respondents' practices. Solo practitioners were significantly older than group practitioners by an average of 5 years (p<.001). Approximately 80% of solo practitioners and 70% of group practitioners were male (p<.001).

3.3.2 Q-2 - Frequency Distribution: How many physicians are there at your primary practice location?

Physicians who indicated that they work in very large institutions positively skewed the distribution of the number of physicians at each primary practice location. The median value offers a better representation of the actual number of physicians per practice location. Half of the respondents indicated that there are 4 or fewer physicians at their primary practice location (median = 4).

Statistics
Number of Physicians
N	Valid	834
	Missing	77
Mean		12.64
Median		4.00
Mode		1
Std. Deviation		36.09
Minumun		1
Maximum		500

3.3.3 Q-3 - Frequency Distribution: In an average week, please estimate the (i) number of hours you work and (ii) number of patients you see:

The average number of hours worked per week and the average number of patient visits per week were approximately 47 and 109, respectively. Several of the physicians who reported seeing between 300-500 patients per week indicated that they are radiologists or pathologists, and these values represent films read per week or tests interpreted per week, rather than direct patient visits per se. In addition, physicians who reported zero patients per week indicated that they are not in direct patient contact (e.g., as in the fields of pathology or radiology) or are in non-clinical practice such as administrative medicine.

Statistics
		Hours per Week	Patients per Week
N	Valid	869	811
	Missing	42	100
Mean		46.850	108.848
Median		50.000	100.000
Mode		60.0	100.0
Std. Deviation		16.980	73.463
Minimum		4.0	.0
Maximum		110.0	500.0

Male respondents reported significantly more work hours per week than did female respondents, with work weeks of 48.5 hours and 41.5 hours, respectively. Male respondents also reported significantly more patient visits per week than did female respondents, with mean values of 115 and 87 patients per week, respectively. Age was significantly correlated with both hours worked and patient visits; i.e., both decrease with increasing age.

3.3.4 Q-4 - Frequency Distribution: How would you rate your general computer proficiency?

The majority of physicians (76.7%) indicated feeling either somewhat proficient or proficient in their own computer skills. Physicians who indicated that they do not feel proficient comprised 18.3% of the respondents, and a small proportion (5%) felt that their computer skills are at an expert level.

3.3.5 Computer Proficiency and Practice Set-up

Chi-square tests indicated that solo practitioners feel significantly less proficient than do those in administrative or clinical group practices (p<0.004). Only 14.7% of group practitioners indicated that they feel not at all proficient, whereas 21.2% of solo practitioners do not feel proficient in their own computer skills.

Linear regression analysis was used to determine if specific physician personal characteristics were predictive of perceived computer proficiency. Independent variables entered into the model included: hours worked per week; patient visits per week; in-office Internet access; solo practice setting; age; gender; hospital privileges; and certification.

Results showed that access, age, and gender were all significant predictors of computer proficiency. Physicians who are younger, male, and have in-office access to the Internet indicated that they are computer proficient. Standardized regression coefficients (Betas) indicated that access is the strongest contributor to the model.

Coeffinients^a

	Model	Unstandardized Coefficients		Standardaized Coefficients	Sig.	t
		B	Std. Error	Beta
a Dependent Variable: Computer Proficiency
1	(Constant)	2.612	.156		16.734	.000
	SOLO PRACTITIONER	-8.842E-02	.053	-.055	-1.667	.096
	HospPriv	-4.915E-02	.061	-.026	-.803	.422
	RCPSC	3.027E-02	.059	.019	.510	.610
	Percent Males	.433	.064	.230	6.715	.000
	Age	-1.620E-02	.002	-.232	-6.619	.000
	Access Location OFFICE	.493	.054	.305	9.060	.000
	Hours per week	9.323E-04	.002	.020	.550	.583
	Patients per Week	-7.655E-04	.000	-.070	-1.876	.061

3.3.6 Q-5 - Frequency Distribution: Describe the level of computerization in your practice:

The vast majority of physicians (84.3%) indicated that they perform billing activities by computer. Scheduling and follow-up are computerized in the practices of 43.2% of respondents. Electronic medical records are used in 24.2% of practices. Only 6.2% of physicians indicated that they have no computerization within their practice.

3.3.7 Computerization in Practice and Practice Set-up

Practitioners who are not using computers in practice work primarily in solo practice settings. Approximately 9% of all solo practitioners indicated that they have no computers within their practice. Billing and reconciliation functions are computerized in over 80% of all practices, regardless of practice set-up. Electronic medical records are found in over 30% of group practices and in fewer than 15% of solo practices.

3.3.8 Q-6 - Frequency Distribution: How many hours per week would you estimate you use the Internet for personal or professional reasons (including sending and receiving e-mail)?

The majority of physicians use the Internet and only 6.9% indicated that they do not participate in any Internet activities. The majority of physicians (63.1%) indicated that they use the Internet between 1-10 hours per week. Physicians who indicated that they use the Internet less than one hour per week or more than 11 hours per week comprised 17.4% and 12.6% of the respondents, respectively.

3.3.9 Q-7 - Frequency Distribution: From which location do you access the Internet?

The vast majority of respondents (93.2%) indicated that they access the Internet from home, and a significantly smaller proportion (51.7%) indicated that they have Internet access in their office. Other specified locations included: hospital, mobile locations (e.g., hotels, while travelling), university, and cottage / vacation settings.

3.3.10 Relationships of Total Internet Use, Professional Internet Use, Practice Set-up with Location of Internet Access

In-office Internet access was found in 45.7% of solo practices and in 56.4% of group settings. Significant positive correlations (p<.001) were found between total number of Internet access locations and professional use of the Internet. All respondents who indicated that they use the Internet less than 1 hour per week have home-only access. No significant differences were found between total Internet use and gender. On average, female respondents use the Internet more for professional activities (46.3%) than do male respondents (42.0%), but the differences were not significant.

Linear regression analyses were conducted to determine if physician and practice characteristics were predictive of total Internet use. Results indicated that 38.3% (R-Square = .383) of the variance in total Internet use could be explained by the model, which included: hours worked per week; patient visits per week; age; gender; hospital privileges; solo practice setting; certification; computer proficiency; and in-office access. Patients per week, computer proficiency, and in-office access were all significant predictors; physicians with fewer patients, better computer skills, and Internet in their office use the Internet more. Computer proficiency (Beta = .462) and in-office Internet access (Beta = .163) were the strongest predictors in the model.

Coeffinients^a

	Model	Unstandardized Coefficients		Standardaized Coefficients	t	Sig.
		B	Std. Error	Beta
1	(Constant)	.944	.249		3.78	.000
	Hours per Week	-4.915E-04	.002	-.007	-.224	.823
	Patients per Week	-2.379E-03	.001	-.153	-4.705	.000
	Age	-6.005E-03	.003	-.059	-1.791	.074
	Percent Males	.106	.087	.040	1.215	.225
	SOLO PRACTITIONER	-7.157E-02	.069	-.031	-1.032	.302
	HospPriv	7.9070E-02	.081	.030	.982	.327
	Certified Specilist (CFPC or RCPSC)	8.874E-02	.080	.035	1.107	.268
	Computer Proficiency	.666	.047	.462	14.065	.000
	Access Location - OFFICE	.371	.075	.163	4.962	.000
a Dependent Variable: Internet Hours per Week

3.3.11 Q-8 - Frequency Distribution: What would you estimate your total Internet usage time is for personal (e.g., banking, e-mail, booking trips, general interest, etc.) and professional reasons (e.g., communicating with patients / colleagues, literature searches, obtaining drug information, etc.)?

Statistics

		Percent Internet Use - PERSONAL	Percent Internet Use - PROFESSIONAL
N	Valid	746	746
	Missing	165	165
Mean		56.84	43.07
Median		60.00	40.00
Mode		50	50
Std. Deviation		29.87	29.83
Minimum		0	0
Maximum		100	100

On average, respondents indicated that 43.1% of their Internet time is spent engaging in professional activities and 56.8% is spent engaging in personal activities. The most common balance between personal and professional activities was a ratio of 60% personal to 40% professional.

3.3.12 Work Load, Computer Proficiency, and Professional Internet Use

Correlation coefficients were calculated between number of hours worked per week, number of patient visits per week, computer proficiency, total Internet hours per week, and percent of Internet use for professional activities. Significant positive correlations (p<.05) were found between hours worked per week, computer proficiency, total Internet hours per week, and professional Internet use. Conversely, significant negative correlations (p<.01) were present between number of patient visits per week and total and professional Internet use.

Correlations

		Hours Worked per Week	Patient Visits per Week	Computer Proficiency	Internet Hours per Week	Percent Internet Use - PROFES-SIONAL
Hours Worked per Week	Pearson Correlation	1.000	.222**	.142**	.072*	.143**
	Sig (2-tailed)	-	.000	.000	.041	.000
	N	871	807	868	800	731
Patient Visits per Week	Pearson Correlation	.222**	1.000	-.065	-.226	-.228
	Sig (2-tailed)	.000	-	.116	.000	.000
	N	807	811	808	744	676
Computer Proficiency	Pearson Correlation	.142**	-.055	1.000	.559**	.193**
	Sig (2-tailed)	.000	.116	-	.000	.000
	N	868	808	893	822	745
Internet Housr per Week	Pearson Correlation	.072*	.226**	0.559**	1.000	.272**
	Sig (2-tailed)	.041	.000	.000	-	.000
	N	800	744	822	824	746
Percent Internet Use - PROFESSIONAL	Pearson Correlation	.143*	-.228**	.193**	1.272**	1.000
	Sig (2-tailed)	.000	.000	.000	.000	-
	N	731	676	745	746	746

** .Correlation is significant at the 0.01 level (2-tailed).
* .Correlation is significant at the 0.05 level (2-tailed).

3.3.13 Internet Time in Professional Activity - Calculated Indicator

As neither total Internet hours nor percent of total Internet time spent in professional activities independently provided an adequate picture of the practice-related time spent online, an indicator variable was created to provide a better indication of what 90% of time in professional online activities means in terms of hours. The indicator was created by multiplying the estimated level of Internet hours per week (Q-6) by the estimated percentage of Internet time spent in professional activities. This created an indicator that accounted for the Internet time spent in professional activities rather than just total Internet time or percentage of professional activities alone.

Linear regression analyses indicated that 33% (R-Square = .330) of the variance in the Internet time in professional use indicator was predicted by the independent variables computer proficiency, solo practice set-up, in-office Internet access, hours worked per week, and patient visits per week. As with overall Internet use, physicians who see fewer patients, feel more proficient, and have in-office access were more likely to have more Internet time in professional activities. Unlike total Internet hours, the indicator of Internet time in professional activities was also predicted by hours worked per week and solo practice set-up. Computer proficiency and in-office Internet access played the largest roles in the predictive model, with standardized regression coefficients of Beta = 0.281 and Beta = 0.271, respectively.

Coeffinients^a

	Model	Unstandardized Coefficients		Standardaized Coefficients	t	Sig.
		B	Std. Error	Beta
1	(Constant)	-9.758	22.919		-.426	.670
	Age	.405	.309	.045	1.310	.191
	Percent Males	-10.483	8.024	-.045	-1.306	.192
	Certified Specialist (CFPC or RCPSC)	2.398	7.359	.011	.326	.745
	Computer Proficiency	35.978	4.399	.281	8.179	.000
	SOLO PRACTITIONER	-15.300	6.404	-.077	-2.389	.017
	Access Location - OFFICE	53.751	6.862	.271	7.833	.000
	Hours per Week	.683	.205	.114	3.236	.001
	Patients per Week	-.314	.047	-.231	-6.737	.000
	HospPriv	1.966	7.476	.008	.263	.793
a Dependent Variable: Internet time in Professional Activities

3.3.14 Q-9 - Frequency Distribution: If you use the Internet for professional reasons, please indicate the types of professional activities in which you employ Internet resources:

Online researching and reviewing medical literature represent the most common professional use of the Internet, employed by 81% of the respondents. The second most common professional use of the Intent is reviewing medical information resources such as clinical practice guidelines (79.4%). Approximately 59% of respondents indicated that they communicate with colleagues electronically using the Internet. Approximately 9% of respondents indicated some 'other' Internet uses. The 'other' activities respondents specified included: continuing medical education; communication with specialty and professional associations; conference information; diagnostic test interpretation / down-loading of images; consulting; and teaching.

3.3.15 Descriptive Statistics - Calculated Indicator for Total Number of Professional Internet Activities Indicated per Respondent (SUM USE)

An indicator was developed for each physician's total number of Internet professional activities (Q-9) by calculating the sum count of the items indicated. This indicator is referred to as Sum Use.

Statistics
SUM USE
N	Valid	710
	Missing	201
Mean		3.854
Median		4.000
Mode		3.0
Std. Deviation		1.783
Minimum		1.0
Maximum		9.0

Approximately two-thirds of respondents (66.2%) indicated that they use the Internet for four or fewer professional activities.

3.3.16 Q-10 - Frequency Distribution: Please indicate what you feel are your biggest barriers to integrating the Internet with your own daily professional activities:

A busy schedule, indicated by 56.7% of respondents, was the most commonly cited barrier to integrating the Internet into daily professional activities. The length of time that is required to search the Internet for desired information and the high volumes of information to sort through were the second and third most common barriers, cited by 52.8% and 46.1% of respondents, respectively. Over half of the respondents (51.8%) specified some 'other' barrier. The majority of the 'other' responses were comments referring to and expanding upon the choices that are provided in the survey question (e.g., no computer skills, too slow, not interested, etc). Some of the specified barriers that did not overlap with the original selection provided in the question included: learning curve / change in practice approach; technical maintenance / breakdowns; lack of physical access to the Internet in all practice locations; no remuneration for set-up costs / maintenance; viruses; lack of infrastructure for integration of data between locations; lack of computer literacy / willingness / buy-in of colleagues; and legal implications.

3.3.17 Descriptive Statistics: Calculated Indicator for Total Number of Barriers to Integrating the Internet with Daily Professional Practice Indicated per Respondent (SUM BARRIERS)

An indicator was developed for each physician's total to number of barriers (Q-10) by calculating the sum count of the items indicated. This indicator is referred to as Sum Barriers.

Statistics
SUM BARRIERS
N	Valid	825
	Missing	86
Mean		3.377
Median		3.000
Mode		3.0
Std. Deviation		1.850
Minimum		1.0
Maximum		10.0

Approximately half of all respondents identified three or more barriers to integrating Internet technology into daily practice activities.

3.3.18 Q-11 - Frequency Distribution: Please indicate which of the following you think might help you use the Internet more in your practice:

Listings of reliable practice-related web sites was the most common need cited by 56.5% of respondents. Internet courses specifically designed for physician by physicians and time-effective tools for incorporating Internet resources into practice were listed as helpful by 49.1% and 41.4% of respondents, respectively. Some of the 'other' needs specified by respondents included: improved access; decreased overall costs (start-up, maintenance, etc.); fewer 'restricted practice sites' (i.e., membership requirements, registrations, access fees, logins, etc.); more physicians to 'come on board'; changes to fee schedule; and systems integration at a provincial level.

3.3.19 Descriptive Statistics - Calculated Indicator for Total Number of Needs Indicated per Respondent to Help Improve Internet Use in Practice (SUM NEEDS)

An indicator was developed for each physician's total number of needs to get Internet into practice (Q-11) by calculating the sum count of the items indicated. This indicator is referred to as Sum Needs.

Statistics
SUM NEEDS
N	Valid	765
	Missing	146
Mean		3.979
Median		4.000
Mode		4.0
Std. Deviation		2.525
Minimum		1.0
Maximum		11.0

Approximately one-third of respondents indicated five or more needs to help improve their Internet use in daily practice.

3.3.20 Q-12 - Frequency Distribution: Please indicate the types of resources that you would like to see made available, improved or expanded on the Internet to assist you in your professional activities:

Access to full-text articles and important study results was the most common desired Internet resource, wanted by 70.1% of respondents. Access to important patient information was indicated as a wanted Internet resource by 60% of respondents. Approximately 54% of respondents indicated that they would like online prescription information such as an online compendium or drug bulletins, and 53% indicated that accredited CME is desirable.

Some of the 'other' desired Internet resources specified by physicians included: drug interaction monitoring / programs; standard reference texts; and disease management tools.

3.3.21 Descriptive Statistics - Calculated Indicator for Total Number of Internet Resources Wanted for Professional Activities Indicated per Respondent (SUM WANTS)

An indicator was developed for each physician's total number of desired Internet resources (Q-12) by calculating the sum count of the items indicated. This indicator is referred to as Sum Wants.

Statistics
SUM WANTS
N	Valid	P
	Missing	76
Mean		4.207
Median		4.000
Mode		3.0
Std. Deviation		2.369
Minimum		1.0
Maximum		11.0

Approximately 40% of respondents indicated five or more resources that they would like to see available on the Internet.

3.3.22 Correlations of Individual Physician Characteristics and Use, Barriers, Needs, and Wants

Indicators of sum use, sum barriers, sum needs, and sum wants were correlated with respondent age, gender, certification (including CFPC or RCPSC), and solo practice. Significant negative correlations were found between sum use, sum wants, and age (p<.001). Certified respondents were correlated with increased number of professional Internet activities, and solo practice was negatively correlated with sum use. No significant correlations were found between gender and sum use, sum barriers, sum needs or sum wants.

3.3.23 Predictive Models for Barriers, Needs, and Wants

Linear regression analyses were conducted to evaluate if sum barriers, needs, and wants could be predicted from certain physician characteristics. Independent variables included in the models were: (i) age, (ii) gender, (iii) in-office access to Internet, (iv) computer proficiency, (v) certified respondent (CFPC or RCPSC), (vi) solo practitioner, (vii) Internet time in professional activities, (viii) hours worked per week, (ix) number of patient visits per week, and (x) hospital privileges (y / n).

The regression models for predicting sum barriers, needs, and wants were significant (p<.001); however, the variability explained by the independent variable for each model was low. R-Square values, unstandardized (B), and standardized (Beta) regression coefficients and significance levels are presented.

Coeffinients^a

		Unstandardized Coefficients		Standardaized Coefficients
	Model	B	Std. Error	Beta	t	Sig.
1	(Constant)	5.148	.545		9.443	.000
	Age	-1.38E-02	.007	-.085	-1.892	.059
	Percent Males	2.417E-02	.185	.006	.131	.896
	Certified Specialists (CFPC or RCPSC)	-7.11E-02	.173	-.018	-.412	.681
	Computer Proficiency<</p> /td>	-.465	.107	-.198	-4.363	.000
	SOLO PRACTITIONER	.111	.149	.031	.745	.456
	Access Location - Office	.118	.164	.033	.717	.474
	Hours per Week	-1.80E-03	.005	-.016	-.363	.717
	Patients per Week	1.253E-03	.001	.052	1.128	.260
	HospPriv	.197	.172	.048	1.145	.253
	Indicator - Internet time in Professional Activities	-8.721E-04	.001	-.046	-.957	.339
a Dependent Variable: SUM BARRIERS

The model predicting sum number of barriers was significant and explained 5.3% of the variance (R-Square = .053). The total number of barriers indicated by respondents was predicted by computer proficiency; i.e., fewer barriers were found with increasing proficiency.

Coeffinients^a

		Unstandardized Coefficients		Standardaized Coefficients
	Model	B	Std. Error	Beta	t	Sig.
1	(Constant)	5.595	.760		7.364	.000
	Age	1.581E-03	.010	.007	.155	.877
	Percent Males	4.077E-02	.258	.007	.158	.874
	Certified Specialists (CFPC or RCPSC)	-.224	.241	-.040	-.932	.352
	Computer Proficiency	-.957	.149	-.287	-6.436	.000
	SOLO PRACTITIONER	.166	.208	.033	.798	.425
	Access Location - Office	-8.58E03	.229	-.002	-.037	.970
	Hours per Week	1.155E-03	.007	.007	.167	.867
	Patients per Week	1.8134E-03	.002	.053	1.171	.242
	HospPriv	.259	.239	.044	1.082	.279
	Indicator - Internet time in Professional Activities	1.931E-03	.001	.072	1.522	.129
a Dependent Variable: SUM NEEDS

Computer proficiency was the only significant predictor in the model for total needs indicated by respondents. Similarly to the total number of barriers, as computer proficiency increased, the total number of needs indicated by respondents decreased.

Coeffinients^a

		Unstandardized Coefficients		Standardaized Coefficients
	Model	B	Std. Error	Beta	t	Sig.
1	(Constant)	4.497	.697		6.452	.000
	Age	-3.04E-02	.009	-.143	-3.247	.001
	Percent Males	-6.24E-02	.237	-.012	-.264	.792
	Certified Specialists (CFPC or RCPSC)	-1.44E-02	.221	-.003	-.065	.948
	Computer Proficiency	.384	.136	.125	2.818	.005
	SOLO PRACTITIONER	3.148E-02	.191	.007	.165	.869
	Access Location - Office	-.142	.210	-.031	-.679	.498
	Hours per Week	1.522E-03	.006	.010	.240	.810
	Patients per Week	1.004E-03	.001	.032	.707	.480
	HospPriv	.125	.219	.023	.570	.569
	Indicator - Internet time in Professional Activities	4.479E-02	.001	.181	3.846	.000
a Dependent Variable: SUM WANTS

Computer proficiency, age, and Internet time in professional activities were all significant predictors of the total number of desired Internet resources indicated by respondents. Increasing number of wanted Internet resources was predicted by increasing proficiency, increasing Internet time in professional activities, decreasing age, and decreasing number of physicians at the primary practice location. Standardized regression coefficients indicated that Internet time in professional activities (Beta = .181) and age (Beta = -.143) played the largest roles in the predictive model.

The specific barriers indicated by physicians varied across differing levels of proficiency, further indicating that barriers were not uniform across all respondents. Physicians who are less proficient indicated barriers directly associated with Internet skills.

The pattern of needs indicated by physicians also varied across computer proficiency levels, further indicating that physicians with varying levels of proficiency have different needs.

3.3.24 Predictive Models for "Internet Adds No Value to Practice" - Calculated Indicator

An indicator was developed to identify those respondents who indicated that the Internet (i) Does not add value to patient care (Q-10), (ii) Prefer not to incorporate the Internet into practice (Q-11), (iii) Internet can not be incorporated into practice effectively (Q-11) or (iv) Internet adds no value to practice (Q-11). This 'No Value' indicator was dichotomous, identifying those who indicated 'yes' to any of the above responses.

Logistic regression analysis was used to determine if specific physician characteristics were indicative of physicians who felt that the Internet adds no value to practice. The same independent variables were used in the logistic regression model as in the linear regression models predicting sum barriers, sum needs, and sum wants.

The logistic regression model was significant and the total variance explained by the model was modest (Nagelkerke R-Square = .141). Physicians in solo practice settings, with lower indicators of Internet time in professional activities were more likely to feel that the Internet adds no value to practice and patient care.

3.3.25 Q-13 - Descriptive Statistics: How confident do you feel that the Internet can be a secure means of exchanging confidential information for the following purposes:

Confidence was ranked on a five-point scale, where 1 indicated least confident and 5 indicated most confident.

Respondents indicated that they feel most confident that the Internet can be a secure means of submitting billing information, with an average confidence score of 3.62. They were least confident in the use of the Internet for shopping and exchanging confidential patient information using EMR.

3.3.26 Descriptive Statistics - Calculated Indicator for Average Security Across All General Internet Activities An indicator reflecting general confidence in security (General Activity Confidence) was calculated for each respondent by determining the mean value of responses for each item in Q-13.

Statistics
GENERAL ACTIVITY CONFIDENCE
N	Valid	866
	Missing	45
Mean		3.116
Median		3.167
Mode		3.0
Std. Deviation		.992
Minimum		1.0
Maximum		5.0

The average level of confidence across all general activities, including: shopping; banking; personal e-mail; professional e-mail; billing; and EMR was calculated per respondent to create a general activity confidence indicator. The mean general activity confidence was normally distributed, with a mean value of 3.116.

3.3.27 Q-14 - Frequency Distribution: Please indicate which security items below would help to increase your level of confidence in security and privacy of Internet usage for transfer of confidential patient information:

Over two-thirds of respondents indicated that they do not know what types of security features would help to increase their level of confidence in Internet security for the exchange of confidential patient information. The majority of the 'other' features listed by respondents were primarily commentaries addressing the fact that there is nothing that can increase their level of confidence in Internet security to the level where they would feel confident putting patient information online (i.e., can never have complete security). A few of the respondents indicated that their confidence in security can be improved with digital IDs, very high bit encryption, VPN use, biometric access, and continual monitoring / updating of security.

3.3.28 Q-15 - Frequency Distribution: Do you have a firewall on the computer that you use most frequently to access the Internet?

Fire Wall
		Frequency	Valid Percent	Cumulative Percent
Valid	No	247	28.3	28.3
	Yes	339	38.9	67.2
	Don't Know	286	32.8	100.0
	Total	872	100.0
Missing	System	39
Total		911

Firewalls are used in 38.9% of the systems respondents use to access the Internet. Approximately 60% of respondents access the Internet without a firewall or do not know if their system has a firewall.

3.3.29 Q-16 - Frequency Distribution: Do you feel that the potential benefits associated with electronic medical records (e.g., province-wide access to patient records, regardless of location of treatment) outweigh any potential concerns with security and privacy?

EMR Benefits
		Frequency	Valid Percent	Cumulative Percent
Valid	No	414	49.1	49.1
	Yes	429	50.9	100.0
	Total	843	100.0
Missing	System	68
Total		911

The respondents were almost equally split in their feelings on the potential benefits of EMR outweighing any potential concerns with security and privacy, with 50.9% indicating that the benefits do outweigh concerns and 49.1% indicating that they do not.

3.3.30 Q-17 - Frequency Distribution: Would you feel confident accessing medical records electronically in your practice if you were provided appropriate assurances of the security of the patient information?

EMR in your practice

		Frequency	Valid Percent	Cumulative Percent
Valid	No	187	22.1	22.1
	Yes	661	77.9	100.0
	Total	848	100.0
Missing	System	63
Total		911

When asked if they would feel confident accessing medical records electronically in their own practice if security assurances were satisfactory, 77.9% of respondents indicated yes.

3.3.31 Q-18 - Descriptive Statistics: How confident would you feel exchanging confidential information with the following organizations? -Your Personal / Practice Information

Confidence in Exchanging Practice Information

	N
	Valid	Missing	Mean	Median	Std. Deviation
Patient Info - OMA	781	130	3.24	3.00	1.26
Patient Info - MOHLTC	781	130	2.94	3.00	1.32
Patient Info - Health Canada	748	163	2.88	3.00	1.28
Patient Info - CMA	766	145	3.22	3.00	1.24
Patient Info - CPSO	785	126	3.24	3.00	1.33
Patient Info - RCPCS	674	237	3.27	3.00	1.28
Patient Info - CFPC	596	315	3.21	3.00	1.30
Patient Info - OHA	653	258	2.70	3.00	1.28
Patient Info - Hospitals	752	159	2.92	3.00	1.28
Patient Info - Physicians	766	145	3.26	3.00	1.21
Patient Info - Other	22	889	3.00	3.00	1.72

Confidence was ranked on a five-point scale, where 1 indicated least confident and 5 indicated most confident.

Respondents indicated that they are most confident exchanging their personal / practice information with the Royal College of Physicians and Surgeons of Canada (mean = 3.27), other physicians (mean = 3.26), the Ontario Medical Association (OMA) (mean = 3.24), and CPSO (mean = 3.24). They indicated that they are least confident exchanging practice information with Health Canada (mean = 2.88) and the Ontario Hospital Association (OHA) (mean = 2.70). Some of the 'other' organizations listed by respondents included: the Canadian Medical Protection Association; faculty members; insurance companies; labs; lawyers; and research facilities.

3.3.32 Q-18 - Descriptive Statistics: How confident would you feel exchanging confidential information with the following organizations? -Your Patients' Information

Confidence in Exchanging Patient Information

	N
	Valid	Missing	Mean	Median	Std. Deviation
Patient Info - OMA	703	208	2.57	3.00	1.33
Patient Info - MOHLTC	713	198	2.66	3.00	1.35
Patient Info - Health Canada	682	229	2.52	2.00	1.31
Patient Info - CMA	687	224	2.62	3.00	1.33
Patient Info - CPSO	710	201	2.88	3.00	1.42
Patient Info - RCPCS	608	303	2.80	3.00	1.40
Patient Info - CFPC	558	353	2.71	3.00	1.36
Patient Info - OHA	604	307	2.39	2.00	1.28
Patient Info - Hospitals	699	212	2.91	3.00	1.33
Patient Info - Physicians	709	202	3.15	3.00	1.32
Patient Info - Other	27	884	2.96	3.00	1.53

Confidence was ranked on a five-point scale, where 1 indicated least confident and 5 indicated most confident.

Respondents indicated that they are most confident exchanging their patients' information with other physicians (mean = 3.15), hospitals (mean = 2.91), and CPSO (mean = 2.88). They indicated that they are least confident exchanging practice information with Health Canada (mean = 2.52) and OHA (mean = 2.39). As with exchange of personal / practice information, some of the 'other' organizations listed by respondents included: the Canadian Medical Protection Association; faculty members; insurance companies; labs; lawyers; and research facilities.

3.3.33 Mean Levels of Confidence for General Internet Activity, Practice Information, and Patient Information Exchange - Calculated Indicators

An indicator reflecting general confidence in security (General Activity Confidence) was calculated for each respondent by determining the mean value of responses for each item in Q-13. Likewise, a similar indicator was calculated for each physician for confidence in security of practice-related information (Practice Information Confidence) and patient-related information (Patient Information Confidence) in columns 1 and 2 of Q-18.

Statistics

	N
	Valid	Missing	Mean	Median	Std. Deviation
GENERAL ACTIVITY CONFIDENCE	866	45	3.116	3.167	.992
PRACTICE INFO CONFIDENCE	801	110	3.114	3.000	1.089
PATIENT INFO CONFIDENCE	739	172	2.755	2.800	1.143

Paired t-tests indicated that for every organization listed, physician confidence is higher for the exchange of practice-related information than it is for patient-related information (p<.001).

3.3.34 Correlations Between Confidence in Internet Security and EMR

Pearson correlation coefficients were calculated between mean values across confidence in Internet security for general activities; exchange of practice information; exchange of patient information; potential benefits of provincial EMR; and whether physicians would use EMR in their own practice with appropriate security assurances and age, computer proficiency, and Internet hours per week.

Correlations

		Age	Computer Proficiency	Internet Hours per Week
GENERAL ACITIVY CONFIDENCE	Pearson Correlation	-.148**	.291**	.329* *
	Sig. (2-talied)	.000	.000	.000
	N	862	863	800
PRACTICE INFO CONFICENCE	Pearson Correlation	-.058	.110**	.177**
	Sig. (2-talied)	.103	.002	.000
	N	796	799	739
PATIENT INFO CONFIDENCE	Pearson Correlation	-.044	.095**	.167**
	Sig. (2-talied)	.235	.010	.000
	N	734	739	690
EMR BENEFITS > Risks	Pearson Correlation	-.017	.114**	.179**
	Sig. (2-talied)	.615	.001	.000
	N	837	841	776
EMR In your practice with apporiate security	Pearson Correlation	-.149**	.226**	.221**
	Sig. (2-talied)	.000	.000	.000
	N	842	846	782
** .Correlation is significant at the 0.01 level (2-talied).

Correlation analysis results indicated that age was negatively correlated with confidence in Internet security for general activities and use of EMR in own practice. Computer proficiency and Internet hours per week were correlated with confidence in security of Internet, regardless of activity or type of information exchanged.

3.3.35 Relationship Between Technical Knowledge of Computer Security and Confidence Levels

Analysis of variance using Scheffé post-hoc comparisons indicated that respondents who are not sure of their systems' security (i.e., firewall protection) were significantly less likely to indicate that they feel the benefits of EMR outweigh the concerns (p<.001). Of the respondents who indicated that they do not know if their computer has firewall protection, only 39% indicated that they feel the benefits of EMR outweigh the potential concerns, whereas 64% of the respondents who do have a firewall and 48% of those who do not have a firewall feel the benefits of EMR outweigh potential concerns. A similar significant relationship (p<.001) was found between knowledge of a system's security features and confidence in Internet security for general activities. Those who indicated that they have firewall protection were most confident, with a mean confidence score of 3.4, followed by those who have no firewall protection and those who do not know, with mean confidence scores of 3.1 and 2.9, respectively.

3.3.36 Predictive Model for Potential Benefits of Provincial EMR

Logistic regression analyses were conducted to evaluate whether provincial EMR benefits were predicted by specific physician individual and practice characteristics, including: age; gender; certifications; number of physicians at primary practice location; hours worked per week; patient visits per week; computer proficiency; Internet hours per week; hospital privileges (yes / no); solo practice; and mean confidence level for general Internet activities.

The model was significant and the total variance explained by the model was 19.7% (Nagelkerke R-Square = .197). Physicians who are confident in Internet security for general activities were twice as likely to feel that the potential benefits associated with EMR outweigh potential concerns. In addition, physicians who use the Internet for more hours per week were 1.3 times as likely to feel that EMR benefits outweigh concerns. None of the other individual or practice characteristics in the model was predictive.

3.3.37 Q-19 - Frequency Distribution: If you were to use the Internet, which of the following general College activities would you be receptive to completing online?

Half of all respondents indicated that they would participate in all listed general CPSO activities. Those who indicated that they would not participate in general CPSO activities online comprised 22.2% of the respondents.

3.3.38 Q-20 - Frequency Distribution: If you were to use the Internet, which of the following College's peer assessment activities would you be receptive to completing online?

The receptiveness to CPSO peer assessment activities online varied, depending on the specified activity. Respondents were receptive to completing an initial questionnaire online, and to receiving various forms of correspondence and general assessment information online. Only 13.1% of respondents indicated that they would transfer patient records for review online. Those respondents who indicated that they would not like to participate in peer assessment activities online comprised 42.3% of the respondents. Many of the physicians who indicated that they did not want to do peer assessments online specified that they feel it is important to do this activity in person.

3.4 MAIL VS. ONLINE RESPONDENTS - DIFFERENCES IN USE, NEEDS, BARRIERS, AND WANTS

Physicians responding to the survey by mail and online were compared for their levels of confidence in Internet security; sum uses; sum barriers; sum needs; sum wants; total Internet hours per week; percent of Internet time for professional use; computer proficiency; and opinion that Internet adds no value to practice. T-tests were used to determine significant differences between the mail respondents and the online respondents.

Online respondents indicated significantly more professional Internet activities (i.e., sum uses) and resources wanted online (i.e., sum wants). No significant differences were found between the mail respondents and the online respondents in the total number of barriers.

Approximately 9% fewer online respondents than mail respondents felt that the Internet adds no value to practice. Computer proficiency, total Internet hours per week, and percent of Internet time in professional activities were all significantly higher in the online respondent cohort.

3.5 NON-COMPUTER / NON-INTERNET USERS - BARRIERS, NEEDS, AND WANTS

Approximately 10% (N = 93) of the respondents indicated that they do not use computers in practice and / or that they do not use the Internet at all.

3.5.1 Frequency Distribution - Non-Computer / Non-Internet User Barriers

Most (65.9%) non-users indicated that their lack of familiarity with Internet use and their lack of computer skills were barriers to Internet use in practice. A busy schedule and no value to patient care were indicated as barriers by 38.6% and 27.3% of non-users, respectively.

3.5.2 Frequency Distribution - Non-Computer / Non-Internet User Needs

Non-users indicated that they feel that general computer / Internet courses (47%), Internet courses designed by physicians (39.8%), courses with tutorials (31.3%), and listings of reliable web sites (31%) would all help them use the Internet in practice.

3.5.3 Frequency Distribution - Non-Computer / Non-Internet User Wants

The resources wants of non-users did not vary from those of users. Most wanted full-text articles (48.5%), access to patient information (42.6%), accredited CME (39.7%), and prescription information (39.7%).

3.5.4 Non-Computer / Non-Internet Use and Individual and Practice Characteristics

Physicians in solo practice settings were significantly more likely (p<.001) to be non-users than those practising in group settings, as 69.0% of non-users were solo practitioners. Non-users indicated significantly more than users (p<.001) that they feel less proficient in their computer skills and that the Internet adds no value to practice. There were no significant gender or certification differences between users and non-users. T-test results indicated that the non-user group was significantly older than the user group by approximately 9.4 years (p<.001).

Logistic regression analysis was used to evaluate whether physician characteristics could be used to predict a non-user. Independent variables in the model included: solo practice; number of hours worked per week; number of patients per week; computer proficiency; general confidence in Internet; age; and gender.

Logistic regression analysis indicated that physicians who practice in a solo setting, are older, and feel less proficient were significantly more likely to be non-users.

3.5.5 Non-Computer / Non-Internet Use and Confidence in Internet Security

T-tests revealed significant differences between users and non-users in their mean confidence levels for general Internet activities and willingness to use EMR in practice, even with the appropriate security assurances. No significant differences were found between users and non-users in their confidence in exchange of practice or patient information, or in opinion on the benefits outweighing concerns for a provincial EMR.

3.6 GEOGRAPHIC VARIATION

The CPSO divides the province into 10 districts for its Council election functions. For the purposes of these analyses, two northern electoral districts were combined, forming a total of 9 geographic practice areas. The counties, regional / district municipalities* and / or territorial districts** included within each of the geographic practice areas are as follows:

• Area 1: Essex, Kent and Lambton
• Area 2: Elgin, Huron, Middlesex, Oxford and Perth
• Area 3: Bruce, Dufferin, Grey, Wellington and Waterloo*
• Area 4: Brant, Haldimand-Norfolk*, Halton*, Hamilton-Wentworth* and Niagara*
• Area 5: Simcoe, Muskoka*, Durham*, Peel* and York*
• Area 6: Dundas, Glengarry, Lanark, Prescott, Renfrew, Russel, Stormont and Ottawa-Carleton*
• Area 7: Frontenac, Haliburton, Hastings, Leeds and Grenville, Lennox and Addington, Northumberland, Peterborough, Prince Edward and Victoria
• Area 8: Municipality of Metropolitan Toronto
• Area 9: Algoma**, Chochrane**, Manitoulin**, Nipissing**, Parry Sound**, Sudbury and Timiskaming**,Kenora**, Rainy River**, Thunder Bay** - Northern Ontario

Analysis of variance with Scheffé post hoc analysis revealed no significant differences across practice areas for computer proficiency; total hours Internet use per week; percent professional use; sum use; sum barriers; sum needs; sum wants; EMR benefits; or confidence in Internet security for general activities, practice or patient information.

3.7 COMMENTS

Over 200 respondents provided comments. All comments were reviewed in detail and recorded verbatim. Detailed comments are not presented here; however, the major themes are summarized. The comments provided can be categorized under one of three headings: (i) Pro-Internet, (ii) Optimistic yet Cautious, and (iii) Anti-Internet. The vast majority of the comments fell into the 'Optimist yet Cautious' category.

3.7.1 Pro-Internet Comments

Many of the comments provided by the respondents regarding the use of Internet resources in practice were extremely positive. Centralized patient information, i.e., lab tests from any lab location, consultation reports, hospital and office patient care details, prescription history / activities were all items mentioned as highly desirable and useful resources. Many physicians indicated that they feel the Internet is the 'way of the future' to improve patient care, and that they feel the profession will adapt and evolve, as they have evolved, to include Internet technology in order to keep pace with the changing face of medicine.

3.7.2 Optimistic yet Cautious Comments

The majority of physicians indicated that they are optimistic about the potential benefits of the Internet and a provincial medical record, but remain cautious because of several key barriers, including:

1. Security: Security of confidential information was the number one concern expressed throughout the comments provided by respondents. Physicians are very concerned about how patient confidentiality can be guaranteed. They are concerned that there is a lack of standardization among organizations with regard to security, and they do not feel that current assurances of security are adequate, as large organizations across the globe are continually having periodic security breaches.
2. Time: Many respondents indicated that they feel they already have very little time to do much more than provide care for their patients. They indicated that using the Internet is a luxury that they cannot afford. Improved speed and search tools were two suggestions repeated throughout the comments.
3. Access: Access to the Internet was an issue in terms of location of access (i.e., want access at office and hospital, etc.), in terms of coordination of access (i.e., my office computer and the hospital computer do not talk to one another), and in terms of practice-accessible information (i.e., the information on the Internet needs to be easy to incorporate into practice).
4. Cost: The expense of hardware, maintenance, technical support, virus protection, and remuneration for use of technology were all indicated as reasons for a cautious approach to Internet integration.
5. Learning Curve: Many physicians indicated that it would take them time to learn a new approach to practice, and that they are somewhat intimidated by the prospect of incorporating technology into their practice.

3.7.3 Anti-Internet Comments

There were physicians who indicated that they are not willing to incorporate Internet technology into practice and who are sceptical of its utility in patient care. Several indicated that they have no confidence in security, that the Internet decreases time with patients and is of no value to patient care, and that they do not have any desire to add the Internet to practice.

4. Conclusion

4.1 ONTARIO PHYSICIANS ARE INTERNET USERS

The results of the study clearly show that Ontario physicians do use the Internet. Encouragingly, 77.9% of Ontario physicians indicated that they use the Internet for professional reasons. The majority of Ontario physicians use the Internet between 1-10 hours per week, with an average of 43% of that time being spent on professional activities. Although there were slight fluctuations across areas, the total Internet use and Internet time in professional activities did not vary significantly across geographical areas.

Physicians indicated that they primarily use the Internet to access medical literature and practice resources such as clinical practice guidelines. Almost 60% of respondents indicated that they use e-mail for professional communication with colleagues. A large number of respondents specified that they use the Internet for a variety of educational activities, including personal CME and teaching. Two-thirds of the respondents indicated that they use the Internet for five or more different professional activities.

A small proportion (10.2%) of the sample respondents indicated that they do not use computers and / or the Internet at all. These individuals were significantly older and more isolated (i.e., 69% were solo practitioners) than the user population, and many indicated that they are near retirement and that they are unlikely, at this point in their careers, to begin a new approach to practice that includes the Internet.

4.2 INTERNET USE VARIES ACCORDING TO PROFICIENCY, PATIENT VOLUME, AND IN-OFFICE INTERNET ACCESS - NOT AGE

In general, males and younger physicians indicated higher levels of computer proficiency and general use. These findings were confirmed through the regression analyses, which indicated that age and gender were important factors affecting the perceived computer proficiency of the physicians. However, age and gender did not seem to be significant factors in total Internet use or Internet time in professional activities. Regression analyses indicated that physicians who feel more proficient and have in-office access were also more likely to use the Internet in general and for professional activities, regardless of age or gender.

Computer proficiency was a significant predictor of the sum number of barriers and needs indicated by respondents. As with Internet time, age and gender were not predictive of perceived barriers or needs. In-office access to the Internet was also a significant predictor for total and professional Internet activity. If the Internet was not available at the point of service, its application and utility to practice were minimal.

Patient visits per week was a strong determinant of total Internet use and Internet time in professional activities. Physicians attending to higher patient volumes were less likely to feel proficient and less likely to use the Internet for any purpose, including professional activities.

Solo practice set-up also appeared to limit Internet use in a similar fashion. Physicians who see high volumes of patients or who are in solo practice settings are likely to be pressed for time. This relationship was clearly reflected when the barriers and needs proposed by respondents were examined.

The relationship that emerged from the analyses implies that time is a limiting factor of proficiency. The more time a physician has to participate in online professional activities, the more proficient he / she becomes and, conversely, if a physician is too busy to use the Internet frequently, he / she is likely to be less proficient and, as a result, will use the Internet less in practice.

4.3 INTERNET USERS' BARRIERS AND NEEDS - TIME AND PROFICIENCY

The top three barriers cited by respondents were busy schedule, length of search times, and high volumes of information to sort through online. All three of these barriers are related to time and timely access to information. The needs listed by respondents echoed this theme. Physicians indicated that they feel that listings of reliable, practice-related web sites would be helpful in gaining timely access to information, and time-effective tools for incorporating Internet into practice would be beneficial in helping them use Internet resources more in their practices.

Respondents who are non-computer / non-Internet users indicated barriers and needs related to a lack of familiarity with computer / Internet technology and to improving computer skills. This group also indicated that time (i.e., busy schedule) is another main barrier to Internet use in practice. Gaining Internet skills and experience is difficult to do when there is little time to devote to these activities.

Computer proficiency was the only significant predictor of the total number of barriers and needs indicated by respondents. Patient volumes, solo practice, and hours worked did not emerge as significant predictors of the total number of barriers and needs indicated by respondents.

4.4 DESIRED ONLINE RESOURCES

The desired online resources indicated by respondents focused on access to full-text articles, accredited CME, and prescription information including online compendiums and drug bulletins. Access to important patient information such as diagnostic test results, hospital discharge summaries, and consulting / referring physician notes was listed as a resource that 60% of respondents indicated they would like to see made available or improved online to assist in daily practice activities.

4.5 USE LEADS TO CONFIDENCE

Although 60% of respondents indicated that online access to patient information would be helpful in practice, approximately half of the respondents indicated that they feel that the potential benefits a provincial EMR do not outweigh their concerns with security and privacy of patient information. Those individuals who were generally more confident with online security and the concept of provincial EMR indicated that they are proficient Internet and computer users. Over 30% of respondents indicated that they do not know if the system from which they access the Internet has a firewall, and 67% of respondents indicated that they are not sure / do not know what types of technical security items would increase their confidence in Internet security. The analysis indicated that those respondents who knew more about the technical aspect of system security, whether or not their systems actually have that security, were more likely to feel confident about the security of confidential information on the Internet.

A significant relationship was found between computer proficiency and a physician's confidence in Internet security, both for current general Internet activities and for future potential province-wide EMR. Once again, it appears that computer proficiency, both in terms of actual users skills and technical knowledge, is a key element in promoting the use of and confidence in online activities.

5. Recommendations

5.1 INCREASE ACCESS TO INCREASE INTERNET USE IN PRACTICE

This study showed that computer proficiency is a key element in professional Internet use. The more physicians can use the Internet in practice, the more they will use the Internet in practice. It is clear from the results that the limiting factors associated with proficiency are related primarily to organizational and system-level problems, and less to individual practitioners. Although overall proficiency was predicted by age and gender, these elements were not predictive of professional use or barriers, nor could they be changed or affected through educational or policy initiatives.

It is recommended that programs and policies designed to increase Internet use in daily practice focus on issues associated with access. To adequately address this issue, all aspects of access must be considered, including: (i) physician time availability to access the Internet for practice, (ii) Internet access at the point-of-service provision, and (iii) timely access to online information with regard to searches and information review.

5.1.1 Physician Time Available to Access the Internet and Point-of-Service Internet Access

The time available to a physician is a determinant of overall Internet proficiency and use. Solo practitioners, high volume practices, and generally busy schedules impede physicians from accessing the Internet in daily practice. In addition, access to the Internet at the point-of-service is primarily an organizational factor related, in part, to practice set-up and location of primary practice (i.e., hospital vs. private office). Practice set-up, patient volume, and in-office access are often outside of the immediate control of the practitioner. Solo practitioners face further cost and maintenance issues associated with integrating computer systems into their practices. Therefore, physicians require assistance to enable them to devote time to learning how to integrate the Internet into practice and developing the necessary proficiencies to promote Internet use and decrease barriers to use. Simply having physicians attend Internet courses is neither what physicians need nor what they want; completing a course may increase physicians' general knowledge, but it will not enable them to have the 'use time' they require to build their level of familiarity with online practice resources so that they may comfortably and frequently use them in practice.

5.1.2 Timely Access to Quality Online Practice Information

Physicians require timely access to quality information when referring to the Internet in practice. High-speed access is critical, search times need to be reduced, and quality of web site information needs to be assured. In Ontario, efforts to improve Internet access in physician offices have been undertaken as part of the Smart System for Health and the ePhysician Project ²¹; however, high-speed access is not yet available in many rural communities and isolated areas throughout the province.

Improving search times and information reliability can be addressed. Providing peer-reviewed practice-related web sites, much in the same way journals provide physicians with peer-reviewed research studies, would be of benefit to physicians. Editorial boards of medical journals are responsible for continually reviewing and publishing new study findings. A similar structure / process could be applied to an Internet-based web site review board, whose primary responsibility is not to produce the practice resource information available online, but to review the legitimacy of the sources of the online materials and the currency / existence of the web site (i.e., is it still there or no longer maintained), and to catalogue and structure the information so that it may be searched in a manner relevant to practitioners (i.e., practice specialty area, disease-based, organ system, etc.). In doing so, a practice-based web site index would be established where all practitioners could gain quick and easy access to relevant practice information.

The CPSO is in the process of considering this recommendation by reviewing the current methods of web site quality rating processes, the organizations which provide such services at the present time, and the potential partnerships for future development of an online, practice-based resource / web site index for physicians.

5.2 CONFIDENCE IN SECURITY AND PRIVACY OF PATIENT INFORMATION

The results of the study clearly indicated that physicians are very concerned about the security and privacy of patient information. However, those individuals who use the Internet more and who have a higher level of understanding of system security features are more confident with online security. With regard to the issue of information privacy and security, knowledge and familiarity build confidence. Efforts to address the issues of access outlined above will promote proficiency and use, and will help build confidence. Likewise, concurrent efforts should be made to help educate physicians (and the public) to system precautions that can / should be taken to help secure all online information. In Ontario, efforts are currently underway to establish provincial data and technology standards for the exchange of patient information.^{22; 23}

5.3 COLLEGE OF PHYSICIANS AND SURGEONS OF ONTARIO (CPSO) AND ONLINE QUALITY IMPROVEMENT RESOURCES

The study shows that half of all respondents are willing to participate in general CPSO activities online. In addition, a substantial proportion state that they would participate in many of the CPSO peer assessment activities online. Based upon these findings, the CPSO has a solid rationale upon which to develop an implementation plan for expanded online activities.

It is recommended that the CPSO develop its online quality improvement program elements (e.g., peer assessment program) with caution. A significant proportion of respondents (43%) indicated that they would not be receptive to participating in assessment activities online, primarily because they feel that this is an important activity for maintaining the personal contact between physician assessors and program participants. In consideration of the important educational value gained by the personal interaction between peers, it is recommended that assessment activities maintain and promote this aspect of the current program.

However, the study indicates that there are aspects of the CPSO quality improvement programs which have the potential to be effectively provided online. The completion of questionnaires, self-assessment modules and exercises, receipt of correspondence, and assessment information are all activities that over 40% of respondents indicated they would complete online if the services were available.

5.4 DISSEMINATION PLAN

The dissemination of the study results is already underway. The results of the study have been presented or are planned for presentation on the following dates and locations:

• January 27, 2003 Workplace Safety and Insurance Board (WSIB) - Toronto, ON
• February 7, 2003 Web Site Advisory Committee, CPSO - Toronto, ON
• February 10, 2003 Quality Assurance Committee, CPSO - Toronto, ON
• March 11, 2003 Executive Committee, CPSO - Toronto, ON
• May 29 / 30, 2003 Council, CPSO - Toronto, ON

In addition, the results of the study will be presented upon request, subject to the availability of the investigators. The investigators would be pleased to provide a formal presentation of the study to the Office of Health and the Information Highway, Health Canada.

Plans for publication include:

• Submission of the study results to a peer-reviewed journal (e.g., Canadian Medical Association Journal).
• Summary article in Members' Dialogue - Official Publication of the College of Physicians and Surgeons of Ontario. This publication is distributed to the 26 000 registered physicians in Ontario.
• Availability of the full study report and executive summary online at the CPSO web site (URL: http://www.cpso.on.ca).

Copies of the results will be made available upon request.

5.5 NEXT STEPS

The results of this study provide valuable information upon which to build online quality improvement and education tools. It is important to address issues of access and proficiency, in order to promote the use and acceptance of Internet resources in practice. The long-term goals emerging from this study are to determine if online educational and assessment programs have an impact on physician performance and patient care. The outcome indicators for measuring changes in physician performance already exist. Since 1980, the CPSO has been conducting evaluations of physician performance as a key component of its mandated quality assurance activities under the Regulated Health Professions Act. The impact of integrating online quality improvement tools on physician performance may be monitored and documented through the performance evaluation programs which are already well established in Ontario.

6. Contact Information

For further information regarding the Use, Needs, and Barriers: Ontario Physicians and the Internet survey, please contact:

Elizabeth F. Wenghofer, M.Sc.
Senior Epidemiologist, Clinical Practice Research Manager
Quality Management Division
College of Physicians and Surgeons of Ontario
80 College Street
Toronto, Ontario, Canada M5G 2E3
Tel: (416) 967-2600 Ext. 450
Fax: (416) 967-2605
Email: ewenghofer@cpso.on.ca

7. Project Evaluation

7.1 EVALUATION OUTLINE

As outlined in the original proposal for the Knowledge Development and Exchange (KDE) Applied Research Initiative of Health Canada's Office of Health and the Information Highway, the evaluation of this study focused on process evaluation and addressed the achievement of stated objectives, accuracy and reliability of the data collected, and efficiency of project administration.

7.2 ACHIEVEMENT OF OBJECTIVES

An integrated evaluation occurred throughout the project to ensure that objectives were successfully achieved. Survey items were specifically designed to directly address outlined objectives, as were all planned analyses. The overall expected outcome of the study was a comprehensive understanding and information concerning Ontario physicians' needs, barriers, and requirements for implementing Internet technology into daily practice, continuing education, and quality improvement. The detailed expected outcomes for each study objective and the success in meeting each objective within the study are listed below:

OBJECTIVE	OUTCOME	OBJECTIVE ACHIEVED?
Do physicians use the Internet?	Proportions and characteristics of physicians who use / do not use the Internet established Internet usage patterns including frequency, access, reasons for use, confidence, and the characteristics of the profession associated with variation in these patterns established	YES
Why do physicians use or not use the Internet?	General level of physician computer proficiency and Internet use determined Proportion of physician personal and professional Internet use determined Types of professional activities determined Concerns and barriers to using the Internet including issues of privacy, confidentiality, cost, knowledge, and comfort established Needs required to promote / improve Internet usage and incorporation into practice including incentives, education, type / format of education, resources, security, and privacy determined Effects of demographic and practice characteristics on proficiency, professional use, barriers, and needs determined for both Internet users and non-users	YES
What do physicians want from Internet resources?	Determined professional online resources wanted by physicians including desired programs, information, courses, communication with CPSO, patient satisfaction information, security, and privacy Effects of demographic and practice characteristics on desired resources determined for both Internet users and non-users General confidence in Internet security, opinions on potential benefits of EMR, and knowledge of security technology established	YES

7.3 Non respondent analysis

Bias of survey studies due to non-response is always a potential concern. There is always the possibility that the respondent population is somehow different from the non-respondent population in demographic structure or attitude, resulting in biased findings.²⁴ It is often difficult, if not impossible, to determine non-respondent characteristics in studies where responses are returned anonymously. In this study, although all responses were strictly confidential and reporting was structured on aggregate information, the study participants were known and the responses monitored. As a result, it was possible to determine if the respondent population varied from that of the non-respondents.

T-test and Chi square analyses were conducted to determine whether the demographic structure of the non-respondent population varied significantly from that of respondents, thereby possibly resulting in bias in the study findings. The two groups were compared for age, gender, and certifications from the College of Family Physicians of Canada (CFPC) and the Royal College of Physicians and Surgeons of Canada (RCPSC).

T-tests indicated that the non-respondent population was significantly younger than the respondent population (p = .016), with a mean difference of 2.6 years. Chi-square tests indicated that the respondent population included a significantly higher proportion of males (75.3%) than the non-respondent population (69.7%). No significant differences were found between respondents and non-respondents in their certifications with the two national colleges.

The study findings indicated that age and gender were both predictive of proficiency; i.e., older and female physicians were likely to feel less proficient than younger or male physicians. An older population may have resulted in an underestimate in levels of proficiency, whereas a more male population may have contributed to an overestimate; however, it is unlikely that this difference has resulted in significant bias to the findings, as the bias in estimates are not in a single direction.

The older, male make-up of the respondent population is logical when considered within the context of the study findings. Increasing age is significantly related to decreasing patient volume and hours worked, resulting in better access to online activities. Males were found to feel more proficient than females and they use online resource more in practice. If these physicians have more time to go online, they were also likely to have more time to respond to this survey.

7.4 Response rate

At 61.4%, the response rate for this study was very good. A desirable response rate lies between 60% and 80%.^{24; 25} Questionnaires mailed to the general public typically result in response rates of 10% to 20%; low response rates weaken survey findings, since small sample sizes may not accurately represent the target population.^25-27 In comparison to previous surveys studying Internet use in physicians, this study had a substantially higher response rate. The Canadian Medical Association's (CMA) 2001 and 2002 Physician Resource Questionnaires (PRQs) reported response rates of 42%²⁸ and 37%²⁹, respectively.

7.5 Representative sampling frame - provincial picture

In order to determine whether the study respondents are representative of the Ontario population, demographic and practice characteristics were compared between the two groups.

Demographic comparisons indicated that the respondent population mirrors that of the Ontario physician population for average age, gender distribution, and certifications.³⁰ The mean hours worked per week and patient visits per week in the respondent population differ somewhat from the Ontario physician population, but there is substantial overlap in the distributions. The respondents work slightly more hours, with an average of 46.9 hours per week (standard deviation (SD) = 16.9 hours) and see fewer patients, with an average of 108.8 visits per week (SD = 73.4 visits). The Ontario physician population works an average of 43.7 hours per week (SD = 20.9 hours)³⁰ in direct patient care activities and has 121.6 visits per week (SD = 104.5 visits).³⁰ Solo practitioners comprised 48.4% of the respondents and they constitute 50.1%30 of Ontario physician population in aggregate. On the whole, the comparison showed that the two populations are comparable and the respondent population is representative of the Ontario physician population. Consequently, it can be concluded that the results presented in this study are highly representative of the Ontario physician population.

7.6 Representative results - national picture

The CMA's 2002 PRQ looked at several of the same questions addressed in this study.¹¹ Comparing the two studies' respondent demographics, practice patterns, and study findings provides some indication as to how representative this study is on a national level.

Direct comparisons of gender or certifications were not possible, as the gender and certification data in the 2002 PRQ data were reported per survey item and not presented for the total PRQ respondent population. Derived from the tables presented,¹¹ the 2002 PRQ respondent population comprised approximately 68% males and approximately 50% RCPSC certified physicians. This study comprised proportionately more males (75.3%) than the 2002 PRQ and approximately the same proportion of RCPSC certified physicians (50.8%). The gender difference between the two study populations is not considered a significant issue, as gender was not found to be a strong predictor of Internet use, barriers or needs, although it was a predictor of computer proficiency. Mean age of respondents was not provided in the 2002 PRQ report.

The patient visits per week were reported in the 2002 PRQ as follow: 46% of physicians see fewer than 100 patient per week, 33.9% of physicians see between 100 - 199 patients per week, and 9.8% of physicians see more than 200 patients per week.¹¹ The distribution was fairly similar to the respondents' in this study, where 40.6% of physicians see fewer than 100 patients per week, 34.2% of physicians see between 100 - 199 patients per week, and 14.2% of physicians see more than 200 patients per week. In this study, the respondents had a slightly larger group of high volume practices, which can be explained, in part, by those physicians reporting exceptionally high patient volumes and indicating that they are radiologists or pathologists, and these values represent films read per week or tests interpreted per week rather than direct patient visits per se.

The 2002 PRQ reported that 89% of physicians use the Internet, 57.4% have Internet access at their practice location, and 27% use EMR, either alone or in combination with paper records.¹¹ In comparison, this study found that 89.9% of physicians use the Internet, 51.7% have in-office access to the Internet, and a further 13.5% indicated access in other practice locations (e.g., hospital, university faculty); 24.2% indicated that they use EMR in their practice, although no differentiation was made between those who use EMR only or in combination with paper records.

From the comparisons that are possible it appears that this study's findings and the 2002 PRQ show similar results. The similarity of the study populations and findings supports the notion that the present study can be generalized to a national level; however, this should be considered with caution. Further evaluation of possible geographic effects on Internet use is recommended, as the 2002 PRQ indicated that differences may exist between urban and rural settings.

7.7 Survey method / administration - feedback from respondents

Contact information was provided in all letters, reminders, on the web site, and in the survey booklet. Telephone, fax, e-mail, and address information was provided for two contact individuals at the CPSO to assist physicians with their questions related to the survey.

Approximately 92 inquiries were received. The majority of physician inquiries were received shortly after the reminders were sent. Most physicians called or e-mailed with requests for another copy of the survey; thirty-one of those called to request another survey and of those, 24 submitted responses.

Five physicians were unable to access the web site. Trouble-shooting on the web site revealed a problem with viewing the web site using older browser versions. This problem was corrected and no further browser difficulties were reported.

Eight physicians reported difficulties entering the secure survey or not remembering / not having access information. Access information was clearly outlined in all letters, reminders, and in the survey booklet. Once access specifications were clarified (i.e., login id = CPSO number and password = birth date in YYYYMMDD format), these physicians were able to access the web site and complete the survey without further incident.

Five physicians called to indicate that they would not be responding, and five called to indicate that they were no longer in practice. The remainder of the ineligible physicians provided notification of their retirement on the returned survey form itself.

The remainder of calls were from physicians wanting to respond by phone (8), checking whether they had already responded (3) or calling to say that the survey was in the mail (11). Several called to verify whether they were eligible to respond (6) as their practice was limited in scope, they worked part-time or locums.

Only three physicians called to express their dissatisfaction with being asked to participate in the survey or with receiving multiple reminders. The vast majority of physicians who called the CPSO were very happy to offer their help with the study.

7.8 Timelines, milestones, and deliverables

Three deliverables were outlined in the original project proposal. All milestones and deliverables were met well within planned timelines (see Appendix 2) and specifically included: (i) the survey instrument, which could be used / shared with other organizations upon request; (ii) the results of the study, to be disseminated through publications and presentations; and (iii) the development of recommendations and a plan of action for the development of online resources and quality improvement activities for the CPSO quality improvement programs.

7.8.1 The Survey Instrument

The first immediate deliverable of the study was the needs assessment survey, developed to elicit physicians' opinions and requirements for promoting the use of Internet resources in practice. The survey instrument was created using valid survey methodology and was to be pilot-tested to ensure usability by the profession. Sharing the survey with organizations (e.g., universities, other provincial regulatory colleges, and physician organizations) outside of Ontario may enable other provinces and territories to conduct their own needs assessments and physician Internet competency evaluations.

A secure area on the CPSO web site was built specifically for this study. The online survey application was tendered for contract through a formal Request for Proposal (RFP) process administered by the CPSO Information Technology Department. Four companies responded to the RFP. blue sands Inc. was chosen for the application development. Overview documentation from blue sands Inc. is provided in Appendix 3. Copies of the survey files, utility files, and overview documentation file are included on a CD-Rom. Online development, application testing, and move to production were completed on schedule. In addition to the survey form itself, a data entry form, a response monitoring report, a mailing label, and a reminder letter generator were built as part of the application. A password management function was also incorporated, in order to ensure that passwords could be reassigned if a user were to experience difficulty logging into the survey.

7.8.2 Publications and Presentations

The publications and presentations constituted the second immediate project deliverable. The effective dissemination of results may enable stakeholder organizations to develop new Internet- based applications and to implement them effectively. The dissemination of the results of this study began immediately after the data analyses were completed and, as outlined in Section 5.4 of this report, the dissemination activities will continue through formal publications and presentations, and information will be provided upon request

7.8.3 Recommendations for CPSO Internet Resource Development

The development of recommendations and a structured plan commenced upon completion of all data analyses. The results have been brought to the attention of the CPSO Web Site Advisory Committee, the CPSO Senior Management Group, and the CPSO Executive Committee. There will also be a formal presentation of the study result to the CPSO Council in May, 2003. The study findings have been received with great interest, and the CPSO Web Site Advisory Committee and the Information Technology Department will use the results of the survey to assist in prioritizing the development of online resources for CPSO's general and quality improvement activities. The formal integration of online tools will enhance an already internationally-recognized and esteemed quality assurance program, and will further support efforts to assure the public and the profession that practising physicians are continually striving to improve the quality of the care they offer to their patients.

7.9 Budget / costs - estimate vs. actual

The study was completed for a total cost of $72,955.19, well within the estimated budget, with actual costs totalling $7,301.01 below the estimated figure.

Reference List

1. Ward JP, Gordon J, Field MJ, Lehmann HP. Communication and information technology in medical education. Lancet 2001; 357(9258):792-6.
2. Manning PR and DeBakey L. Continuing medical education: the paradigm is changing. J Contin Educ Health Prof 2001; 21(1):46-54.
3. Pereira J, Peden J, Campbell K. Instructional technology in medical education: lessons learnt. J Telemed Telecare 2000; 6 (Suppl 2):S56-8.
4. Harris JMJ, Salasche SJ, Harris RB. The Internet and the globalisation of medical education. BMJ 2001; 323(7321):1106.
5. Marshall JN, Stewart M, Ostbye T. Small-group CME using e-mail discussions. Can it work? Can Fam Physician 2001; 47:557-63.
6. Casebeer L, Bennett N, Kristofco R, Carillo A, Centor R. Physician Internet medical information seeking and on-line continuing education use patterns. J Contin Educ Health Prof 2002; 22(1):33-42.
7. Gorman PJ, Meier AH, Rawn C, Krummel TM. The future of medical education is no longer blood and guts; it is bits and bytes. Am J Surg 2000; 180(5):353-6.
8. IMS HEALTH, The College of Family Physicians of Canada, and The Royal College of Physicians and Surgeons of Canada. Computer Technology in Physician Offices. Pointe-Clair, PQ, IMS HEATLH. 1999.
9. White C, Sheedy V, Lawrence N. Patterns of computer usage among medical practitioners in rural and remote Queensland. Aust J Rural Health 2002; 10(3):137-46.
10. Kalsman MW and Acosta DA. Use of the Internet as a medical resource by rural physicians. J Am Board Fam Pract 2000; 13(5):349-52.
11. Results of the CMA 2002 Physician resource questionnaire: computer use and the Internet. CMAJ 2002; URL: http://www.cmaj.ca/cgi/content/full/167/5/521/DC1
12. Chin T. The plugged-in crowd: How many physicians are online? American Medical News 2000; URL: http://www.ama-assn.org/sci-pubs/amnews/pick_00/tesa1113.htm
13. American Medical Association. AMA study: Physicians' use of internet steadily rising. AMA Press Release 2002; URL: http://www.ama-assn.org/ama/pub/print/article/1616-6473.html
14. Jadad AR, Sigouin C, Cocking L, Booker L, Whelan T, Browman G. Internet use among physicians, nurses, and their patients. JAMA 2001; 286(12):1451-2.
15. Grol R and Wensing M. Implementation of quality assurance and medical audit: general practitioners' perceived obstacles and requirements. Br J Gen Pract 1995; 45:548-552.
16. O'Leary DS. Reordering performance measurement priorities: a modest proposal for setting priorities, from the president of the Joint Commission on Accreditation of Healthcare Organizations. Health Affairs 1998; 17:38-39.
17. Wakefield DS, Helms CM, Helms L. The peer review process: the art of judgment. J Healthcare Qual 1995; 17:11-15.
18. Soh KB. Job analysis, appraisal and performance assessments of a surgeon - a multifaceted approach. Singapore Med J 1998; 39:180-185.
19. Ram P, van der Vleuten C, Rethans JJ, Grol R, Aretz K. Assessment of practicing family physicians: comparison of observation in a multiple-station examination using standardized patients with observation of consultations in daily practice. Acad Med 1999; 74:62-69.
20. Dillman DA. Mail and Telephone Surveys, Toronto, John Wiley & Sons, Inc.; 1978.
21. Office of Health and Information Highway. Health Infostructure in Canada - Provincial and Territorial Plans and Priorities 2002 - Ontario. Health Canada 2002; URL: http://www.hc-sc.gc.ca/hcs-sss/ehealth-esante/infostructure/pt/2002/on_e.html#pa
22. Ontario Hospital eHealth Council. Smart systems for health network rolling out to Ontario hospitals. eHealth Update 2002; URL: http://www.oha.com/OHA/MEDIA.NSF/
    1ec23d8b20bec4820525690f00201d75/8da5fcc34a24c93485256bf90061fa73/$FILE/eHealthUpdateMarch2002.pdf
23. Ontario Hospital eHealth Council. An update on data and technology standards in Ontario. eHealth Update 2002; URL: http://www.oha.com/OHA/MEDIA.NSF/1ec23d8b20bec4820525690f00201d75/1c1a60fb60b2f86985256bf90062604d?OpenDocument
24. Crosby FE, Ventura MR, Feldman MJ. Examination of a survey methodology: Dillman's Total Design Method. Nurs Res 1989; 38:56-58.
25. Anema MG and Brown BE. Increasing survey responses using the total design method. J Contin Educ Nurs 1995; 26:109-114.
26. Smeeth L and Fletcher AE. Improving the response rates to questionnaires. BMJ 2002; 324(7347):1168-9.
27. Edwards P, Roberts I, Clarke M, et al. Increasing response rates to postal questionnaires: systematic review. BMJ 2002; 324(7347):1183.
28. Martin S. Older MDs rushing to the Internet. CMAJ 2002; 166:234.
29. Martin S. What paperless office? CMAJ 2002; 167:182.
30. Wenghofer EF. Results of the 2000 CPSO Annual Membership Survey (unpublished); 2000.

Appendix 1: Survey Copy

Use, Needs and Barriers:
Ontario Physicians and the Internet

Integrated Internet Technology into Physician Quality Improvement Initiatives

Section A -- About Your Practice

Q-1

Which best describes your practice set-up?

(Please select one response)

• SOLO PHYSICIAN
• GROUP PRACTICE WHERE COSTS, OVERHEAD STAFF, ETC., ARE SHARED (ADMINISTRATIVE GROUP)
• GROUP PRACTICE WHERE COSTS, OVERHEAD, STAFF, ETC., ARE SHARED AND YOU ATTEND TO AND/OR COVER EACH OTHERS' PATIENTS (CLINICAL GROUP)

Q-2

How many physicians are there at your primary practice location?

NUMBER OF PHYSICIANS AT PRIMARY PRACTICE LOCATION:

Q-3

In an average week, please estimate the:

NUMBER OF HOURS YOU WORK:

NUMBER OF PATIENTS YOU SEE:

Q-4

How would you rate your general computer proficiency?

(Please select one response)

• NOT AT ALL PROFICIENT
• SOMEWHAT PROFICIENT
• PROFICIENT
• EXPERT LEVEL

Q-5

Describe the level of computerization in your practice:

(Please check all that apply)

• NO COMPUTERS - PLEASE PROCEED TO QUESTION 10
• BILLING AND RECONCILIATION
• ELECTRONIC MEDICAL RECORDS
• SCHEDULING AND FOLLOW-UP

SECTION B - YOUR CURRENT USE OF THE INTERNET

Q-6

How many hours per week would you estimate you use the Internet for personal or professional usage (including sending and receiving email)?

(Please select one response)

• < 1 HOUR
• 1-5 HOURS
• 6-10 HOURS
• 11-20 HOURS
• OVER 20 HOURS
• I DO NOT USE THE INTERNET AT ALL à PLEASE PROCEED TO QUESTION 10

Q-7

From which location do you access the Internet?

(Please check all that apply)

• HOME
• OFFICE
• OTHER (PLEASE SPECIFY):

Q-8

What would you estimate your total Internet usage time is for personal (e.g., banking, email, booking trips, general interest, etc.) and professional use (e.g., communications with patients/colleagues, literature searches, obtaining drug information, etc.)?

PERSONAL USE
_ _ _%

PROFESSIONAL USE
_ _ _%

TOTAL 100%

Q-9

If you use the Internet for professional reasons, please indicate the types of professional activities in which you employ Internet resources:

(Please check all that apply)

• COMMUNICATION WITH COLLEAGUES
• COMMUNICATION WITH PATIENTS
• MEDICAL INFORMATION RESOURCES (E.G., CLINICAL PRACTICE GUIDELINES)
• DRUG INFORMATION
• RESEARCH/MEDICAL LITERATURE (E.G., MEDLINE, ON-LINE JOURNALS)
• PROVIDING INFORMATION TO PATIENTS (E.G., SUPPORT SITES FOR PATIENTS)
• ELECTRONIC SUBMISSIONS OF OHIP BILLINGS
• ELECTRONIC SUBMISSIONS FOR OTHER INSURANCE BILLINGS
• RECEIVING LAB RESULTS
• ACCESSING MEDICAL RECORDS FROM HOME/OUTSIDE OFFICE
• OTHER (PLEASE SPECIFY):

Q-10

Please indicate what you feel are your biggest barriers of integrating the Internet with your own daily professional activities:

(Please check all that apply)

• LACK OF FAMILIARITY WITH INTERNET USE (I.E., SEARCH SKILLS, GENERAL COMPUTER SKILLS, ETC.)
• HIGH VOLUMES OF INFORMATION TO SORT THROUGH
• LACK OF ABILITY TO DETERMINE QUALITY/LEGITIMACY OF INTERNET SITES
• LENGTH OF TIME REQUIRED TO SEARCH THE INTERNET FOR DESIRED INFORMATION
• COST OF INTERNET SERVICES (EITHER ASSOCIATED WITH COMPUTER HARDWARE OR INTERNET PROVIDER FEES)
• LACK OF CONFIDENCE IN INTERNET SECURITY FOR USE IN PROFESSIONAL MATTERS
• DOES NOT ADD VALUE TO PATIENT CARE
• IMPRACTICAL TO DO "SEARCHES" WITH PATIENT IN THE OFFICE
• BUSY SCHEDULE
• OTHER (PLEASE SPECIFY):

Q-11

Please indicate which of the following do you think might help you use the Internet more in your practice:

(Please check all that apply)

• COURSES THAT INCLUDE FRIENDLY TAKE-HOME TUTORIALS
• COURSES THAT TEACH GENERAL COMPUTER AND INTERNET SKILLS AND SEARCH TECHNIQUES
• INTERNET COURSES SPECIFICALLY DESIGNED BY PHYSICIANS FOR PHYSICIANS
• LISTINGS OF RELIABLE PRACTICE RELATED WEB SITES
• TIME EFFECTIVE TOOLS FOR INCORPORATING INTERNET RESOURCES INTO PRACTICE
• COST DISCOUNTED INTERNET SERVICES (I.E., LOWER MONTHLY FEES)
• ONE-ON-ONE SESSION WITH A PHYSICIAN WHO USES THE INTERNET IN PRACTICE (I.E., A "HERE IS HOW I USE THE INTERNET" DEMONSTRATION AND Q&A SESSION)
• SOFTWARE TO ASSIST PHYSICIANS WITH INTERNET USE
• ON-LINE HELP
• UNLIMITED TECH SUPPORT
• SOFTWARE-SPECIFIC HELP
• PREFER NOT TO INCORPORATE THE INTERNET INTO PRACTICE
• CANNOT BE DONE EFFECTIVELY
• OTHER (PLEASE SPECIFY):

Q-12

Please indicate the types of resources that you would like to see made available, improved or expanded on the Internet to assist you in your professional activities:

(Please check all that apply)

• FULL TEXT ARTICLES AND IMPORTANT STUDY RESULTS
• ACCESS TO IMPORTANT PATIENT INFORMATION (E.G., DIAGNOSTIC TEST RESULTS, HOSPITAL DISCHARGE SUMMARIES, CONSULTING/REFERRING PHYSICIAN NOTES, ETC.)
• RESOURCES WHICH ALLOW YOU TO DETERMINE THE CREDIBILITY OF THE INFORMATION FOUND ON-LINE
• TOOLS THAT COLLECT FEEDBACK ON YOUR PATIENT'S SATISFACTION
• TOOLS THAT HELP YOU ASSESS/REFLECT ON YOUR OWN PRACTICE
• INTERACTIVE ENHANCEMENT COURSES
• MOCOMP, MAINPRO AND/OR MAINPRO(C) ACCREDITED CME
• DIAGNOSTIC SUPPORT TOOLS/DECISION SUPPORT TOOLS
• PRESCRIPTION INFORMATION (I.E., ON-LINE COMPENDIUM, DRUG BULLETINS, ETC.)
• COMMUNICATION FORUMS/CHATROOMS TO CONSULT WITH OTHER PHYSICIANS ON CLINICAL/PRACTICE
• OTHER (PLEASE SPECIFY):

Section C: Security, Confidentiality and Internet use

Q-13

How confident do you feel that the Internet can be a secure means of exchanging confidential information for the following purposes:

(Please circle appropriate number; 1=least confident, 5=most confident, 9=no opinion)

use	feeling
SHOPPING (I.E., PROVIDING CREDIT CARD INFORMATION ON-LINE)	1	2	3	4	5		9
BANKING (I.E., PAYING BILLS, TRANSFER FUND, INVESTING)	1	2	3	4	5		9
PERSONAL EMAIL	1	2	3	4	5		9
PROFESSIONAL EMAIL (I.E., PATIENT OR PRACTICE INFORMATION)	1	2	3	4	5		9
SUBMISSION OF BILLING INFORMATION TO OHIP OR OTHER SOURCES	1	2	3	4	5		9
ELECTRONIC MEDICAL RECORDS (I.E., CONTAINING CONFIDENTIAL PATIENT INFORMATION)	1	2	3	4	5		9

Q-14

Please indicate which security items below would help to increase your level of confidence in security and privacy of Internet usage for transfer of confidential patient information:

(Please check all that apply)

• 128-BIT ENCRYPTION
• MULTIPLE PASSWORDS FOR ENTRY INTO WEBSITE/PROGRAM
• NOT SURE/DO NOT KNOW
• OTHER (PLEASE SPECIFY):

Q-15

Do you have a firewall on the computer that you use most frequently to access the Internet?

• YES
• NO
• DO NOT KNOW

Q-16

Do you feel that the potential benefits associated with electronic medical records (e.g., province wide access to patient records regardless of location of treatment) outweigh any potential concerns with security and privacy?

• YES
• NO

Q-17

Would you feel confident accessing medical records electronically (including your own medical history) in your practice if you were provided appropriate assurances of the security of patient information?

• YES
• NO

Q-18

How confident would you feel exchanging confidential information with the following organization(s)?

(Please circle appropriate number; 1=least confident, 5=most confident, 9=no opinion)

	Your Personal/Practice Information								Your Patient's Information
1. Ontario Medical Association	1	2	3	4	5		9		1	2	3	4	5		9
2. Ontario Ministry Of Health And Long Term Care	1	2	3	4	5		9		1	2	3	4	5		9
3. Health Canada	1	2	3	4	5		9		1	2	3	4	5		9
4. Canadian Medical Association	1	2	3	4	5		9		1	2	3	4	5		9
5. College Of Physicians And Surgeons Of Ontario	1	2	3	4	5		9		1	2	3	4	5		9
6. Royal College Of Physicians And Surgeons Of Canada	1	2	3	4	5		9		1	2	3	4	5		9
7. College Of Family Physicians Of Canada	1	2	3	4	5		9		1	2	3	4	5		9
8. Ontario Hospital Association	1	2	3	4	5		9		1	2	3	4	5		9
9. Hospitals	1	2	3	4	5		9		1	2	3	4	5		9
10. Other physicians	1	2	3	4	5		9		1	2	3	4	5		9
11. Other (please specify)	1	2	3	4	5		9		1	2	3	4	5		9

Section D: College Of Physicians And Surgeons Of Ontario And Internet Utilization

Q-19

If you were to use the Internet, which of the following general College activities would you be receptive to completing on-line:

(Please check all that apply)

• APPLICATION FOR LICENCE
• ANNUAL RENEWAL OF CERTIFICATE OF REGISTRATION (INCLUDING CONFIRMATION OF RENEWAL TRANSACTION)
• ANNUAL MEMBERSHIP SURVEY
• MEMBERSHIP ELECTION BALLOT SUBMISSION
• AD HOC SURVEYS
• I WOULD NOT PARTICIPATE IN GENERAL COLLEGE ACTIVITIES ON-LINE

Q-20

If you were to use the Internet, which of the following College's Peer Assessment activities would you be receptive to completing on-line:

(Please check all that apply)

• COMPLETION OF INITIAL QUESTIONNAIRE
• RECEIVING NOTIFICATION OF YOUR ASSIGNED ASSESSORS (AND GENERAL INFORMATION ABOUT THEM, I.E., BRIEF BIO)
• RECEIPT OF GENERAL PEER ASSESSMENT INFORMATION
• PARTICIPATION IN SELF ASSESSMENT MODULES
• TRANSFER OF PATIENT RECORDS FOR REVIEW BY ASSESSORS
• RECEIPT OF ASSESSMENT RESULTS AND FEEDBACK FROM ASSESSORS
• COMPLETING POST-ASSESSMENT QUESTIONNAIRES ABOUT YOUR ASSESSMENT EXPERIENCE
• UNLIMITED ACCESS TO "WHAT STAGE IS MY ASSESSMENT PROCESS AT?"
• IMMEDIATE ACCESS TO COLLEGE STAFF COORDINATING THE ASSESSMENT
• OTHER (PLEASE SPECIFY):
• I WOULD NOT PARTICIPATE IN THE COLLEGE'S PEER ASSESSMENT ACTIVITIES ON-LINE

We are interested in your comments concerning Internet use/barriers in your practice. Please feel free to tell us your thoughts.

COMMENTS:

Thank you for your time.

BACK COVER:

If you have any questions or require assistance, please contact:

Elizabeth Wenghofer
Quality Management Division
College of Physicians and Surgeons of Ontario
80 College Street
Toronto, ON M5G 2E2
Tel (416) 967-2600 Ext. 450
Toll Free 1-(800)-268-7096 ext.450
Fax (416) 967-2605
Email: ewenghofer@cpso.on.ca

Appendix 2: Project Milestones and Deliverables

Deliverable / Milestone		Start Date	Target Date	Complete Date
1. Physician Selection (test sample)/Survey Tracking Database and Reporting Requirements/Survey Item Development/Survey Layout Design (Mail Survey and On-line)		24 APR 02	14 JUN 02	14 JUN 02
2. On-line Survey Technical Requirements Development and Application Development RFP Process		06 MAY 02	26 JUL 02	26 JUL 02
3. Survey Pilot, Pilot Analysis, Design Editing		22 JUL 02	12 AUG 02	12 AUG 02
4. Survey/Reminder Printing		14 AUG 02	23 SEP 02	23 SEP 02
5. Application Development		19 AUG 02	04 OCT 02	04 OCT 02
6. Application Testing and Survey Packaging		07 OCT 02	25 OCT 02	01 NOV 02
7. Survey Administration, Data Collection and Follow-up	Mail Out Survey	04 NOV 02	04 NOV 02	04 NOV 02
	Mail Out Received (estimate)	08 NOV 02	N/A	N/A
	Mail Reminder Postcard	11 NOV 02	12 NOV 02	12 NOV 02
	Response Deadline	N/A	22 NOV 02	N/A
	1st Reminder Letter to non-respondents only - 3 wks after initial mail out; enclose letter with a 2nd survey	N/A	25 NOV 02	28 NOV 02
	2nd Reminder Letter to non-respondents only - 7 wks after initial mail out; letter only	N/A	09 DEC 02	13 DEC 02
	Collection Cut-off	N/A	20 DEC 02	10 JAN 03
	Data input deadline	N/A	20 DEC 02	13 JAN 03
	Data Cleaning completed (i.e. analysis set up)	06 JAN 03	13 JAN 03	20 JAN 03
8. Data analysis, Final Report and Dissemination Plan including Evaluation		20 JAN 03	28 FEB 03	28 FEB 03

Appendix 3: On-Line Survey Application Documentation

OHIH Survey Overview

blue sands Inc.

Internet Integration and the Membership Survey Facility

1. Survey Overview

This survey was primarily delivered using packaged survey generation software from Perseus Survey Solutions (http://www.perseus.com). This software allows the user to generate the HTML necessary to provide a web-based version of a questionnaire, and store information in any number of commercially available databases. The software also can be used to generate paper surveys and the table structure necessary to capture survey information. Details of the custom CPSO implementation follow.

2. CPSO Implementation

In the CPSO web implementation, the database used to store the survey responses was MS-SQL Server 7.0. Utilizing Perseus-supplied options and ODBC, we configured the Perseus Portal to populate a SQL table with all responses. We also configured the portal to re-populate the responses when a user went into the survey to review their answers. Instructions on how to configure these options are provided in the Perseus documentation.

In order to provide additional flexibility in formulating the questions, the Perseus-generated HTML was modified manually. Specific information on how to perform these modifications is outlined in this document. (Section 5. HTML Modifications)

All questions were initially set to "mandatory". Subsequent testing determined that the processing time associated with the web pages determining the status of each question was unreasonable on slow pc's. As a result all questions, except for the first one, were set to "optional".

3. Operating Environment

The following software components were used for the survey.

• Windows 2000 Server, SP2
• Microsoft SQL Server 7.0, SP4
• Perseus SurveySolutions XP Enterprise Version 5.0
• IIS 5.0

4. Accessing the Survey File

The survey definition is contained in the survey.que file located on the Survey Overview CD. In order to open this file you must obtain a copy of Perseus SurveySolutions XP Version 5.0.

5. HTML Modifications

The CPSO web version of the survey required functionality that is not provided for by the Perseus product. Specifically Questions 18 and 19 need to be modified to allow for the submission of a free form text field. This field is the last selection option for each of these questions and is labeled as "Other". In order to add these fields a few manual changes need to be made to the Perseus generated HTML. The following instructions will provide you with the information necessary to make these modifications.

1. If you are using a database such as MS-SQL Server to hold the responses you will need to add two additional columns to the response table. To do this you must first create the table in a test/development environment by submitting a survey response. You should then edit this table and add the columns listed in the table below, remove the test response that was entered to create the table originally, then move this altered table to your production database.

Column Name <	Type
Q18Specified_11	Varchar(255)
Q19Specified_11	Varchar(255)

You may also run the ohih_survey.sql script that is contained in the Survey Overview CD.

2. Edit the HTML file that is generated by Perseus and make the following edits.

a) Locate the string "PdcFrmFld" and replace the entire line it is contained in with the following:

Q1; Q2_1; Q3_1; Q3_2; Q4; Q5_1; Q5_2; Q5_3; Q5_4; Q6; Q7_1; Q7_2; Q7_3; Q7Specified_3; Q8_1; Q8_2; Q9_1; Q9_2; Q9_3; Q9_4; Q9_5; Q9_6; Q9_7; Q9_8; Q9_9; Q9_10; Q9_11; Q9Specified_11; Q10_1; Q10_2; Q10_3; Q10_4; Q10_5; Q10_6; Q10_7; Q10_8; Q10_9; Q10_10; Q10Specified_10; Q11_1; Q11_2; Q11_3; Q11_4; Q11_5; Q11_6; Q11_7; Q11_8; Q11_9; Q11_10; Q11_11; Q11_12; Q11_13; Q11_14; Q11_15; Q11Specified_15; Q12_1; Q12_2; Q12_3; Q12_4; Q12_5; Q12_6; Q12_7; Q12_8; Q12_9; Q12_10; Q12_11; Q12Specified_11; Q13_A_1; Q13_A_2; Q13_A_3; Q13_A_4; Q13_A_5; Q13_A_6; Q14_1; Q14_2; Q14_3; Q14_4; Q14Specified_4; Q15; Q16; Q17; Q18_A_1; Q18_A_2; Q18_A_3; Q18_A_4; Q18_A_5; Q18_A_6; Q18_A_7; Q18_A_8; Q18_A_9; Q18_A_10; Q18_A_11; Q18Specified_11; Q19_A_1; Q19_A_2; Q19_A_3; Q19_A_4; Q19_A_5; Q19_A_6; Q19_A_7; Q19_A_8; Q19_A_9; Q19_A_10; Q19_A_11; Q20_1; Q20_2; Q20_3; Q20_4; Q20_5; Q20_6; Q21_1; Q21_2; Q21_3; Q21_4; Q21_5; Q21_6; Q21_7; Q21_8; Q21_9; Q21_10; Q21_11; Q21Specified_10; QComments; "/>

b) Locate the string "PdcFldTyp" and replace the entire line it is contained in with the following:

T; N; T; T; T; N; N; N; N; N; N; N; N; N; T; F; F; N; N; N; N; N; N; N; N; N; N; N; T; N; N; N; N; N; N; N; N; N; N; T; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; T; N; N; N; N; N; N; N; N; N; N; N; T; N; N; N; N; N; N; N; N; N; N; T; N; N; N; N; N; N; N; N; N; N; N; N; N; N; T; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; T; M; "/>

c) Locate the string "PdcCurPgeFrmFld" and replace the entire line it is contained in with the following:

Q1; Q2_1; Q3_1; Q3_2; Q4; Q5_1; Q5_2; Q5_3; Q5_4; Q6; Q7_1; Q7_2; Q7_3; Q7Specified_3; Q8_1; Q8_2; Q9_1; Q9_2; Q9_3; Q9_4; Q9_5; Q9_6; Q9_7; Q9_8; Q9_9; Q9_10; Q9_11; Q9Specified_11; Q10_1; Q10_2; Q10_3; Q10_4; Q10_5; Q10_6; Q10_7; Q10_8; Q10_9; Q10_10; Q10Specified_10; Q11_1; Q11_2; Q11_3; Q11_4; Q11_5; Q11_6; Q11_7; Q11_8; Q11_9; Q11_10; Q11_11; Q11_12; Q11_13; Q11_14; Q11_15; Q11Specified_15; Q12_1; Q12_2; Q12_3; Q12_4; Q12_5; Q12_6; Q12_7; Q12_8; Q12_9; Q12_10; Q12_11; Q12Specified_11; Q13_A_1; Q13_A_2; Q13_A_3; Q13_A_4; Q13_A_5; Q13_A_6; Q14_1; Q14_2; Q14_3; Q14_4; Q14Specified_4; Q15; Q16; Q17; Q18_A_1; Q18_A_2; Q18_A_3; Q18_A_4; Q18_A_5; Q18_A_6; Q18_A_7; Q18_A_8; Q18_A_9; Q18_A_10; Q18_A_11; Q18Specified_11; Q19_A_1; Q19_A_2; Q19_A_3; Q19_A_4; Q19_A_5; Q19_A_6; Q19_A_7; Q19_A_8; Q19_A_9; Q19_A_10; Q19_A_11; Q20_1; Q20_2; Q20_3; Q20_4; Q20_5; Q20_6; Q21_1; Q21_2; Q21_3; Q21_4; Q21_5; Q21_6; Q21_7; Q21_8; Q21_9; Q21_10; Q21_11; Q21Specified_10; QComments; "/>

d) Locate the string "PdcCurPgeFldTyp" and replace the entire line it is contained in with the following:

T; T; T; N; N; N; N; N; N; N; N; N; T; F; F; N; N; N; N; N; N; N; N; N; N; N; T; N; N; N; N; N; N; N; N; N; N; T; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; T; N; N; N; N; N; N; N; N; N; N; N; T; N; N; N; N; N; N; N; N; N; N; T; N; N; N; N; N; N; N; N; N; N; N; N; N; N; T; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; N; T; M; "/>

e) Locate the 5th occurrence of following string "</table></blockquote>" and insert the following block of HTML code immediately above it.

<td><input type="radio" name="Q18_A_11" id="Q18_A_11" value="1" /></td>

<td><input type="radio" name="Q18_A_11" id="Q18_A_11" value="2" /></td>

<td><input type="radio" name="Q18_A_11" id="Q18_A_11" value="3" /></td>

<td><input type="radio" name="Q18_A_11" id="Q18_A_11" value="4" /></td>

<td><input type="radio" name="Q18_A_11" id="Q18_A_11" value="5" /></td>

<td><input type="radio" name="Q18_A_11" id="Q18_A_11" value="9" /><>/td>

</tr>

f) Locate the 6th occurrence of following string "</table></blockquote>" and insert the following block of HTML code immediately above it.

<td><input type="radio" name="Q19_A_11" id="Q19_A_11" value="1" /></td>

<td><input type="radio" name="Q19_A_11" id="Q19_A_11" value="2" /></td>

<td><input type="radio" name="Q19_A_11" id="Q19_A_11" value="3" /></td>

<td><input type="radio" name="Q19_A_11" id="Q19_A_11" value="4" /></td>

<td><input type="radio" name="Q19_A_11" id="Q19_A_11" value="5" /></td>

<td><input type="radio" name="Q19_A_11" id="Q19_A_11" value="9" /></td>

</tr>