COMPETE II - Computerization of Medical Practices for the Enhancement of Therapeutic Effectiveness

Health and the Information Highway Division, Health Canada
December 2003

Table of Contents

• 1.0 Introduction
• 2.0 Project Description
• 3.0 Achievements
  • 3.1 Goals Reached
  • 3.2 Additional Successes
  • 3.3 Unreached Goals
  • 3.4 Documents or Products Generated
• 4.0 Main Impact
  • 4.1 Human Resources Impact
  • 4.2 Privacy and Protection of Information
  • 4.3 Policy and Research Implications
• 5.0 The Future
• 6.0 Communications

1.0 Introduction

The COMPETE II Project was conducted by the Centre for Evaluation of Medicines, St Joseph's Hospital and the CHIPP-funded portion of the study ran from June 2001 to September 30th 2003. The project is a continuation of a research program designed to investigate electronic medical records (EMRs) and computerized decision support as emerging health technologies in Canada. COMPETE I investigated the impacts of EMR selection, implementation and level of use on physician, staff and patient work flow, satisfaction and quality of care, as well as developing electronic prescribing guidance and quality assessment tools, code of conduct for health data privacy, advanced implementation and support procedures and EHR-based data extraction procedures. COMPETE II is a randomized controlled trial to investigate the impact of a Web-based, continuously updated, patient-specific diabetes tracker available to patient and physician plus an automated telephone reminder service for patients, on access, quality, satisfaction and continuity of care. COMPETE II continues on and final data analysis, which CHIPP did not fund, will not be complete until early 2004. There were three primary regions involved in the trial - Hamilton area, Sault Ste Marie, Ottawa. We recruited 47 providers, (physicians and nurse practitioners) and 512 patients (mean age 61 yr, 50% female, 78% completed high school education) to the COMPETE II diabetes study.

2.0 Project Description

The COMPETE II Mission Statement was:

To develop an effective, integrated clinical decision-support tool for patients and health care providers. We will develop and test a diabetes tracker as a model for evidence-based chronic disease management. We will build the diabetes tracker on a base of a standardized core data set (CDS) for health which we will advance to serve as a national prototype. In collaboration with our partners and federal and provincial lead health organizations, this project will provide a clear pathway to an integrated national electronic health record network.

At the outset of the project, we stated: "Although many aspects of clinical and technical integration of care will be tested, the key measures of success of the project will be:

1. Development and evaluation of a standardized, core data set for health with diabetes extensions. This extended core data set (ECDS) will serve a key health Infostructure function: the ability to standardize patient data from multiple electronic health record (EHR) systems for the purposes of sharing information and for providing clinical decision support in a sustainable and scalable manner.
2. Development and evaluation of a diabetes tracker tool that will assist patients and providers to communicate and manage this chronic disease according to individualized best evidence. The tracker will leverage current interface technologies, including telephony, fax, Internet and PDA, to collect, organize and feedback information, advice and strategies to patients and clinicians in a secure environment seamless with their current workflow."

The primary objective of the project was to determine whether technology-based, integrated delivery systems shared by key providers (family physicians and nurses) and patients, could improve access to care, quality of care, and efficiency of care. Because family physicians provide 80% of the care received by Canadian patients, they are the logical focus for the integration of health care services. The COMPETE II CHIPP project was meant to specifically remedy the current irony where those mainly responsible for prevention, diagnosis, management and co-ordination of care - primary care physicians, have been largely excluded from technological support for better health care.

The COMPETE investigators developed and manage Canada's original primary care research network in electronic health records (EHRs), which has successfully implemented full EHRs in community practices with electronic laboratory communication and basic drug and disease decision support. The network continues to expand its membership and its use of information communication technology to enhance both clinical and technical integration and quality of care.

The COMPETE II project developed and is evaluating an integrated, Web-based clinical decision-support tool for diabetes shared by patients and health care providers. The COMPETE II diabetes tracker (CIIDT) is a model for evidence-based chronic disease management in Canada and internationally. The CIIDT is built on a base of a standardized cored data set (CDS) for health which serves as a national prototype for a sharable electronic health record. The project is integrated the diabetes tracker with leading electronic health records (EHRs) to allow the seamless flow of information between the decision support and the EHR, for improved clinical efficiency. The Web-based tracker provides both patients and clinicians immediate, secure access to the patient's 14 key diabetes variables over time compared to targets, as well as short supportive recommendations and the latest, best evidence on each topic in an intuitive colour-coded format. Patients use a voice biometric to receive regular automated telephone reminder messages regarding their appointments, prescriptions and lab tests.

Through its foundation work on data standards, core data sets for EHR, XML data integration, EHR vendor participation recruitment, clinical decision support development and evaluation, and maintenance of its frontline clinical users network, COMPETE II provides a clear pathway to an integrated national electronic health record network. The research expertise of the investigators ensures that each of the major components receives a rigorous evaluation of its impact, both positive and negative.

A randomized controlled trial centred in family physician offices in three Ontario regions - Hamilton, Sault Ste Marie and Ottawa, comparing access to the tracker system versus usual care is nearing completion. The evaluation examines clinical processes and diabetes outcomes (quality of care), usability and ease of use of the tracker and telephone system, health data privacy, continuity of care, quality of life, scalability and sustainability of the intervention. Since new health care technologies are usually poorly evaluated despite their high costs, potential harm and change management needs, this project will yield information previously unattainable but essential to clinicians, patients, researchers and policy makers.

Partners and associates include St. Josephs Healthcare of Hamilton, McMaster University Group Health Co-operative of Sault Ste. Marie, University of Ottawa Primary Care Informatics Group, Diabetes Hamilton, Tagge Medical Solutions Inc., Brogan Consulting Inc, Killdara, Optium Digital Solutions, Clinicare, PEPPER, OSCAR, York Med/Purkinje, MacMedical and P and P Systems.

3.0 Achievements

3.1 Goals Reached

What elements in the originally approved work plan were achieved?

1. Project Management Setup and Control
   1. Vision and direction - maintained, but we scaled back on original vision plan, as the original scope exceeded the scale-back in budget from CHIPP.
   2. Project management and meetings - worked well, maintained order and direction, continuity.
   3. Project management - suboptimal. External, part-time project management without at least moderate clinical or technical competence, is not recommended. Very difficult to find high quality project management in this complex area of research.
   4. Technical infrastructure - Real-time, clinical Web server, knowledge algorithms, computer-assisted telephone interviewing for evaluation, automated telephone reminder system, data integration between 3 of the EHRs and the central data repository, secure access for providers and patients, SQL research-ready database - all developed, tested and all successfully completed except full data integration. Our technical infrastructure is so complex and reliant on so many groups, most external to the coordinating research centre, that support and upgrading is a frequent issue. Provincial health IT infrastructure is required for sustainability.
2. Project Scoping and initiation
   CHIPP application process unfamiliar, very steep learning curve. Partner contacts/contracts developed with key partners. Developed a strong evaluation framework. Good review of technical framework and resources that are available. The rigor of detail necessary for project accounting was a significant expense to the project at the cost of the research.
3. Information Management Scenario and Design
   Achieved. Complex information system developed for patients and providers. Neither group was using the Web for specific health advice before. Various problems with delayed (>2 yr) provincial Internet infrastructure, local EHR limitations, styles of workflow, lack of computer use, forgetting passwords, vagaries of down-time for a university-based Web server, etc, meant that constant manual vigilance to maintain information flow and communications, was necessary. Nonetheless, the power of such an individualized, up-to-date, geography-neutral system, to reach all regions of Canada no matter how remote, has major potential.
4. Diabetes Tracker Development, Testing and Integration training program, help desk, training manuals etc went together well, worked effectively. The development of the Software for the tracker was a lengthy process. The tracker is cumbersome in areas but it works and delivers information to both patients and physicians. The programming expertise required to develop such complex systems, is rare, not generally available in the public sector in health and the private sector is unstable. Two private sector partners dissolved or disappeared during the short life span of the project. Large "brand name" software companies were considered and interviewed, but tended to be totally bereft of content knowledge, inflexible in approach and prohibitively expensive. This is a major barrier to high quality e-health development in Canada.
5. Implementation
   Recruitment goals were readily met for both Physician and patient due to the history of COMPETE, the involvement of the investigators (virtually all influential clinicians) and the awareness of and the attention to the needs of participants. Usage of the CII Web tracker interfaces and the phone system by both physicians and patients were less than perfect, as expected. For virtually every step of the study, we developed work-around procedures in case "Plan A" did not work. This is always a necessity when working at the leading edge of any novel area of research and reinforces how critical high quality evaluation is to have a balanced conclusion of the effectiveness and cost-effectiveness of the intervention.
6. Evaluation
   Evaluation of e-health interventions is the forte of the project and few groups anywhere have the level of expertise and experience in rigorous evaluation of emerging electronic health technologies. Although CHIPP did not fund the full evaluation of our randomized controlled trial, many of the interim results of the evaluation tools are presented in the project evaluation report. Our evaluation includes comparison of end-study with baseline questionnaires from patients and providers for 6 main domains, analyses of the research data repository for actual clinical and process outcomes, analyses of support line files, analyses of summary patient health profiles collected by computer-assisted methods and formal chart review.

What were the contributing key factors?

• Experienced, dedicated staff familiar with uncertainties, need for flexibility and creativity and collaboration in this area.
• Excellent, supportive partners. The partners participated through the entire process and assisted in making the project a success.
• Experienced, expert investigators able to lead and troubleshoot.

What were the obstacles or challenges that had an impact on your achievements and how did you deal with them?

Some significant challenges:

1. Conflict of interest between our project manager (PM) and our developer. The project manager was privy to the day to day running and financial management of the project but was also a key employee of the software developer. Eventually PM fired.
2. Lack of standard development processes and procedures. Clinical Decision Support software is a mission critical, high risk software application. Developing this type of software requires high quality software development processes, such as the Rational Unified Process and the use of Unified Modelling Language processes and procedures. Yet, our developers were not versed or trained in the use of these processes. Hence, the development and testing processes took much longer than anticipated. In the end, we had to rely on human interventions and constant checking of the algorithm outputs to ensure that the software was working as required.
3. Death of a key resource. Our chief informatics expert died unexpectedly soon after development of the software was started. We had to make do with existing resources to make things work. Luckily, most of the documentation work had already been done.
4. Lack of secure messaging infrastructure as promised by the Smart Systems for Health Agency. They are behind in their timelines for providing secure communications channels to physician offices. We ended up using sub-optimal work-arounds. In addition, we were unable to recruit certain sites where the existing quality of the communication channels was inadequate.
5. Onerous documentation requirements from Health Canada. The documentation was non-standard and was not appropriate to managing a large-scale research project. We could not use standard tools available to meet the requirements and this caused excessive delays and costs, which did not add value to the project. We understand the need for accountability but more streamlined processes are required.
6. Electronic Health Record vendors are not yet sufficiently motivated or experienced enough to develop the interfaces required to support the integration between EHRs and the COMPETE II Clinical Data Repository (CDR). We ended up having the physicians fax us their clinical notes and they were entered into the CDR manually at the research centre. EHR integration at the scale intended in Canada, will require much greater direction from policy makers and much greater communication with groups like ours working at the clinical "coal face".
7. Lack of clinical standards. The lack of well-established and usable standards in documenting clinical encounters in primary care (symptoms, physical findings, diagnoses, lab results, medications, procedures, etc) was also a draw back for the project. Although we were able to select certain standards, most EHR vendors were not able to meet the requirements. As above, data standards need to be set at a national level for this "data anarchy" to improve.
8. Lack of human resources with standards knowledge and expertise. Perhaps the biggest constraint that is difficult to overcome is the lack of qualified human resources with knowledge and expertise in implementing standards, both messaging and data/vocabulary standards. This was overcome by using approaches that were as close to the standards as possible. The HL7 version 3 messaging standard was still evolving at the time of the study, so it would have been difficult to implement it properly.

3.2 Additional Successes

What elements were achieved beyond the originally approved work plan?

1. Proof that there is need for a structured core data set. Simple is best. The process of integration is very complex and costly. Need several key individuals - technical and clinical experts - to work on process together over time.
2. Patient input to data collection is very helpful and a necessity.
3. A better understanding of the role and importance of patients in the clinical decision support process. In the next project, we will place more emphasis on patients and getting patients involved sooner.
4. Evaluation scope and depth succeeded beyond expected.

3.3 Unreached Goals

What elements from the originally approved work plan were not accomplished or sustained?

1. The integration process was not fully completed. Although data was successfully transferred from local EHRs to the central data repository through firewalls, staging tables, XML conversion, etc, the final step of advancing through the knowledge algorithms and displaying in the Web tracker interface, could not be completed due to lack of time and resources. This full integration process turned out to be more complicated than envisaged but is primarily a problem of lack of simple data field standards.
2. An originally intended Web to fax component had to be dropped due to expense. Given the lack of use of computers directly by many older patients, telephony and fax may be the most viable way of reaching all patients currently.

3.4 Documents or Products Generated

See provisional list below.

Document /Product Name	Available in Paper and /or Electronic Form	Licence Fee Required for use (Yes/No)	Previously Provided to Health Canada (Yes/No)	Appendix Name /Number
Template(s) for vendor contract(s)	COMPETE II Physician contract - blank COMPETE II Partner Contribution Form	No No	No No	CDa CD
User Guide(s) and/or Training Manual(s)	COMPETE II Physician's User Guide COMPETE II Patient's User Guide	Yes Yes	Yes Yes	CD CD
Template(s) for equipment testing	COMPETE II Test Patient templates	Yes	No	CD
Policy and Procedure Manual(s)	COMPETE II Code of Conduct COMPETE II Physician Site Checklist	No Yes	Yes Yes	CD No
Job Descriptions and / or recruitment material	COMPETE II Recruitment Presentation COMPETE II Sales and Marketing Plan (Change Management Process)	No Yes	Yes No	CD CD
Software Applications, includes: EHR application Security /access alert software Telehealth scheduling software Other	COMPETE II Diabetes Tracker webshots - Patient COMPETE II Diabetes Tracker webshots - Physician COMPETE II Data Flow - Clinical Diagram COMPETE II Data Flow - Technical Diagram Clinical Integration for COMPETE II COMPETE II Data Integration Review COMPETE II Diabetes Tracker Algorithms	Yes Yes No No No No Yes	No No No No No No No	CD CD CD CD CD CD CD
Standards, includes: Data (includes minimal data sets) Image Messaging Other	COMPETE II extended Core Data Set	No (public domain)	Yes	CD
Clinical Training Protocols	COMPETE II Physician's User Guide	Yes	Yes	CD
Clinical Program Protocol(s)	COMPETE II Evaluation Plan	Yes	Yes	CD
Video Conference Protocols and Etiquette Guide
Quality Assurance Procedures	Sample of COMPETE II Randomization Allocations TAGGE Flow chart and scripts	No No	No No	CD CD
Confidentiality and Privacy documents	COMPETE II Code of Conduct COMPETE II Confidentiality Letter	No	No	CD
Consent Forms	COMPETE II Patient Consent Package	No	No	CD
Sustainability Plan	COMPETE II Commercialization Notes- July 11, 2002	Yes	No	CD

^a The name of the document on the CD is the same name that is in the 2^nd column entitled Available in Paper/Electronic Format.

4.0 Main Impact

Your project received CHIPP funding because of its expected impact on delivery of health care services. We would like to find out in your own words just what impact your project had. Some things to think about in responding to this section are:

• What difference is your project making in your community or your region?

The COMPETE II project is changing the way policy makers think about the care of diabetes and, by corollary, other chronic diseases. There is greater realization that there are many deficiencies in patient care and that new tools are required to improve care. There is a greater realization that traditional methods of creating physician behavior change have not worked and that technology may be an appropriate solution that needs to be investigated further. The unexpected enthusiasm for such tools and relative lack of concern about data privacy on the part of patients, is a wake-up call to the power of motivating self-management electronically.

Given the lack of access of many communities to tertiary health care resources or best evidence regarding chronic diseases, the potential power of electronic guidance linked to the patient's own medical file, is great.

The COMPETE II meshes extremely well with community-based diabetes outreach projects such as Diabetes Hamilton. This has considerable generalizability to other communities. COMPETE II would also enhance any organized diabetes self-management program.

• What difference is your project making to your patients, to health care professionals, or to your organization's management and structure?

The COMPETE II Diabetes Tracker has helped streamline care of patients with diabetes by ensuring that they get their lab tests done before they see their doctor (80% of patients in the intervention arm got their lab tests done before their regular visit). Patients are pleased to have access to their own information and appreciate receiving practice advice customized for them in an understandable manner.

Physician data is not yet evaluated but we suspect that the lack of full integration of updated tracker information with their usual EHR, will be a drawback.

Given the very short timelines of the CHIPP project, COMPETE II is considered a pilot project which very much supports proof of concept that electronic technologies that support integration of care of chronic diseases for patients and providers, have considerable potential for benefit. Since all electronic systems tend to be expensive, need frequent upgrading, updating and support, the real question is cost-effectiveness.

The main impact of COMPETE II hopefully will be to spawn further research on this type of care model fine-tuned to provide more case management to assist providers and more flexible, priorities-based interventions for patients.

4.1 Human Resources Impact

Please indicate what short and long term impacts your project has had on human resources. Some things to think about are:

• What new skills have staff/health care providers developed? Are these skills transferable to other jobs or situations?

Providers and patients have hopefully learned the value of some structure in patient data charting. The traditional, totally text-based medical record is completely incompatible with clinical decision support, quality assurance and patient safety review, and evidence-based health surveillance and policy direction.

Staff have gained skills in the increased complexity of project management, financial management, chart review, complex data analysis, Web-based system maintenance and security, etc.

• Have new positions been developed?

Several positions developed but unfunded. This is the major problem with one-time grants.

• Has the new technology increased demand on service providers? If so, how is it being managed?

A deliberate goal of the project was to have patients regularly see their family physician. For some, this might be an increase in their usual visit pattern. These visits in turn are meant to generate referrals for evidence-based preventive care. Although perhaps a burden initially, the full use of the tracker makes the complex management of 14 variables faster, easier to grasp and much easier to maintain over the long-term. Furthermore, where resources allow, the tracker would allow management of diabetes by qualified nursing or pharmacist staff with offloading of physician time to deal with more complicated issues.

• Has job satisfaction changed?

Patient satisfaction has increased on average. Most providers are unlikely to report a difference, given the short lifespan of the project. Any benefits are offset by the need to learn new processes. Longer term interventions with sufficient time for iterative improvement, are required.

Job satisfaction is important but a secondary outcome compared to quality of care.

• Has productivity changed?

Again unlikely to demonstrate a clear difference as benefit in terms of more information on the patient being available at each encounter is likely offset by learning how to use the Web interface and initially having many patient variables off-target and needing attention. The true question is whether over the longer term (> 2 years) would such a system improve care with at least neutral effect on time input.

Productivity is important but only if it improves quality of care.

• Was there resistance to change?

There is always resistance to change. One of the best motivators for change, which we were unable to use prospectively, is to demonstrate baseline time inefficiency, low quality or unnecessary expense which could be remedied by the intervention.

4.2 Privacy and Protection of Information

We would like you to provide information about what may have happened in the area of privacy and the protection of personal health information as a result of your project. Some things to think about are:

• Have there been changes in the behaviour of health professionals/staff with respect to the protection of personal health information?

Yes, security is a higher priority as patients are using the Internet to access some of their data.

• Have changes been made to your policies and procedures?

COMPETE already maintains a CSA-adherent code of conduct which was modified to account for COMPETE II activities.

• Have you completed a Privacy Impact Assessment? If yes, what did it show? Did you change any policies or procedures as a result of the Privacy Impact Assessment?

Carried out informally. A real-time clinical Web server functioning off-site of the clinical setting is a major unresolved privacy issue for the future.

• Did you participate in the CHIPP Privacy and Security Survey conducted in August 2002? If yes, have you implemented the recommendations provided to you?

Yes we did participate, but we have yet to receive any feedback or recommendations.

• Did you receive any patient views/concerns/compliments on the handling and use of their personal information?

Most patients were very pleased with the process, with our experience in the area and with their ability to view their data. No real complaints even though they had trouble remembering their passwords. Their views on health data privacy were explicitly surveyed and are presented in our evaluation report.

• Did any of your privacy/confidentiality rules/guidelines help or hinder your health care providers (all staff) to provide patient service?

Only to the extent that forgetting a password or the Web server being unexpectedly down, wastes time and decreases use of the system.

• Was patient privacy raised as an issue in any way during the trial?

Specifically a focus of the research as outlined above.

• What tools were used for obtaining and managing patient consent?

Combination of fax, paper, telephone (electronic/paper/combination)

• Were there instances where patients refused consent and, if yes, what were the reasons given?

Yes, a few patients refused. Reasons given were very typical of any area of research -they report that they are too old or sick, they are not interested, they did not believe that they had diabetes, or they did not want to spend the time to answer the questionnaires (approximately an hour at baseline).

• Will you be conducting regular Privacy Impact Assessments? If not done routinely, under what circumstances will revised Privacy Impact Assessments be conducted?

No

• Have you participated in discussions with representatives of provincial health infostructure networks on privacy and protection of personal health information?

Yes

• Do you feel your project has influenced privacy policy and process development in your jurisdiction? If yes, in what way?

Yes. Influence would be mostly word of mouth or by a few requests for our Code of Conduct. We have participated in several national projects and guidelines commissions examining health data privacy issues in the era of computerized health databases and EHRs.

4.3 Policy and Research Implications

What results do you feel can be used to influence key groups, based on the knowledge and experience gained by your project? Consider stakeholders such as government policy makers at the federal, provincial, and territorial levels; health planners; health administrators; and health, health care, and technology development researchers.

Please include your recommendations for next steps based on these major findings. For example, this could include privacy protection, supplier readiness, physician readiness, additional research, and potential pilot projects. Here is a suggested structure:

Major finding #1: (one paragraph)

• Policy implications;
• Applied research implications; and
• Recommendations for next steps.

Please create as many "Major finding" clusters as you need.

These are described throughout this report, the evaluation report and several of our presentations in our portfolio.
The major implications of our work are that providers, particularly physicians, are the key stakeholder in any electronic health strategy and the continued disregard for their needs, their complex workflow requirements and unprecedented information demands, will jeopardize Canada's entire EHR strategy. Only two major groups work directly with clinicians to advance their EHR and decision support needs - both are academic and unsupported by e-health strategy groups. We strongly recommend support for ongoing investigation, implementation and evaluation of clinical e-health systems.

5.0 The Future

What are your plans for maintaining or developing your project once CHIPP funding ends? If you have already prepared a sustainability plan, please attach it as an appendix.

Sustainability plan in appendix. Sustainability is a major problem without a coordinated federal or provincial approach to EHRs and clinical decision support systems. The current exclusive interest in infrastructure is counterproductive to Canada's e-health development which should be very much focused on the frontlines of health care.

6.0 Communications

Clearly, communications is important in generating support for projects -- support in the community and among other interested groups. What methods or tools did you use to communicate with your stakeholders? Please provide a copy of the documents you will list below as appendices.

(To create a new row, simply position your cursor in the last cell and press 'tab'.)

Methods or Tools	Date	Targeted Audience	Documents or Presentations Produced	Appendix Name/ Number
Con-ferences	E-Health May 23 - 27, 2003 Int'l Society for Pharma-coepid- emiology 2003 Scheduled for 2004 conferences X 3	Physicians and stakeholders interested in e-heath Int'l researchers, drug policy makers & advisors	Proceedings papers, presentations	Abstract published, presentation
Media Events	Press Conference March 2002	Everyone living and working in the Diabetes world	COMPETE II Press kit
Pub-lications	2002 - 03	Local, national, international	Local newspapers, newsletters, COMPETE Web site, International Journal Medical Informatics, Pharma-coepidem-iology journal
Open House	June 2002
Marketing / Advertis-sement	2002, 2004 (intended)	General audience	Newspapers, Web, network
Website (please provide the Website address)	www. compete-study.com Continuous	Physicians, partners, Diabetes patients, Interested parties	COMPETE II team and partners' bio's presentations made updates on recruitment. Core data set, manuals created
Conference calls	September 2002 - September 2003 [weekly team telemeetings]		Conference Call minutes