Building Working Knowledge: Exchanging Web-Based Concepts and Tools

Health and the Information Highway Division, Health Canada 2003

Table of Contents

• 1. Introduction
• 2. Web-Based Concepts and Tools
• 3. Evaluation
• 4. List of Workshop Presentations and Lab Sessions
• 5. References

1. Introduction

Canadian provincial health data bases have created "information-rich" research environments (Roos and Roos, 2001). In health services research and epidemiology, the growth of large administrative databases and the teamwork required for their use pose significant management challenges. Researchers across Canada are using these data without taking sufficient advantage of economies of scale. These tools, designed with the support of the Knowledge Development and Exchange Applied Research Initiative and presented on the Manitoba Centre for Health Policy (MCHP) web site (www.umanitoba.ca/centres/mchp/concept), are designed to improve productivity nationally and internationally; they include conceptual definitions and frameworks with broad applicability to population-based health services research.

2. Web-Based Concepts and Tools

Aiming for a consistent approach to research design, MCHP has created a publicly-accessible, web-based knowledge repository (the Concept Dictionary and accompanying glossary and protocols) comprising various concepts, methods, and tools appropriate for working with administrative data. The potential for error and for "re-inventing the wheel" is reduced, helping the Centre to carry out its mandate to provide accurate and timely information to health care decision-makers, analysts, and providers; i.e., to offer services which are effective and efficient in improving the health of Manitobans. By making explicit the assumptions associated with use of a research measure, the repository encourages the use of standard terminology and methodology, helps to preserve institutional knowledge and prevent duplication of effort, facilitates collaboration, and aids project management (Burchill et al., 2000).

The University of Monash (Australia) has "mirrored" the Concept Dictionary and is developing additional SAS macros to be posted on the Internet. More recently, the UCLA ATS Statistical Computing Group added the MCHP web site to the UCLA
Statistical Computing Portal, a web site incorporating links to and searches across
Statistical Computing sites from around the world.

Over the past ten years a vast amount of epidemiological, institutional, and technical information has been amassed (Burchill et al., 2000). The external knowledge repository is currently accessible from the main page of the MCHP web site, the "Research Resources Index", which contains links to online research tools (Figure 1) (Burchill et al., 2000; Roos et al., 2003). Readers of this report might visit this web site (http://www.umanitoba.ca/centres/mchp/concept/) to fully appreciate the ease of accessibility and navigability supporting the wealth of information related to research with administrative databases. All of these tools were improved significantly with the support of Health Canada funding.

Figure 1 MCHP Research Resources Index

Concept Dictionary.

Standard data dictionaries typically describe the structure of a given database system, including "standard names for variables, formats, interpretation of the variables, and an overview of known problems and recommendations" (Burchill et al., 2000). The Concept Dictionary goes well beyond the Data Dictionary model by providing operational definitions of analytical concepts developed and used by local health researchers and programmers, describing in detail new variables and frameworks for analysis. Researchers have called for "metadata" - "data about data" - to describe the content, source, vintage, accuracy, quality, and other characteristics of data (Croner, 2003). Organized alphabetically, the Concept Dictionary responds to this need by currently including descriptions for well over 100 concepts, with SAS programs to implement the concept where appropriate, and discussions of alternate formulations and associated problems (Burchill et al., 2000). Local expertise and references are noted, and links to related terms are incorporated from other groups (e.g., ICES). Concepts have also been incorporated as hyperlinks in other research aids (such as research protocols) and in teaching materials (both site-specific courses and the Epidemiology Supercourse) (LaPorte et al., 2002).

Meta-Index and Glossary.

The concepts are organized according to the Medical Sub-Heading (MeSH) system of the National Library of Medicine. A glossary provides a shorter description of important terms (keyed to specific reports); 961 glossary terms and abbreviations were recently noted.

Protocol for Conducting Administrative Research.

This "intellectual middleware" provides a methodological "checklist" for the main steps required to carry out a study using administrative databases. It provides an alternative entry point to the wealth of information available in the Concept Dictionary and Glossary. The protocol currently organizes links to existing documentation into eight categories (e.g., proposal development, database preparation, project management). The protocol not only serves as a starting point for beginning investigators, but also enables more experienced researchers to acknowledge and document departures from standard methodological conventions.

Project Documentation.

Project-based web sites build on the Protocol. They have proven to be a valuable tool for managing and tracking a project through each of its stages. For complex projects particularly, they help to ensure that all project participants apply methodological criteria similarly, that concepts are suitably operationalized, and that analyses are appropriately comparable. A standard format for such web sites is evolving, one which complements and links to the existing concept-based knowledge repository. Several examples of project documentation (with modifications to accommodate confidentiality restrictions) are now posted to the public web site.

3. Evaluation

To date, MCHP has emphasized content over technology in web site development by combining functions cost-effectively in an external web site. Documentation and dissemination overlap to keep marginal costs low. Given the high cost of developing Internet-based educational materials (as was emphasized in a recent Canadian Association of University Teachers bulletin), the integration of this material into lectures and courses has been particularly cost-effective. Over the last year, the growth in usage has been substantial. MCHP began monitoring web site utilization in January 1998 using access logs from the University of Manitoba Apache WWW server. This approach permits counting the number of "hits" and unique "hosts" accessing and loading information from specified web site locations. A hit is counted once for every time a browser goes to the server and requests a new web page. A host represents a unique internet ID, either the ip (internet protocol) number or the resolved host name.

Care is required when interpreting information generated by automated computer monitoring. Hits, for example, are counted once for each graphic file within a web page, and once for
every time internet crawlers or "bots" are automatically sent by search engines as they update their own sites. Graphic files have been excluded from the counts in the graphs below (Figures 2 and 3), but Internet crawlers are not always easily identified. As well, certain hosts may represent a network of firewalled users who are not distinguishable using this method (for example, one host may represent 50 different users). On the other hand, because of the "mirroring" of the Concept Dictionary in Australia and "mirroring" of Epidemiology Supercourse lectures in approximately 25 servers worldwide, these figures probably represent a substantial underestimate of international interest.

Figures 2 and 3 track the average number of hits and hosts per day, respectively, for the last four years, for both the entire externally available web site and for the Research Resources Index, which includes the Concept Dictionary. Hits and hosts for the Research Index grew markedly over this period. The total average number of unique hosts accessing the MCHP web site has almost doubled for each consecutive year, suggesting that this trend may continue for 2003. The total average number of hosts per day increased to 402 in February 2003, up from 354 the previous month, representing a substantial increase over the 242 hosts per day documented for February the previous year. Average total number of hits for February rose to 2,098, an increase of about 22% from the 1,714 tracked the previous month, and almost 50% higher than the 1,415 hits in February of the previous year. Hits and hosts in March 2003 increased over those in February.

Our work over the last year has been recognized with the acceptance of our paper "Working More Productively" for publication in Health Services Research.

4. List of Workshop Presentations and Lab Sessions

Finally, a number of well-attended workshop presentations and lab sessions were carried out with the aid of Health Canada funding. These workshop presentations and lab sessions are noted below:

Roos L. Concept Dictionary lab session (presented after the Socio-Economic Dimensions of Housing and Health Workshop, Winnipeg, MB, January 2003).

Soodeen R. Concept Dictionary lab session (presented to representatives from the University of Calgary and Calgary Health Region, Winnipeg, MB, February 2003).

Roos L. The Concept Dictionary (presented at the Western Regional Training Centre for Health Services Research Workshop, University of Manitoba, Winnipeg, MB, October 2002).

Soodeen R. The Concept Dictionary lab session (presented at the Spatialisation and Health: Characterising Place - A Problem-Solving Workshop, Banff, AB, November / December 2002).

Soodeen R. Concept Dictionary lab session (presented at the Western Regional Training Centre for Health Services Research Workshop, University of Manitoba, Winnipeg, MB, October 2002).

5. References

Acheson ED. Medical Record Linkage. London: Oxford University Press, 1967.

Burchill C, Roos LL, Fergusson P, Jebamani L, Turner K, Dueck S. Organizing the Present, Looking to the Future: An online Knowledge Repository to Facilitate Collaboration. Journal of Medical Internet Research 2000; 2(2):e10.

Chamberlayne R, Green B, Barer ML, Hertzman C, Lawrence WJ, Sheps SB. Creating a Population-Based Linked Health Database: A New Resource for Health Services Research. Can J Public Health 1998; 89(4):270-273.

Croner MC. Public Health, GIS, and the Internet. Annu Rev Public Health 2003; 24:57-82.

LaPorte RE, Sekikawa A, Sa E, Linkov F, Lovalekar M. Infopoints: Whisking Research into the Classroom. British Medical Journal 2002; 324(7329): 99.

Newcombe HB. Handbook of Record Linkage. New York: Oxford University Press, 1988.

Roos LL, Nicol, JP. A Research Registry: Uses, Development, and Accuracy. Journal of Clinical Epidemiology 1999; 52 (1): 39-47.

Roos LL, Roos NP. Of Space and Time, of Health Care and Health. J Health Serv Res Policy 2001; 6(2):120-122.

Roos LL, Soodeen R, Bond R. Working More Productively: Tools for Administrative Data. Health Services Research 2003; in press.