Procedure for the Development and Management of Food Microbiological Methods

Development of Methods
April 2008

Help on accessing alternative formats, such as Portable Document Format (PDF), Microsoft Word and PowerPoint (PPT) files, can be obtained in the alternate format help section.

---

Part 1. Definitions and Explanations of Methods
and General Procedures

Microbiological Methods Committee
Evaluation Division,
Bureau of Microbial Hazards,
Food Directorate,
Health Products and Food Branch (HPFB),
Sir Frederick G. Banting Research Centre [PL 2204E]
Ottawa, Ontario, K1A 0K9

e-mail: Don_Warburton@hc-sc.gc.ca

1. Introduction

The document entitled "Procedure for the Development and Management of Food Microbiological Methods" will be composed of several parts and will be published as separate documents in Volume 1 of this Compendium:

Part 1: Definitions and Explanations of Methods and General Procedures.
Part 2 (A): Data Requirements, Submission Templates and Examples for Quantitative Methods.
Part 2 (B): Data Requirements, Submission Templates and Examples for Qualitative Methods.
Part 3: Guidelines for the Evaluation of Quantitative Food Microbiological Methods.
Part 4: Guidelines for the Evaluation of Qualitative Food Microbiological Methods.
Part 5: Guidelines for the General Requirements to Verify Validated Food Microbiological Methods for Implementation in Routine Testing.

2. Purpose

To define the procedures to be followed in the development and management of food microbiological methods for inclusion in The Compendium of Analytical Methods. These methods are developed and/or validated by Health Canada (HC) and the Canadian Food Inspection Agency (CFIA), as well as other Regulatory Agencies, universities and private companies and are used in the administration of the Food and Drugs Act and Regulations. This document has been reviewed, revised and accepted by the Microbiological Methods Committee (MMC) and replaces the version dated April 1999.

3. Definition of Terms

Additional definitions may be found in other Parts of this document.

3.1 Method:

An analytical procedure applied to a complete food, in-line food samples, the food ingredient or to the food plant environment, and sometimes to clinical samples.

Generally, the methods dealt with in The Compendium of Analytical Methods are end product methods, i.e., they are applied to the finished product or to foods offered for sale. In addition, these methods may be used to help establish databases, primarily on finished products but also on ingredients. Environmental samples are analysed to determine if foods have been produced under sanitary conditions. These methods may contrast with in-process methods which are designed for use by manufacturers to monitor and ensure there has been adequate process control.

Methods used should meet the requirements of the specific testing. For example, whether testing end product, ingredients or environmental samples, only those methods that meet the required criteria of the testing (including sensitivity and specificity, etc.) should be used.

3.2 Food Category:

Food Categories are defined as: (1) Raw Meats, (2) Processed Meats, (3) Poultry Products, (4) Fish and Seafood Products, (5) Eggs and Egg Derivatives, (6) Fruits and Vegetable Based Products, (7) Dairy Products, (8) Chocolate and Bakery Products, (9) Animal Feed, (10) Other Foods (e.g., alcoholic beverages, cereals, spices, processed foods such as salad dressings, etc.), and (11) Environmental Samples (from food plant environments).

3.3 Food Type:

Food Types are defined as selected foods within a Food Category.

Example :	Food Category:	Dairy Products
	Food Types:	Raw Heat processed Frozen Fermented Dry Others	(Raw milk cheese) (Evporated Milk) (Ice Cream) (Yogurt) (Skim Milk Powders) (Processed Cheese)

3.4 Method Selection:

All methods have been categorized in accordance to their degree of validation. Official Methods (MFOs), HPB Methods (MFHPBs), and Laboratory Procedures (MFLPs) each meet the minimum criteria of a Compendium method specified for that category. For any analysis, the Compendium method that is the most effective (as described below) should be used. Although MFLPs may not have been validated to the same extent or degree as HPB Methods, they often have greater analytical sensitivity and specificity according to available data from recognized international agencies.

Laboratories should use the most effective method (i.e., the best or most suitable) so that: (1) the method is fit for purpose (refer to Section 3.5) and can be used to analyse the food category in question so that the analytical results are reliable; (2) the analyses are completed in a timely manner; (3) the health and safety of Canadians is ensured; and (4) the shelf-life of the product is not jeopardized.

Note:

It is imperative that the "Application Section" of each method be reviewed before use to determine the method's applicability to the food type or environmental sample in question. Applicability of methods to new food types requires validation. It is strongly recommended that the validation data be submitted to the MMC for possible amendment of the method.

3.5 Fitness for Purpose:

Degree to which data produced by a measurement process enables a user to make technically and administratively correct decisions for a stated purpose as described in Section 3.4 (see reference 8.1).

3.6 Operational Method Development (OMD):

OMD is a term that refers to the development, validation and comparison of a method to a standard method by two or more laboratories. Over a period of time (usually 1 - 2 years) data is collected showing the suitability of a method for status of a Laboratory Procedure (or other classification). This produces additional data to enhance the status of the method, extend its scope and may be considered a limited collaborative study.

3.7 Comparative and Collaborative Studies:

A Comparative Study compares the method being developed (such as a "rapid kit") to the standard method and usually involves multiple labs analysing natural contaminated samples that have been obtained locally or within a specific region. Each lab then analyses the samples following their own schedule, but following a set protocol. Although naturally contaminated samples are preferred, with some target microorganisms (e.g., E. coli O157:H7) artificially contaminated samples are needed to obtain sufficient numbers of samples. Comparative studies are recommended for the evaluation of "large scope" quantitative methods (e.g., standard plate counts, aerobic plate counts, etc.). Refer to Section 4.5.

A Collaborative Study compares the method being developed (such as a "rapid kit") to the standard method and usually involves multiple labs analysing aliquots of artificially contaminated samples provided by the organizing or reference lab.

A comparative study involving naturally contaminated samples is preferable. However, this is not always practical. Therefore a comparative study involving artificially contaminated samples is acceptable. Each lab then analyses the samples in a specific time frame following a set protocol. Aliquots of naturally contaminated samples can also be sent to each participant.

3.8 Government Laboratories, Standard Methods or equivalent:

"Government Laboratories" refers to any Canadian Federal or Provincial Laboratory and may include associated laboratories that come under their jurisdiction.

"Standard Methods or equivalent" may include any method from methods organizations, such as the AOAC International, Bacteriological Analytical Manual of the US Food and Drug Administration (BAM), American Public Health Association (APHA), International Commission on Microbiological Specification for Foods (ICMSF), International Dairy Federation (IDF), and the International Organization for Standardization (ISO), European Commission, USDA FSIS and others as deemed acceptable by the MMC.

4. General Package Requirements for Submission of Methods

The following general requirements are applicable to qualitative and quantitative bacterial methods. While an overview of the basic requirements is described here, for the specific and detailed requirements applicable to quantitative and qualitative methods refer to Part 3 and Part 4 of this document, respectively. The requirements for methods that detect viruses and parasites, as well as those microbial analytes where there is no reference method, will be reviewed on a case by case basis.

When submitting a method for review, a complete data package must be submitted. The package must include:

1. The title of the method or procedure,
2. The originator of the method (the developer of the method and/or the company marketing the method),
3. The complete validation protocol used (refer to Part 2 of this document for specific requirements for written protocols),
4. Detailed raw data, preferably in Health Canada format (refer to Part 2 of this document),
5. Summary data in the Health Canada format (refer to Part 2 of this document),
6. An indication of the specific Food Categories and Types that the submission covers, and
7. More information or clarification may be requested.

4.1 General Data Requirements for Submissions of Qualitative Methods:

The following requirements are applicable to qualitative bacterial methods. The requirements for methods that detect viruses and parasites, as well as those microbial analytes where there is no reference method, will be reviewed on a case by case basis.

Provide data from as many sources as possible, and provide details of each study. If submitting AOAC data do not include fractional positives in HC format. Submit fractional positive data in the AOAC format.

If the submission is for "multiple foods":

Supply information for a minimum 60 positives and 30 negatives for each food category (dairy, raw meat, processed meats, vegetables, etc.) with a minimum total of 360 positives and associated negatives (a minimum of 180 negatives) for 6 different types of categories if applicable. The submission should specify the Food Categories and Types (e.g., vegetables: tomato, cilantro; dairy: yogurt, raw milk cheese etc.) for which the method was validated. If the submission is for "multiple foods and environmental samples" then environmental samples must also be included. If possible, ensure that at least 3 food types are included for a specific food category so a minimum of 20 positives and 10 negatives for each food type are included. See the examples in Part 2 of this document.

If the submission is for only one food category:

For example if your submission is for dairy only, then supply information on 60 positives each from at least 6 different food types in that category (i.e., raw milk cheese, processed cheese, ice cream, yogurt, etc.), if applicable. See the examples in Part 2 of this document.

If the method targets environmental samples only then provide data as if the submission is for one food category (i.e., a minimum of 60 positives and 30 negatives).

4.2 General Data Requirements for Submissions of Quantitative Methods:

See Part 3 for details.

4.2.1 Linearity Studies (Method Comparison study)

If the submission is for "multiple foods": Supply information for a minimum of five food categories at five levels of the target microorganism replicated the same number of times (i.e., 5 to 10) for each level, using naturally contaminated samples, if possible, or artificially contaminated samples, for a total of 25 to 50 data points for each food category, tested by both methods (reference vs alternative).

If the submission is for only one food category (including environmentals): For example if your submission is for processed meats only, then supply information or a minimum of one food type representing the selected category for five levels of the target microorganism replicated the same number of times (i.e., 5 to 10) for each level, using naturally contaminated samples, if possible, or artificially contaminated samples, for a total of 25 to 50 data points for this food category, tested by both methods (reference vs alternative).

The selected levels should uniformly cover the whole range of interest (i.e., Regulated limits for 3-class sampling plans covering " m " and " M " values).

4.2.2 Limit of detection (LOD) and quantification (LOQ)

Supply information regarding the values obtained for the limit of detection (LOD) and the limit of quantification (LOQ), and the procedure selected to assess these values.

4.2.3 Selectivity (inclusivity/exclusivity)

Provide information on the selectivity of the method using 30 positive pure strains of the analyte being studied to evaluate the inclusivity and 20 negative pure strains other than the analyte being studied to evaluate the exclusivity.

Note:

This criterion is not applicable for total plate counts or similar methods targeting a broad selection of microorganisms (See Section 4.5).

4.2.4 Transferability

Provide data that demonstrate that the method has been transferred to another laboratory without any significant deviation in bias (trueness) and precision using a 95% Confidence Interval (CI).

4.2.5 Inter-laboratory Studies

Validation of quantitative methods targeting a specific analyte (e.g., E. coli, S. aureus, coliforms, and verotoxin, etc.) which is to be spiked at known levels is achieved by a collaborative study and/or a comparative study. In that case provide the data obtained from a collaborative study originating from a minimum of eight participating laboratories, using artificially contaminated samples at three levels tested in duplicate by both methods for a minimum of 48 paired data (96 results). Compute and provide: (1) the relative accuracy, (2) the repeatability standard deviation, (3) the reproducibility standard deviation and (4) comparison of the between and within laboratory variances.

For methods evaluating total microbial loads (e.g., aerobic plate counts, enumeration of molds, and total sporeformers, etc.), a more general approach, such as a comparative study, is more appropriate. A comparative study would ideally involve at least five laboratories. This can be reduced to as few as three under certain circumstances (emergencies, expensive equipment, etc.). If the submission is for all foods, five food categories are tested, with a minimum of 30 paired results per food category (reference/alternative), using preferably naturally contaminated samples. Provide data analysis of the results demonstrating (1) acceptable linearity and (2) equivalency of variances and means between the two methods.

5. CATEGORIES OF METHODS

5.1 Laboratory Procedure:

A method not validated by an interlaboratory study or not meeting the attributes or specifications of an HPB Method but may be used to determine compliance with various Standards and Guidelines. A Laboratory Procedure has the designated code of "MFLP-" and date of publication in the upper right hand corner of the method and is published in Volume 3 of The Compendium of Analytical Methods.

Note:

The validation data required to become an MFLP method is described in Section 4.

Attributes of different Laboratory Procedures are as follows:

1. Originates and/or is evaluated by generating analytical data in a Canadian Government Laboratory (or equivalent; refer to Section 3.8);
2. Is validated or verified by at least one other HPFB or CFIA laboratory (or equivalent) or the validation data has been published in a peer reviewed journal;
3. Conforms to a specific format. A copy of the method with supporting raw and summary data must be supplied, reviewed and accepted by the MMC before the status of "Laboratory Procedure" is given; and
4. Is judged by the MMC to have the potential for meeting appropriate standards of a regulatory method.

5.1.1 Specifications for qualitative Laboratory Procedures (Table 2) include an appropriate degree of accuracy, precision and specificity established by at least two (HPFB or CFIA) laboratories (or equivalent); and practicality for its intended purpose. Specifications or performance criteria for qualitative methods should meet or exceed the following, when compared to a standard or reference method:

1. Relative Sensitivity ≥ 98%,
2. Relative Specificity ≥ 90.4%,
3. False negative rate <2.0%,
4. False positive rate < 9.6%,
5. Efficacy ≥ 94%, and
6. Level of detection: the method must be comparable to or exceed the lower limit of detection of the standard method (usually 3-5 cfu/25 g) or must be able to detect ≤ 3 cfu per g or mL. Each method will be compared against these criteria on a case-by-case basis.

Note:

Other associated documents on the procedures for in-house development and/or validation of established methods may be found in Volume 1 of the Compendium of Analytical Methods.

5.1.2 Specifications or performance criteria for quantitative methods should meet or exceed the following, when compared to a standard or reference method:

1. Linearity: Intercept = 0 (estimation includes the value zero (95% CI)
   Slope = 1 (estimation includes the value of one (95% CI)
2. Selectivity: Inclusivity > 98% (n = 30 strains)
   Exclusivity < 10% (n = 20 strains)
3. Regulated specifications included in the range studied (see reference 8.5).
4. Transferability of the method achieved within specified limits.

5.2 HPB Method:

A method that may be used to determine compliance with various Standards and Guidelines. An HPB Method has the designated code of "MFHPB-" and date of publication in the upper right hand corner of the method and is published in Volume 2 of The Compendium of Analytical Methods.

Note:

The validation data required to become an MFHPB method (i.e., to move from a MFLP to a MFHPB method) is described in Section 4. It is similar to that needed for MFLP status but is generated separately from the data that was used to obtain MFLP status.

An HPB Method is a fully validated and documented analytical method. Attributes of an HPB Method are as follows:

1. Originates and/or is evaluated by generating analytical data in a Canadian Government Laboratory (or equivalent; refer to Section 3.8);
2. Is validated by either:
   1. a comparative study or a collaborative study (refer to Sections 3.7 and 4.2.5), involving five to eight (quantitative) or ten to 15 (qualitative) independent laboratories; OR
   2. operational method development (OMD) processes (refer to Section 3.6); OR
   3. data on method performance collected from a variety of sources over a period of time, including scientific peer-reviewed publications, and other agencies listed in Section 3.8;
3. Conforms to a specific format. A copy of the method with supporting raw and summary data must be supplied, reviewed and accepted by the MMC before the status of "HPB Method" is given;
4. Is judged by the MMC to meet appropriate standards of a regulatory method; and,
5. Has been published as a Laboratory Procedure for a minimum of one year. See Table 1 for additional details on upgrading a Laboratory Procedure to an HPB Method.

5.2.1 Specifications for a qualitative HPB Method (Table 2) include an appropriate degree of accuracy, precision and specificity demonstrated by inter-laboratory study; and practicality for its intended purpose. Specifications or performance criteria should meet or exceed the following, when compared to a standard or reference method (usually a published MFHPB or MFO):

For qualitative methods:

1. Relative Sensitivity ≥ 98%,
2. Relative Specificity ≥ 90.4%,
3. False negative rate < 2.0%,
4. False positive rate < 9.6%,
5. Efficacy ≥ 94%, and
6. Level of detection: the method must be comparable to or exceed the lower limit of detection of the standard method (usually 3-5 cfu/25 g) or must be able to detect ≤ 3 cfu per g or mL. Each method will be compared against these criteria on a case by case basis.

5.2.2 For quantitative methods

For collaborative studies:

1. Linearity : Verification of the model obtained from the transferability study.
   
   Intercept = 0 (estimation includes the value zero (95% CI)
   Slope = 1 (estimation includes the value of one (95% CI)
   
   
2. Trueness : For each of the three levels, bias estimates must demonstrate no significant differences (95% CI).
   
   
3. Reproducibility and repeatability limits are evaluated against Horwitz criterion (see reference 8.2)
   
   
4. Regulated specifications are included in the range studied (see reference 8.5).

For comparative studies:

1. Acceptable linearity (Xref = Yalt);
   
   
2. Equivalency of variances between the two methods demonstrated using ANOVA (see reference 8.4) at α:0.05; and
   
   
3. Acceptable bias between the two methods (non-significant) demonstrated using the T-test (see reference 8.3) at α:0.05.

5.3 Official Method:

Presently defined in Section A.01.010 of the Food and Drug Regulations as a method of analysis or examination designated as such by the Director for use in the administration of the Act and its Regulations. Director is defined as the Assistant Deputy Minister (Section A.01.010). Section A.01.011 states that the Director shall, upon request, furnish copies of Official Methods.

An Official Method has the designated code of "MFO-" and date of publication in the upper right hand corner of the method and is published in Volume 1 of The Compendium of Analytical Methods. An Official Method possesses the same attributes of and specifications of an HPB Method (see above). However, it is more concise, does not give as many details as an HPB Method and does not provide optional steps.

6. STATEMENT OF POLICY REGARDING FOOD MICROBIOLOGICAL METHODS USED BY THE HEALTH PRODUCTS AND FOOD BRANCH

Through the procedures outlined below, the Bureau of Microbial Hazards (BMH) will aim at gradually achieving the greatest possible conformity with methods adopted by standard methods organizations such as the AOAC International, Bacteriological Analytical Methods of the US Food and Drug Administration (BAM), American Public Health Association (APHA), International Commission on Microbiological Specification of Foods (ICMSF), International Dairy Federation (IDF), and the International Organization for Standardization (ISO), wherever this is consistent with the safety of the Canadian consumer.

Reasons for adopting the above policy are as follows:

1. The above organizations are internationally recognized organizations whose methods are distributed and used worldwide.
2. Many AOAC methods are presently used in the USA and Canada by public health and agricultural organizations charged with responsibility for food safety.
3. It is the aim of the Canadian government to remove, where appropriate, obstacles to free trade, including differences in methodology. The policy formulated above is consistent with this aim.
4. The HPFB and BMH has for more than three decades actively participated in the activities of the AOAC and other organizations and supported their aims.
5. Particularly in the area of chemical and nutritional methodology, AOAC methods are already commonly used in Canada. In the sphere of responsibility of the BMH, AOAC methods are used for detection of extraneous material in foods and paralytic shellfish poisons.
6. Several scientists of the BMH are active members of the AOAC Microbiology Methods Committee, thereby enabling BMH scientists to have methods developed by them considered for validation in AOAC collaborative studies and adoption as Official Final Action Methods. Some are also involved in ICMSF, ISO, and APHA.

Methods originating from other standard organizations will be accepted as Laboratory Procedures (MFLPs) if MMC criteria are met, and, as appropriate, will proceed through normal validation and up-grading to higher levels (HPB Methods).

7. THE MICROBIOLOGICAL METHODS COMMITTEE:

7.1 The Microbiological Methods Committee (MMC):

The term MMC refers to the committee, which is comprised of two components; the Steering Committee (SC) and the Technical Groups (TGs). There is representation from both Health Canada and the Canadian Food Inspection Agency and other agencies in the SC and the TGs. Refer to the "Terms of Reference" published in Volume 1 of the Compendium of Analytical Methods for detailed information.

The consolidation of all federal food inspection services into a single federal food inspection agency has reinforced Health Canada's responsibility for food safety activities including the development and provision of analytical methods for standard-setting.

The MMC provides methodology which can be used in support of HC and CFIA's mandate overseeing the safety of the Canadian food supply, complying with the Food and Drugs Act and Regulations, guidelines, data gathering activities, Good Manufacturing Practices and Hazard Analysis Critical Control Points (HACCP). The MMC has an active role in the harmonization of methods and coordination of method related studies, such as data-gathering activities, surveys, and HACCP activities. Specific roles of the MMC in method development are described in the "Terms of Reference" which will be published in Volume 1 of this Compendium.

7.2 Technical Groups:

Technical Groups (TGs) are committees set up to review data, methods, published papers, and to make recommendations to the MMC on a particular method or issue. The TGs may be composed of specialists from HC and CFIA, as well as other organizations.

8. REFERENCES:

8.1 International Union of Pure and Applied Chemistry (IUPAC). 1997. Compendium on Analytical Nomenclature. Third edition.

8.2 Horwitz, W and R. Albert. 2006. The Horwitz Ratio (HorRat): a useful index of method performance with respect to precision. J AOAC Int. 89:1095-109.

8.3 ISO 3301:1975. Statistical interpretation of data -- Comparison of two means in the case of paired observations International Organization for Standardization. Geneva. ISO Central secretariat. Case Postale 56 CH-1211. Genève 20. Switzerland

8.4 ISO 2854:1976. Statistical interpretation of data -- Techniques of estimation and tests relating to means and variances International Organization for Standardization. Geneva. ISO Central secretariat. Case Postale 56 CH-1211. Genève 20. Switzerland

8.5 HPFB. 2006. Health Products and Food Branch Standards (HPFB) and Guidelines for Microbiological Safety of Food - Interpretive Summary. In: Volume 1. Compendium of Analytical Methods. Ottawa. http://www.hc-sc.gc.ca/fn-an/res-rech/analy-meth/microbio/index-eng.php

Table 1. Purpose of Publication of an MFLP Prior to Upgrading to a MFHPB Method.

The publication of an MFLP for a minimum of one year in the Compendium of Analytical Methods¹ is intended to determine if the method will perform satisfactorily under a variety of operating conditions (such as environment, locality, and personnel changes etc.), normally not considered during a collaborative study.

This is especially important in the following scenarios:

1. When the validating study is completed using samples not covered by the Food and Drugs Act or other standards and regulations;
2. When a limited variety of food types were used;
3. When the scope of the initial study does not cover the "application" of the method;
4. When there is any question as to whether all the criteria of HPB Method status have been met;
5. When the method has been validated by a limited number of laboratories; and
6. When >75% artificially-contaminated samples are used in the collaborative study.

As needed, the MMC will give guidance to the author of the method indicating the specific needs or requirements needed for upgrading of the method to a higher status.

As applicable, after the year's publication, the MMC will review the information and data collected during this year to determine if the MFLP meets all criteria to become an MFHPB method. All further data generated will be reviewed on a case-by-case basis by the MMC.

Table 2. An Example of Calculating the Criteria for a Qualitative Method

N E W M E T H O D	Standard Method
		Positives	Negatives
	Positives	A (360)	B (38)
	Negatives	C (0)	D (400)

Example values are in brackets above.

1. the number of samples found to be positive by both methods
   
   
2. the number of samples found to be positive by the "new method" but negative by the "standard method"
   
   
3. the number of samples found to be positive by the "standard method" but negative by the "new method"
   
   
4. the number of samples found to be negative by both methods

Relative Sensitivity:	A _____	360 ____	= 100%
	(A+C)	360
Relative Specificity:	D _____	400 ____	= 91.3%
	(B+D)	438
False Positive Rate:	B _____	38 ____	= 8.7%
	(B+D)	438
False Negative Rate:	C _____	0 ____	= 0%
	(A+C )	360
Test Efficacy:	A+D __________	760 ____	= 95.2%
	(A+B+C+D)	798

¹ Published on the Health Canada website: www.hc-sc.gc.ca/food-aliment