Introduction

This module describes the process of monitoring quality assurance of assay testing and interpreting results from serosurveys.

Topics covered in this module

• Monitoring quality of assay testing procedures
• Common issues with assay procedures
• Interpretation of results from EIAs

Note: Refer to below modules for related information

• Specimen collection procedures and transport from the field to the laboratory module
• Specimen processing, storage, and shipment module(link)
• Developing and running assay protocols module

Contents

1. Quality Assurance Considerations
   1. Overview of Quality Assurance
   2. Monitoring Assay Quality
   3. Routine Retesting for Quality Assurance
2. Common Problems with Assay Procedures
3. Interpreting EIA Results
   1. Assign outcome value to results
   2. Determining Immunity
4. References
5. Toolkit

Quality Assurance Considerations

Overview of Quality Assurance

Throughout the course of the enzyme immunoassay (EIA) protocol, quality measures related to the assay should be implemented to ensure the precision, reliability and validity of the results.

• Calibration protocols using a reference IgG antibody standard should be performed to ensure accurate and precise results. This should be done at the beginning of testing and if testing survey specimens with different lots of the same assay kit.
  • This can be done with a known panel developed internally or using WHO International standards.
  • Commercially available EIAs contain calibration instructions that should be followed at the beginning of testing and if using different plate readers. These protocols will need to be developed by the laboratory if using an “in house” EIA.
• With even the most reliable tests and most rigorously developed lab protocols, the imperfect nature of EIAs means some level of variability and discordance will occur with retesting protocols. To deal with these, the following re-testing protocols are recommended to ensure the reliability of the serosurvey lab protocol.
• Establish a regular retesting protocol of random or fixed interval specimens on each plate to confirm the protocol is consistent throughout testing.
• Establish a protocol for handling discordant results. (See Interpreting EIA results)

Table 1. Summary of different methods for quality assurance

Monitoring Assay Quality

• It is important to assess quality of the assay over time. Options for external validity assessment include running international standards, a known serum panel, or a validation subset with another EIA or gold standard assay.
• Commercially available EIAs should contain calibrators and controls that have been validated against a reference standard. These are tested on every plate to ensure that the results are valid.
  • The reference values should be included with each test kit. The values for the calibrators and controls must lie within the limits stated for the kit.
  • If the values specified for the calibrators and controls are not achieved, the test results may be inaccurate and testing should be repeated.
  • To allow for rapid identification of problems, recommend adding assay validity checks to the EIA Results template to check the observed values against the reference values and to flag when the specified values are not achieved. Refer to Figure 2 and the “CALIBRATION_ANALYST” tab in the tool, EIA TEMPLATE BLANK, for an example.

Figure 2. In the “Calibration” tab of the template the analyst can record the Optical Density (OD) values for the calibrators and controls (PC positive control; NC, negative control) in the blue cells. The template automatically calculates the standard curve and determines if the assay was valid or not based on the values of the calibrators and controls.

• Tracking calibrators and negative and positive controls
  • Another way to assess the quality of EIAs is to track the values for calibrators and controls of multiple kits from the same lot.
  • This allows visualization of trends, such as decreasing calibrator values or outliers if one plate’s values were higher or lower than other plates.
    • If using multiple laboratories for testing or multiple laboratory technicians, can also compare these data across laboratories or technicians.

Figure 3. Monitoring Optical Density (OD) values for calibrators and controls of the same lot across plates to identify potential problems with assay runs. Although some plate-to-plate variation is normal and expected, the flatter the line, the more uniform the values of the calibrators and controls over time.

• Using internal controls
  • Another way to monitor the quality of specimen testing is to use an internal control on every plate.
    • This could be obtained from a blood bank or some other biorepository. It should be a well-characterized specimen with known antibody concentration and sufficient volume to be used on all EIA plates run for the serosurvey.
    • This could also be done using the International reference standards.
  • Since this will always be the same specimen, monitoring the antibody concentration obtained from every plate allows you to assess if the value is changing over time or across technicians or laboratories.
    • For example, if there is an outlier value, that plate should be reviewed.

Routine Retesting for Quality Assurance

• The goal of routine retesting is to systematically assess intra-plate (within the same plate) and inter-plate (on different plates) variability.
  • This provides a sense of how accurate and reliable the assay results are.
  • Can choose to do as many retests as desired but may want to keep <10% to remain manageable. Can also choose to start by doing more retests for quality assurance and decrease overtime if the serosurvey will be testing large numbers of specimens and there is no evidence of problems with assay quality or testing performance.

Figure 4. Example plate map demonstrating how specimens to be retested can be selected. Yellow cells indicate calibrators and controls provided in the EIA kit as well as in-house internal controls. Blue cells indicate specimens retested for intra-plate quality assurance. Green cells indicate specimens to be retested on another plate for inter-plate quality assurance.

Figure 5. Example scatterplot evaluating intra-assay correlation. The x-axis is the quantitative value for the first well and the y-axis is the value for the second well on the same plate. The red line indicates equivalence. The light gray lines indicate equivocal and positive thresholds. The point circled in red reflects a specimen that was categorized as equivocal in the first well and categorized as positive in the second well (using 200 mIU/mL as the positive threshold). It is important to focus on values around these thresholds since minor quantitative changes may result in qualitative differences in terms of seropositivity.

Common Problems with Assay Procedures

Monitoring specimen testing can help identify problems with assay procedures. Refer to Table 1 for examples of common problems.

Table 1. Common specimen testing issues

Interpreting EIA Results

Assign outcome value to results

Decisions about assigning final outcome values should be specified clearly in the study protocol and adhered to throughout the analysis to ensure validity in final prevalence calculations.

• If specimens are tested only once, use that result as the final value assigned. If specimens are tested more than once, establish a protocol for concordant and discordant results.
• Concordant seropositive or seronegative: Assign agreed upon value.
• Discordant:
  • The study protocol should clearly specify which result to use for a specimen in the analysis
    • Ex. 1: Use the first available result.
    • Ex. 2: Replace initial equivocal result with repeat result if the specimen was positive or negative on re-testing.
• Concordant, but equivocal:
  • The study protocol needs to clearly guide how these outcomes should be categorized in prevalence analysis. Three possible strategies:
    • Include equivocal results as seropositive results
      • Biologically plausible: The EIA threshold for a positive result is set at a level of IgG that is higher than the minimum level required for detection.
    • Include equivocal results as seronegative results
      • Conservative approach that will lead to some underestimation of seroprevalence, which may be preferred to overestimation.
    • Keep equivocal results as a separate category
      • Permits a reviewer to conduct a sensitivity analysis, if desired, by combining the equivocal results with either the positive or negatives.
• Refer to the module, Developing assay protocols, for more information on handling equivocal specimens.
• Note: The decisions regarding recategorizing equivocal results and the numbers of recategorized specimens should be reported clearly in reports and publications such that readers can perform a sensitivity analysis based on these results if desired.
• WHO Serosurvey Manual chapter 4, Section 4.5 Interpreting EIA Results

Figure 6. Example of summary flowchart to assign qualitative values to tested specimens

Determining Immunity

Immunity is a complex process, and IgG antibody detection by EIA provides only a surrogate measure of immunity.

• While assay cut-offs were established to maximize test specificity, most assays were designed to evaluate an individual’s immune response and not for population antibody prevalence estimation.
• A negative or equivocal assay result is typically interpreted as susceptibility to disease, but this may not necessarily be the case.
  • Imperfect sensitivity may be a concern with serum IgG EIAs, meaning that the assay may misclassify someone as seronegative when in fact they are seropositive.
  • An equivocal IgG EIA results may meet threshold cutoff for immunity by neutralization assays (e.g., Plaque reduction neutralization assays or PRN found in the module, Developing and Running Assay Protocols Module).
  • Circulating antibody levels may be lower after vaccination than those occurring after infection. Therefore, low level antibodies resulting in equivocal results would be expected with commercial EIA assay.

Some antigens have established thresholds for protection such that immunity can be thought to provide protection.

• For measles, the gold standard is the plaque reduction neutralization assay, and the threshold for protection is believed to be >120mIU/mL.
• For rubella, the gold standard is the hemagglutinin inhibition assay, and the threshold for protection is believed to be >10-15 IU/mL.
• Reference: World Health Organization Manual for the Laboratory-based Surveillance of Measles, Rubella, and Congenital Rubella Syndrome, Chapter 9

Some assays may not have established cut-offs to determine immunity, requiring more complex methodologies for interpretation such as mixture modeling¹.

References:

World Health Organization Guidelines on the Use of Serosurveys in Support of Measles and Rubella Elimination, Chapter 4 Laboratory Methods

World Health Organization Manual for the Laboratory-based Surveillance of Measles, Rubella, and Congenital Rubella Syndrome

• Chapter 9. Laboratory testing for determination of population immune status

International Reference Standards

1. Vyse, Andrew J., N. J. Gay, L. M. Hesketh, R. Pebody, P. Morgan-Capner, and E. Miller. 2006. “Interpreting Serological Surveys Using Mixture Models: The Seroepidemiology of Measles, Mumps and Rubella in England and Wales at the Beginning of the 21st Century.” Epidemiology and Infection 134 (6): 1303–12. https://doi.org/10.1017/S0950268806006340.

Toolkit