Introduction
This module describes the process of monitoring quality assurance of assay testing and interpreting results from serosurveys.
This module describes the process of monitoring quality assurance of assay testing and interpreting results from serosurveys.
Topics covered in this module
Throughout the course of the enzyme immunoassay (EIA) protocol, quality measures related to the assay should be implemented to ensure the precision, reliability and validity of the results.
Goal |
Monitoring Assay Quality |
Routine |
|
Type of specimen or reference used |
Manufacturer-provided calibrators |
Internal controls |
Random retesting of serosurvey specimens |
Description |
Reference values provided by manufacturer to check assay |
Developed in-house or using International Reference Standards
|
Between plate (inter) and within plate (intra) assay correlation of specimens tested more than once |
How evaluated |
Tracking values over time or comparing between laboratories or technicians
|
Calculate the following using the quantitative values • Coefficient of variation • Correlation coefficient (R2) |
Table 1. Summary of different methods for quality assurance
Figure 2. In the “Calibration” tab of the template the analyst can record the Optical Density (OD) values for the calibrators and controls (PC positive control; NC, negative control) in the blue cells. The template automatically calculates the standard curve and determines if the assay was valid or not based on the values of the calibrators and controls.
Figure 3. Monitoring Optical Density (OD) values for calibrators and controls of the same lot across plates to identify potential problems with assay runs. Although some plate-to-plate variation is normal and expected, the flatter the line, the more uniform the values of the calibrators and controls over time.
Intra-plate (within plate) variability can be assessed by randomly repeating a few specimens on the same plate.
Inter-plate (between plate) variability can be assessed by randomly repeating a few specimens on a different plate.
Coefficient of variation is calculated by dividing the standard deviation (σ) of a set of measurements by the mean (µ) of the set of measurements and multiplying by 100 to get %.
Cv=σ/μ*100
This calculation could be done for the optical density (OD) values or the antibody concentration calculations. The smaller the coefficient of variation, the more reliable the estimate. One rule of thumb guidance is <10% acceptable, 10-14% questionable, >15% unacceptable and should be reviewed.
Correlation coefficient (R2) denotes the relationship between the two measurement values. The closer the R2 is to 1, the more tightly correlated the two measurement values are. This can be visually seen on a scatterplot or calculated using a data program (e.g. Excel, STATA, R).
Figure 4. Example plate map demonstrating how specimens to be retested can be selected. Yellow cells indicate calibrators and controls provided in the EIA kit as well as in-house internal controls. Blue cells indicate specimens retested for intra-plate quality assurance. Green cells indicate specimens to be retested on another plate for inter-plate quality assurance.
Figure 5. Example scatterplot evaluating intra-assay correlation. The x-axis is the quantitative value for the first well and the y-axis is the value for the second well on the same plate. The red line indicates equivalence. The light gray lines indicate equivocal and positive thresholds. The point circled in red reflects a specimen that was categorized as equivocal in the first well and categorized as positive in the second well (using 200 mIU/mL as the positive threshold). It is important to focus on values around these thresholds since minor quantitative changes may result in qualitative differences in terms of seropositivity.
Monitoring specimen testing can help identify problems with assay procedures. Refer to Table 1 for examples of common problems.
Issue | How to identify |
---|---|
Duration of time from placing first specimen on the plate to the last specimen on the plate is long. | Intra-plate retesting demonstrates systematically lower values for the repeat tests at the end of the plate than in their original well (Figure 4). |
Clog in the plate washer or one loose-fitting pipette in a multi-channel pipette. | Values along the same row of a plate are consistently high or low, particularly if comparison is of repeated testing values. |
One laboratory technician is not carefully following the assay protocol. | Monitoring the inter-plate retesting, intra-plate retesting, and values of the calibrators and controls by a laboratory technician can help pinpoint whether there is more variability seen with a particular technician’s plate runs. |
Too much time elapsed during one step of the EIA. | Laboratory technicians should report if the timing at any step of the EIA exceeds that allotted by the kit instructions. Another way to identify this is if the values for that plate are systematically higher than the values of the repeat tests that are done on a retest plate. |
Specimens have degraded from too many freeze-thaw cycles. | If possible, specimens should only be thawed once for testing and any retests are done quickly thereafter. If the retested results are always found to have lower antibody concentrations than the original values, this could indicate that specimens are degrading.Toolkit-Data Procedure EIA Assay Template Example |
Differences in quality between lots of EIAs.
A lot is a set of multiple EIA testing kits that were all calibrated and validated together, so they have the same reference values. |
While there is some inherent variability expected between lots, they should all be able to provide valid results. When starting a new lot, consider retesting a subset of specimens already tested from the previous lot to assess concordance. If using internal controls, compare the antibody concentrations from plates that used different lots.
Note: the OD values will differ between kits, but the antibody concentrations should not. |
Table 1. Common specimen testing issues
Decisions about assigning final outcome values should be specified clearly in the study protocol and adhered to throughout the analysis to ensure validity in final prevalence calculations.
Figure 6. Example of summary flowchart to assign qualitative values to tested specimens
Immunity is a complex process, and IgG antibody detection by EIA provides only a surrogate measure of immunity.
Some antigens have established thresholds for protection such that immunity can be thought to provide protection.
Some assays may not have established cut-offs to determine immunity, requiring more complex methodologies for interpretation such as mixture modeling1.
International Reference Standards
1. Vyse, Andrew J., N. J. Gay, L. M. Hesketh, R. Pebody, P. Morgan-Capner, and E. Miller. 2006. “Interpreting Serological Surveys Using Mixture Models: The Seroepidemiology of Measles, Mumps and Rubella in England and Wales at the Beginning of the 21st Century.” Epidemiology and Infection 134 (6): 1303–12. https://doi.org/10.1017/S0950268806006340.
Section | Toolkit material | Context |
---|---|---|
Monitoring quality – assay validation | Toolkit-Data Procedure EIA Assay Template Blank | Template for use with Euroimmun measles or rubella EIA kits |
Monitoring quality – assay validation | Toolkit-Data Procedure EIA Assay Template Example | Template for use with Euroimmun measles or rubella EIA kit |
Monitoring quality – retesting | Toolkit-Example Protocol for Assay Retesting | Protocol developed for use in a measles and rubella serosurvey |