Biochemical or biomarker tests can facilitate treatment decisions while being less physically invasive than other procedures such as biopsy or radiography. Point-of-care testing (POCT) further reduces the burden of testing by making the test and test results easily accessible, either at the patient’s home or at a primary health care centre, thus shortening wait times for test results (Yager, Domingo, and Gerdes 2008).
Due to their molecular nature, biomarker-based tests present some challenges: Chemicals, hormones and antigens cannot be directly observed and are usually too small to be quantified by microscope. Well-established methods to indirectly observe such molecules, or analytes, include mass spectrometry, immunoassays and variants. This document will focus on non-isotopic assays (e.g. ELISA) with characteristics that may transfer well to POC applications.
Some of the limitations of immunoassays, and especially some of the additional compromises associated with POCTs, may not be well understood by users of the tests. On the other hand, developers of the tests may not appreciate the potential ensuing clinical consequences of a poorly developed or characterised test. Therefore, there is a need to regulate the use of POCTs to encourage standardization and transparency in their development. Although the process of regulating diagnostic tests is not as rigorous as for pharmaceutical interventions, guidelines have been published by a number of major organisations such as the Clinical and Laboratory Standards Institute (CLSI) (also see the International Organization for Standardization (ISO), Geneva).
CLSI guidelines recommend:
In turn, scientific articles comment and build upon the guidelines: The rapid evolution of diagnostic technologies requires that some fundamental methods be reintroduced, and other be adapted for newer applications. We are particularly interested in the statistical methods used to characterise POCT performance, including calibration curve-fitting and analytical validation, as part of pre-market test development.
Unlike lab-based tests, POCTs must be easy to use and feasible as a single-sample-at-a-time platform (also known as a random access system). Once a patient sample has been properly collected *, there are a number of steps to obtaining a concentration estimate. Table 1 compares the general steps for traditional lab-based tests running concurrent calibrators versus an ideal POCT system.
* We will not cover proper sample collection in this document.
| Traditional lab-based batch assays including calibrators | Quantitative POCT needing minimal user intervention |
|---|---|
| - Store sample appropriately until processing | (Sample would have been introduced immediately into the device) |
| - Process the collected sample | (Processing happens automatically within the chip) |
| - Measure the generated response | - Measure the generated response |
| - Estimate calibration curve from response and concurrent calibrators | (Calibration curve model, previously determined by manufacturer, is stored in the response reader’s software) |
| - Patient sample concentration is estimated from response and calibration curve | (Patient sample concentration is estimated from response and calibration curve by the built-in software—indistinguishable from the measurement step for the user) |
| - Interpret the result based on estimate, margin of error and reference values | - Interpret the result based on estimate, margin of error and reference values |
In the final stages of test development, calibration curve-fitting is followed by assessment of test performance. The measurements from the new test, over time and across different labs or instruments, are compared (using standard samples) to estimate reliability. The estimated concentrations of patient samples are compared to an existing reference method to assess agreement. Some of the statistical characteristics of the test are summarized from those validation studies and reported to test users and regulators. Underlying those downstream summary metrics are many different statistical methods and several iterations data collection and reduction.
A precursory review of the literature —textbooks and CLSI guidelines—and some anecdotal recent publications left us with the impression that there was a lack of detailed information on procedures for statistical characterization of POCTs, especially the development of master calibration curves.
Statistical procedures used to estimate concentration from observed response have mainly evolved in the context of lab-based assays where control samples are usually included in parallel with the experimental samples. These are especially plate-based batch assays where physical space considerations figure prominently in study design. We found that there is very little guidance for POCT developers for curve-fitting.
For test performance statistics, the CLSI guidelines are a good starting point, but are not specific enough for POCTs. Even the literature on making test performance claims for traditional assays is sparse for test developers; the onus seems to be on test users for whom there is plenty of guidance. Manufacturers of other random-access type biomarker testing technologies do not seem to publish detailed methods and results from test development so we cannot glean much from their experience based on published material alone. We believe that statistical methods for POCTs (and other random-access systems) deserve more attention. Hence in this Grand Challenges Canada funded project, we sought to review and complement the available, but overly general, recommendations for test characterization and validation by focusing on their application to POCTs.
Our results are presented here as a series of brief narrative reviews and detailed tutorials.
Yager, Paul, Gonzalo J Domingo, and John Gerdes. 2008. “Point-of-care diagnostics for global health.” Annual Review of Biomedical Engineering 10 (January): 107—–1144. doi:10.1146/annurev.bioeng.10.061807.160524.