Abstract
Background: Molecular tests (ie, real-time polymerase chain reaction [RT-PCR]) and antigen tests are used to detect SARS-CoV-2. RT-PCR tests are generally considered to be the standard for clinical diagnosis of SARS-CoV-2 due to accuracy and reliability but can take longer to return results than antigen tests. Our aim was to examine if point-of-care (POC) testing for SARS-CoV-2 infection would provide a flexible resource to help achieve workplace safety. We compared test results and time-to-test results between a POC RT-PCR test and a send-out PCR test in a program implemented in summer 2020.
Results: POC testing shortened the time to results to 110 minutes in the POC setting from the 754 minutes for send-out tests. The specificity of POC RT-PCR single POC testing was 98.7% compared with send-out RT-PCR testing and was confirmed at 99.8% in a validation analysis. The sensitivity of the POC testing was 100% compared with send-out RT-PCR, although in a validation analysis, sensitivity appeared as 0% because only the 12 positive or indeterminate samples on the first analysis were retested and the majority were false-positives that were correctly ruled out.
Conclusions: POC testing for SARS-CoV-2 with RT-PCR technology is possible at reduced time compared with send-out PCR testing.
Introduction
As the COVID-19 pandemic continues to evolve, workplace testing for the virus continues to be a topic of great importance to public health. Workplace screening is mandated for unvaccinated workers under new Occupational Health and Safety Administration regulations,1 and testing programs continue in high-risk settings such as schools.2,3
Point-of-care (POC) testing for SARS-CoV-2 infection provides a flexible resource to help achieve workplace safety during this pandemic. The mainstay of POC testing has been the use of rapid antigen tests for COVID-19, although rapid molecular tests have also been developed.4⇓–6 Both of these technologies are limited by sensitivity, with reported sensitivity of 78% to 84% for antigen tests, and sensitivity of as low as 73% and as high as 100% for molecular tests.4 Of note, a more recent evaluation of an antigen test reported higher sensitivity of 93.3% with specificity of 99.2%.7 Sensitivity is highest with symptomatic cases and with earlier testing during the course of illness.4,5,7 Given the relationship between symptoms and sensitivity of these tests, there exists the possibility of false-negative testing during workplace screening for asymptomatic patients.
Real-time polymerase chain reaction (RT-PCR) tests are generally considered to be the standard for clinical diagnosis of SARS-CoV-2 due to their accuracy and reliability. To date, RT-PCR testing has been available through core laboratories, with time delays due to transport time and batch-processing schedules at the laboratory. POC RT-PCR testing offers the promise of rapid detection available from POC testing and the higher sensitivity of an RT-PCR assay.
A novel POC RT-PCR assay has been developed that has been reported to have high sensitivity (95%) and high specificity (100%) when compared with lab-based RT-PCR testing (as with antigen and molecular tests, sensitivity was reported to be even higher with higher levels of viral load).8 However, this analysis was based on retrospective analysis of laboratory samples, not on actual test performance in a workplace screening program.
Competitive sports provide a setting where potential SARS-CoV-2 outbreaks could occur, as there are opportunities for infectious disease transmission via person-to-person contact and airborne routes.9 Further, the nature of contact sports such as professional basketball provides significant challenges for following recommended practices on physical distancing and wearing face coverings to prevent exposure to SARS-CoV-2.
To keep the players, on-court personnel, and support personnel safe from COVID-19 during the 2020 National Basketball Association (NBA) Summer League, the NBA and the NBA Players Association developed a program of universal screening of players and league and team staff, including daily viral testing, as part of their closed-campus occupational health program. This program included daily laboratory-based RT-PCR testing for SARS-CoV-2.10 The RT-PCR test had an anticipated 12-hour to 24-hour delay between sample collection and test reporting even with dedicated testing resources. This lag could have created a gap in time in which players who were later determined to be infected could expose others to the virus.
The NBA program provided an ideal setting for testing whether a POC RT-PCR test could be used to replace a send-out test in a workplace screening application given the high number of personnel involved, the need for regular screening of personnel, and the importance of a rapid turnaround time for reporting test results.
Methods
Study Sample
This study population sample consisted of NBA support personnel at Disney World during the 2020 NBA Summer League subject to the NBA Universal Screening. NBA support personnel were required to participate in the NBA-developed program of universal SARS-CoV-2 screening of players and league and team staff (the NBA Universal Screening Program). This program included daily RT-PCR testing for SARS-CoV-2 using the Roche cobas assay system.10 The NBA set up a dedicated lab for this program to ensure that its testing efforts did not adversely impact test availability for the surrounding community.
Patient Consent
Patients in this Universal Screening Program were invited to participate in this study assessing the test performance of the POC RT-PCR testing device. Personnel who met study eligibility criteria were asked to review the research information sheet documenting the study procedures and risks and benefits of the study. A waiver of documentation of consent and a Health Insurance Portability and Accountability Act Alteration of Authorization were used to obtain verbal consent. All participants were over the age of 18. The study was reviewed and approved by Advarra Institutional Review Board.
Investigational SARS-CoV-2 Test Procedures
Viral testing was conducted at the POC for sample collection and testing. The study health care provider (HCP) collected a dual nostril midturbinate sample according to the manufacturer's collection instructions. One sample was sent for the laboratory-based RT-PCR test, and 1 sample was allocated to testing with the POC device.
Visby describes the POC device as a “single-use disposable, fully-integrated, rapid, automated RT-PCR in vitro diagnostic test.”11 At the time of the study, the device was used under an investigational device exemption. It has now received Food and Drug Administration emergency use authorization and is approved for use with nasopharyngeal, midturbinate, and anterior nasal swabs.12 .
POC testing was performed following instructions documented in the quick reference guide provided by Visby. The POC device tested here was performed under Clinical Laboratory Improvement Amendments (CLIA) procedures. POC test validation included testing of 1 external positive control (positive for the infection) and 1 negative control (negative for the infection) at each site each month and with each new shipment or lot of the Visby device. Specimen collection and testing workflow are summarized in Figure 1.
Specimen collection and testing workflow. Abbreviations: HCP, health care provider; POC, point-of-care; RT-PCR, real-time polymerase chain reaction.
All clinical trial data were securely stored on an IQVIA database. Participant identifying information was removed by IQVIA. Study data transferred to the research team were identified solely by a linked study ID number.
Validation Study
Because the Visby device was experimental at the time of this study, an additional protocol was run to further adjudicate positive and indeterminate test results. Samples that were determined to be positive or indeterminate on the first analysis were retested with new POC devices for 2 replicates. The prespecified interpretation of the validation study was reported as positive based on 2 of 3 positive results.
Results
Study Sample
Overall, we collected 957 samples from study participants analyzed by both testing protocols (participants may have provided more than 1 sample during the screening program). For the laboratory RT-PCR test, 896 were considered valid for analysis (94%), while 911 were valid for analysis for the single-test RT-PCR POC device (96%). (A test is considered valid if a result, either positive or negative, is reported by the testing equipment.) In total, there were 852 collected samples that were considered valid for analysis using both assays (89%).
Performance Comparison with Gold Standard
The laboratory RT-PCR test served as the standard result for this study. Sensitivity and specificity of the single-test POC test were determined compared with the standard. Out of the 852 entries, only 1 was positive with the laboratory RT-PCR. In contrast, 12 were identified as positive by the POC test. Results are reported in Table 1. Overall, the single POC test had a sensitivity of 100% and a specificity of 98.7%.
Comparison of Single-Test Point-of-Care Results with the Reference Laboratory Tests
Results Reporting Time
The major advantage of a single-test POC test would be in faster results reporting. As can be seen in Figure 2, the turnaround time for the POC RT-PCR test is much shorter than for the laboratory test. On average, the POC test required 110 minutes (SD 37 minutes) to results, while the laboratory test required 754 minutes (SD 207 minutes) (P = .00). The POC test had an almost 7-fold increase in diagnostic speed.
Distribution of turnaround times for both the point-of-care (POC) and the laboratory real-time polymerase chain reaction (RT-PCR) test. Analysis of 852 samples with valid test results on both the single-test POC and laboratory RT-PCR test (see discussion in the text).
Validation
A validation protocol was developed to further test the performance of the POC test. On a single-test evaluation using the POC device, there were 12 positive results. When retested under the validation protocol, the number of positive tests on a 3-test series was reduced from 12 to 2. This validation study reduced the number of false-positive test results but also changed the true-positive result to a false-negative result. For this sample, 1 result was negative and 1 result was invalid across the 3 tests. The prespecified scoring methodology was used to determine that the final result was negative. The resulting sensitivity and specificity of the POC test are 0% and 99.8% in the validation study.
In these cases, as additional tests were required for validation, the turnaround time was also impacted. An additional 30 to 60 minutes were required per sample, depending on whether the confirmatory tests were conducted in parallel or in series. This corresponds to a 27% to 54% increase in turnaround time when compared with the single-test POC protocol.
Discussion
Workplace POC testing for the SARS-CoV-2 virus is an important element of pandemic response in critical environments such as essential workplaces, schools, airports, and in professional sports. There remains a debate about the optimal testing strategy for screening populations.13 Here, we report the results of a POC RT-PCR test that can provide another tool that can be considered in developing a virus screening strategy. Overall, in this study, the single-test POC RT-PCR test had sensitivity of 100% and specificity of 98.7%. Further, this level of diagnostic performance was available in an average of 110 minutes compared with 754 minutes for send-out RT-PCR testing, a 7-fold difference.
The POC device tested here was performed under CLIA procedures. Requiring the use of trained personnel for test administration required significant staffing for large-scale screening (in this case, 1 technician could administer 5 to 6 devices at a time including specimen preparation, test performance, and data entry).
As a workplace screening device, it is critical that there is an overall data strategy to collect, track, and report test results. In the NBA program, a robust data architecture was developed using a clinical research organization. The POC device has analog results reporting, so the testing staff had to manually enter the results into the database, allowing for potential for reporting error. Conversion of the device to a digital reporting format could address these issues.
From a clinical epidemiology perspective, the high number of false-positive tests obtained with a single test is expected in a low prevalence population. For example, in testing a population with a 1% prevalence rate of SARS-CoV-2 infection, with a test performance of 95% sensitivity and 98% specificity, the test would be expected to report 9.5 true-positive results and 19.8 false-positive results for a positive predictive value of 32%. At a prevalence of 5%, the test performance would improve to 47.5 true-positive results and 19 false-positive results for a positive predictive value of 71%. With these same test parameters, negative predictive value remains above 99% at a prevalence of 1% or 5%, suggesting the benefits of the negative test results for screening programs.
Retesting positive samples in a low-prevalence population may reduce the false-positive results in population screening efforts. Confirmation of results of false-positive tests in the validation study with a confirmatory test resulted in a decrease in the number of false-positives (in this case also misclassifying the true-positive result), while increasing the total screening time and therefore potentially hindering the efficiency of a POC testing program. Of note, a retrospective analysis of SARS-CoV-2 samples previously published used only a single-test POC protocol and did not attempt a validation study on the positive samples.8 The result of a validation exercise would be directly impacted by the prevalence of the infection in the population being tested (the retrospective study was conducted on 61 positive and 17 negative samples for a prevalence of 78%)8.
The final important parameter for a workplace testing program is the cost of the underlying technology (in this case, referring to the cost of acquiring the test material, not the market price paid by insurance firms). In the United State, publicly available cost data on diagnostic testing are hard to identify. However, on the global market, antigen test reagents cost from $2.50 to $3.00 per test, while reagents for PCR testing cost from $10.00 to $19.80 per test. PCR testing is performed on equipment that ranges in price from $15,000 to $153,000.14 The value of POC testing should be assessed based on test performance, time, and cost.
Limitations
We tested 1 population under 1 set of test conditions that were designed for high-volume screening. Performance, in terms of time to perform the POC test, may differ in other clinical settings.
In a quality review of the testing program, there were 2 observations that could have affected the results. First, several of the positive test results were found on the same afternoon in sequence after a positive sample. Given the sensitivity of PCR assays, it is possible that contamination of the work area contributed to these results in the field setting of the testing program. Second, the test results were photographed for later evaluation, and the results of 2 tests reported as positive at the site were read as negative from the photographs. Since the POC device was being tested as an investigational device, better documentation and training associated with a marketed product could have addressed these errors.
Finally, 94% of the laboratory and 96% of the single-test POC samples were considered valid for analysis when samples were obtained under direct observation. Collection of valid samples for self-administered tests in a workplace environment may not achieve this high level of performance.
Conclusion
Single-test POC testing for SARS-CoV-2 with RT-PCR technology is possible with excellent test characteristics compared with laboratory testing, and with much faster time to result reporting.
Acknowledgments
Kelly Lacob and Teresa Abraham contributed to the development of the testing protocol, and Visby Medical provided logistic support for this study. IQVIA merged the RT-PCR testing data with the NBA data for this analysis.
Notes
This article was externally peer reviewed.
Funding: Funding provided by the National Basketball Association.
Conflict of interest: None.
To see this article online, please go to: http://jabfm.org/content/35/1/96.full.
- Received for publication July 9, 2021.
- Revision received September 20, 2021.
- Accepted for publication September 22, 2021.
