Abstract
Background: A systematic review of clinical decision rules to identify patients at low risk for community-acquired pneumonia (CAP) has not been previously presented in the literature.
Methods: A systematic review of MEDLINE for prospective studies that used at least 2 signs, symptoms, or point-of-care tests to determine the likelihood of CAP. We included studies that enrolled adults and adolescents in the outpatient setting where all or a random sample of patients received a chest radiograph as the reference standard. We excluded retrospective studies and studies that recruited primarily patients with hospital-acquired CAP.
Results: Our search identified 974 articles, 12 of which were included in the final analysis. The simple heuristic of normal vital signs (temperature, respiratory rate, and heart rate) to identify patients at low risk for CAP was reported by 4 studies and had a summary estimate of the negative likelihood ratio (LR−) of 0.24 (95% CI, 0.17 to 0.34) and a sensitivity of 0.89 (95% CI, 0.79 to 0.94). The simple heuristic of normal vital signs combined with a normal pulmonary examination to identify patients at low risk for CAP was reported by 3 studies, and had a summary estimate of LR− of 0.10 (95% CI, 0.07 to 0.13) with an area under the receiver operating characteristic curve of 0.92.
Conclusions: Adults with an acute respiratory infection who have normal vital signs and a normal pulmonary examination are very unlikely to have CAP. Given a baseline CAP risk of 4%, these patients have only a 0.4% likelihood of CAP.
Community-acquired pneumonia (CAP) is a significant source of morbidity and mortality for adults in the United States. Episodes requiring hospitalization occur at an estimated annual incidence rate of 25 to 36 cases per 10,000 adults, trailing only live births as a reason for hospitalization.1,2 In 2015, there were over 50,000 deaths due to pneumonia (about 1.6 deaths per 10,000 persons) and when combined with influenza, it is the eighth leading cause of death in the United States.3
The recommended test for diagnosing CAP is by chest radiograph (CXR).4 However, obtaining a CXR in all outpatients with acute cough would be costly, inconvenient, and would expose many patients to radiation unnecessarily. A way to limit unnecessary testing of these patients is by stratifying risk of CAP based on the clinical presentation. Patients identified as low risk for CAP could avoid the need for CXR and antibiotics, reducing overall health system costs, antibiotic use, and unnecessary radiation exposure.
A clinical decision rule (CDR) is an algorithm, point score, or simple heuristic (ie, rule of thumb”) that categorizes patients as having a risk for disease as confirmed by a valid reference standard test. These are typically developed with a multivariate analysis using independent predictors such as demographics, signs, symptoms, and point-of-care (POC) tests. An example of a CDR is the Strep Score (widely known as the Centor Score), which uses tonsillar exudates, swollen tender anterior cervical nodes, lack of a cough, and history of fever to categorize the risk of strep throat.5
While the use of signs and symptoms to diagnose CAP has been well studied,6⇓⇓–9 the focus has generally been on diagnosing CAP rather than on ruling it out. For the outpatient primary care physician, especially one who does not have ready access to chest radiography, the question of ruling out CAP is equally relevant. In this study, we systematically reviewed the literature to identify CDRs that may be used to rule out CAP in otherwise-healthy adults, also known as “low-risk criteria” for CAP. Our goal is to identify the combination(s) of signs (including vital signs), symptoms, and POC tests with the lowest negative likelihood ratio (LR−), to help physicians confidently rule out CAP without having to obtain a CXR in the outpatient setting.
Methods
Inclusion and Exclusion Criteria
Our search was designed to identify studies that used a CDR to diagnose, predict, or rule out CAP in the outpatient setting. A study was included if it used a CXR or computed tomography scan as the primary reference standard and was given to all patients enrolled in the study. If the reference standard was used in a random or systematic sample of low-risk CAP patients to minimize radiation exposure, the study was also included. Studies had to gather data prospectively and only studies that recruited adults or adolescents in an outpatient setting (emergency department, urgent care, primary care, or outpatient clinic) were included.
Studies where a majority of the enrolled patients had hospital-acquired or ventilator-associated pneumonia, were immunocompromised, or that were conducted in special populations such as military or nursing homes were excluded. We excluded studies that were not prospective, such as case-control studies, case reports, and outbreak investigations. An exception was made if the case-control study enrolled symptomatic patients prospectively, such as a consecutive series where patients with similar symptoms but with and without CAP were matched.
Search Strategy
We performed a systematic review of articles published in MEDLINE from inception to January 2017 (Appendix A). Any systematic reviews that appeared in our results were reviewed for relevant articles that fit our inclusion and exclusion criteria. We also searched the reference lists of any article that were reviewed in full text form.
Each author independently reviewed the titles and abstracts to identify any articles that required a full text review. Any article identified by at least 1 reviewer had its full text reviewed. The full text review and all data abstraction methods described hereafter were performed independently, in parallel, by 2 authors with a third author who helped resolve any discrepancies. We used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)10 to document our search process.
Data Abstraction and Analysis
We first abstracted the study characteristics, including the combination of signs, symptoms, or POC tests that were used to diagnose CAP. Articles were assessed for bias using the Quality Assessment Tool for Diagnostic Accuracy Studies (QUADAS-2) framework adapted for our study.11 The evaluation consists of 4 areas: patient selection, index test, reference standard, and flow and timing. For each, a set of questions assess the article for bias, answering yes, no, high, low, or unknown. A final overall assessment for each article was given a low, moderate, or high risk for bias.
When available, true positive (CDR+, CAP+), false positive (CDR+, CAP−), true negative (CDR−, CAP−) and false negative (CDR−, CAP+) rates of each CDR were recorded. When not directly provided, they were calculated using data from the study. These data were used to calculate positive and negative likelihood ratios (LR+ and LR−) for CDRs reporting a dichotomous outcome of CAP versus no CAP, and stratum-specific likelihood ratios (SSLRs) for CDRs reporting more than 2 possible outcomes (eg, low-, moderate-, and high-risk groups). Post-test probabilities were calculated for standardized low-prevalence (4%) and high-prevalence (20%) populations, to represent typical outpatient primary care and emergency department populations respectively, using summary estimates of likelihood ratios for high-performing CDRs.6,12
Lastly, we calculated 3 risk groups (low, moderate, and high) as part of a post-hoc analysis of any CDR that was based on a multichotomous score.6,7 The risk groups were assigned based on the distribution of likelihood ratios from the studies' published data.
If 3 or more studies reported data for a single CDR, we performed a bivariate meta-analysis to calculate summary estimates and 95% confidence intervals for sensitivity, specificity, LR+, LR−, and the area under the receiver operating characteristic curves (AUROCCs). These were performed using the mada (Meta-Analysis of Diagnostic Accuracy) package in R, version 3.4.3 (R Foundation for Statistical Computing, Vienna, Austria) using the Reitsma method.13
Institutional Review Board approval was classified as exempt as this was a secondary analysis of previously published data, and no funding was provided for this research.
Results
Our initial search strategy identified 974 articles (Figure 1). The title and abstract review eliminated 906 articles. Sixty-eight articles had full text review and after identifying an additional 42 articles through a bibliography review, we excluded 98. The most common reasons to exclude a study were because it was not prospective, was a literature review or guideline, did not report any usable patient data, or did not present data that met our criteria for a CDR. An updated search before writing (January 2018) did not yield additional studies to add. A final total of 12 studies were included.6⇓⇓–9,14⇓⇓⇓⇓⇓⇓–21
PRISMA flow diagram of search strategy and selection of articles.
Study Characteristics and Quality Assessment
Study characteristics are summarized in Table 1. Almost half were performed in the United States6,7,14,16,20 and no other country appeared more than once. Enrollment of patients occurred from 1984 to 2010, with a mean age between 32 and 65 years. Six studies were performed in the emergency department setting and the other 6 in primary care. Sample sizes ranged from 246 to 2820 patients.
Characteristics of Included Studies
The QUADAS-2 assessment of bias for the 12 studies is presented in Appendix B1. Overall, 6 studies were determined to be at low risk of bias and the other 6 at moderate risk of bias. Those that were moderate risk were only included after consideration of their limitations.6,14,16,17,20,21 Two were case control studies16,21 but enrolled patients prospectively in a consecutive series, and a CXR was performed on each patient. In 1 study,17 not all patients received the reference standard CXR; those with a low probability of CAP were randomized to receive one or not. The final 3 studies that we included6,14,20 enrolled patients when a CXR was ordered because pneumonia was suspected or the physician considered the probability of pneumonia to be greater than zero. We chose to accept this limitation because we are interested in when it is appropriate to order a CXR for CAP in this clinical situation.
Clinical Decision Rules Identified
Studies evaluated CDRs with between 3 and 10 elements (Appendix B2). The 4 individual signs or symptoms that appeared in at least half of the studies were elevated temperature, elevated heart rate, crackles on auscultation, and decreased breath sounds. Elevated temperature was the most common shared sign or symptom, found in 10 of the 12 studies. The definition of elevated temperature was not consistent, with some using greater than 37.8°C, some greater than 38.0°C, and 2 described as just “fever.” The other 3 signs or symptoms were used in 6 studies.
Three studies evaluated a simple point score (see Appendix B3 for calculation of the scores, which had varying points assigned to each of the symptoms). These scores and their classification of patients into post hoc low-, moderate-, and high-risk groups is summarized in Table 2.6,7,9 In the study of Diehr and colleagues,7 patients with a score from −3 to 0 had a SSLR of 0.47, and a 1.2% probability of CAP given a baseline prevalence of 2.6%. The high-risk group had a 27.3% probability of CAP and a SSLR of 14.0.
Clinical Decision Rules That Used a Point Score to Diagnose Pneumonia
Heckerling and colleagues6 used a derivation set (Illinois) to develop a point score and then validated it with data from 2 other locations (Nebraska and Virginia). The Illinois derivation set had an AUROCC of 0.82 (95% CI, 0.78 to 0.86), while the Nebraska and Virginia validation sets had AUROCCs of 0.82 (95% CI, 0.74 to 0.90) and 0.76 (95% CI, 0.66 to 0.86), respectively. Low-, moderate-, and high-risk groups were created post hoc for the derivation set and validation set, and were pooled for the entire study. Overall, patients presenting with no or 1 abnormal finding in the pooled set had a low risk of CAP, with a probability of 4.0% given a baseline prevalence of 29.2% and a SSLR of 0.19. The high-risk group with 4 or 5 abnormal findings had a 64.0% probability of CAP and a SSLR of 8.3.
The study by Van Vugt9 and colleagues incorporated a POC test, c-reactive protein (CRP). The overall model had an AUROCC of 0.77 (0.73 to 0.81). The low-risk group with a score of zero, had 0.7% probability of CAP given a prevalence of 5.2%, with a SSLR of 0.14. The SSLRs for moderate and high-risk groups were 0.76 and 4.3, respectively.
The remaining 9 studies reported the accuracy of 18 simple heuristics that predict a dichotomous outcome of CAP versus no CAP (Appendix B4). A CDR using the presence of normal vital signs to exclude CAP was the most common, appearing in 4 studies,15,16,20,21 while a CDR using normal vital signs plus a normal pulmonary examination to exclude CAP appeared in 3 studies.15,16,21
Fourteen CDRs have good sensitivities (above 75%) while 12 lacked specificity (below 60%). The highest sensitivity was 100%, for a clinical decision tree that used CRP less than 10 μg/mL or CRP 11 to 50 μg/mL and absence of dyspnea or daily fever to exclude CAP.8 Only 2 studies reported CDRs that were both sensitive and specific (86%/72% and 81%/64%), and both used normal vital signs to exclude CAP (temperature, heart rate, and respiratory rate).16,21
Normal vital signs accompanied by no findings on a pulmonary examination rules out CAP well, with 3 studies of this CDR having LR−s 0.09 to 0.11.15,16,21 A meta-analysis of the 4 studies that reported normal vital signs15,16,20,21 as low-yield criteria for CAP had a summary estimate of sensitivity of 0.89 (95% CI, 0.79 to 0.94) and a summary LR− of 0.24 (95% CI, 0.17 to 0.34) (Table 3). The summary receiver operating characteristic (ROC) curve (Appendix C) has good discrimination and a narrow confidence interval, with an AUROCC of 0.89. For the studies reporting the combination of normal vital signs plus normal findings on the pulmonary examination to exclude CAP,15,16,21 the overall sensitivity was 0.96 (95% CI, 0.92 to 0.98) and the LR− 0.10 (95% CI, 0.07 to 0.13). The summary ROC curve for this CDR also had a narrow confidence range around the summary estimate, with an AUROCC of 0.92.
Summary Estimates of Meta-Analysis for the Diagnostic Accuracy of Clinical Decision Rules that Diagnose (Rule in) CAP
Discussion
In this systematic review, we identified 2 potentially useful low-risk criteria for use in the outpatient setting to identify patients at very low risk of CAP: normal vital signs, and normal vital signs plus normal pulmonary findings. The latter simple heuristic performed very well, with very high sensitivity (0.96), low LR− (0.10), and an AUROCC of 0.92.
Only 1 study included a physician's overall clinical impressions as part of the CDR.19 It also included CRP and oxygen saturation, and had unhelpfully high LR− (0.35 to 0.74). Further study of the physician's overall clinical impression would be helpful, including studies to identify which components of the impression had the greatest impact on decision making.
In our systematic review, 5 studies incorporated CRP in their CDR.8,9,18,19,21 Among those, a decision tree developed by Steurer et al8 ruled out CAP in all patients with a CRP less than 10 μg/mL or when CRP was between 11 and 50 μg/mL in patients without persistent dyspnea or fever. This is a potentially very useful CDR that requires prospective validation. In a large, multi-country, prospective study of over 2500 patients where the prevalence of CAP was 5%, the addition of CRP to the score improved discrimination based on the area under the ROC and diagnostic accuracy.9
Of the 12 studies included, 3 were CDRs using a point score. For each rule, the low-risk group had a negative likelihood ratio less than 0.2. In a standardized low prevalence setting (4%), intended to simulate the prevalence of CAP among patients with acute cough in primary care, these scores would reduce the likelihood of CAP in the low-risk group to approximately 0.4% to 0.8% (Table 4). Only 1 of the 3 point scores,6 developed by Heckerling and colleagues, was externally validated, had a large sample size, and of all the CDRs produced in our review had the highest AUROCC. However, it has not been validated in over 20 years. Thus, validation of these CDRs in a contemporary population of patients with acute lower respiratory tract infections or clinically suspected CAP would be helpful.
Simulated Primary Care (4%) and Emergency Department (20%) Prevalence Rates for Selected CDRs
The use of “big data” and machine learning methods with electronic health records (EHRs) will likely be important in the future.22,23 Although EHRs contain a vast amount of patient data, they are often incomplete and are unable to automate the CDR.24 The CDRs found in this systematic review (normal vital signs or normal vital signs and normal pulmonary examination) are simple and allow a physician to quickly rule out CAP and avoid unnecessary CXRs without the need of access to an EHR or other algorithms. These quick decisions are valuable in high turnover settings such as emergency departments or urgent care, which are also where physicians are more likely to encounter adults with a cough.
We are primarily limited by the quality of the included studies, with half of the included studies judged to be a moderate risk of bias. To limit this impact, we only included prospective studies where all participants received the same reference standard. In addition to using MEDLINE, we also reviewed other systematic reviews, the references of included articles, and ran a Google search to assure completeness of the search.
Conclusions
The simple heuristic of normal vital signs and normal pulmonary examination greatly reduces the likelihood of CAP in outpatient with acute cough illness (LR−, 0.1). Clinicians can use this to confidently identify patients with acute cough who do not require a CXR, in the absence of other factors such as signs of malignancy or prolonged symptoms. A simple heuristic that also incorporates CRP is promising, as are several point scores,6,7,9 but these require prospective validation in contemporary outpatient populations.
Appendix A
Search Strategy for MEDLINE:
(“clinical criteria”[TIAB] OR “diagnostic value” [TIAB] OR “predictive value”[TIAB] OR “rule out”[TIAB] OR decision[TIAB] OR prediction[TIAB]) AND (“pneumonia”[MeSH Terms] OR pneumonia[TIAB] OR pneumoniae[TIAB]) AND (community[TIAB] OR emergency[TIAB] OR urgent[TIAB] OR primary[TIAB] OR acute[TIAB] OR “general practice”[TIAB]) NOT (“hospital-acquired”[TIAB] OR “hospital-associated”[TIAB] OR “healthcare-associated”[TIAB] OR nosocomial[TIAB] OR stroke[TIAB] OR klebsiella[TIAB] OR tuberculosis[TIAB] OR surgery[TIAB] OR ventilator[TIAB] OR “intensive care unit”[TIAB] OR “ICU”[TIAB] OR retrospectively[TIAB] OR retrospective[TIAB] OR “case-control”[TIAB] OR “case report”[TIAB] OR “case series”[TIAB] OR gastrointestinal[TIAB] OR immunocompromised[TIAB] OR HIV[TIAB] OR cancer[TIAB]) AND hasabstract[text].
Appendix
Assessment of Study Quality Using the Quality Assessment Tool for Diagnostic Accuracy Studies (QUADAS-2) Framework
Individual Signs, Symptoms, and Point Care Tests Used in Clinical Decision Rules (CDRs) to Diagnose or Rule Out Pneumonia
Diagnostic Accuracy of Clinical Decision Rules Using Signs, Symptoms, and Point of Care Tests to Diagnose (Rule In) Pneumonia
Summary Receiver Operating Characteristic Curves for Clinical Decision Rules Using any Abnormal Vital Signs (A), and Any Abnormal Vital Sign and Abnormal Pulmonary Exam (B) to Diagnose (Rule in) Community Acquired Pneumonia.
Appendix C
Dier, 1984
Heckerling, 1990
Van Vugt, 2013
Notes
This article was externally peer reviewed.
Funding: none.
Conflict of interest: none declared.
To see this article online, please go to: http://jabfm.org/content/32/2/234.full.
- Received for publication July 31, 2018.
- Revision received October 23, 2018.
- Accepted for publication October 24, 2018.
References
In this issue
Jump to section
Related Articles
Cited By...
- Diagnosing community-acquired pneumonia via a smartphone-based algorithm: a prospective cohort study in primary and acute-care consultations
- Diagnosing Community-Acquired Pneumonia: diagnostic accuracy study of a cough-centred algorithm for use in primary and acute-care consultations
- Challenges in the virtual assessment of COVID-19 infections in the community
- The Most Frequently Read Articles of 2019
- Response: Re: Signs and Symptoms That Rule Out Community-Acquired Pneumonia in Outpatient Adults: A Systematic Review and Meta-Analysis
- Multiple Research Methodologies Can Advance the Science of Family Medicine