Abstract
Objective: The goal of this study was to evaluate the quality of evidence supporting primary care–relevant Choosing WiselyTM recommendations using the Strength of Recommendation Taxonomy (SORT).
Methods: All Choosing Wisely “top 5” lists published by American medical specialty societies through June 2014 were reviewed for relevance to primary care. Both authors independently applied SORT to generate an evidence letter grade for each of the included recommendations, relying on citations supplied by the nominating organizations.
Results: Of 310 recommendations, 224 were identified as being relevant to primary care. We rated 43 (19%) as SORT level of evidence A, 57 (25%) as B, and 124 (55%) as C.
Conclusion: We found that a majority of primary care–relevant Choosing Wisely recommendations are based on expert consensus or disease-oriented evidence. Further research is warranted to strengthen the evidence base supporting these recommendations in order to improve their acceptance and implementation into primary care.
The American Board of Internal Medicine Foundation's Choosing WiselyTM campaign, which challenges medical specialty societies to create “top 5” lists of nonbeneficial services that their members commonly provide to patients, has been widely adopted in the United States.1 The first 3 top 5 lists were created by a working group of family physicians, general internists, and pediatricians.2 Since then, 70 different medical provider organizations have joined the campaign and contributed recommendations.3
There is little information about the adoption of Choosing Wisely among primary care physicians. Rosenburg and colleagues4 retrospectively analyzed claims data from a single insurer on 7 of the earliest Choosing Wisely recommendations and found no significant change in use of the services from 2010 through 2013. The prevalence of nonbeneficial services in primary care settings remains high; for example, 46.5% of fee-for-service Medicare beneficiaries who underwent low-risk, noncardiac procedures from 2006 to 2011 received preoperative cardiac testing.5 Implementing Choosing Wisely in primary care could yield substantial benefits for patients and reduce medical costs. However, it may be difficult to convince primary care clinicians to change established practices if, as some have asserted,6 supporting evidence for assessments of services considered “low value” is limited.
The goal of this study was to evaluate the quality of evidence supporting primary care–relevant Choosing Wisely recommendations using a strength of recommendation taxonomy (SORT) developed for US family medicine journals.7
Methods
K.W.L. reviewed all Choosing Wisely top 5 lists published by American medical specialty societies through June 2014 for relevance to this study. Family physicians are appropriate surrogates for primary care because family medicine's scope of practice includes care for women, children, and adults in all settings. Therefore, our final list of primary care–relevant Choosing Wisely recommendations included all services likely to be provided by family physicians (eg, antibiotics, electrocardiography, basic laboratory tests) or referred by family physicians (eg, cardiac stress testing, mammography, dual-energy radiograph absorptiometry). Services generally ordered and/or performed by subspecialists (eg, chemotherapy, complicated surgical procedures) were excluded.
Both authors independently applied SORT to generate an evidence letter grade for each of the included recommendations, relying on citations supplied by the nominating organizations. Both authors have extensive experience and training in applying SORT criteria to peer-reviewed literature and have served as faculty for family medicine residency programs for >5 years. Differences in assigned letter grades were resolved by consensus. After the list of evidence ratings was complete, J.R.Y. categorized the recommendations by body system or discipline. We analyzed the overall distribution of evidence ratings and the proportions of ratings within body system/discipline categories.
Results
Of 310 recommendations reviewed, 224 were identified as being relevant to primary care. After reaching consensus on the 34 recommendations about which we initially disagreed, we rated 43 (19%) as SORT level of evidence A, 57 (25%) as SORT level of evidence B, and 124 (55%) as SORT level of evidence C (Table 1). The most common reasons for disagreement were finding stronger evidence outside of what the organization submitted to justify a higher grade, disagreement about what constituted a high-quality cohort study, and 1 reviewer being able to access the full text of an article through his academic institution that the other could not. Eleven of the SORT level C recommendations were unlikely to be upgraded by further research studies because they were self-evident (eg, reduce radiation exposure whenever possible) or so vague or broad that they could not be proven (eg, discuss risks of interventions with patients). Across medical disciplines, only the orthopedic category had a predominance of SORT level A recommendations (6 of 11, or 55%), whereas several body system categories (cardiovascular, gastrointestinal, pulmonologic, rheumatologic, urologic) contained no SORT level A recommendations. Selected Choosing Wisely recommendations, our SORT ratings, and the rationale for those ratings are provided in Table 2.
Primary Care–Relevant Choosing WiselyTM Recommendations Sorted by Evidence Rating and Body System
Selected Choosing WiselyTM Recommendations with Strength of Recommendation Taxonomy Ratings and Rationale
Discussion
Although a previous analysis categorized the “evidentiary rationales” (level of certainty, comparison of risks and benefits with those of alternatives, comparative cost or cost-effectiveness) for the Choosing Wisely top 5 lists,8 to our knowledge this study is the first to apply a widely accepted evidence rating system to the subset of Choosing Wisely recommendations relevant to primary care. A limitation of our study is that the selection of a service as relevant to primary care was somewhat subjective. We also cannot generalize our findings to the complete list of Choosing Wisely recommendations, including primary care—relevant recommendations, published after June 2014.
We found that a majority of primary care–relevant Choosing Wisely recommendations are based on expert consensus or disease-oriented evidence. In light of other factors that drive unnecessary medical interventions, such as patient satisfaction and fee-for-service reimbursement, this may make it more difficult to convince clinicians to change established practices. Further research is warranted to strengthen the evidence base supporting these recommendations in order to improve their acceptance and implementation into primary care.
Notes
This article was externally peer reviewed.
Funding: none.
Conflict of interest: KWL receives salary support from the American Academy of Family Physicians, which participates in the Choosing Wisely campaign.
Disclaimer: The ideas expressed in the article are solely those of the authors and do not reflect the views of the U.S. Army, the Department of Defense, or the U.S. Government.
- Received for publication January 4, 2016.
- Revision received April 14, 2016.
- Accepted for publication April 18, 2016.
