Evaluating the Evidence for Choosing WiselyTM in Primary Care Using the Strength of Recommendation Taxonomy (SORT)
==================================================================================================================

* Kenneth W. Lin
* Joseph R. Yancey

## Abstract

*Objective:* The goal of this study was to evaluate the quality of evidence supporting primary care–relevant Choosing WiselyTM recommendations using the Strength of Recommendation Taxonomy (SORT).

*Methods:* All Choosing Wisely “top 5” lists published by American medical specialty societies through June 2014 were reviewed for relevance to primary care. Both authors independently applied SORT to generate an evidence letter grade for each of the included recommendations, relying on citations supplied by the nominating organizations.

*Results:* Of 310 recommendations, 224 were identified as being relevant to primary care. We rated 43 (19%) as SORT level of evidence A, 57 (25%) as B, and 124 (55%) as C.

*Conclusion:* We found that a majority of primary care–relevant Choosing Wisely recommendations are based on expert consensus or disease-oriented evidence. Further research is warranted to strengthen the evidence base supporting these recommendations in order to improve their acceptance and implementation into primary care.

*   Consensus
*   Medicine
*   Primary Health Care
*   Research
*   Medical Societies
*   United States

The American Board of Internal Medicine Foundation's Choosing WiselyTM campaign, which challenges medical specialty societies to create “top 5” lists of nonbeneficial services that their members commonly provide to patients, has been widely adopted in the United States.1 The first 3 top 5 lists were created by a working group of family physicians, general internists, and pediatricians.2 Since then, 70 different medical provider organizations have joined the campaign and contributed recommendations.3

There is little information about the adoption of Choosing Wisely among primary care physicians. Rosenburg and colleagues4 retrospectively analyzed claims data from a single insurer on 7 of the earliest Choosing Wisely recommendations and found no significant change in use of the services from 2010 through 2013. The prevalence of nonbeneficial services in primary care settings remains high; for example, 46.5% of fee-for-service Medicare beneficiaries who underwent low-risk, noncardiac procedures from 2006 to 2011 received preoperative cardiac testing.5 Implementing Choosing Wisely in primary care could yield substantial benefits for patients and reduce medical costs. However, it may be difficult to convince primary care clinicians to change established practices if, as some have asserted,6 supporting evidence for assessments of services considered “low value” is limited.

The goal of this study was to evaluate the quality of evidence supporting primary care–relevant Choosing Wisely recommendations using a strength of recommendation taxonomy (SORT) developed for US family medicine journals.7

## Methods

K.W.L. reviewed all Choosing Wisely top 5 lists published by American medical specialty societies through June 2014 for relevance to this study. Family physicians are appropriate surrogates for primary care because family medicine's scope of practice includes care for women, children, and adults in all settings. Therefore, our final list of primary care–relevant Choosing Wisely recommendations included all services likely to be provided by family physicians (eg, antibiotics, electrocardiography, basic laboratory tests) or referred by family physicians (eg, cardiac stress testing, mammography, dual-energy radiograph absorptiometry). Services generally ordered and/or performed by subspecialists (eg, chemotherapy, complicated surgical procedures) were excluded.

Both authors independently applied SORT to generate an evidence letter grade for each of the included recommendations, relying on citations supplied by the nominating organizations. Both authors have extensive experience and training in applying SORT criteria to peer-reviewed literature and have served as faculty for family medicine residency programs for >5 years. Differences in assigned letter grades were resolved by consensus. After the list of evidence ratings was complete, J.R.Y. categorized the recommendations by body system or discipline. We analyzed the overall distribution of evidence ratings and the proportions of ratings within body system/discipline categories.

## Results

Of 310 recommendations reviewed, 224 were identified as being relevant to primary care. After reaching consensus on the 34 recommendations about which we initially disagreed, we rated 43 (19%) as SORT level of evidence A, 57 (25%) as SORT level of evidence B, and 124 (55%) as SORT level of evidence C (Table 1). The most common reasons for disagreement were finding stronger evidence outside of what the organization submitted to justify a higher grade, disagreement about what constituted a high-quality cohort study, and 1 reviewer being able to access the full text of an article through his academic institution that the other could not. Eleven of the SORT level C recommendations were unlikely to be upgraded by further research studies because they were self-evident (eg, reduce radiation exposure whenever possible) or so vague or broad that they could not be proven (eg, discuss risks of interventions with patients). Across medical disciplines, only the orthopedic category had a predominance of SORT level A recommendations (6 of 11, or 55%), whereas several body system categories (cardiovascular, gastrointestinal, pulmonologic, rheumatologic, urologic) contained no SORT level A recommendations. Selected Choosing Wisely recommendations, our SORT ratings, and the rationale for those ratings are provided in Table 2.

View this table:
[Table 1.](http://www.jabfm.org/content/29/4/512/T1)

Table 1. 
Primary Care–Relevant Choosing WiselyTM Recommendations Sorted by Evidence Rating and Body System

View this table:
[Table 2.](http://www.jabfm.org/content/29/4/512/T2)

Table 2. 
Selected Choosing WiselyTM Recommendations with Strength of Recommendation Taxonomy Ratings and Rationale

## Discussion

Although a previous analysis categorized the “evidentiary rationales” (level of certainty, comparison of risks and benefits with those of alternatives, comparative cost or cost-effectiveness) for the Choosing Wisely top 5 lists,8 to our knowledge this study is the first to apply a widely accepted evidence rating system to the subset of Choosing Wisely recommendations relevant to primary care. A limitation of our study is that the selection of a service as relevant to primary care was somewhat subjective. We also cannot generalize our findings to the complete list of Choosing Wisely recommendations, including primary care—relevant recommendations, published after June 2014.

We found that a majority of primary care–relevant Choosing Wisely recommendations are based on expert consensus or disease-oriented evidence. In light of other factors that drive unnecessary medical interventions, such as patient satisfaction and fee-for-service reimbursement, this may make it more difficult to convince clinicians to change established practices. Further research is warranted to strengthen the evidence base supporting these recommendations in order to improve their acceptance and implementation into primary care.

## Notes

*   This article was externally peer reviewed.

*   *Funding:* none.

*   *Conflict of interest:* KWL receives salary support from the American Academy of Family Physicians, which participates in the Choosing Wisely campaign.

*   *Disclaimer:* The ideas expressed in the article are solely those of the authors and do not reflect the views of the U.S. Army, the Department of Defense, or the U.S. Government.

*   Received for publication January 4, 2016.
*   Revision received April 14, 2016.
*   Accepted for publication April 18, 2016.

## References

1.  1.Wolfson D, Santa J, Slass L. Engaging physicians and consumers in conversations about treatment overuse and waste: a short history of the Choosing Wisely campaign. Acad Med 2014;89:990–5.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1097/ACM.0000000000000270&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=24979166&link_type=MED&atom=%2Fjabfp%2F29%2F4%2F512.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000338210000017&link_type=ISI) 

2.  2.Good Stewardship Working Group. The “top 5” lists in primary care: meeting the responsibility of professionalism. Arch Intern Med 2011;171:1385–90.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1001/archinternmed.2011.231&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=21606090&link_type=MED&atom=%2Fjabfp%2F29%2F4%2F512.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000293642800016&link_type=ISI) 

3.  3.ABIM Foundation. Choosing Wisely: about the campaign. Available from: [http://www.choosingwisely.org/wp-content/uploads/2015/04/About-Choosing-Wisely.pdf](http://www.choosingwisely.org/wp-content/uploads/2015/04/About-Choosing-Wisely.pdf). Accessed March 31, 2016.
    
    

4.  4.Rosenburg A, Agiro A, Gottlieb M, et al. Early trends among seven recommendations from the Choosing Wisely campaign. JAMA Intern Med 2015;175:1913–20.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1001/jamainternmed.2015.5441&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=26457643&link_type=MED&atom=%2Fjabfp%2F29%2F4%2F512.atom) 

5.  5.Colla CH, Morden NE, Sequist TD, Schpero WL, Rosenthal M. Choosing Wisely: prevalence and correlates of low-value health care services in the United States. J Gen Intern Med 2015;30:221–8.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1007/s11606-014-3070-z&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=25373832&link_type=MED&atom=%2Fjabfp%2F29%2F4%2F512.atom) 

6.  6.Howard DH, Gross GP. Producing evidence to reduce low-value care. JAMA Intern Med 2015;175:1893–4.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=26457957&link_type=MED&atom=%2Fjabfp%2F29%2F4%2F512.atom) 

7.  7.Ebell MH, Siwek J, Weiss BD, et al. Strength of recommendation taxonomy (SORT): a patient-centered approach to grading evidence in the medical literature. Am Fam Physician 2004;69:548–56.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=14971837&link_type=MED&atom=%2Fjabfp%2F29%2F4%2F512.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000188830200011&link_type=ISI) 

8.  8.Gilwa C, Pearson SD. Evidentiary rationales for the Choosing Wisely top 5 lists. JAMA Intern Med 2014;311:1443–4.