Skip to main content

Advertisement

Log in

Can Choice of the Sample Population Affect Perceived Performance: Implications for Performance Assessment

  • Original Article
  • Published:
Journal of General Internal Medicine Aims and scope Submit manuscript

ABSTRACT

BACKGROUND

There is accelerating interest in measuring and reporting the quality of care delivered by health care providers and organizations, but methods for defining the patient panels for which they are held accountable are not well defined.

OBJECTIVES

To examine the potential impact of using alternative algorithms to define accountable patient populations for performance assessment.

RESEARCH DESIGN

We used administrative data regarding Community Health Center (CHC) visits in simulations of performance assessment for breast, cervical, and colorectal cancer screening.

PARTICIPANTS

Fifteen CHC sites in the northeastern US.

MEASURES

We used three different algorithms to define patient populations eligible for measurement of cancer screening rates and simulated center-level performance rates based on these alternative population definitions.

RESULTS

Focusing on breast cancer screening, the percentage of women aged 51–75 eligible for this measure across CHCs, if using the most stringent algorithm (requiring a visit in the assessment year plus at least one visit in the 2 years prior), ranged from 28% to 60%. Analogous ranges for cervical and colorectal cancer screening were 18–59% and 26–62%, respectively. Simulated performance data from the centers demonstrate that variations in eligible patient populations across health centers could lead to the appearance of large differences in health center performance or differences in expected rankings of CHCs when no such differences exist. For instance, when holding performance among similar populations constant, but varying the proportion of populations seen across different health centers, simulated health center adherence to screening guidelines varied by over 15% even though actual adherence for similar populations did not differ.

CONCLUSIONS

Quality measurement systems, such as those being used in pay-for-performance and public reporting programs, must consider the definitions used to identify sample populations and how such populations might differ across providers, clinical practice groups, and provider systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1

Similar content being viewed by others

References

  1. Institute of Medicine. To Err is Human: Building a Safer Health System. Washington, DC: National Academy Press; 2000.

    Google Scholar 

  2. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001.

    Google Scholar 

  3. Rosenthal M, Landon BE, Normand SL, Frank RG, Epstein AM. Pay for performance in commercial HMOs. N Engl J Med. 2006;355:1895–1902.

    Article  CAS  PubMed  Google Scholar 

  4. Landon BE, Normand ST, Blumenthal D, Daley J. Physician clinical performance assessment: prospects and barriers. JAMA. 2003;290:1183–9.

    Article  CAS  PubMed  Google Scholar 

  5. Agency for Healthcare Research and Quality. Screening for Breast Cancer: Summary of Recommendations/ Supporting Documents, Available at: http://www.ahrq.gov/clinic/uspstf/uspsbrca.htm, Accessed October 10, 2009.

  6. Agency for Healthcare Research and Quality, Screening for Colorectal Cancer, Available at: http://www.ahrq.gov/clinic/uspstf/uspscolo.htm, Accessed October 10, 2009.

  7. Agency for Healthcare Research and Quality, Screening for Cervical Cancer, Available at: http://www.ahrq.gov/clinic/USpstf/uspscerv.htm, Accessed October 10, 2009.

Download references

Acknowledgements

This project was supported by grant numbers 1R01CA112367–01 from the National Cancer Institute and 1 U01 HS13653 from the Agency for Healthcare Research and Quality, with support from the Health Resources and Services Administration.

The authors thank Yang Xu, M.S., for statistical programming, Emily Corcoran for editorial assistance, and Steve Taplin, M.D., for comments on an earlier version of this manuscript. We also thank the participating Community Health Centers without whom this research could not have been completed.

Conflict of Interest

None disclosed.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Bruce E. Landon MD, MBA.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Landon, B.E., O’Malley, A.J. & Keegan, T. Can Choice of the Sample Population Affect Perceived Performance: Implications for Performance Assessment. J GEN INTERN MED 25, 104–109 (2010). https://doi.org/10.1007/s11606-009-1153-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11606-009-1153-z

Keywords

Navigation