Skip to main content

Main menu

  • HOME
  • ARTICLES
    • Current Issue
    • Abstracts In Press
    • Archives
    • Special Issue Archive
    • Subject Collections
  • INFO FOR
    • Authors
    • Reviewers
    • Call For Papers
    • Subscribers
    • Advertisers
  • SUBMIT
    • Manuscript
    • Peer Review
  • ABOUT
    • The JABFM
    • The Editing Fellowship
    • Editorial Board
    • Indexing
    • Editors' Blog
  • CLASSIFIEDS
  • Other Publications
    • abfm

User menu

  • Log out

Search

  • Advanced search
American Board of Family Medicine
  • Other Publications
    • abfm
  • Log out
American Board of Family Medicine

American Board of Family Medicine

Advanced Search

  • HOME
  • ARTICLES
    • Current Issue
    • Abstracts In Press
    • Archives
    • Special Issue Archive
    • Subject Collections
  • INFO FOR
    • Authors
    • Reviewers
    • Call For Papers
    • Subscribers
    • Advertisers
  • SUBMIT
    • Manuscript
    • Peer Review
  • ABOUT
    • The JABFM
    • The Editing Fellowship
    • Editorial Board
    • Indexing
    • Editors' Blog
  • CLASSIFIEDS
  • JABFM on Bluesky
  • JABFM On Facebook
  • JABFM On Twitter
  • JABFM On YouTube
Research ArticleArticle

Measuring and Improving Quality in the US: Where Are We Today?

Elizabeth A. McGlynn
The Journal of the American Board of Family Medicine September 2020, 33 (Supplement) S28-S35; DOI: https://doi.org/10.3122/jabfm.2020.S1.190398
Elizabeth A. McGlynn
From the Kaiser Permanente Research, Kaiser Permanente, Pasadena, CA; Kaiser Permanente Bernard J. Tyson School of Medicine, Pasadena, CA.
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • References
  • Info & Metrics
  • PDF
Loading

Abstract

In the 50 years since the American Board of Family Medicine (ABFM) was established, the United States has gone from a shared perspective that high-quality care was being routinely delivered to becoming aware of the significant and pervasive problems with quality. Efforts to stimulate improved quality have included public reporting, pay for performance, and value-based purchasing. In addition, maintenance of certification, systematic reviews of research, practice guidelines, electronic health records, and quality improvement programs have offered support for different dimensions of quality. Despite these programs and infrastructure, there is little evidence that quality has improved systematically in the United States. There are areas in which quality is better but many other areas in which quality has remained the same or even worsened. The focus on financial incentives as a primary tool for motivating improvement may not be productive and there is little evidence from research that quality varies with payment or incentives. Quality is a systems issue and requires system solutions. The ABFM has had a long commitment to assessing quality and has an opportunity to lead the way in reimagining quality measurement and assessment.

  • Certification
  • Pay for Performance
  • Quality Improvement
  • Quality of Health Care
  • Value-Based Purchasing

The American Board of Family Medicine’s (ABFM’s) 50th anniversary offers an opportunity to reflect on the United States’ journey in measuring and improving quality. Fifty years is a long time and the blink of an eye for medicine and for the quality journey. Fifty years ago, Denton Cooley implanted the first artificial heart and AIDS is believed to have first migrated to the United States. Advances in science and technology in that year included the Apollo 11 moon landing and walk, the first Concorde test flight, and the first automated teller machine. Medicare and Medicaid were only a few years old when Richard Nixon declared, “We face a massive crisis in health care costs.”1

A Brief, Selected History of Quality Measurement and Performance

Quality of care has been the subject of research and policy debates for a long time as illustrated by selected examples. Florence Nightingale, a leading statistician of her time, documented that unsanitary conditions in military hospitals were the major reason for preventable deaths among soldiers fighting in the Crimean War in 1853. In 1917, Ernest Codman, a pioneer in measuring and classifying medical errors and in assessing the long-term outcomes of surgery, publicly reported on surgical outcomes at Massachusetts General Hospital and called on all hospitals to do the same. A study of care received by members of the Teamsters Union in New York found that 57% of hospital care met recommended standards.2 Lyons and Payne,3 using a novel, peer-reviewed episode of illness method, found that postdischarge care in Hawaii met standards 41% of the time. In 1976, Rhee4 reported on variations in quality among 454 physicians in 18 specialties ranging from 45% for stroke to 91% for cesarean section.

Research on quality has examined both overuse (delivering care for which the expected benefits do not outweigh the expected risks) and underuse (failure to deliver care that has been shown to be beneficial). Wennberg5 reported on variations in the rates with which common surgical procedures were performed across the United States in the Medicare population. Brook and colleagues6 at RAND and University of California, Los Angeles (UCLA) led studies of the appropriateness with which diagnostic and surgical procedures were used. The early studies using this method7⇓⇓⇓⇓⇓–13 found that about one third of procedures were not clinically appropriate, that is, the potential health risks to patients undergoing the procedures were equal to or greater than the potential health benefits.14 However, there was no relationship between appropriateness and the rates of utilization.15⇓–17 Similar rates of inappropriate procedure use were found in the United Kingdom,18 Israel,19 Canada,20 and Sweden21 with very different models of health care financing, and overuse and underuse could co-occur within the same county.22

In the late 1990s, a literature review found consistent and pervasive deficits in quality.23 Despite this body of research, no strong signal had yet emerged that the United States had suboptimal quality. This began to change in the late 1990s and early 2000s with a series of reports and publications. In 1999, the Institute of Medicine (IOM), renamed the National Academy of Medicine in 2015, published To Err is Human, raising attention about the problem of errors in the delivery of medical care.24 The committee estimated that up to 98,000 people died in the United States each year as a result of medical errors. The IOM followed this with Crossing the Quality Chasm, which provided a broader framework for assessing and addressing quality problems.25

Along with the IOM reports, 4 articles from a national study also contributed to breaking through the perception barrier. The study enrolled a random sample of 6700 adults in 12 geographic areas broadly representative of the United States.26 Participants completed a health history survey and gave investigators permission to obtain medical records from all clinicians and institutions from which they had received medical care in the 2 years before enrollment. Using methods previously developed at RAND, the team developed 439 quality measures for 30 acute and chronic conditions representing the leading causes of death and illness, and preventive care. Scores were constructed by identifying all quality indicators for which an individual was eligible and counting the number of those indicators that were received or offered. The first article, published in 2003, reported that American adults were receiving about 55% of the recommended care for the leading causes of illness and death.26 Performance on the quality indicators was similar for preventive, acute, and chronic care but varied by condition. The second article reported that overall quality ranged from 51% in Little Rock, Arkansas to 59% in Seattle, Washington.27 Quality varied by condition and no community was always the best or worst on any of the dimensions examined. The communities varied in population growth rates, average income, poverty levels, rates of uninsured, hospital beds and physicians per 1000 population, and penetration of managed care, but there was no discernable relationship between these economic factors and quality. The third article reported that differences in quality between demographic groups were smaller than the gap between observed performance and optimal performance.28 The fourth article reported that children in the households participating in the national study of adults were receiving 47% of recommended ambulatory care.29 The best performance was observed for acute problems (68%) followed by chronic problems (53%) and then preventive care (41%). Quality for specific conditions ranged from 92% for upper respiratory infection to 34% for adolescent preventive care.

Policy Responses to Address Gaps in Quality

Different approaches to addressing gaps in quality have been undertaken since then. Notably, all these approaches rely on quality measurement as the basis for reporting, incentives, or contracting. These policy approaches were built on top of existing mechanisms for ensuring quality such as professionalism, licensure, and board certification.

Public Reporting of Quality Performance

Public reporting may affect quality through transparency, consumer choice, and reputation. Making quality performance results publicly available provides transparent information about variations in performance. Transparency enables consumers or their agents to use quality reports to choose health plans, hospitals, and doctors. Transparency and consumer choice also increase the motivation of clinicians, hospitals, and systems to improve performance because of concerns about reputation.

In 1986, the Health Care Financing Administration (now the Centers for Medicare and Medicaid Services [CMS]) produced national public reports on hospital mortality for 17 medical and surgical conditions.30 New York state followed with reports on hospital mortality rates for coronary artery bypass graft surgery.30 Since then public reports on quality have been released at various levels in the system—hospital, health plan, medical group, nursing home, physician. The Agency for Healthcare Research and Quality (AHRQ) commissioned a comprehensive review of the literature on public reporting published between 1980 and 2011. Public reporting was associated with an increase in quality improvement activities, some improvements in some quality measures, but little evidence that it affected choice of health care providers by patients.31 The authors found considerable heterogeneity in outcomes and moderate quality evidence making it difficult to draw definitive conclusions. A more recent review examining the impact of public reporting on clinical outcomes found mostly a positive effect on mortality (risk ratio, 0.85; 95% CI, 0.79 to 0.92) although the authors noted considerable heterogeneity among studies.32 Consumer choice of health plans, hospitals, and physicians is a complex task which may explain why transparency has not been associated with major changes in consumer choices.33

Pay for Performance

Fee-for-service payment is generally believed to incentivize clinicians and systems to provide more services than necessary (overuse). Capitation payments raise concerns that clinicians and systems are incentivized to withhold needed care (underuse). Pay for performance (P4P) was developed to reward quality within either type of health care payment system. In P4P, payments to clinicians, hospitals, and systems can be adjusted by adding a quality-based incentive payment (or making a portion of overall payment contingent on quality performance). Private purchasers were early adopters of this approach. For example, in 2001 the California Integrated Healthcare Association created one of the first P4P programs for physician groups. Blue Cross Blue Shield of Massachusetts introduced the Alternative Quality Contract in 2009 with a similar focus on physician groups. CMS collaborated with Premier on the Hospital Quality Incentive Demonstration Program from 2003 to 2009 targeting hospital care for 3 conditions: acute myocardial infarction, heart failure, and pneumonia. CMS also ran the Physician Group Practice Demonstration. States have tried various P4P programs for Medicaid and Children’s Health Insurance Program providers. The design of P4P programs varies and can include both positive and negative incentives (bonuses and penalties).

Research on P4P has found mixed results. A study of the Premier hospital P4P program found early short-term effects that converged with the control group about 5 years into the program.34 A review of P4P similarly found short-term (2 to 3 years) positive effects on processes of care with longer term effects uncertain.35 Positive studies tended to be those in the United Kingdom or in areas with very low baseline performance. No consistent effects were found for intermediate or long-term health outcomes. Although considerable heterogeneity exists among programs, the review concluded that this does not change the mixed assessment of success.35 Although the literature on P4P finds mixed or modest effects, 1 US36 and 1 UK37 study found that performance declined after selected P4P incentives were removed.

Value-Based Purchasing

This extends P4P with a more explicit focus on simultaneously assessing quality and spending. Bundled payments—a lump sum payment for a specific episode of care for a condition—were an early approach to value-based purchasing; private purchasers were early adopters. The approach has been extended to populations, such as those engaged in Medicare’s Accountable Care Organizations. Another variation on this approach is value-based insurance design, in which patient cost sharing is adjusted to incentivize high-value care (or disincentivize low-value care).

Research on these programs has found similar marginal effects.38 An evaluation of Medicare’s Hospital Value Based Purchasing Program found no differences in process of care, patient experience, or mortality.39 The Medicare Payment Advisory Commission has recommended combining the 4 hospital quality payment programs into a single Value Incentive Program to bring the programs into alignment with the Commission’s principles which include a broad measures of overall performance rather than focus on a limited number of conditions.40

Researchers and reviewers have noted a variety of issues that might explain the modest effects associated with quality payment programs such as: design and choice of quality measures, magnitude and design of incentives, method of attributing patients to providers, and the effect on vulnerable populations.

Enablers for Quality: Additional Efforts to Support Quality

The previous section highlighted specific programs that use quality measures and systematic assessments of performance to incentivize quality improvement. A parallel effort is developing needed infrastructure or support for improved quality.

Maintenance of Certification

Time limited board certification reflects a belief that knowledge and skills should be periodically reevaluated because the science of medicine continues to evolve. The ABFM program includes 3 components relevant to quality: self-assessments of knowledge, a secure examination, and performance improvement modules. This approach combines mechanisms to raise awareness of potential gaps in knowledge, a high-stakes examination to demonstrate knowledge, and practice-specific exercises to examine and improve performance.41

Systematic Evidence Reviews

Systematic reviews are one approach to dealing with the challenge of the volume, variable study designs, and quality of published research. Such reviews summarize what is known about the potential benefits and harms of drugs, devices, and other health care services. An IOM committee developed standards for systematic reviews to encourage high quality reviews.42 AHRQ supports the Evidence-based Practice Center program, a group of 12 centers that conduct systematic reviews for both the federal government and other entities.43 The Cochrane Collaboration is another source of high-quality systematic reviews.44

Practice Guidelines

Practice guidelines go beyond systematic reviews by recommending evidence-based practices to manage patients. IOM produced a companion study about the standards for producing good guidelines including the role of scientifically valid systematic reviews.45 To make practice guidelines actionable, many systems have incorporated decision support systems and alerts that provide information about guideline-recommended actions at the point of care.

Electronic Health Records

When the American Recovery and Reinvestment Act of 2009 was passed with incentives for electronic health record (EHR) adoption, there was optimism that EHRs would contribute both to assessing quality performance and providing tools to ensure reliable delivery of high-quality care. The idea of seamless data flows that would help physicians and systems ensure they were delivering recommended care, offer shared decision making tools, and have real-time feedback on quality performance was appealing. Others have written eloquently about the failure of our current experience to meet the goal,46 but it likely that we will redesign rather than eliminate EHRs from medical practice.

Quality Improvement Programs

Quality improvement programs have been organized at different levels in the health care system. The approaches include Plan-Do-Study-Act (PDSA) cycles, Lean, 6 Sigma, and total quality management. Typically, these approaches focus on changing a discrete practice (rather than larger systems changes) and are undertaken in local settings. Quality improvement also tends to be iterative and limited to the specific context in which the improvement strategy is undertaken, making systematic assessment of the impact of these programs challenging.

Putting It All Together: Are We Making Progress?

Over the last 50 years, we have become more aware that recommended care is not always offered or received, and we have implemented a variety of programs and supports to address that problem. We are measuring more dimensions of quality, more often, for more parts of the health care system. We are realigning incentives to reward better performance. We are trying to make it easier to know what works best for which patients. We are making that information more readily available at the point of care. There are individually focused and organizationally driven programs to improve quality. The public is more aware of variations in quality and some are more engaged in ensuring they receive the care they need or in choosing where to go for care based on publicly available data. So, we must be doing better, right?

This is a hard question to answer. We have not had any large-scale, national assessment of quality since 2003. Much of the public reporting in quality focuses on subsets of the population defined by the setting of care (eg, hospitals), the payer (eg, Medicare, managed care), or a limited number of conditions (eg, heart attacks). We have some insights from AHRQ which publishes a regular report on quality and disparities.47 The report includes a variety of process, outcome, and patient-experience measures from multiple federal data sources. The collection of indicators and report construction make it difficult to garner any overall sense of the current state of quality. Levine and colleagues48 examined changes in a select number of process measures using data from the federal Medical Expenditures Panel Survey and found little change over the period from 2002 to 2013.

The National Committee for Quality Assurance (NCQA) has the longest standing public report on quality across multiple sectors. The current version of Healthcare Effectiveness Data and Information Set (HEDIS) includes 57 measures in multiple categories (prevention, chronic care management, overuse, patient experience) and is reported for commercial, Medicaid and Medicare enrollees. An examination of the 2018 State of Health Care Quality report underscores the variability in progress.49 For example, breast and cervical cancer screening have shown little improvement, hovering around the low 70%s, whereas colorectal cancer screening among Medicare Advantage enrollees improved from 52.6% to 69.6% from 2004 to 2017. Blood pressure control among commercial Health Maintenance Organization (HMO) enrollees has improved from 39% to 62.2% from 1999 to 2017 and glycated hemoglobin (HbA1c) control has stayed about the same for both commercial and Medicare enrollees at around 60%.

The collection of studies on public reporting, pay for performance and value-based purchasing also lead to the conclusion that major improvements in quality have not occurred as a result of those initiatives. There have been pockets of improvement (“islands of excellence”) but few systematic, sustained, and substantial improvements.

Where Do We Go from Here?

Improving quality seems to be a daunting task. We may have been focused on the wrong approaches, specifically, relying on payment incentives to create the conditions for improvement despite considerable research that finds little relationship between the way care is paid for, whether in the United States or other countries, and the appropriateness or quality of care delivered. It is not unreasonable to consider whether payment methods create barriers to improvement, but it is perhaps unreasonable to assume that changes in payment alone will address the problem. Another approach has been to focus on the measures including reducing the number of measurements,50 aligning measures with other regulatory programs or developing different measures (generally under the heading of “measures that matter”). Most of these efforts are designed to reduce the burden of measurement (rather than improve quality) but this would require a level collaboration across state, federal, and private entities that has not yet emerged.

Improving quality requires a systems approach. This is particularly evident in primary care where the complexity of matching the unique characteristics of an individual patient to the wide array of necessary or recommended interventions in the context of a series of short, unplanned, and uncertainly scheduled visits exceeds the capacity of the human brain. Moreover, patients initiate much of their own care, move between coverage programs and providers, and have preferences that inform (rightfully) the care they receive—all which make delivery and measurement of quality care more complex. Health care systems that have made significant and sustained progress in quality have devoted time, people, and resources to creating systems—and still encounter challenges every day in managing improvement.51

The observation that quality is a systems issue leads some to wonder which components have the largest effect on performance—physicians, medical groups, health plans, patients, and other economic factors. One study found that the variance attributable to physicians ranged from 39% to 49%, compared with 13% to 24% for medical groups and 12% to 26% for the service area, depending on the measure.52 Studies using different measures, settings, and geographies come to different conclusions53⇓–55 but all entities contribute to observed results.

An example from Kaiser Permanente illustrates the multiple components that may be required to achieve high performance. Between 2000 and 2013, Kaiser Permanente–Northern California improved blood pressure control among patients with hypertension from 44% to 90%.56,57 The region undertook a coordinated effort with 6 critical components: leadership commitment, developing and building a hypertension registry, creating and updating an evidence-based guideline that included a drug treatment algorithm, routine feedback to medical centers and physicians on performance, medical assistant blood pressure visits without copays, and incorporating single-pill combination medication. This systems approach—with the work of everyone on the team aligned—has been replicated in other clinical areas with similar significant improvements in performance.

Because most physicians do not practice in organizations like these large systems, alternative mechanisms may be needed to create virtual or other systems that could support physicians on their quality journeys. This represents a potential role for Boards that are responsible for assessing physician quality through performance improvement modules that constitute a part of maintenance of certification. The ABFM, for example, could identify a comprehensive set of quality indicators that reflect excellence in primary care, develop systems to routinely assess physician performance on those indicators (including novel ways of extracting data), and provide feedback to physicians to facilitate improvement. They could certify programs or entities that provide good value in coaching physicians in quality improvement. Boards could also highlight where financial barriers exist to delivering quality care.

At the same time, Boards could invest in reimagining quality measurement, particularly for primary care physicians, and work with regulators and accreditors to provide safe harbors for those willing to be part of the needed innovations.58,59 There is considerable dissatisfaction with the current state of measurement and assessment but few serious efforts exist to make significant changes. This is a ripe area for physician leadership—and if physicians do not or cannot step up to lead these efforts, we may see more of the same ineffective efforts continue.

Notes

  • This article was externally peer reviewed.

  • Funding: None.

  • Conflicts of interest: None.

  • Received for publication October 28, 2019.
  • Revision received May 5, 2020.
  • Accepted for publication May 5, 2020.

References

  1. 1.↵
    1. Blumenthal D,
    2. Morone J
    . The heart of power: health and politics in the oval office. Berkeley, CA: University of California Press; 2009, 226.
  2. 2.↵
    1. Morehead MA,
    2. Donaldson R,
    3. Sanderson RS,
    4. et al
    . A study of the quality of hospital care secured by a sample of teamster family members in New York City. New York, NY: Columbia University, School of Public Health and Administrative Medicine; 1964.
  3. 3.↵
    1. Lyons TF,
    2. Payne BC
    . The Quality of Physicians’ Health-Care Performance: A comparison against optimal criteria for treatment of the elderly and younger adults in community hospitals. JAMA 1974;227:925–8.
    OpenUrlCrossRefPubMed
  4. 4.↵
    1. Rhee SO
    . Factors determining the quality of physician performance in patient care. Med Care 1976;14:733–50.
    OpenUrlCrossRefPubMed
  5. 5.↵
    1. Wennberg J,
    2. Gittelsohn A
    . Small area variations in health care delivery. Science 1973;182:1102–8.
    OpenUrlAbstract/FREE Full Text
  6. 6.↵
    1. Brook RH,
    2. Chassin MR,
    3. Fink A,
    4. Solomon DH,
    5. Kosecoff J,
    6. Park RE
    . A method for the detailed assessment of the appropriateness of medical technologies. Int J Technol Assess Health Care 1986;2:53–63.
    OpenUrlCrossRefPubMed
  7. 7.↵
    1. Winslow CM,
    2. Kosecoff JB,
    3. Chassin M,
    4. Kanouse DE,
    5. Brook RH
    . The appropriateness of performing coronary artery bypass surgery. JAMA 1988;260:505–9.
    OpenUrlCrossRefPubMed
  8. 8.↵
    1. Winslow CM,
    2. Solomon DH,
    3. Chassin MR,
    4. Kosecoff J,
    5. Merrick NJ,
    6. Brook RH
    . The appropriateness of carotid endarterectomy. N Engl J Med 1988;318:721–7.
    OpenUrlCrossRefPubMed
  9. 9.↵
    1. Bernstein SJ,
    2. McGlynn EA,
    3. Siu AL,
    4. et al
    . The appropriateness of hysterectomy. A comparison of care in seven health plans. Health Maintenance Organization Quality of Care Consortium. JAMA 1993;269:2398–402.
    OpenUrlCrossRefPubMed
  10. 10.↵
    1. Hilborne LH,
    2. Leape LL,
    3. Bernstein SJ,
    4. et al
    . The appropriateness of use of percutaneous transluminal coronary angioplasty in New York State. JAMA 1993;269:761–5.
    OpenUrlCrossRefPubMed
  11. 11.↵
    1. Leape LL,
    2. Hilborne LH,
    3. Park RE,
    4. et al
    . The appropriateness of use of coronary artery bypass graft surgery in New York State. JAMA 1993;269:753–60.
    OpenUrlCrossRefPubMed
  12. 12.↵
    1. Bernstein SJ,
    2. Hilborne LH,
    3. Leape LL,
    4. Park RE,
    5. Brook RH
    . The appropriateness of use of cardiovascular procedures in women and men. Arch Intern Med 1994;154:2759–65.
    OpenUrlCrossRefPubMed
  13. 13.↵
    1. Kleinman LC,
    2. Kosecoff J,
    3. Dubois RW,
    4. Brook RH
    . The medical appropriateness of tympanostomy tubes proposed for children younger than 16 years in the United States. JAMA 1994;271:1250–5.
    OpenUrlCrossRefPubMed
  14. 14.↵
    1. Anderson RJ,
    2. Rice TH,
    3. Kominski GF
    1. McGlynn EA,
    2. Brook RH
    . Ensuring quality of care. In Anderson RJ, Rice TH, Kominski GF, eds. Changing the U.S. health care system: key issues in policy and management. 2nd ed. San Francisco, CA: Jossey-Bass; 2001.
  15. 15.↵
    1. Siu AL,
    2. Sonnenberg FA,
    3. Manning WG,
    4. et al
    . Inappropriate use of hospitals in a randomized trial of health insurance plans. N Engl J Med 1986;315:1259–66.
    OpenUrlCrossRefPubMed
  16. 16.↵
    1. Chassin MR,
    2. Kosecoff J,
    3. Park RE,
    4. et al
    . Does inappropriate use explain geographic variations in the use of health care services? A study of three procedures. JAMA 1987;258:2533–7.
    OpenUrlCrossRefPubMed
  17. 17.↵
    1. Leape LL,
    2. Park RE,
    3. Solomon DH,
    4. Chassin MR,
    5. Kosecoff J,
    6. Brook RH
    . Does inappropriate use explain small-area variations in the use of health care services? JAMA 1990;263:669–72.
    OpenUrlCrossRefPubMed
  18. 18.↵
    1. Gray D,
    2. Hampton JR,
    3. Bernstein SJ,
    4. Kosecoff J,
    5. Brook RH
    . Audit of coronary angiography and bypass surgery. Lancet 1990;335:1317–20.
    OpenUrlCrossRefPubMed
  19. 19.↵
    1. Pilpel D,
    2. Fraser GM,
    3. Kosecoff J,
    4. Weitzman S,
    5. Brook RH
    . Regional differences in appropriateness of cholecystectomy in a prepaid health insurance system. Public Health Rev 1992–1993;20:61–74.
    OpenUrl
  20. 20.↵
    1. McGlynn EA,
    2. Naylor CD,
    3. Anderson GM,
    4. et al
    . Comparison of the appropriateness of coronary angiography and coronary artery bypass graft surgery between Canada and New York State. JAMA 1994;272:934–40.
    OpenUrlCrossRefPubMed
  21. 21.↵
    1. Bernstein SJ,
    2. Brorsson B,
    3. Aberg T,
    4. Emanuelsson H,
    5. Brook RH,
    6. Werko L
    . Appropriateness of referral of coronary angiography patients in Sweden. SECOR/SBU Project Group. Heart 1999;81:470–7.
    OpenUrlAbstract/FREE Full Text
  22. 22.↵
    1. Carlisle DM,
    2. Valdez RB,
    3. Shapiro MR,
    4. Brook RH
    . Geographic variation in rates of selected surgical procedures within Los Angeles County. Health Serv Res 1995;30:27–42.
    OpenUrlPubMed
  23. 23.↵
    1. Schuster MA,
    2. McGlynn EA,
    3. Brook RH
    . How good is the quality of health care in the United States? Milbank Q 1998;76:517–63.
    OpenUrlCrossRefPubMed
  24. 24.↵
    1. Kohn LT,
    2. Corrigan JM,
    3. Donaldson MS
    Institute of Medicine (US) Committee on Quality of Health Care in America. Kohn LT, Corrigan JM, Donaldson MS, eds. To err is human: building a safer health system. Washington, DC: National Academies Press; 1999.
  25. 25.↵
    Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academies Press; 2001.
  26. 26.↵
    1. McGlynn EA,
    2. Asch SM,
    3. Adams J,
    4. et al
    . The quality of health care delivered to adults in the United States. N Engl J Med 2003;348:2635–45.
    OpenUrlCrossRefPubMed
  27. 27.↵
    1. Kerr EA,
    2. McGlynn EA,
    3. Adams J,
    4. Keesey J,
    5. Asch SM
    . Profiling the quality of care in 12 communities: results CQI study. Health Aff (Millbank) 2004;23:247–56.
    OpenUrl
  28. 28.↵
    1. Asch SM,
    2. Kerr EA,
    3. Keesey J,
    4. et al
    . Who is at greatest risk for receiving poor-quality health care? N Engl J Med 2006;354:1147–56.
    OpenUrlCrossRefPubMed
  29. 29.↵
    1. Mangione-Smith R,
    2. DeCristofaro AH,
    3. Setodji CM,
    4. et al
    . The quality of ambulatory care delivered to children in the United States. N Engl J Med 2007;357:1515–23.
    OpenUrlCrossRefPubMed
  30. 30.↵
    1. Hannan EL,
    2. Barach PR,
    3. Jacobs JP,
    4. Lipshultz SE,
    5. Laussen PC
    , eds. Public reporting of cardiac data: pros, cons, and lessons for the future. Pediatric and Congenital Cardiac Care 2015;1:467–77.
    OpenUrl
  31. 31.↵
    1. Totten AM,
    2. Wagner J,
    3. Tiwari A,
    4. O’Haire C,
    5. Griffin J,
    6. Walker M
    . Closing the quality gap: revisiting the state of the science (vol. 5, public reporting as a quality strategy). Evid Rep Technol Assess, No 208.5. Rockville, MD: Agency for Healthcare Research and Quality; 2012.
  32. 32.↵
    1. Campanella P,
    2. Vukovic V,
    3. Parente P,
    4. Sulejmani A,
    5. Ricciardi W,
    6. Specchia ML
    . The impact of public reporting on clinical outcomes: a systematic review and meta-analysis. BMC Health Serv Res 2016;16:296.
    OpenUrlCrossRefPubMed
  33. 33.↵
    1. Schlesinger M,
    2. Kanouse DE,
    3. Martino SC,
    4. Shaller D,
    5. Rybowski L
    . Complexity, public reporting, and choice of doctors: a look inside the blackest box of consumer behavior. Med Care Res Rev 2014;71:38S–64S.
    OpenUrlCrossRefPubMed
  34. 34.↵
    1. Werner R,
    2. Kolstad JT,
    3. Stuart KA,
    4. Polsy D
    . The effect of pay-for-performance in hospitals: lessons for quality improvement. Health Aff (Millbank) 2011;30:690–8.
    OpenUrl
  35. 35.↵
    1. Mendelson A,
    2. Kondo K,
    3. Damberg C,
    4. et al
    . The effects of pay-for-performance programs on health, health care use, and processes of care: a systematic review. Ann Intern Med 2017;166:341–53.
    OpenUrlCrossRefPubMed
  36. 36.↵
    1. Lester H,
    2. Schmittdiel J,
    3. Selby J,
    4. et al
    . The impact of removing financial incentives from clinical quality indicators: longitudinal analysis of four Kaiser Permanente indicators. BMJ 2010;340:c1898.
    OpenUrlAbstract/FREE Full Text
  37. 37.↵
    1. Minchin M,
    2. Roland M,
    3. Richardson J,
    4. Rowark S,
    5. Guthrie B
    . Quality of care in the United Kingdom after removal of financial incentives. N Engl J Med 2018;379:948–57.
    OpenUrlCrossRefPubMed
  38. 38.↵
    1. Chee TT,
    2. Ryan AM,
    3. Wasfy JH,
    4. Borden WB
    . Current state of value-based purchasing programs. Circulation 2016;133:2197–205.
    OpenUrlAbstract/FREE Full Text
  39. 39.↵
    1. Ryan AM,
    2. Krinsky S,
    3. Maurer KA,
    4. Dimick JB
    . Changes in hospital quality associated with hospital value-based purchasing. N Engl J Med 2017;376:2358–66.
    OpenUrlCrossRefPubMed
  40. 40.↵
    Medicare Payment Advisory Commission. Report to the Congress: Medicare payment policy. Washington, DC: MedPAC; 2019.
  41. 41.↵
    Institute of Medicine. Finding what works in health care: standards for systematic reviews. Washington, DC: The National Academies Press; 2011.
  42. 42.↵
    1. Peterson LE,
    2. Blackburn BE,
    3. Puffer JC,
    4. Phillips RC
    . Family physicians’ quality interventions and performance improvement through the ABFM diabetes performance in practice module. Ann Fam Med 2014;12:17–20.
    OpenUrlAbstract/FREE Full Text
  43. 43.↵
    Agency for Healthcare Research and Quality. About the evidence-based practice centers program. Rockville, MD. 2015. Available from: https://www.ahrq.gov/research/findings/evidence-based-reports/overview/index.html. Accessed October 27, 2019.
  44. 44.↵
    Cochrane Collaboration. Cochrane Database of Systematic Reviews. Cochrane Library. 2019. Available from: https://www.cochranelibrary.com/cdsr/about-cdsr. Accessed October 27, 2019.
  45. 45.↵
    1. Mancher M,
    2. Miller Wolman D,
    3. Greenfield S,
    4. Steinberg E
    1. Graham R
    , Mancher M, Miller Wolman D, Greenfield S, Steinberg E, eds. Clinical practice guidelines we can trust. Washington, DC: The National Academies Press; 2011.
  46. 46.↵
    1. Wachter R
    . The digital doctor: hope, hype, and harm at the dawn of medicine’s computer age. McGraw-Hill Education; 2015.
  47. 47.↵
    Agency for Healthcare Research and Quality. 2018. National Healthcare Quality and Disparities Report. 2019. Available from: https://www.ahrq.gov/research/findings/nhqrdr/nhqdr18/index.html. Accessed October 27, 2019.
  48. 48.↵
    1. Levine DM,
    2. Linder JA,
    3. Landon BE
    . The quality of outpatient care delivered to adults in the United States, 2002 to 2013. JAMA Intern Med 2016;176:1778–90.
    OpenUrl
  49. 49.↵
    National Committee for Quality Assurance. State of health care quality, 2018. 2019. Available from: https://www.ncqa.org/report-cards/health-plans/state-of-health-care-quality-report/. Accessed October 2, 2019.
  50. 50.↵
    Institute of Medicine. Vital signs: core metrics for health and health care progress. Washington, DC: The National Academies Press; 2015.
  51. 51.↵
    1. Schilling L,
    2. Chase A,
    3. Liu AY,
    4. et al
    . Kaiser Permanente’s performance improvement system, part 1: from benchmarking to executing on strategic priorities. Jt Comm J Qual Saf 2010;36:484–98.
    OpenUrl
  52. 52.↵
    1. Rodriguez HP,
    2. Scoggins JF,
    3. von Glahn T,
    4. Zaslavsky AM,
    5. Safran DG
    . Attributing sources of variation in patients’ experiences of ambulatory care. Med Care 2009;47:835–41.
    OpenUrlCrossRefPubMed
  53. 53.↵
    1. O’Connor PJ,
    2. Rush WA,
    3. Davidson G,
    4. et al
    . Variation in quality of diabetes care at the levels of patient, physician, and clinic. Prev Chronic Dis 2008;5:A15.
    OpenUrlPubMed
  54. 54.↵
    1. Selby JV,
    2. Schmittdiel JA,
    3. Lee J,
    4. et al
    . Meaningful variation in performance: what does variation in quality tell us about improving quality? Med Care 2010;48:133–9.
    OpenUrlCrossRefPubMed
  55. 55.↵
    1. Zaslavsky AM
    . Using hierarchical models to attribute sources of variation in consumer assessments of health care. Statist Med 2007;26:1885–900.
    OpenUrl
  56. 56.↵
    1. Jaffe MG,
    2. Lee GA,
    3. Young JD,
    4. Sidney S,
    5. Go AS
    . Improved blood pressure control associated with a large-scale hypertension program. JAMA 2013;310:699–705.
    OpenUrlCrossRefPubMed
  57. 57.↵
    1. Jaffe MG,
    2. Young JD
    . The Kaiser Permanente Northern California story: improving hypertension control from 44% to 90% in 13 years (2000–2013). J Clin Hypertens 2016;18:260–1.
    OpenUrl
  58. 58.↵
    1. McGlynn EA,
    2. Schneider EC,
    3. Kerr EA
    . Reimagining quality measurement. N Engl J Med 2014;371:2150–3.
    OpenUrlCrossRefPubMed
  59. 59.↵
    1. McGlynn EA,
    2. Kerr EA
    . Creating safe harbors for quality measurement innovation and improvement. JAMA 2016;315:129–30.
    OpenUrlPubMed
PreviousNext
Back to top

In this issue

The Journal of the American Board of Family   Medicine: 33 (Supplement)
The Journal of the American Board of Family Medicine
Vol. 33, Issue Supplement
September-October 2020
  • Table of Contents
  • Table of Contents (PDF)
  • Cover (PDF)
  • Index by author
  • Front Matter (PDF)
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word on American Board of Family Medicine.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Measuring and Improving Quality in the US: Where Are We Today?
(Your Name) has sent you a message from American Board of Family Medicine
(Your Name) thought you would like to see the American Board of Family Medicine web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
14 + 1 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Citation Tools
Measuring and Improving Quality in the US: Where Are We Today?
Elizabeth A. McGlynn
The Journal of the American Board of Family Medicine Sep 2020, 33 (Supplement) S28-S35; DOI: 10.3122/jabfm.2020.S1.190398

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Share
Measuring and Improving Quality in the US: Where Are We Today?
Elizabeth A. McGlynn
The Journal of the American Board of Family Medicine Sep 2020, 33 (Supplement) S28-S35; DOI: 10.3122/jabfm.2020.S1.190398
Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Abstract
    • A Brief, Selected History of Quality Measurement and Performance
    • Policy Responses to Address Gaps in Quality
    • Enablers for Quality: Additional Efforts to Support Quality
    • Putting It All Together: Are We Making Progress?
    • Where Do We Go from Here?
    • Notes
    • References
  • References
  • Info & Metrics
  • PDF

Related Articles

  • No related articles found.
  • PubMed
  • Google Scholar

Cited By...

  • Measuring and publishing quality improvement
  • Towards a Quality Agenda for Family Medicine
  • How Should Board Certification Evolve?
  • Google Scholar

More in this TOC Section

  • CERA: A Vehicle for Facilitating Research in Family Medicine
  • Current and Future Challenges to Publishing Family Medicine Research
  • Putting Trainees at the Center of the Family Medicine Research Workforce of Tomorrow
Show more Article

Similar Articles

Keywords

  • Certification
  • Pay for Performance
  • Quality Improvement
  • Quality of Health Care
  • Value-Based Purchasing

Navigate

  • Home
  • Current Issue
  • Past Issues

Authors & Reviewers

  • Info For Authors
  • Info For Reviewers
  • Submit A Manuscript/Review

Other Services

  • Get Email Alerts
  • Classifieds
  • Reprints and Permissions

Other Resources

  • Forms
  • Contact Us
  • ABFM News

© 2025 American Board of Family Medicine

Powered by HighWire