How Does Prior Experience Pay Off in Large-Scale Quality Improvement Initiatives?

For the ABCS data, we reviewed data, including trajectories of change over time, computing descriptive statistics related to variability and removing practices that exhibited extreme, implausible jumps or otherwise implausible trajectories that could not be addressed by Cooperatives. Practices were also removed that had submissions with non-standard measurement periods. For all other variables and outcomes, data quality checks were performed in an iterative manner, with our team identifying anomalies or irregularities (e.g., excessive missingness, implausible or non-sensical inputs) and Cooperatives addressing those on their end and resubmitting corrected data.

Abbreviations: ABCS, Aspirin, Blood Pressure, Cholesterol, and Smoking; EHR, electronic health record.

Abbreviations: ESCALATES, Evaluating System Change to Advance Learning and Take Evidence to Scale.

Notes: Percentages may not add up to 100% due to missing data. Variables with >15% missing data indicated with an.* ¹Location designation determined using rural-urban commuting area codes. ²Safety net includes Federally Qualitied Health Centers, rural health clinics, Indian Health Services clinics, and other federally owned clinics. ³Other ownership includes nonfederal, private/non-clinician, and those indicating “other” without specifying an ownership type.

Notes: The table shows mean baseline and post-intervention levels, and standard deviations of ABCS, AR, and CPCQ as well as levels of hours and months of facilitation during the intervention for all practices in the sample as well as stratified by Cooperative level of experience. For clinical measures, preliminary data quality assessment revealed large increases or decreases for some practices. To eliminate the influence of such outliers, we excluded practices with outcome change below the 5th percentile or above the 95th percentile from all our analysis. For overall pre-post ABCS, AR, and CPCQ changes, bold denotes statistical significance at the 5% level.

Sources: EvidenceNOW EHR records.

Abbreviations: SD, standard deviation; CPCQ, Change Process Capacity Questionnaire; ABCS, Aspirin, Blood Pressure, Cholesterol, and Smoking; EHR, electronic health record.

Notes: The table shows unadjusted and adjusted regression estimates and p-values (in parentheses) of differences in outcome changes between Cooperative groups. Practices from Cooperatives with low experience are the reference group. Adjusted estimates are based on regressions that include practice characteristics (practice location; practice ownership; practice size; practice patient characteristics). Bold denotes statistical significance at the 5% level. All standard errors are clustered at the Cooperative level using bootstrapping.

Sources: EvidenceNOW EHR records and practice survey.

Abbreviations: EHR, electronic health record; CPCQ, Change Process Capacity Questionnaire.

Abbreviations: FQHCs, Federally qualified health centers; RHS, Rural health services; IHS, Rural health services.

Question 8 was reverse-coded, then responses were rescaled to range from 0 to 1 and averaged to produce the practice-level AR score, ranging from 0 to 1.