INTRODUCTION

Engaging patients with chronic diseases in self-management activities may improve their well-being and prevent costly medical complications. The potential health and economic returns of patient involvement in care has driven an expansion of programs that “assist consumers and their support system to become engaged in a collaborative process designed to manage medical/social/mental health conditions more effectively.”1 Commonly referred to as disease management programs, they are implemented by a number of health insurers and commercial health care companies. These programs primarily rely on telephonic intervention, are independent of the patient’s usual health care setting, and have demonstrated some reductions in health care utilization and costs.24 Conversely, primary care providers (PCP) are increasingly offering chronic care management programs within their practices.57 Both insurer disease management and provider care management include patient education, goal setting, and self-management support. However, the latter has the advantage of integration and coordination with the patient’s PCP, which may be key to improving health outcomes.812

Although each of the two aforementioned approaches has been evaluated independently, there is a lack of studies comparing them directly. In addition, the literature is inconsistent with regard to outcomes in natural care delivery settings.1315 There appears to be much to learn about the effectiveness of these approaches and how they compare to each other.16 Initiatives in primary care, such as the Patient Centered Medical Home (PCMH), encourage transforming the organization and delivery of primary care,1719 by, for example, adding care managers.20,21 Given that practices will likely continue and increase participation in PCMH and similar efforts, evidence of care management’s effectiveness in this setting is needed.

In 2010, a large Michigan health insurer with a health plan-delivered disease management program conducted a 2-year pilot program to support care management delivered in primary care settings. Subsequently, the Agency for Healthcare Research and Quality funded an evaluation study to compare the effectiveness between the two care management models, in terms of patient targeting and engagement in care management, as well as clinical and utilization outcomes. Patient engagement and utilization results are reported elsewhere.22,23 The focus of this paper is to compare the relative effectiveness of the two delivery modes on patient clinical outcomes. We hypothesized that provider-delivered care management (PDCM) is more effective than the health-plan delivered care management (HPDCM), due to the integration of care management in the primary care setting.

METHODS

Care Management Interventions

In efforts to improve the health of its members, the health plan in this study routinely administers HPDCM for members with at least one of five chronic diseases: congestive heart failure, chronic obstructive pulmonary disease, coronary heart disease, diabetes, or asthma. Employers contract with the health plan to deliver this program to qualified members. HPDCM is provided by dedicated nurses employed by and located at the health plan (centralized), who reach out to and engage eligible members via phone calls. Details of the HPDCM program are summarized in the last column of Table 1.

Table 1. Description of the PDCM and HPDCM Care Management Approaches

Aligned with its goal to promote the implementation of PCMHs,24 the health plan recognized that care management delivered within a primary care setting may increase engagement of members in their care and result in improvements in health outcomes and reductions in health care utilization and costs. Therefore, this health plan funded five physician organizations (POs) for a pilot PDCM program that was implemented from April 2010 through March 2012. The POs were responsible to develop new, or expand existing, care management programs within their local context. However, both PDCM and HPDCM included a dedicated care manager and care management activities (i.e., chronic disease education, self-management support, motivational interviewing, goal setting, and health coaching). The PDCM programs varied from both HPDCM and each other in their targeting strategy, structure, program features and scope.22 Details of PO-specific approaches are shown in the first five columns in Table 1. Some key differences between PDCM programs were the mode of care management delivery (phone or in person), care manager background (RN or PharmD), and the focus of care management efforts (patients with medication management issues or complex diseases).

Study Design

The need for ongoing program delivery prevented random assignment of POs and their associated members to PDCM or HPDCM. Thus, to compare the PDCM pilot (intervention) with the HPDCM program (comparison), we adopted a quasi-experimental two-group pre-post design. To create the comparison group, we selected patients belonging to primary care practices that either were not participating in the pilot, but were affiliated with one of the participating POs, or were similar to pilot practices in geographic area and size, but were affiliated with non-pilot POs. The first type shared the same central administrative support from the pilot POs and the second type was in close proximity with the pilot practices, which made both somewhat comparable to intervention practices. The pre-post periods covered 12 months before and after the pilot start and end dates.

Study Populations

Both groups included patients who were health plan members eligible for coverage with HPDCM and confirmed as having at least one of the five chronic diseases covered in HPDCM. PDCM included patients assigned to a PCP in one of the PDCM pilot practices and reported by the pilot POs as having received an attempted or actual contact for outreach by the pilot practice. HPDCM patients were assigned to a PCP in one of the selected comparison practices, identified as meeting health plan high-cost risk criteria (per claims-based algorithms), and outreached or had an outreach attempt by a health plan-employed nurse for HPDCM.

Data Collection

Care Management Processes

On a monthly basis, POs reported care management activities to the health plan, including 1) the number of patients “accepted” into the pilot, i.e., those confirmed by the practice as its patient and having one of more of the five chronic diseases; 2) patients “targeted” for the pilot, i.e., those outreached/offered care management; and 3) patients “engaged” in the pilot, i.e., those with one or more care management sessions/encounters with the care manager. For HPDCM, outreach and engagement status were obtained from nurse-recorded data in the health plan’s information system.

Practice Characteristics

We obtained data from the health plan for several practice characteristics (Table 2), including the risk score of attributed patients. The average risk score (standardized to have a mean of one) is a claims-derived risk score for high health care costs (Ingenix Impact IntelligenceTM). As an indicator of practices’ level of PCMH implementation, which would likely influence their ability to provide care management and other population-based health services, we obtained the 2009 PCMH scores collected by the health plan in conjunction with the plan’s PCMH designation program.24

Table 2. Practice- and Patient-level Characteristics Between PDCM and HPDCM Groups

Patient Clinical Data

Because POs had different electronic health record (EHR) and registry systems, we constructed a data collection form so that each PO could query their data system using a common metric (e.g., ICD-9 diagnosis codes for the five diseases). Data extracted included age, sex, height, weight, body mass index (BMI), systolic and diastolic blood pressure (SBP and DBP), total cholesterol, high- and low-density lipoprotein (HDL and LDL), hemoglobin A1c (HbA1c), B-type natriuretic peptide blood test, left ventricular ejection fraction, dates of diagnoses for any of the five chronic diseases above, and medications prescribed. All data spanned 1 April 2009 to 31 December 2012 (one year before, during and nearly one year after the pilot).

Statistical Methods

The primary outcomes were the proportions of patients with LDL < 100 mg/dL, HDL ≥ 40 mg/dL (male) or HDL ≥ 50 mg/dL (female), total cholesterol ≤ 200 mg/dL, SBP < 140 mmHg and DBP < 80 mmHg, obesity (BMI ≥ 30), extreme obesity (BMI ≥ 40), and the average weight loss (kg), as these measures are commonly used by health plans for quality improvement. For patients with type 2 diabetes, following the American Diabetes Association guidelines, we defined an indicator for optimal outcome if the patient met these criteria: HbA1c < 7 %, SBP < 140 mmHg, DBP < 90 mmHg, and LDL < 100 mg/dL.25 B-type natriuretic peptide blood test and left ventricular ejection fraction had too few observations for meaningful analysis.

Consistent with the quasi-experimental two-group pre-post design, we used the difference-in-differences (DID) method to ameliorate potential selection bias (Fig. 1). The dashed line indicates the counterfactual outcome in the treatment group had there been no treatment. Thus, using the DID model, we subtracted differences in outcomes between PDCM and HPDCM prior to the interventions, i.e., the systematic differences, from the difference in outcomes between PDCM and HPDCM after the interventions to derive the comparative effectiveness. Hence, the baseline differences were not attributed to the programs. Due to difficulties in collecting EHR data from non-pilot POs and the limited number of potential comparison patients, we could not use propensity score or other matching methods to select patients. However, matching and regression adjustment both controlled for observable confounders.26,27

Figure 1.
figure 1

Causal effects in the difference-in-differences model.

We employed generalized linear models, controlling patient and practice characteristics and accounting for the clustering of patients in practices and practices in POs. Each model included the main effect terms for PDCM, the pre-post indicator, and the interaction between them. The interaction term gave the DID estimate indicating whether PDCM led to more improvement in patient outcomes compared to HPDCM. We estimated the marginal predictions over the entire study population to estimate the causal average mean difference in changes in the outcomes. The 95 % confidence intervals for all effect measures were based on percentiles via 1000 bootstrap replications using practice as the bootstrapping unit to account for clustering.28

We carried out the analyses first among patients who had been outreached (including those engaged) and then among only patients who had been engaged, because the act of outreaching to patients might have an effect on clinical outcomes, as facilitation of patient involvement in care has been found to be related to general adherence patterns.29 For “outreach,” we used the first date of outreach as the start date; and for “engagement,” we used the date of the first care management encounter as the start date.

RESULTS

The five POs selected 52 affiliated primary care practices to participate in the PDCM pilot. One practice changed its PO affiliation within the first year and was excluded from our study. One PO with 20 small practices could only produce meaningful data for ten patients, and was also excluded from analyses. Figure 2 summarizes the flow of the number of patients/practices in the remaining 31 PDCM practices and the combined HPDCM comparison group. The largest drop in sample size in the PDCM group was due to the inability of POs to confirm a chronic disease identified by the plan. The largest drop in sample size in the HPDCM group was due to a number of patients who were attributed by the health insurer to a specific PO, but were not found in the POs’ EHRs. Among patients with at least one chronic disease documented in the EHRs, 87 % of the PDCM and 76 % of the HPDCM patients had some valid outcomes data for analyses. A total of 4,000 patients were clustered in 165 practices (31 in PDCM and 134 in HPDCM).

Figure 2.
figure 2

Flow diagram of study population. * Eligibility based on HPDCM’s criteria (identified by the health plan via administrative data as having at least one of five chronic diseases, and having coverage for HPDCM), and having been accepted by the PO into the pilot. POs had different selection criteria for outreaching to patients. One pilot PO was excluded due to lack of EHR data. ** Five chronic diseases were included in the pilot: congestive heart failure, chronic obstructive pulmonary disease, coronary heart disease, diabetes, and asthma. *** Some patients were found in PO’s EHR, but they did not have the primary outcome data for this study.

Table 2 summarizes the practice-level and patient-level characteristics in PDCM and the two types of comparison groups. Compared to the HPDCM group, the PDCM group had a higher number of PCPs, higher number of adult patients attributed to PCPs, fewer average years since medical school graduation among PCPs, and higher proportion of practices with PCMH designation in the year before the pilot intervention, likely due to the PO’s purposeful selection of high-performing practices to participate in the pilot. Generally, the PDCM patients were slightly younger, had higher number of chronic diseases, lower proportions of male patients, and higher proportions of patients with asthma and heart failure. In the following analyses, we adjusted for these significant differences.

Whether we used the first type of HPDCM group (n = 767) or the second type (n = 870) as comparison in the DID analyses, the results were similar. Thus, to improve efficiency, we combined the two groups together as one comparison group for the final results.

Among all outreached patients with at least one chronic disease (Table 3), the proportion of patients with LDL < 100 mg/dL increased by 3 % for the PDCM group (95 % CI: 1 % to 6 %) and 1 % for the HPDCM group (95 % CI: −2 % to 5 %). However, the difference in these improvements, i.e., the DID estimate of 2 %, was not statistically significant (95 % CI: −2 % to 6 %). Among all HPDCM patients, effects of engagement were generally larger than effects of outreach; in the PDCM group, however, no effect was statistically significant among engaged patients.

Table 3. Clinical Outcomes for Patients with at Least One Chronic Disease in PDCM and HPDCM Groups

Among outreached diabetic patients (Table 4), the proportion of PDCM patients with LDL < 100 mg/dL increased by 4 % (95 % CI: 1 % to 7 %). For the HPDCM, there was a significant improvement in the proportion of patients with optimal outcomes after outreach as compared to before (3 %, 95 % CI: 2 % to 6 %). There was a significant difference in the proportion of obesity between the two groups (4 %, 95 % CI: 0.3 % to 8 %) favoring HPDCM. Among engaged diabetic patients, the proportion of HPDCM patients with LDL < 100 mg/dL increased by 12 % (95 % CI: 0.1 % to 23 %). Again, effects among engaged patients were generally larger in the HPDCM group. However, the DID estimates were not statistically significant for any outcomes in engaged patients.

Table 4. Clinical Outcomes for Diabetic Patients in PDCM and HPDCM Groups

DISCUSSION

In this comparison of PDCM to HPDCM, we found that neither care management delivery mode had clear advantages over the other during the first year of follow-up, in terms of HbA1c, blood pressure, LDL, and weight loss. In fact, there was minimal evidence of effectiveness of either program for these outcomes. The minor improvement in cholesterol (three more patients out of 100 achieving LDL < 100 mg/dL) in the short term might not be significant at the population level, but it is consistent with, if not more significant than, the recent trend.30 Our lack of significant findings may be related to several factors. First, only about 30 % of the study population was older than 60 years, representing a somewhat younger, chronically ill adult population. The majority of the study patients might not have needed intensive care management, with about half of them already meeting SBP, DBP and LDL criteria at baseline. Second, improvements in clinical outcomes can take long periods of time, and 12 months post-care management may not have been a sufficient amount of time to detect measureable changes. In a systematic review, Norris and colleagues31 found many studies documented improvements within 12 months of the launch of care management, although longer studies did show larger effects. We did not have access to other intermediate outcomes, such as changes in patient behaviors, which may have improved despite no appreciable clinical improvement. Third, we relied on data from different EHRs and registries, which had varying degrees of data quality. Fourth, PDCM pilot practices appeared to have a slightly higher proportion of patients meeting recommended guidelines prior to the intervention than HPDCM practices. Thus, for some outcomes, PDCM patients had less room to improve than HPDCM patients. Fifth, we analyzed our data based on the intent-to-treat principle in clinical trials. In real world studies, because of diverse patient populations, not all patients were targeted for all outcomes. However, we had limited information on the specific contents and goals of care management for individual patients, which precluded the possibility for subgroup analyses based on specific targets. Sixth, we defined engagement as having at least one care management encounter. Since we did not measure the “dose” of care management received by patients, we could not carry out a sensitivity analysis for different definitions of engagement. Finally, these data were collected from specific POs in Michigan, limiting the generalizability to other settings.

We did not collect data on costs of running the programs; thus, a full cost-effectiveness analysis could not be carried out. However, even though there were not many significant differences in effectiveness between the two groups in the short term, there was a significant cost saving in healthcare expenditure, including out of pocket spending after 2.5 years among PDCM patients.23

An unexpected finding in the process of data collection was that among patients who met the health insurer’s eligibility criteria and were found in POs’ EHRs, quite a few patients (30 % for PDCM and 42 % for HPDCM) did not have a documented chronic disease in the EHRs.32 This suggested that many EHRs may not be ready for clinical research,33 or that the insurer's administrative data contained spurious diagnoses,34 or both. We also had to exclude patients who met the inclusion criteria and had a known start date for outreach, but did not have any clinical measures such as HbA1c or LDL. Compared to the patients included in our analyses, the excluded patients (n = 331 for PDCM and 517 for HPDCM) might be systematically different. Based on the health plan data alone, the excluded PDCM patients were slightly younger and had fewer chronic diseases than the included PDCM patients; and the excluded HPDCM patients had a lower prevalence of diabetes, but higher prevalence of asthma and COPD than the included HPDCM patients (data not shown). It is likely that, given the characteristics of the excluded patients, including them would not increase our chance of detecting significant effects. The discrepancy between health plan data versus practice EHR data tells a cautionary tale regarding studies using administrative data that are not research ready.

Practices in the pilot POs employed both objective criteria and subjective clinical judgment in selecting patients for outreach. Thus, the outreached patients in PDCM versus HPDCM were not as comparable as they would be in a randomized controlled trial. The DID approach we adopted relies on the assumption that the clinical outcome trends in the PDCM group would be the same as the trends in the HPDCM group, had the PDCM patients been exposed to the HPDCM intervention. This assumption might be violated if some unobserved causes led to different trends regardless of the intervention.

It would be ideal to compare our findings to similar studies in which a practice-based care management is compared to a health plan-delivered care management program. However, we were unable to locate any such published research. Existing literature typically finds positive clinical improvements comparing care management with usual care in carefully controlled settings,810,13 in contrast to this study’s real world implementation. Research on factors related to successful implementation of care management that leads to clinical improvement is limited. When care management is embedded into the primary care practice, it will produce greater participation and improved clinical results as compared to usual care.5,11 Our qualitative work35 provides evidence that the PDCM approaches were highly variable across POs, likely with variable success. When averaged across all PDCM practices, the effects of specific strategies might have been masked or dampened

From an implementation standpoint, the more embedded the care manager, the greater the success in engaging patients in working with the care manager. Although we did not quantitatively examine other factors, our qualitative work strongly leads us to suspect that other factors significantly impacted clinical outcomes within the 1-year time frame of this study.36 These other factors include the dose and effectiveness of the care management interventions by the care manager and the potential for improvement of the clinical targets and behavioral goals of those patients. Another significant factor is the degree of effective team functioning among practice members, especially noticeable when a care manager is embedded in the practice. Further exploration of these elements is definitely warranted to move the field forward.

In summary, our study suggests that while both health plan- and provider-delivered care management may demonstrate some minimal improvement in clinical values, when compared to each other using a DID approach, neither was found to be superior to the other. Since this comparison is unique in the literature, further research should explore potential differences in care management strategies that may account for program effect on specific outcomes.