Abstract
Purpose: We examined the effect of admission for myocardial infarction, heart failure, or pneumonia during the first academic quarter compared with all other quarters in teaching versus nonteaching hospitals on length of stay, cost, and mortality.
Methods: Using data 2011 Nationwide Inpatient Sample, multivariable modeling with an interaction term was used to test teaching hospital effect by academic quarter. Logistic regression was used for mortality and log-transformed linear models for cost and length of stay.
Results: Charlson Index scores were similar in teaching and nonteaching hospitals. Patients admitted to teaching hospitals for myocardial infarction in the first quarter had a higher risk-adjusted mortality (1.217; confidence interval, 1.147–1.290) than those admitted to a nonteaching hospital during the same quarter (0.849; confidence interval, 0.815–0.885). Mean cost heart failure admissions averaged $584 more, and the mean length of stay was longer (0.10; P = .0127), during the first academic quarter. These effects were not present for quarters 2 through 4.
Conclusions: This study suggests small increases in mortality among patients admitted with myocardial infarction in the first academic quarter compared with all other quarters in teaching versus nonteaching hospitals. Increased cost and longer stay were seen for those admitted with heart failure.
- Heart Failure; Hospitalization; Hospitals
- Teaching; Length of Stay; Linear Models; Logistic Models; Myocardial Infarction; Pneumonia
The “July effect” is a phenomenon in academic medicine that refers to the influx of new trainees and the subsequent presumed effects on the quality of patient care and health outcomes. Within academic medical centers in the United States, July represents the time of year when teaching hospitals undergo a cohort turnover, with the graduation or advancement of experienced residents and their replacement with inexperienced medical school graduates. The plausible consequences of such a mass transition have been the focus of research over the past 2 decades, but studies have failed to reveal a conclusive relationship.
Much of the prior research has focused on surgical outcomes, including complication rates and in-hospital mortality. Most studies investigating cardiac surgery did not demonstrate differences in risk-adjusted morbidity or mortality in teaching versus nonteaching hospitals based on period of the academic year; however, lower mortality rates were seen, despite higher complication rates, in the first academic quarter (ie, July, August, and September) among patients undergoing coronary artery bypass grafting.1⇓–3 Additional studies found longer surgical times, and longer episodes of cardiac ischemia, during the early part of the academic year compared with the later part.4,5 No differences in overall postoperative complication or in-hospital mortality rates were noted in teaching versus nonteaching hospitals based on operative month among spinal surgeries, major cancer surgeries (including head and neck surgery), or pediatric neurosurgeries.6⇓⇓–9 Furthermore, no differences in length of stay (LOS) or hospital charges were noted in fields including pediatric neurosurgery or head and neck cancers.8,9 A review of Medicare recipients undergoing 1 of 7 different procedures also failed to identify an increased mortality rate in teaching hospitals in July compared with all other months.10 However, a study focused on repair of hip fractures revealed a 12% increased relative risk of mortality in the first 2 months of the academic year in teaching hospitals compared with nonteaching hospitals.11
Few studies have focused solely on medical admissions. A study comparing mortality and LOS among patients admitted to an intensive care unit found no evidence of the “July effect” by academic quarter in teaching versus nonteaching hospitals, with similar results for a stratified analysis of surgical and nonsurgical patients.12 However, a study of patients with preselected discharge diagnoses admitted to hospitals in the Minneapolis/St Paul area showed a decline in charges for patients with internal medicine diagnoses who were admitted to teaching hospitals over the academic year, but an increase in LOS and charges for those admitted with surgical diagnoses.13
The largest observational study to date examined all patient admissions, including for both medical and surgical diagnoses, over a 5-year period from the Nationwide Inpatient Sample (NIS) database and investigated trends in LOS and risk-adjusted mortality rates in teaching versus nonteaching hospitals. That study noted longer hospitalizations and higher mortality rates in the first quarter of the academic year in teaching hospitals compared with all other quarters in teaching versus nonteaching hospitals, but it did not control for case mix index.14 A 2011 meta-analysis revealed worsening mortality and efficiency of care in July but noted several methodological limitations and marked study heterogeneity that limited conclusions from the study.15 In addition, many published studies are limited by small sample sizes or an inability to use nonteaching hospitals as control sites. Yet the systematic review pointed out that the studies with higher-quality designs and larger samples were the ones that more often showed increased mortality and decreased efficiency during the residents transition.16⇓⇓⇓⇓⇓⇓⇓⇓–25
Whether the academic changeover phase is truly associated with lower quality of care and/or an increase in adverse events remains unknown. Many previous studies were methodologically limited by inadequate adjustments for differences in case mix or seasonal trends. Inadequate adjustment for patient mix may be alleviated by limiting admissions to a subset of patients with medical conditions that may be expected to be sensitive to variations in the quality of care delivered. To minimize effects of case mix heterogeneity, we examined the effect of hospital admission for acute myocardial infarction (AMI), heart failure (HF), or pneumonia (PNA) in the first quarter of the academic year compared with all other quarters in teaching versus nonteaching hospitals.
Methods
Data Source
We obtained data for this study from the 2011 NIS, the largest publicly available all-payer inpatient health care database in the United States, developed as part of the Health Care Cost and Utilization Project by the Agency for Healthcare Research and Quality. The database contains data from approximately 8 million hospital stays from roughly 1000 hospitals, sampled to approximate a 20% stratified sample of US community hospitals, and is both internally and externally validated on a yearly basis to ensure consistency and accuracy. Participating hospitals were defined by the American Hospital Association to be “all non-Federal, short-term, general, and other specialty hospitals, excluding hospitals of institutions.” Included are both public hospitals and academic medical centers. Short-term rehabilitation hospitals, long-term nonacute care hospitals, psychiatric hospitals, and alcoholism/chemical dependency treatment facilities are excluded from the database.
Patients
All patients with a primary diagnosis code for AMI, HF, or PNA, as defined by the Clinical Classification Software groups specified in the DxCCS variable, were included in the sample. These diagnoses were specifically chosen because they are considered to be sensitive to the quality of care delivered, as evidenced by their inclusion in the Centers for Medicare and Medicaid Services Readmissions Reduction Program. Because patients transferred from other hospitals often have more severe disease and poorer outcomes,26⇓–28 patient transfers were excluded from the analysis. Patients were stratified by academic admission quarter: quarter 1 (Q1; July to September) versus quarters 2 to 4 (Q2 to Q4: September through June). Patient comorbidities were assessed using the Charlson comorbidity score.29 Patient race (African American, Hispanic, or other) and insurance type (private, Medicare, Medicaid, uninsured) were included as possible confounders.
Outcomes
We examine 3 outcomes, established a priori, that may be affected by the July effect and that were available for all admissions: 2 process measures (LOS and total cost) and 1 outcome measure (mortality).
Statistical Analysis
Multivariable modeling with SAS version 9.4 (SAS Institute, Inc., Cary NC) was used to test the study hypotheses. Models were adjusted for sex, age, race, admission type (AMI, HF, or PNA), Charlson comorbidity score, hospital location (rural vs urban), and patients' insurance status, and was adjusted for effects of correlation within each hospital. Since a seasonal variation in outcomes has been noted regardless of hospital teaching status, we compared outcomes for teaching and nonteaching hospitals by quarter and based our hypothesis testing on the presence of a significant interaction effect between teaching status and academic season. We used logistic regression for dichotomous outcomes and log-transformed linear models for cost and LOS because of their distributional characteristics. All analyses were conducted to test the null hypothesis that outcomes (LOS, total cost, and mortality) are not significantly different in the first academic quarter in teaching hospitals compared with all other quarters in teaching versus nonteaching hospitals. Differences between teaching and nonteaching hospitals were measured with a binary variable (1 = teaching; 0 = nonteaching). The effect of quarter was tested as an interaction between being admitted to a teaching hospital and admission quarter, with statistical significance defined as P < .05 for the interaction term.
Results
The 2011 NIS database included information for 544,617 patients with a primary diagnosis code of AMI, HF, or PNA; 42% of those were admitted to teaching hospitals (Table 1). Average patient age was 67.2 years. Because of the large sample size, we observed a significant difference in all categories, although not all differences were considered to be clinically significant. When comparing patients cared for at nonteaching and teaching hospitals, patients admitted to nonteaching hospitals were more often white (70% vs 57%), whereas teaching hospitals had higher rates of patients representing other races. Teaching and nonteaching hospitals had similar proportions of patients with comorbid conditions, as measured by the Charlson comorbidity score. Teaching hospitals had a higher proportion of patients with Medicaid (12% vs 8%) and private insurance (20% vs 17%), where nonteaching hospitals care for slightly more Medicare patients (68% vs 61%).
Baseline Characteristics of Patients Admitted to Teaching Hospitals for Acute Myocardial Infarction, Heart Failure, or Pneumonia, from the 2011 Nationwide Inpatient Sample
Patients admitted to teaching hospitals during the first academic quarter had a higher risk-adjusted mortality (1.076; 95% confidence interval [CI], 1.041–1.11) compared with those admitted to a nonteaching hospital during the same quarter (0.938; 95% CI, 0.918–0.959) (Table 2). This effect was not present for quarters 2, 3, and 4. When stratified by primary diagnosis code, only mortality for patients admitted with AMI remained significant (1.217; 95% CI, 1.147–1.290). Adjusted mean cost per admission was noted to be an average of $373 more in the first academic quarter in teaching hospitals compared with all other quarters. Stratification by primary diagnosis code revealed a $584 difference in HF admissions, whereas admissions for AMI and PNA were no longer significant (Table 3). Adjusted mean LOS was longer for all admissions combined (0.01; P < .0001) during the first academic quarter, and for only HF admissions (0.10; P = .0127) when stratified by primary diagnosis code (Table 4).
Adjusted Odds Ratio for Mortality in First Academic Quarter Compared with All Other Quarters in Teaching Versus Non–Teaching Hospitals, from the 2011 Nationwide Inpatient Sample
Adjusted Mean Cost per Admission by Quarters in Teaching Hospitals, from the 2011 Nationwide Inpatient Sample
Adjusted Mean Length of Stay per Admission by Quarters in Teaching Hospitals, from the 2011 Nationwide Inpatient Sample
Discussion
The “July effect” is a theory in academic medicine referring to the potential decline in quality of care and patient outcomes associated with the matriculation of new resident trainees. While most previous studies have focused on surgical diagnoses, this is the first study that we are aware of to examine a subset of medical diagnoses using a large sample of nationally representative hospitals, using nonteaching hospitals as a control, and controlling for patient characteristics and seasonal variation. Our findings suggests an increased risk of mortality in the first academic quarter in teaching hospitals, with patients admitted with a diagnosis code for AMI being most vulnerable to the July effect. This effect is not simply an “academic hospital” effect, because our design controlled for an overall difference between teaching and nonteaching hospitals, as well as for seasonal trend. We observed a similar effect on cost and LOS. A significantly higher mean cost and LOS per admission occurred in teaching hospitals in the first academic quarter compared with all other quarters. Many of these effects with respect to cost and LOS seemed to be driven by patients with a primary diagnosis of HF. While the cost difference for AMI was not statistically significant, this may be the result of a combination of a smaller effect size and smaller sample size. Further assessment is needed to determine whether patients with a primary diagnosis of PNA are insensitive to the July effect or whether a possible effect is obscured by seasonal variation in PNA cases.
Previous studies have shown conflicting results, likely as a result of multiple factors. Those that focused on a single site likely reflected patient demographics and practice patterns specific to the area, whereas those using nationwide data could show a more broad representation. Results also vary depending on specialty and subspecialty, as clinical care and resident supervision undoubtedly differ between specialties. Finally, some studies did not use nonteaching hospitals as a control group and therefore cannot separate seasonal variation or other confounding factors from an increased incidence of poor outcome.
Strengths of this study include the use of the NIS, the largest all-payer national inpatient database, to provide an analysis on a nationwide level, avoiding regional practice pattern biases that may have been present in smaller, local studies. The NIS also provides patient and hospital information that can be used to control for potential confounders that may affect patient outcomes, such as differences in comorbid conditions. While our study is similar to the 2005 economics study by Huckman and Barro,14 we were able to better control for case mix by limiting our analysis to 3 common diagnoses.
Limitations of this study are those typically associated with cross-sectional analysis of archival data. This was a 1-year, cross-sectional analysis that was limited to 3 diagnoses. The inclusion of additional years or diagnoses may yield different results. Coding inaccuracies have been reported in the NIS but are more likely to be systematic errors and not be specific to month of admission or hospital teaching status.30 It seems that patient severity, as measured by Charlson comorbidity scores, was similar between teaching and nonteaching hospitals; however, patients admitted to teaching hospitals may have been more likely to have other clinical circumstances not reflected in the Charlson score or otherwise in the NIS that may have contributed to increased mortality, longer LOS, and higher costs. However, this effect is expected to be present for all academic quarters, so its impact on our results should be minimal. Teaching hospital status was defined by the American Hospital Association's Annual Survey of Hospitals and does not reflect the extent of resident participation in clinical care. Finally, while this study supports the notion of a July effect in medical admissions, the effect is small and only observable because of the very large sample size in this study; individual hospitals may not have measurable differences.
Conclusion
This study suggests an increased risk of death, longer LOS, and increased costs for patients with the diagnoses of AMI, HF, or PNA admitted to teaching hospitals during the first academic quarter compared with all other quarters in teaching versus nonteaching hospitals.
Notes
This article was externally peer reviewed.
Funding: none.
Conflict of interest: none declared.
To see this article online, please go to: http://jabfm.org/content/30/2/189.full.
- Received for publication June 30, 2016.
- Revision received October 10, 2016.
- Accepted for publication October 17, 2016.
