Abstract
Background: Patient experiences are important components of the patient-centered medical home. Competing demands of primary care and resident education contribute to inefficiencies at teaching sites, which may contribute to poor patient experiences. Educational commitments of residencies may also produce positive experiences. The objective of this study was to compare patient experiences in teaching versus nonteaching sites.
Methods: Patients across 6 primary care sites (3 teaching and 3 nonteaching) completed surveys. Patient experiences assessed using (1) Consumer Assessment of Health care Providers and Systems (CAHPS) across 3 domains: access to care, communication, clerk/receptionist courtesy (scores range from 1 [worst] to 6 [best]) and (2) Patient Assessment of Chronic Illness Care, measuring chronic care experiences (scores range from 1 [worst] to 5 [best]).
Results: Approximately 70% of participants were female and >40% were Latino. The adjusted mean score for patient-reported access at teaching sites was 4.35; at nonteaching sites it was 5.14 (P = .01). The total mean score for chronic disease was 4.02 for teaching sites and 3.79 for nonteaching sites (P = .01). Four of the 5 Patient Assessment of Chronic Illness Care subscale scores were better at teaching sites.
Conclusions: Worse access scores among teaching sites may reflect the complexities of residencies. Better chronic care scores are encouraging and potentially indicate focused training. Residencies might continue chronic disease training while considering efforts to improve access.
- Health Services Research
- Medical Home
- Medical Residency
- Patient-Centered Care
- Patient Satisfaction
- Quality of Health Care
Primary care has proven to be beneficial for people and societies and serves as the foundation for better overall health care.1⇓–3 The patient-centered medical home (PCMH) has become a critical part of efforts to improve primary care and is meant to serve as an advanced model for improved quality, value, and patient centeredness.4 PCMHs may be especially useful in poor, medically underserved neighborhoods with large disease burden, where medical care is often fragmented. These communities may benefit most from a strengthened primary care–based health care system.
The Bronx, New York, is home to underserved, predominately minority communities with a low educational level and poor health statistics.5 The proportion of residents living below the poverty level is 31% in the Bronx, compared with 21% in New York City (NYC) overall, and the overall proportion of college graduates in the Bronx is substantially lower than that in NYC (14% vs 27%, respectively). It is discouraging to note that only 26% of residents in the Bronx report having a regular doctor, and many Bronx residents use emergency departments for sick visits or health advice.5 More than ever, PCMH efforts to improve patient care have the potential for a positive impact in communities such as the Bronx.
The notion of patient centeredness is a core component of the PCMH and refers to a more complete understanding of a patient's life and a focus on individual social/cultural factors when caring for the patient.6,7 Patient centeredness often is assessed by evaluating patient-reported experiences in health care.7 Patient experiences may depend in part on the primary care facility at which care is received. Because of the competing demands of education and patient care, it often has been noted that busy teaching sites in particular have limited access8,9 and long wait times8,10,11 and often run inefficiently8,11,12 because of a high volume of residents in training, their own competing priorities,12,13 and the overall complexity of the teaching environment, which may be particularly true in the poor, underserved communities of the Bronx. However, residency programs are also centers of expertise.14,15 Specifically because of their commitment to education and a focus on training,14 teaching sites may also offer certain advantages. Despite the challenges of academic medicine, several primary care residencies around the nation have focused teaching efforts, in particular related to evidence-based chronic care management.13
This study sought to examine patient-reported experiences of provider communication, access to care, clerk/receptionist courtesy, and chronic disease management at 6 primary care sites in the Bronx. We also sought to estimate the differences in patient experiences by teaching versus nonteaching primary care sites.
Methods
Study Design, Participants, and Setting
As part of PCMH evaluation efforts at Montefiore Medical Center/Albert Einstein College of Medicine, cross-sectional data were collected from 6 primary care facilities in the Bronx, New York, from June 2011 to April 2012. Three of the sites are teaching facilities and 3 are nonteaching facilities. For the purpose of this study, teaching sites are defined as facilities that have residents and fellows who are routinely precepted by clinical faculty. Nonteaching sites do not have residents/fellows but include attending physician providers and/or nurse practitioners/physician assistants. The 6 primary care sites were selected from among 23 sites in the main hospital network. Two of the sites were selected by hospital clinical leaders to introduce the PCMH. All sites were selected to broadly represent teaching and nonteaching practices; large, medium-sized, and small practices; and both family medicine and medicine/pediatrics staffed sites. No statistics were used in the selection process. All teaching sites in the network are medium-sized or large, and all provide care to larger proportions of patients insured by Medicaid or uninsured patients compared with the nonteaching sites. The sample was limited to English- or Spanish-speaking adult patients ≥18 years old who presented for a scheduled appointment and who had at least 2 visits to the facility during the preceding year. Patients who had at least 2 visits were considered to be established, continuity patients. Recruitment yielded a total sample of 1752 participants across the 6 primary care sites. This study was approved by the institutional review board of the Montefiore Medical Center/Albert Einstein College of Medicine.
Data Collection and Measurement
Participants were randomly sampled Monday through Friday during regular office hours using the last digit of their medical record numbers. Potential participants were approached in facility waiting areas by trained, bilingual research assistants (RAs). RAs recruited and obtained consent from participants after describing the study and ascertaining eligibility. Data were collected via self-administered surveys in either English or Spanish, depending on participant preference; surveys were completed by participants in the facility waiting rooms. RAs were available on site to answer questions or assist participants with completion of the survey if needed (eg, participants who had low literacy or were visually impaired). We surveyed approximately 300 participants per site, except for one site at which we surveyed approximately 100 participants because the total number of patients attending that site is proportionally smaller compared with the other sites. The total response rate was approximately 60%. The most common reasons for refusal to participate were “not interested” and “not enough time.”
The survey assessed participants' report of the patient centeredness of their health care by measuring patient experiences. Subscales were specifically drawn from the Agency for Health care Research and Quality Consumer Assessment of Health care Providers and Systems (CAHPS) Clinician and Group survey16 and the Patient Assessment of Chronic Illness Care (PACIC), developed by Glasgow et al,17 both well-established and validated (in English and Spanish) instruments.
Measures
Sociodemographic measures include age, sex, education level, race, ethnicity (Hispanic/Latino or not), language spoken (English/Spanish), and insurance payer (straight Medicaid, Medicaid managed care, private insurance, self-pay, Medicare).
Overall Health Status
Participants rated their overall health by answering the question, “In general, how would you rate your overall health?” Responses range from excellent (1) to poor (5).16
Patient Experiences
Items were drawn from the CAHPS Clinician and Group survey, a validated tool that assesses patients'/consumers' experiences with health care.16,18,19 We used CAHPS subscales to measure perceived patient experiences regarding (1) access to care; (2) provider communication; and (3) courtesy of clerks and receptionists. These specific item sets were chosen because they are believed to be the most relevant and applicable to the patient population of interest. The access to care subscale includes 5 questions about the feasibility of making timely appointments, readily getting medical questions answered, and wait times. The provider communication subscale asks 6 questions about how well providers communicate with, listen to, and interact with patients. Courtesy of clerks/receptionists assesses patients' perceptions of the helpfulness, courtesy, and respectfulness of facility clerks and receptionists (2 items). Each item was scored on a 6-point scale ranging from 1 (never) to 6 (always). The higher the score, the better and more positive the patient experience.
Participants were given a list of the most common chronic conditions and asked to indicate whether they had been told by a doctor that they had any of the listed conditions. The list of chronic conditions is as follows: hypertension/high blood pressure, angina/coronary artery disease, congestive heart failure, diabetes, asthma/emphysema/chronic obstructive pulmonary disease, rheumatoid arthritis, osteoarthritis/degenerative joint disease, cancer (any type other than skin), depression, acid reflux/stomach ulcers, and migraine headaches. For the subset of participants who reported an ongoing chronic illness, we also assessed their experiences regarding chronic disease care with the PACIC developed by Glasgow et al.17 The PACIC is a validated measure of a patient's perception about the management of his or her chronic disease.17,20,21 PACIC subscales measure perceived patient experiences regarding (1) patient activation; (2) delivery system and decision support; (3) goal setting; (4) problem solving and contextual counseling; and (5) follow-up coordination. Patient activation assesses the extent to which patients feel able to take responsibility for their care.17,22 The patient activation subscale asks patients about whether they feel that their input is solicited when making health care decisions. For example, one item states, “Over the past 12 months, when I received care, I was asked for my ideas when we made a treatment plan.” Delivery system design/decision support refers to actions that organize care and provide information that improves patients' understanding of their care.17 The goal setting subscale assesses whether patients feel they receive enough information that allows them to set specific collaborative goals about their chronic disease.17 Problem solving/contextual counseling refers to patients' perceptions about whether their social and cultural environment is considered when formulating treatment plans.17 For example, one item states, “Over the past 12 months, when I received care, I was sure that my doctor or nurse thought about my values, beliefs, and traditions when they recommended treatments to me.” Finally, the follow-up/coordination subscale considers reinforcement of ambulatory treatment through contact with patients to assess progress and coordinate care.17 Each item is scored on a 5-point scale ranging from 1 (none of the time) to 5 (always). Scores for individual items in a subscale were tallied and averaged to deduce the total subscale score; however, subscale scores were not calculated when >20% of the items lacked a response. A total PACIC score also was calculated by tallying all items in each of the 5 subscales and deriving the average. Again, total PACIC scores were not calculated when >20% of the items lacked a response. The higher the score, the better and more positive the patient experience. The possible range for each PACIC subscale was 1 to 5; the actual range for each subscale was also 1 to 5.
Statistical Analysis
Stata 10 statistical software (StataCorp, College Station, TX) was used for analysis. We began by reviewing descriptive statistics for all participants then broken out by teaching and nonteaching site. We then reviewed univariate distributions of subscale questions by site. For age, initial data were collected for 6 different age categories ranging from 18 to 74 years of age. During the exploratory phase, it was noted that participants in the younger and older age categories tended to report similar scores on the CAHPS and PACIC subscales. Therefore, for the purposes of analysis the age categories were combined into 3 separate categories: 18 to 34 years; 35 to 54 years, and 55 to 74 years.
For the access to care subscale, missing data were handled as defined in the CAHPS instructions and account for intentionally omitted responses that follow specific skip patterns.16 We did not impute missing data for sociodemographic variables (age, sex, race/ethnicity, survey language, education, and insurance payer). In multivariate analyses, cases lacking information on any involved variable were excluded.23 Percentages of missing data for each sociodemographic variable are listed in Table 1.
For all the other subscales, missing data were managed by using ipsative mean imputation in which values for missing items within a subscale were imputed by calculating the mean of the responses to those items for which there was a response.23 Imputation was done only for subscales for which at least 80% of the items had responses. Cronbach α was calculated for each of the CAHPS and PACIC subscales (as well as for the total PACIC score) and ranged from 0.83 to 0.96.
After handling missing data, mean scores for each subscale were compared by teaching versus nonteaching site via bivariate analyses using t tests. Mixed-effects linear regression models were subsequently built, adjusting for possible confounders, with a random effect at the site level to account for clustering of respondents within sites. Age, sex, education, and overall health status were included in the regression model because they are variables typically adjusted for in most analyses of CAHPS data.16 Race/ethnicity, survey language, and insurance payer also were chosen for inclusion in the model because they were deemed important and relevant to the unique and diverse patient population of the Bronx and have been found to be important contributors to patient experiences.24
The normality and homoscedasticity of the error distributions of the models were confirmed graphically. In addition, we verified that results were not materially changed by reanalysis using Huber-White robust standard errors. Last, sensitivity analyses were run to assess whether the effects of teaching versus nonteaching sites were appreciably altered. They were not (difference changed by <5%).
We defined statistical significance as P ≤ .05. Because the percentage of missing data varied for the sociodemographic variables included in the multiple linear regression models, sample sizes necessarily decreased when adjusting for possible confounders (range of observations lost, 91–226). The percentage of missing data for the sociodemographic variables (except race) was about the same for teaching and nonteaching sites. For race, nonteaching sites had more missing data compared with teaching sites (20.5% vs 14%, respectively). Numbers of observations were lower overall for the PACIC subscales because these questions were answered only by patients who reported a chronic illness.
Results
Sample Characteristics
There were 975 participants from teaching sites and 777 participants from nonteaching sites. Recall that the sample size for each site was approximately 300, except for one nonteaching site, for which the sample size was smaller (approximately 100) because of that site's smaller population. Participants ranged in age from 18 to 74 years. Most participants were 55 to 74 years old (∼40%); however, an only slightly lower percentage (36%) ranged in age from 35 to 54 years. Approximately 25% of the population was 18 to 34 years old, which is not unexpected given the decreased propensity of younger populations to seek medical care. Approximately 70% of all participants were female. These results are shown in Table 1.
More than 40% of participants identified as Latino, with a higher percentage of Latinos attending teaching sites versus nonteaching sites. Of the sample, 40% identified as African American and 10% as white, whereas 1.5% identified as Asian, 1% as Native Hawaiian/Pacific Islander/Native American/Alaskan Native, and 2% as multiracial. A larger number of participants reported their race as being “other” (26%), but more than half of these also identified as Latino. About 20% of all participants reported having less than a high school education, more at teaching sites compared with nonteaching sites. Thirteen percent of the sample chose to take the survey in Spanish instead of English, and most of these participants were patients at teaching sites.
Overall Health Status
For teaching sites, the mean score for overall health status was 2.97; for nonteaching sites, the mean score was 2.85. Recall that responses range from excellent (1) to poor (5). Overall health status is outlined in Table 2.
Patient Experiences per CAHPS
Details of patient experiences per the CAHPS are shown in Table 3. For teaching sites, the adjusted mean score for access to care (adjusting for age, sex, education, overall health status, race/ethnicity, survey language, and insurance payer) was 4.35 (range, 1–6), which was significantly lower (worse) than the mean score (5.14) for nonteaching sites (P = .01). For provider communication, there were no differences in the adjusted mean values between teaching (5.60) and nonteaching sites (5.52). There were also no differences in adjusted mean values for clerk/receptionist courtesy between teaching and nonteaching sites (5.20 and 5.52, respectively). Crude means for all the CAHPS subscales were similar to the adjusted means.
Patient Experiences per PACIC
Details of patient experiences per the PACIC also are shown in Table 3. For the subset of patients with chronic illness, the adjusted mean total PACIC score was significantly higher (better) for teaching compared with nonteaching sites (4.02 and 3.79, respectively; P = .01). For 4 of the 5 subscales of the PACIC, adjusted mean scores were higher for patients at teaching sites compared with those at nonteaching sites: patient activation (4.26 vs 3.18; P < .001), goal setting (3.85 vs 3.65; P = .051), problem solving (4.36 vs 4.06; P = .01), and follow-up/coordination (3.59 vs 3.36; P = .03). Delivery system mean scores were not different between teaching and nonteaching sites (4.39 vs 4.34, respectively; P = .60). Again, crude means for all the PACIC subscales as well as the total PACIC score were similar to the adjusted means.
Of note, within the regression models, participants in the oldest age category (55–74 years) reported significantly better scores for all CAHPS subscales and all but one PACIC subscale (patient activation) compared with the youngest age category. In addition, participants who rated their own overall health more favorable were also significantly more likely to report better scores for all CAHPS and PACIC subscales. Given that both age and overall health rating were consistently significant in all regression models for the CAHPS and PACIC subscales, an interaction term was added to the models; however, the interaction did not achieve statistical significance for any of the dependent variables.
Discussion
To our knowledge, this study is among the first to compare results of patient-reported health care experiences by teaching versus nonteaching sites. The results of data collection reveal that patient-reported access to care scores are significantly worse among patients at teaching sites compared with those at nonteaching sites in this urban network. These results hold true when controlling for several important demographic factors including age, sex, education, overall health status, race/ethnicity, language spoken, and insurance payer. Not only are these variables relevant and diverse among the Bronx patient population, but previous work also noted that several of these variables are important to consider when assessing patient reports of their experiences.24⇓–26
The significantly lower scores for access to care among patients at teaching sites may reflect the known difficulties that often are associated with busy, sometimes inefficient, residency practices. Residents often have competing priorities and busy schedules that can limit time available for ambulatory care, depending on their other rotations and program years. Hectic schedules may result in long patient wait times and difficulties making appointments with resident providers. In addition, long, busy, or thorough precepting sessions may also contribute to longer wait times for patients, often resulting in increased patient aggravation. Of note, a recent study with a comparable patient population found similar mean scores for access to care, clerk/receptionist courtesy, and communication as those reported by patients at our nonteaching sites in the Bronx.24 Another study with a relatively less ethnically diverse and older patient population found moderately higher scores for access and communication; however, clerk/receptionist courtesy scores were similar to those reported in this study.27
In a more positive light, patient-reported experiences of chronic care management were significantly better among patients at teaching sites. Compared with previous studies of patients with chronic disease in the United States and abroad, the overall adjusted PACIC scores were better for both teaching and nonteaching sites in this study.28⇓⇓–31 For the individual subscales of the PACIC, findings vary by specific chronic disease; however, scores are still notably better for patients at teaching and nonteaching sites in this study.29⇓–31 In addition, we recognize that while the differences between teaching and nonteaching sites for the overall PACIC and 3 subscales are statistically significant, the differences are numerically small when considering a scale from 1 to 5 and thus may not be clinically relevant. The statistical significance may be in part a reflection of the moderately large sample size. However, better PACIC scores among patients at teaching sites might also be a result of recent efforts among primary care residency programs to improve training in chronic care management.15,32 The results of this study can provide motivation not only for further studies but also, and more important, for a continued focus on chronic disease management training, especially since primary care residents continue to report inadequate training in chronic disease management.33
The fact that overall health status and age were important determinants of patient-reported experiences for both CAHPS and PACIC items is not surprising. It follows that how a patients perceives their general health might affect how they then perceives health care experiences. In terms of age, the finding that the oldest patients (55–74 years old) also tend to report the most favorable experiences might be explained by older patients who have spent longer amounts of time (compared with younger patients) attending a given primary care site and having grown accustomed to the appointment accessibility, facility staff, and chronic disease management style of the site providers.
Strengths and Limitations
One of the main strengths of this study is that it is among the first to compare experiences of patients at primary care teaching sites and nonteaching sites. These results will not only serve as baseline data for longitudinal analyses but also will provide a basis of comparison for other primary care residency programs. Another strength of this study is its adequate sample size. We set out to achieve a sample size of 267 participants from 5 of the 6 sites and approximately 100 participants for the sixth site. We achieved our projected sample size at all 6 sites. Our relatively high response rate was achieved in part because we recruited participants in facility waiting rooms as opposed to relying on mailed surveys or telephone interviews, which are likely to result in much lower response rates (particularly among low-income minority patients34) compared with our study's response rate.35 Finally, the sample of patients in this study is representative of the overall population of residents in the areas of the Bronx where the primary care facilities are located.5
The main limitation of the study relates to missing data. In particular for some of the demographic variables (race, insurance payer, and education), >10% of data were missing (Table 1). Upon review of the data, we did not find any particular patterns of nonresponse that would suggest systematic omission of certain questions. Nonetheless, there is still the possibility that there are omitted structural or organizational variables (such as differences in physician compensation and/or physical facilities/amenities) in our analysis that might actually account for some or all the differences we see between the teaching and nonteaching sites. While such variables could not be addressed in this study, they should be considered in future research. However, previous research has excluded differential item functioning for CAHPS items.18 The study is also limited because we have little information about nonresponders (those who refused to participate in the study) other than their stated reasons for refusal. We have no demographic, health, or utilization data for these individuals. Therefore, it is not possible to truly know whether nonresponders are systematically different from responders in these respects. Last, while this study included a relatively large sample size of patients, including only 6 primary care sites is a significant limitation that restricts both the internal validity and external generalizability of our results.
Conclusion
Among 6 primary care facilities in the Bronx, we found that patient experiences differ between teaching and nonteaching sites. When adjusting for various sociodemographic variables, patients at teaching sites reported worse access to care per CAHPS scores. However, patients at teaching sites also reported significantly better chronic disease management scores on the PACIC scale. Given the increasing prevalence of chronic illness, these results are encouraging and can propel residency programs to begin/continue to comprehensively train residents in chronic disease management. However, a directed focus on assessing and improving access to care is likely also warranted. Patient centeredness is a crucial element of the medical home. The application and process of the PCMH seeks to improve primary care and, by extension, the overall health of communities. In an effort to improve PCMH efforts, busy primary care teaching facilities with ongoing PCMH implementation might consider shifting their efforts toward assessment and improvement of patient access to care.
Acknowledgments
We gratefully acknowledge support from Montefiore Medical Center and the cooperation of health center staff and patients.
Notes
This article was externally peer reviewed.
Funding: DNC's time was supported by a New York State Empire Clinical Research Investigator Program award (to AEB) during her time as a research fellow at Albert Einstein College of Medicine in the Department of Family and Social Medicine.
Conflict of interest: none declared.
- Received for publication July 31, 2013.
- Revision received October 31, 2013.
- Accepted for publication November 27, 2013.
