Abstract
Background: We analyzed data from a cohort of 782 older patients assembled in 1999 to 2000 to determine whether their baseline assessments of the quality of their primary care measured using the Components of Primary Care Index (CPCI) were associated with subsequent changes in health-related quality of life and/or survival.
Methods: Longitudinal growth curve models were used to analyze changes in Quality of Well-Being scores over an average of 2.07 years. Cox proportional hazards models were used to identify variables associated with mortality over an average of 8.91 years (6,966 person-years). To reduce confounding by severity of illness, subjects were stratified into 3 groups based on disability and use rates. Within subgroups, we controlled for number of chronic illnesses and scores on the General Health subscale of the Medical Outcomes Study Short Form-36. We also controlled for baseline age, gender, marital status, income, body mass index, educational attainment, duration of the relationship with current primary care physician, and number of visits to the primary care physician in the year before enrollment. Analyses took into account clustering of patients within primary care physician.
Results: Neither total CPCI nor any CPCI subscale score was associated with Quality of Well-Being Self-administered Scale change over time or survival.
Conclusions: Assuming that effective primary care results in better health-related quality of life and longer survival and that the CPCI captures important primary care attributes, older patients' level of satisfaction with the quality of their primary care may not be a good surrogate measure of effectiveness.
Patients are a potential source of information about the quality of the primary care services they receive, and several instruments have been developed to obtain this information from them.1⇓–3 One of these, the Components of Primary Care Index (CPCI), was developed by Flocke for use in the Direct Observation of Primary Care study.4 The CPCI is a reliable and valid measure of the quality of primary care services as perceived by patients.1 Based on the 1994 definition of primary care proposed by the Institute of Medicine in 1994,5 it produces 8 subscale scores, Comprehensiveness, Accumulated Knowledge, Coordination, Preference for Regular Primary Care Physician (PCP), Interpersonal Communication, Advocacy, Family Context, and Community Context, and an estimated proportion of primary care visits made to the patient's usual PCP. The psychometric properties of the CPCI have been found to be similar in older as compared with younger patient populations (personal communication with Dr. Cheryl Aspy, author of an article describing these analyses recently submitted for publication).
Two of the most important goals of health care are to prevent premature death and to preserve or improve health-related quality of life (HRQoL). The availability of primary health care services is a predictor of a variety of positive health outcomes across countries and across states within the United States.3,6⇓–8 On the assumption that health care can be improved further through market forces, many recent efforts to control the costs of health care have included attempts to strengthen the ability of patients to choose their healthcare providers based on their assessments of quality.9,10
The primary purpose of this study was to determine whether patient assessments of the quality of their primary care, measured by CPCI subscale scores, were associated with better subsequent HRQoL and/or improved survival, that is, whether patients can accurately assess the effectiveness of the care. The study was conducted in a cohort of older patients assembled for this purpose in 1999 and followed to March 31, 2010.
Methods
Study Data
The data used in this study were obtained from the Oklahoma Longitudinal Assessment of the Health Outcomes of Mature Adults (OKLAHOMA) studies data set. Previous publications have described the methodology used to obtain this data in somewhat greater detail.11,12 The study was reviewed and approved by the Institutional Review Board of the University of Oklahoma Health Sciences Center, and all subjects consented to participate, first by telephone and then by signing a written consent form.
Between January 1, 1999, and December 31, 2000, 23 family physician members of the Oklahoma Physicians Resource/Research Network (OKPRN) created, from their billing records, lists of patients ≥65 years of age seen by them within the prior 18 months. Patients were then excluded if they had switched physicians, died, were in nursing homes, or were felt by this PCP to be too confused to sign consent. Eligible patients received a letter from their physician inviting them to participate. Two weeks later, the project coordinator followed up with these patients by telephone. Those who agreed to participate were asked to complete a questionnaire sent to them 2 weeks before their enrollment visit.
The questionnaire included questions about demographic information, health habits, symptoms, medical conditions, activities of daily living skills (ADL; 14-point scale), instrumental activities of daily living (IADL; 14-point scale), the Medical Outcomes Study Short Form–36 (SF-36), self-rated health (5-point Likert and 100-point rating scales), the Quality of Well-Being Self-administered Scale (QWB-SA), and the CPCI. Participants were also asked about the length of their relationship with their current primary care physician, and they were asked to estimate the number of visits to the PCP, other primary care providers, subspecialists, and emergency rooms and number of hospitalizations during the past year.
The questionnaire also included the CPCI instrument, which is composed of 43 questions from which one can calculate 8 subscale scores. The number of questions used to calculate each subscale score ranged from 2 to 9. Each question was scored from 1 (strongly disagree) to 6 (strongly agree). Scores were averaged across questions yielding subscale scores of from 0 to 6. A total score was calculated by summing across the 8 subscale scores. The assumption was that these individual subscales, when summed, reflected an individual's overall perception of primary care. Two questions were used to estimate the proportion of primary care visits made to the patient's usual PCP (0–1).
Two research nurses enrolled participants at their family physicians' offices at times scheduled specifically for this purpose. The nurses reviewed the study protocol, obtained informed consent, and checked the questionnaire for completeness. Each year on the anniversary of their initial enrollment, participants were invited to re-enroll. Those who agreed completed a follow-up questionnaire and were reconsented and briefly examined again by a research nurse. Numbers of participants enrolled in Years 1 to 4 were 848, 597, 401, and 319, respectively.
Participant deaths were determined at the end of the 5-year study using information provided by their designated contacts and PCPs and from the Social Security Death Index (http://ssdi.rootsweb.ancestry.com/). Subsequently, deaths have been tracked using the Social Security Death Index.
Participants who changed PCPs during the original 4-year study were excluded from these analyses. We also excluded those for whom we could not calculate ADL or IADL scores because these scores were used to control for severity of illness (see subsequently). If a participant did not answer the minimum number of items on a particular CPCI subscale, the score for that subscale was not calculated as per the scoring protocol developed by Flocke (Flocke, personal communication, 2008).
Controlling for Severity of Illness
To reduce the expected impact of severity of illness on potential associations between CPCI scores and outcomes, we used 2 strategies. First, we categorized participants a priori into clinically recognizable subgroups based on baseline ADL/IADL composite score <27/28 (disabled subgroup) and, for the rest, number of PCP visits in the year before enrollment (>4 = high users; ≤4 = low users). We then created a morbidity index by ascribing 1 point to the presence of each of the following chronic illnesses reported by participants at baseline and summing them: depression, diabetes mellitus, stroke, liver disease, Parkinson disease, autoimmune disease, lung disease, heart disease, or cancer. In addition, we controlled for patients' baseline General Health subscale score on the SF-36. Because of the wide range of possible numbers of physician visits within the disabled and high-using groups, we controlled for numbers of physician visits during the year before enrollment as well.
Health-Related Quality of Life
Descriptive statistics were calculated for baseline variables for the entire cohort and for each of the 3 severity of illness subgroups. A correlation matrix was constructed for independent variables and QWB-SA scores. Linear regression models were then created within each severity subgroup to further evaluate baseline variables associated with baseline QWB-SA scores. Linear regression was also used to evaluate associations between PCP and baseline QWB-SA scores. Hierarchical (repeated measures within participants within PCPs) longitudinal growth curve analyses were performed within each severity subgroup with QWB-SA scores in Years 2 to 4 regressed on promising independent variables found in the linear regression models. This analysis method examines associations between independent variables and change in the dependent variable over time. In SAS, this procedure is called “PROC MIXED.” Variables were removed from the models if they added little and were clinically unimportant. We then added each CPC1 subscale score to each of the models to assess their independent impact on change in QWB-SA scores over time.
Mortality
Chi square and independent t tests were used to analyze bivariate associations between baseline variables and death. Logistic regression models were created within each severity subgroup to further evaluate associations between baseline variables and mortality (in SAS, PROC GENMOD). Logistic regression was also used to assess strength of association between PCP and mortality. Hierarchical (participants within PCPs) Cox proportional hazards models (in SAS, PROC PHREG) were then created for the entire cohort and within each subgroup by considering promising independent predictors of mortality and then removing variables with little or no contribution to the models and no clinical relevance. Individual CPCI subscale scores were then entered into these models.
Statistical Software and Adjustments for Skewing of CPCI Scores and for Multiple Analyses
All analyses were performed using SAS 9.1, Version 5.1.2600. The degree of skewness for the total CPCI scores was −0.37 with subscale skewness ranging from −0.59 for Family to −1.00 for Coordination and Preference for Regular Doctor. Because the CPCI total and subscale scores were skewed to the left, in addition to the standard analyses, we used an exponential transformation (exp [x]) to achieve a more normal distribution. After exp transformation, the distribution of scores approached a normal distribution.
Because of multiple comparisons (total CPCI, 8 subscales, and proportion of visits to usual PCP; 3 severity of illness strata), we chose to consider associations with P values < 0.01 to be statistically significant.
Results
Study Population
Eight hundred fifty-four individuals completed the initial questionnaire. Sixty-six participants who changed PCPs during the 4 years of the study were excluded as were 6 participants who answered <50% of the items on the CPCI questionnaire or who could not be classified because of failure to complete the ADL or IADL scales. Patients who changed PCPs had lower total CPCI scores than those who did not (36.3 versus 38.2; P = 0.007). The final study population then included 782 individuals. Baseline characteristics of the study population are shown in Table 1.
Severity of Illness Subgroups
Table 2 shows the differences across the 3 “severity of illness” subgroups. Our a priori categorization rules achieved our goal, which was to define clinically recognizable subgroups with different levels of severity of illness, HRQoL, and mortality rates. Nearly all CPCI subscale means were higher in the middle group (high users). This appeared to be due primarily to an association between CPCI subscale scores and frequency of PCP visits (P < 0.0001).
Outcomes by Physician
Total CPCI scores by PCP ranged from 32.1 to 41.5. There was no association between average total CPCI scores by PCP and baseline QWB-SA, final QWB-SA, or mortality. In fact, the 2 PCPs whose patients had the highest rates of survival and whose HRQoL improved over time had average to low mean CPCI scores.
Associations between Baseline Variables and QWB-SA Scores
There were positive associations between higher baseline QWB-SA scores and younger age, male sex, being married, having had more education, higher income, higher body mass index, better function (ADL, IADL, all SF-36 subscales), better self-rated health, no hearing loss, absence of several chronic conditions (heart disease, stroke, diabetes, and hypertension), and fewer physician and outpatient visits in the previous year. In the linear regression model including all participants, higher baseline QWB-SA was associated with younger age, lower morbidity score, higher IADL score, higher self-rated health, and fewer visits to physicians in the previous year.
Linear regression models revealed no associations between baseline QWB-SA scores and any of the other baseline variables in the disabled subgroup. In the nondisabled groups, the only predictor of higher baseline QWB-SA scores was fewer total outpatient visits in the year before enrollment. In the high-user subgroup, higher baseline QWB-SA scores were also associated with higher self-rated health and Emotional Health subscale score (SF-36).
After stratification into the severity of illness subgroups, and after controlling for baseline age, sex, race, income, education, body mass index, marital status, general health, years with PCP, number of chronic illnesses, General Health, and number of primary care visits, neither CPCI total nor any of the CPCI subscale scores was associated with changes in QWB-SA scores over time in any of the subgroups based on hierarchical longitudinal growth curve analyses (Table 3). A higher proportion of visits with the participants' usual PCP was associated with more positive trends in QWB-SA scores in the healthiest subgroup (P = 0.02), but these associations did not reach our specified level of statistical significance. Analyses using transformed CPCI scores were not significantly different from those using untransformed scores.
Associations between Baseline Variables and Mortality
Table 1 displays the independent associations between baseline population characteristics and mortality. Two hundred sixty-four (34%) participants had died as of March 31, 2010. The data set then, with respect to mortality, includes up to a maximum of 11 calendar years and 6966 person-years of follow-up. Survival was associated with younger age, higher income, better function, higher body mass index, better self-rated health, and higher QWB-SA scores at baseline (Table 1). Logistic regression modeling of survival and explanatory variables within the disabled subgroup demonstrated age as the only predictive variable. Predictors of survival in the high-using group included female sex and younger age at baseline. Within the healthiest subgroup, survival was associated with younger age and communication scores in the CPCI instrument.
After controlling for baseline age, sex, race, income, education, body mass index, marital status, general health, years with PCP, morbidity score, and number of visits to the PCP in the year before enrollment, Cox proportional hazards models identified no associations whatsoever between CPCI scores and earlier mortality in any of the defined subgroups (Table 4). Analyses using transformed CPCI scores were not significantly different from those using untransformed scores.
Discussion
The concept of evidence-based medicine is predicated on the assumption that the goal of health care is to improve outcomes. Two of the more important health outcomes are enhanced quality of life and prolongation of survival. To the extent that effectiveness is defined in terms of those outcomes, our results suggest that older patients' ratings of their primary care on the CPCI questionnaire are not good measures of effectiveness of care. If this is true, then patient satisfaction scores should probably not be relied on as measures of clinical effectiveness, although they might still be regarded as subjective indicators of other aspects of quality. Of course, we can say nothing about the relationship of CPCI scores to other important outcomes such as enhanced personal growth and development, enhanced family stability, or a more comfortable dying process.
These results should not be too surprising. Patients often, very naturally, value immediate comfort more than future health, particularly the cost of future health. Primary care physicians in the private sector are under pressure to please patients because they are, after all, customers. The subject matter is complex, and so physician personality traits and confidence can be easily mistaken for clinical competence.
Studies of student ratings of teachers' performance are illuminating. Students have been found to be accurate judges of certain aspects of teaching including “how clear, interesting, respectful, and fair” a teacher is as well as how well the teacher was able to motivate them to learn the material. However, students are not able to provide reliable information on the quality of the course objectives, the content of the course, or the course assignments.13 Translating this to the clinical arena, patients are the best judges of their experience of care but can probably not be relied on to accurately assess the appropriateness of clinical evaluations and recommendations.
This study was limited in several ways. Patient satisfaction surveys are notorious for both ceiling (leftward skew) and halo (generally positive feelings toward PCPs raise all subscale scores) effects. We transformed the scores to address the skew, we looked at total CPCI scores as well as the subscales, and we controlled for duration of the patient–PCP relationship, but those were only partial remedies.
Severity of illness and its reflection in patients' need for and experience with certain primary care functions (eg, coordination) are difficult to measure. Although we used a 2-step process, first stratifying and then controlling for number of chronic conditions, similar but not identical to commonly used methods14,15 and the General Health subscale of the SF-36, we are not confident that we removed all the confounding caused by this factor. Weiner and colleagues16 have described a more comprehensive measure, but unfortunately, we did not have all the data required for this measure.
There was a high dropout rate over the 4 years of the study, resulting in a large number of missing values for HRQoL beyond baseline. This reduced our chance to find associations. The growth curve analysis used all the data that were available, however. The QWB-SA is fairly sensitive to small changes, but perhaps not sensitive enough to detect weak associations between CPCI scores and changes in HRQoL over short periods of time. We believe that we captured all or very nearly all deaths, so those calculations should not have been affected.
We chose to exclude individuals who changed doctors during the first 4 years of the study, reasoning that those patients would be less likely to have time to benefit from the care they were evaluating. A potential disadvantage of this decision is that poor rating of the functions of primary care might prompt patients to change physicians; perhaps this is further truncating the spread of scores that already seem to be skewed toward the positive rating. In fact, in a previous analysis of OKLAHOMA studies data, the accumulated knowledge, communication, and family orientation subscale scores did predict change in PCP.11
This is the first study to attempt to determine whether the CPCI can discriminate between better or worse primary care. Only a few other published studies have reported the results of analyses of associations between patient assessments of the quality of their primary care and outcomes. Safran,2 using an instrument similar to the CPCI, found, in a cross-sectional analysis of employed adults, that patient perceptions of their physicians' whole-person knowledge about them was associated with adherence to physician recommendations regarding behavioral risk factors.17 There were also small but statistically significant associations between trust, communication, thoroughness of examinations, physician's knowledge of the patient, and integration of care and patient-reported improvements in health status over the previous 4 years. In this study, the researchers used the physical and mental health subscale scores from the Medical Outcomes Study's SF-12, chronic medical diagnoses from a list of 21 conditions, and behavioral risk factors.
In a separate study using the same instrument, however, there was no consistent association between patients' assessment of primary care quality and clinician or practice performance on the HEDIS quality-of-care measures. The authors concluded that “clinical quality and patient experience are distinct but related domains that require separate measures and improvement initiatives.”18
Kerse and colleagues19 studied adults being cared for in primary care practices in New Zealand. Patients completed a waiting room survey about attributes of their relationship with their PCP before a visit. They were then interviewed by phone 4 days later and queried about adherence to medications prescribed at the index visit. They found that PCP–patient concordance was associated with subsequent adherence. Most recently, Bertakis and Azari20 found that patient-centered primary care encounters were associated with lower costs of care over the subsequent year. However, patient satisfaction was not associated with more objectively rated patient-centered care based on scored videotaped encounters.
The strengths of our study include its prospective cohort design and our 2-step process of controlling for severity of illness (stratification, then statistical adjustment within the strata). For the survival analyses, our average period of follow-up was reasonably long, and a substantial proportion (34%) had died. Although the average duration of follow-up for quality of life was relatively short, the participants were all over the age of 65 years, making short-term changes in quality of life more likely.
Conclusions
This study casts doubt on the ability of older patients to accurately judge the effectiveness of their primary care. It does not, however, mean that patient satisfaction is worthless. Obviously, we would prefer that patients live longer and better and feel positively about the care they are receiving.
Notes
This article was externally peer reviewed.
Funding: Provided by the Presbyterian Health Foundation and the University of Oklahoma Department of Family and Preventive Medicine.
Conflict of Interest: none declared.
- Received for publication December 21, 2010.
- Received for publication April 18, 2012.
- Revision received March 28, 2011.
- Revision received May 3, 2012.
- Accepted for publication April 13, 2011.
- Accepted for publication May 4, 2012.
References
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.