Abstract
Background: Primary care physicians often fail to diagnose low bone density. This pilot study assessed 2 interventions for their effect on bone mineral density testing.
Methods: Five practices in the Iowa Research Network were randomized: 2 to chart reminder alone (CR), 2 to chart reminder plus mailed patient education (CR+PtEd), and one to usual care. A total of 204 women aged 65 years or older were recruited from within these practices. Bayesian hierarchical analyses were used instead of traditional statistical methods to take advantage of collateral data and to adjust for differences between clinics at baseline.
Results: After the intervention, the rates of completed bone mineral density testing were 45.2% in the CR+PtEd group, 31.4% in the chart remainder only group, and 9.7% in the usual care practice. Bayesian analysis adjusted for patient and clinic characteristics, which made use of collateral data, gave an odds ratio of 5.47 for the effect of CR+PtEd group. The Bayesian P was .029 and the one-sided 95% credible interval for the odds ratio was greater than 1.2. The effect of CR+PtEd was confirmed by sensitivity analyses. Traditional hierarchical analysis adjusted for practice characteristics could not be used to estimate statistical significance because there were not enough clinics to accommodate a model that included all the important covariables.
Conclusions: Specific chart reminders to physicians combined with mailed patient education substantially increased the levels of bone density testing and could potentially be used to improve osteoporosis screening in primary care. Bayesian hierarchical analysis makes it possible to assess practice-level interventions when few practices are randomized.
Low bone density or osteoporosis is common and affects nearly half of postmenopausal women attending primary care practices who have had no previous diagnosis of osteoporosis.1 These conditions lead to substantial morbidity and mortality, including fractures, disability, and death.2 Bone mineral density (BMD) screening is the only way to reliably assesses low bone density.3–5 The National Osteoporosis Foundation guidelines,6 which are endorsed by 9 specialty societies and the US Preventive Services Task Force, recommend BMD testing for all women aged 65 and older. However, nearly one-half to two-thirds of women attending primary care practices have not received BMD testing,1,7,8 although rates are improving.9 It is estimated that Medicare could save millions of dollars with increased BMD testing.10
There have been relatively few randomized studies testing strategies to improve BMD testing in primary care, and these studies have had mixed results.11–14 The present study tested 2 reminder interventions delivered within 5 private family medicine office practices. The first was a chart reminder to perform BMD testing on all women older than 65 years. This method was chosen because there is good evidence that physician reminders generally result in more patients receiving preventive services.15,16 The other approach was to combine the chart reminder with information mailed directly to patients about the recommendation that all women aged 65 and older receive osteoporosis screening. Providing patients with information has been suggested as an approach to “activate the patient”17,18 and was also found to be effective by Stone et al's15 meta-analysis, with adjusted odds ratios of 1.29 to 1.53 for interventions targeting immunizations and other screening services. It is similar to the highly successful direct-to-consumer approach currently used by the pharmaceutical industry for drug advertising.19,20
We conducted this pilot study to test whether a chart reminder with or without a patient education intervention could improve rates of BMD testing among women 65 years of age and older attending community family medicine practices. Results were analyzed and compared using conventional regression analyses, hierarchical methods, and Bayesian methods.
Methods
In the spring of 2001, physicians from Iowa family physician practices who had expressed an interest in being a part of the Iowa Research Network (IRENE) were telephoned to ascertain their potential interest in participating in a “women's health study.” The IRENE is a primary care research network consisting of approximately 277 primary care clinicians in 150 practices throughout the state of Iowa. IRENE's mission is to create new knowledge with relevance to rural primary care clinicians and their patients, with the outcome of improving the care of patients. The physicians were told that the study would be explained during a seminar presented at their office. Based on the available budget, it was only possible to recruit 6 of the practices that had expressed an interest in the study and were sufficiently close to Iowa City for the research team to travel to the offices to abstract medical records.
A study orientation seminar was arranged at a time convenient to the practice. During the spring and early summer of 2001, the primary investigator (PI) (B.T.L.) or the lead research assistant (S.S.) visited each practice. A 30-minute presentation was made covering postmenopausal health issues and medications. General plans for the research project were discussed, including the idea that practices would be randomized to an intervention or usual care group for an “important women's health issue.” Physicians were not told what this health issue was. Practices were also advised to designate member of the staff to identify and mail a preprepared packet containing information about the study to the practice's female patients aged 65 years and older who were scheduled for an annual examination visit.
Physicians provided their informed consent to participate. Practices were not provided with any compensation for participation, but printing, stationery, and mailing costs were covered by the project.
Interventions
At the time when practices were randomized, one practice decided there were insufficient personnel available to place the chart reminder on the chart. Therefore, this practice was eliminated from the randomization process. Using a random number generator, the remaining 5 practices were randomized to one of 3 groups: 2 practices to physician chart reminder alone, 2 practices to physician chart reminder plus patient education, and 1 practice to usual care.
The chart reminder was a sticky note following National Osteoporosis Foundation guidelines that practices could place on the charts wherever they thought it would be most effective (Figure 1). To simplify operations, office personnel were to place these reminders on the charts of all women aged 65 years and older who were scheduled to come in for annual exams, not just those women in the study. Most practices chose to place the reminder on the left side of the record near the lists of medical problem(s), medications, immunizations, and other preventive health screenings. None of the practices used electronic medical records.
Osteoporosis screening chart reminder.
Mailed patient education packets included the 2-sided card from the National Osteoporosis Foundation titled “Osteoporosis: Can It Happen to You?” with basic information about osteoporosis and a list of questions ascertaining risk factors. Attached to the card was a brief information sheet from the PI that explained the preventable and treatable nature of osteoporosis, that the National Osteoporosis Foundation recommends BMD testing for all women 65 years of age and older, that testing is covered by Medicare, and that requested that women talk with their family physician about having a test. The mailings were accompanied by a cover letter signed by the physicians in the practice.
Patients
Female patients aged 65 years and older scheduled for upcoming annual examination visits were identified by an office coordinator. They were mailed packets containing a cover letter describing the study; the informed consent document; and a 6-page survey that asked about demographics, health habits, reproductive history, medication use, medical history, and previous health screening. Patients were not provided with any compensation, and only one recruitment mailing was done. All participating patients gave their informed consent. The study and methods were approved by the University of Iowa Institutional Review Board.
Patient Follow-up Survey
Two to 3 weeks after the annual examination visit, a 2-page follow-up survey was sent to women asking about their perceived health, whether they brought up any concerns about specific health issues with their physician, and whether their physician discussed the value of a BMD test. Information from the surveys and chart reviews was verified and double-entered into a database.
Chart Reviews
Structured chart reviews of consenting patients were conducted at each practice by a study team consisting of the PI and several research assistants trained by the PI in medical record review. The median time between the intervention and the chart review was 6.7 months because the practices were located across the state of Iowa. Information collected included birth date; date of the index visit (the annual examination visit); number of annual examination visits to the practice during the preceding 26 months; height; weight; current use of estrogen, osteoporosis medications, and calcium; specific medical conditions; and dates and results of all bone density tests. The chart review form was reviewed for completeness by a second reviewer. Mailings were conducted from 2001 to 2002 and chart reviews were conducted from 2001 to 2003.
Analytic Methods
The primary assessment of the association between intervention and BMD testing was done with a Bayesian hierarchical analysis. The main outcome was the binary variable, BMD testing (yes or no) in the interval between the annual examination visit and the time of the chart review. The analysis was performed both with and without adjusting for patient-level and practice-level covariates to assess whether confounding could substantially influence the results. The covariates considered for inclusion in the analysis were patient-level variables from chart reviews and follow-up surveys, including previous BMD testing for each patient and the BMD testing rate for the practice during the 14 months before intervention. The covariates tested in the final model were those that were significant in a stepwise logistic regression analysis at the P < .10 level.
Results from the Bayesian hierarchical analysis were compared with usual hierarchical methods and logistic regression analysis. The usual hierarchical methods also allow researchers to control for patient-level and practice-level characteristics. Logistic regression analysis was done at the patient level; the binary variable BMD testing was the outcome and intervention and covariables were used as independent variables. The assumption in logistic regression analysis is that practices would have similar underlying rates of BMD testing if they had similar previous rates and similar patients.
Bayesian Analysis
In contrast to the frequentist approach, the Bayesian approach estimates the probability that the treatment effect is within a given range. From the Bayesian analysis we obtained the probability that the intervention did not increase the rates of BMD testing. This probability is similar to how one-sided P is often interpreted. Bayesian analysis can take advantage of collateral information obtained outside of the study to estimate the intrinsic variation among the practices in the rates of osteoporosis screening. The intrinsic variation depends only on the unmeasured patient or practice characteristics. It does not depend on within-clinic random variation, on variation caused by measured practice or patient characteristics, or on the intervention. There were 2 types of collateral information about this variation: (1) the 14-month rates of BMD screening in 15 practices not included in the present study,21 and (2) clinical judgment. Our clinical judgment was that the 95% confidence interval for the intrinsic practice rates was no wider than from 8% to 40%, which translated into a standard deviation for clinic rates of testing (σe) of 0.5. A sensitivity analysis was performed to test how the Bayesian results depended on clinical judgment.
We compared results of logistic regression, frequentist hierarchical analysis (SAS GLIMIX version 9.1, SAS Institute, Inc., Cary, NC), and Bayesian hierarchical analysis with and without collateral data (WinBUGS version 1.4, Cambridge, United Kingdom). The Bayesian analyses assumed vague, independent priors on all parameters.
Results
We received 204 baseline patient surveys out of the 578 distributed (35% response rate), 195 medical records reviewed, and 193 follow-up surveys received (95%). The patient consent rate varied from 27% to 47% by practice, with no difference in the overall response rates in the intervention versus usual care practices. The average age of the women was 74.0 years, body mass index was 27.3 kg/m2, and their estimated total daily calcium intake from dairy and supplements was 1508 g; 62.8% of the women were married, 61.3% had a total family income less than $35,000, 63.4% had high school education or less, 49.0% took aspirin, and 69.1% stated they had previously had a BMD test.
Table 1 shows summarized information from the medical record reviews. There were no differences in mean age, mean number of medical conditions, or past rates of BMD testing among women in the 3 groups. The usual care group had significantly lower educational status and mean number of past annual examination visits during the 26 months before chart review.
Patient Characteristics in the Three Osteoporosis Screening Study Groups
For both intervention groups, the rate of BMD testing after the interventions was higher than the rate during the 14 months preceding the intervention: 31.4% versus 22.6% for the chart reminder (CR) alone (P = .2164 using the McNemar χ2 test), and 45.2% versus 22.6% for the CR+PtEd group (P = .0133 using the McNemar χ2 test). The rate of testing remained constant in the usual care group: 9.7% for both intervals.
On the follow-up survey there was significant variation in the percentage of women who asked their physician about a BMD test (32.4% of the CR group, 39.3% of the CR+PtEd group, and 13.3% of the usual care group; P < .05) and who discussed BMD tests with their physician (45.1% of the CR group, 59.0% of the CR+PtEd group, and 16.7% of the usual care group; P < .001). Discussion of issues not targeted by the intervention, such as heart disease, cancer, stroke and depression, did not significantly differ by group.
Table 2 shows the results of testing intervention effectiveness using each of 4 analytic approaches after taking into account patient- and practice-level covariables. These characteristics include regularity of menses before menopause, total medications, self-reported history of osteoporosis, and prior BMD testing. Odds ratios (ORs) for CR and CR+PtEd groups were similar for all 4 analyses. The estimated ORs for CR (OR, 2.27) and CR+PtEd (OR, 4.99) were identical for the logistic regression and frequentist hierarchical analysis with covariables. P values could not be computed with the frequentist hierarchical analyses because there were not enough clinics to accommodate a model that included all the important covariables. The hierarchical Bayesian analysis with collateral data showed that the estimated OR was 2.37 (P = .156) for the CR group and 5.47 (P = .029; one-sided 95% credible interval for the odds ratio was greater than 1.2) for the CR+PtEd group. When collateral data were not used in the hierarchical Bayesian analysis there was less certainty in the estimates of the variation in the clinic effects and, consequently, less statistical significance for the intervention effects.
Osteoporosis Screening Intervention Effectiveness According to Four Types of Analyses
The Bayesian analysis was rerun assuming the variation in practice effects, σe2, was known (Table 3). The Bayesian equivalent to P, assuming σe was known, depended on the exact value for σe. The clinical assumptions made about the intrinsic variation of the practice effects was equivalent to a maximum value of σe = 0.5. P equivalents for the effect of CR+PtEd were .033 if σe = 0.1, .038 if σe = 0.3, and .076 if σe = 0.5.
Sensitivity Analyses to Between-Clinic Standard Deviation, σe
Discussion
We tested whether a physician CR alone or a combined intervention of a CR plus mailed patient education materials increased the likelihood that women would have a BMD test. Screening rates after the intervention were 9.7% in the usual care group, 31.4% in the CR group, and 45.2% in the combined CR+PtEd group. We found that the combined intervention of CR+PtEd encouraging BMD testing significantly increased the rates of BMD testing compared with usual care (OR, 5.47; Bayesian equivalent of one-side P = .029), but we were not able to detect a significant effect of the physician CR alone.
We found 2 other cluster, randomized intervention studies that targeted both patients and physicians to improve BMD testing in primary care.11,12 These studies were able to make use of electronic clinical information, did not obtain informed consent, and were significantly larger than ours. Both used hierarchical methods for analysis but not a Bayesian approach. In the study by Lafata et al,11 15 primary care clinics with 10,354 women were randomized by clinic to usual care, mailed reminders to patients only, or mailed reminders with physician prompts. BMD testing rates after intervention were 10.8% in the usual care group, 21.4% in the group who were mailed patient reminders, and 28.9% in the mailed patient reminder plus physician prompt group.11 Each intervention increased BMD testing rates compared with usual care.11 There was a significant interaction between age and BMD testing, such that the effect was more pronounced at higher ages.11 The other study found no effect of mailed patient education tested in a 2 × 2 fashion with academic detailing to physicians.12 In that study, 828 physicians with 13,455 Medicare patients were randomized to usual care, academic detailing alone, mailed patient education, or academic detailing plus mailed patient education.12 No difference between groups was found for the main outcome of a composite endpoint of BMD testing or initiation of a medication for osteoporosis.12 Other studies that have attempted to increase BMD testing rates have used substantially different types of interventions such as patient focus groups14 or various formats for BMD reports.13
Our results were similar to Lafata et al11 in that we found a strong effect for a combined intervention of mailed patient education combined with physician chart reminder for osteoporosis screening. One possible explanation for the differences between our results and those of Solomon et al12 is that our letters to patients were personalized and were from the practice physicians, and the educational materials accompanying the letters included clear information that BMD testing is recommended for all women aged 65 years and older. Letters from a physician's office may be more trusted by patients than letters from another source. Unlike Solomon et al's study,12 which did not test chart reminders, both our study and that of Lafata et al11 used a physician chart reminder that was available at the time the patient was seen by the physician. Our chart reminder was especially simple to implement, ie, it went on the charts of all women aged 65 years and older and recommended that the patient should be tested for BMD.
Much research has examined what types of interventions work best to improve preventive services. A meta-analysis of physician prompts showed rate differences of 5.8% for Papanicolaou smears to 18.3% for immunizations.16 Indeed, the effectiveness of physician reminders is so powerful that Balas et al16 argue that the use of control groups raises ethical questions. The Veterans Administration has shown remarkable adherence to patient care guidelines with the use of electronic physician reminders.22 A meta-analysis of interventions to increase preventive services showed that organizational change gave the largest effect, with provider reminders and patient education also showing significant effects.15 Combining 2 or more effective strategies worked better than single ones.15 With the increasing emphasis on improving the delivery of preventive and other health services, systems integrating components of the chronic care model have been recommended.23
Limitations
This was a small, cluster randomized trial conducted in a family physician practice-based research network with limited funding. Because the practices and patients in this study were self-selected, the generalizability of our findings are unknown; the intervention's effectiveness may be less in other settings. Patients who agreed to be in the study may be more amenable to receiving education and testing for osteoporosis, even though at the time of consent they were not told that the study focused on osteoporosis screening. It is somewhat surprising that we were able to demonstrate an effect of the combined intervention given the high rate of osteoporosis awareness in these practices as judged by the relatively high rates of previous BMD testing.
The fact that the usual care group had a lower rate of previous screening than the intervention practices is not something that could have been known before the intervention. Accessing this data requires informed consent, it is costly, and none of these practices had electronic records, from which this type of information might be able to be obtained in aggregate. The range of osteoporosis screening values during the 14 months before the intervention in this study was 9.7% for the usual care office and 22.6% for the intervention offices; this was not significantly different from the average rate of BMD testing during a 14-month period among women aged 65 years and older in 15 clinics which did not participate in this study, which was 16.5% (SD, 15.5%).
The medical literature has numerous examples of practice variation. Thus, it is very important that any study of practice-based interventions take into account baseline rates for outcomes at which an intervention is directed. Thus, in conducting our Bayesian analysis we controlled for previous BMD testing at both the patient and the practice level, and we used collateral data from 15 offices that did not participate in this particular study (they participated in a cross-sectional study of colon cancer screening).21 Including those women with a previous BMD test should make it more difficult to see an effect of the intervention, yet the rates of BMD testing increased dramatically and were highly significant for the CR+PtEd group and did not increase at all for the usual care group. We are not aware of clinic factors other than the intervention that could have accounted for the increased rates of BMD testing. All of our results were consistent in showing a very similar intervention effect for chart reminders to physicians combined with patient education.
Statistical Methods
In this study it was necessary to adjust for covariables because the analysis was influenced by a practice with a low baseline testing rate that was randomized to the usual care group, and these patients may be less “health conscious” than those patients randomized to the intervention groups, as judged by fewer annual exams. With the available data, and because we wanted to control for patient and practice characteristics, it was not possible to perform a usual hierarchical analysis because of a sample size inadequate for estimating all study parameters. For this reason, the primary analysis was a Bayesian hierarchical analysis that made use of collateral data. This analysis showed a strong and statistically significant effect for the CR+PtEd group (OR, 5.47; Bayesian equivalent of one side P = .029), which was confirmed by sensitivity analyses.
The best solution for the analytic problems in this study is to increase the sample size, but this would be costly. With limited federal funding for research to improve clinical practice,24 it is necessary to use the most efficient methods possible to evaluate interventions. Bayesian methods are a promising approach for improving the efficiency of trials that randomize patients at a practice level. They allow for consideration of covariables as well as the use of collateral data, and they are the only type of analysis that can answer the key clinical question, “What is the probability that the intervention is effective?” Although interpreting this type of analysis requires judgment, there are 2 reasons that the comfort level of persons interpreting these analyses may increase. First, the sensitivity analyses used to explore the influence of assumptions on results is now used in a number of settings.25–27 Second, the greater availability of electronic databases could result in the availability of better collateral data. Thus, Bayesian methods should be an increasingly acceptable tool and the need for large numbers of clinics to participate in some trials may be reduced.
Conclusion
We found that the combination of relatively simple interventions—ie, mailing patient education information and placing chart reminders directly on patients’ charts—could potentially be used to improve osteoporosis screening among women aged 65 years and older. The intervention strategies, with planned physical examinations, could easily be applied to a number of preventive health screening issues and other medical problems. The Bayesian analysis used for this research is broadly applicable to the analysis of interventions among patients attending primary care practices where it is important to account for the clustering of patients within practices.28,29
Acknowledgments
We gratefully acknowledge the support of the offices of the participating IRENE physicians.
Notes
This article was externally peer reviewed.
Funding: Financial support was provided by the American Academy of Family Physicians Foundation through an Advanced Research Training Grant awarded to Barcey Levy and the Department of Family Medicine, University of Iowa Roy J. and Lucille A. Carver College of Medicine, Iowa City.
Previous presentations: Portions of the data presented in this study were presented at the North American Primary Care Research Group 31st Annual Meeting, Banff, Alberta, Canada (October 25–28, 2003); the 4th Annual Convocation of the American Academy of Family Physicians Research Network, Kansas City, MO (March 18–21, 2004); the American Academy of Family Physicians Research Network 7th Annual Convocation of Practices, Virginia Beach, VA (March 1–4, 2007); and the 2007 Agency for Health care Research and Quality National Practice Based Research Conference, Bethesda, MD (May 16–18, 2007).
Conflict of interest: none declared.
See Related Commentary on Page 352.
- Received for publication April 10, 2008.
- Revision received August 20, 2008.
- Accepted for publication August 25, 2008.
References
In this issue
Jump to section
Related Articles
Cited By...
- Improving bone mineral density screening in patients with inflammatory bowel disease: a quality improvement report
- Practice-based Research Networks (PBRNs) Are Promising Laboratories for Conducting Dissemination and Implementation Research
- Re: Interventions to Improving Osteoporosis Screening: An Iowa Research Network (IRENE) Study
- Response: Re: Interventions to Improving Osteoporosis Screening: an Iowa Research Network (IRENE) Study
- Response: Re: Interventions to Improving Osteoporosis Screening: an Iowa Research Network (IRENE) Study
- Re: Interventions to Improving Osteoporosis Screening: An Iowa Research Network (IRENE) Study
- Practice-based Research is Community Engagement
- Fourth Journal of the American Board of Family Medicine Practice-based Research Theme Issue