Improving Mammography Screening Using Best Practices and Practice Enhancement Assistants: An Oklahoma Physicians Resource/Research Network (OKPRN) Study

Methodology

During a 15-month period, we first used the best practices research methodology²² and then a multicomponent implementation intervention for breast cancer screening within the Oklahoma Physicians Resources/Research Network. Exemplars (those having mammography rates exceeding 80% of eligible patients) were identified using audit data from 48 Oklahoma Physicians Resources/Research Network clinicians from a concurrent unrelated project directed at colorectal cancer screening. An investigator (JWM) identified and queried 5 of these physician exemplars around the state and characterized their methods, identifying both general principles and specific techniques (see Table 1).

Once a set of principles and techniques was identified from interviews with exemplars and a literature review, 16 physicians (from 16 different practices) were solicited for participation in a randomized, controlled trial. Most practitioners were in solo practice (10 of 16). The others were in academic practice or in a group practice. In the intervention sites (4 solo, 2 academic, 2 group) the practice enhancement assistants (PEAs) worked with the enrolled practitioner and his/her assigned nurse to make selected changes that the practitioner felt would improve his/her ability to screen and refer women for mammograms. The clinicians in academic practice had lower volumes than the other clinicians because of fewer clinic days. There was no overlap between intervention and usual care practices because there were 16 clinicians from 16 different practices.

Physicians were randomly assigned to the intervention and usual care groups using an internet randomizer.²⁵ The 8 physicians assigned to the intervention group received (1) audit results and a comparison with the network benchmark (benchmark data were obtained through chart audits from another project); (2) academic detailing of exemplar principles and information from the medical literature; (3) services of a practice facilitator for 9 months; and (4) information technology support if requested. Practices were free to choose (or not) from the identified exemplar strategies or to modify them as necessary to fit the practice constraints of their individual settings. The PEAs were trained in the exemplar methods and were skilled in quality improvement techniques. The “Plan, Do, Study, Act” rapid cycle quality improvement process was used to implement incremental changes and make adjustments as required.²⁶ The PEAs spent at least 2 days per month at each practice and helped the practitioners design their interventions and facilitate the “Plan, Do, Study, Act” process. The role of the PEA was to provide information and feedback to guide the practice redesign activities. The remaining 8 practices comprised the usual care group and received no feedback or practice change facilitation.

Data Collection

A research assistant audited all practices at baseline and again after the 9-month intervention to determine the rates of mammography (defined as documentation of a mammogram within the past 2 years) before and after the intervention period. Data items for the chart audit included patient demographic characteristics, such as age, race/ethnicity, insurance type, education, marital status, number of visits in the past 12 months, whether or not a mammogram had been recommended, and whether mammography screening (binary; 1 = yes, 0 = no) had occurred. When documentation showed that a mammogram had been recommended within the time period of interest it was noted on the data form. However, these patients remained in the denominator. Patients for whom documentation indicated that they had refused a mammogram were removed from the denominator.

Eligible patients were women 50 years of age or older who had at least one visit to the practice in the 12 months before the study implementation and who did not have a diagnosis of breast cancer. A target sample size was 100 randomly selected charts per physician for each data collection period (or as many as were eligible in the case of physicians who did not practice full-time). For the audit after intervention, women who were 50 years of age or older who had seen the clinician during the last 6 months of the intervention period (or in the last 6 months of the 9-month period after the pre-audit for the usual care group) and who did not have a diagnosis of breast cancer were included. Lists of eligible patients were generated from billing records or electronic medical record systems and then systematically sampled to achieve the desired number of 100 charts. If review of the chart revealed that this patient belonged to another clinician, it was skipped and the next chart on the list was substituted. All eligible charts were abstracted if the available number was less than 100.

At the conclusion of the intervention, the PEAs summarized the methods used to improve mammography screening by each practice. In addition, the PEAs were asked to rate the practices' commitments to changing their office systems to improve delivery of preventive services. This information was used to provide more detail regarding the interventions that were implemented. Table 2 summarizes the exemplar methods used by each intervention practice.

Data Analysis

Descriptive statistics were calculated for both the pre and post chart audit data. Comparisons were made between groups for the baseline data to determine the degree to which the randomization process resulted in groups that were equivalent on the demographic characteristics. Given that the randomization was at the clinician level, comparisons of mammography rates were made at that level using t tests on the calculated difference scores between percentage rates before and after the intervention. Because of the clustering of patients within clinician practices, we also conducted a Rao-Scott²⁷ χ² test, which produces an adjusted χ² with 1 degree of freedom that accounts for the design effect caused by clustering, as an additional confirmation. We also calculated the interclass correlation coefficient. Availability of data for race/ethnicity, marital status, and education varied widely by provider resulting in some missing data (68% of usual care group, 55% of the intervention group for race/ethnicity; 40% of the usual care group and 25% of the intervention group for marital status) such that reasonable comparisons could not be made. Therefore, these items were not included in the analyses. There were 28 missing values for insurance codes, and 70 “other” insurance types that included “none” or “unknown type” or “Indian Health Service.” When comparing Medicaid and Medicare patients by intervention or usual care groups, there were 349 and 358 patients, respectively, in each group. For the analysis after the audit, insurance status was not a factor in mammography rates.

Protection of Human Subjects

The study was submitted to the University of Oklahoma Health Sciences Center Institutional Review Board and approved. All patient data were de-identified and all practices within the network had current HIPAA compliant Business Associate agreements with the University of Oklahoma Health Sciences Center that permitted research assistants to view their patient records. Participating physicians signed statements of informed consent.

Results

Table 3 contains the results of the comparison of the demographic before and after intervention and outcome variables by group. There was no difference in mean age or percentage of patients who were insured privately or by other means at either data collection period. Mammography rates also did not differ at baseline. Groups, however, did differ in the change rates for mammograms offered at the study's conclusion (P = .043). Thirty-six percent of eligible women in the usual care group were offered a mammogram and 35% of those eligible had documentation that a mammogram had been performed. Fifty-four percent of eligible patients in the intervention group were offered a mammogram and 52% of those eligible had documentation that the mammogram had been completed. The percent of patients with current mammograms at baseline was identical for both groups (40%). The t test comparing mean change scores resulted in t = 2.78, 14 degrees of freedom, and P = .015, suggesting that the intervention group increased their mammography rates more than the usual care group.

Differences between the intervention and usual care groups using the Rao-Scott χ² analysis to account for clustering also revealed a significant difference between the 2 groups (x_RS² = 17.49; degree of freedom = 1; P < .0001). The intraclass correlation coefficient was .114 based on 8 clusters per group with an average size of 75.8. Only 2 of the usual care practices showed an improvement, whereas 3 had rates that dropped across the study period. Mammography documentation for the intervention group ranged from a low of 21% to a high of 75%; however, 6 of the 8 practices improved their mammography rates, 2 remained the same, and none declined.

In all the practices that improved their rates, the physician was highly motivated (estimated by the PEA based on leadership and involvement in the intervention) to improve, and a system for screening and referral was put into place. That did not mean that the strategy required physician action. Most practices used strategies that empowered the medical assistant to check charts at the time a patient presented to the clinic and determine whether the patient was eligible for a mammogram, and if so to schedule it. One practice generated a list of women who did not have evidence of a current mammogram in the chart and sent each one a reminder that it was time to schedule their mammogram; this resulted in a 49% increase in their mammography rate. Another practice with a high rate of improvement used an information technology strategy that involved tracking all referrals and providing that information to the physician so that follow-up could be initiated when a referral appointment was not kept. This strategy resulted in a 56% increase.

Discussion

The purpose of this randomized controlled trial was to determine the impact of a multicomponent intervention on mammography rates, including a practice facilitator and “best practice” methods. The results suggest that this intervention can improve mammography rates in a range of practice settings. Given the low mammography rates in the state of Oklahoma, having proven strategies that increase the number of women who choose mammography screening could potentially reduce the burden of breast cancer within the state through early detection, when treatment is more likely to be efficacious. These findings are consistent with other studies that have tested multicomponent interventions that included modified academic detailing, patient education, physician reminders, audit and feedback, prompt and reminder systems;²⁸ physician education, and a practice facilitator to help redesign office routines including flow sheets; physician education, provider prompts, patient reminders, printed patient education materials, and transportations assistance.^20,29

It should be noted that the overall current mammography rates in both the intervention and usual care practices were lower than the reported overall state rate, which is based on telephone interviews with patients. One likely explanation is that women who receive gynecological care from a sub-specialist may also receive referrals from that specialist for mammography and the reports of results are returned to the referring physician rather than the family physician. Lack of documentation of this in the chart (even though possibly known to the physician) would be reflected by no offer of a referral and no current mammogram noted. Lack of documentation of referrals and patient refusals could also have contributed. It is likely that some patients selected for the chart audits were not continuity patients, in that they could qualify for the audit by age, sex, and having had only one visit during the relevant time period. The likelihood of a mammography referral in the case of a first visit prompted by an acute problem would be low. It is also possible that the time limit (eg, a qualifying visit could occur within days of the audit) precluded an opportunity for some patients to schedule and receive a mammogram.

Despite having access to the multicomponent translational intervention, 2 of the intervention practices did not improve their mammography rates. When practice characteristics including years in practice, age, type of practice, and patient characteristics (including age and insurance type) were compared, no significant differences were found between those who improved and those who did not. For one of the practices that did not change, the physician attributed the low rate of mammograms to the lack of initiative by the patients and felt that it was their responsibility to self-refer for mammography. Consistent with that attitude, only 21% of eligible patients in that practice were offered a mammogram. The other practice that did not improve, although an aggressive method was implemented, required the nurse to review each patient chart for current mammogram status and to signal the physician if a mammogram were needed. No change was observed in the overall rate at this practice, which was just under 60%. They also had an adherence to recommendation rate around 98% and the age of the patients did not differ from the overall mean, so it is not clear why the rate did not improve.

These practices were typical of many small practices where the office staffing consists of a receptionist, billing clerk, nursing or medical assistant, and a clinician. Of the 4 practices that improved the most (17 to 43 percentage points), 2 were in academic practices, one was in a group practice, and one was in solo practice. Given a desire to increase screening and referral for mammography, our study suggests that rates can be improved by selecting a process to identify those patients who need a mammogram and making a referral (or appointment for the mammogram) at that time. The consistency among successful improvers was the office-wide commitment to and participation in the process, once again reinforcing the power of a goal in outcome achievement.

The study is limited by the small number of practices and the possibility that those selected to participate may not be representative of other practices in the research network. In addition, given the awareness of the intervention sites that their performance would be observed by the PEA and analyzed, the Hawthorne effect may have contributed to the outcome. It is also possible that having only one person interview the exemplars resulted in the exclusion of a strategy from the best practices suggestions that might have been even more effective.

Despite these limitations, this study found that using a multicomponent translational intervention consisting of academic detailing, audit feedback with benchmarking, and practice facilitation resulted in significant improvement in mammography rates. Although baseline motivation to improve is a factor, it should be noted that each of the intervention components was designed to motivate the practices toward incremental system changes that would ultimately improve their rates of performance.