Abstract
Introduction: The aim of the Rural Older Adult Memory (ROAM) pilot study was to evaluate the feasibility of screening and diagnosing dementia in patients aged 75 years or older in 6 rural primary care practices in a practice-based research network.
Methods: Clinicians and medical assistants were trained in dementia screening using the ROAM protocol via distance learning methods. Medical assistants screened patients aged 75 years of age and older. For patients who screened positive, the clinician was alerted to the need for a dementia work-up. Outcomes included change in the proportion of patients who were screened and diagnosed with dementia or mild cognitive impairment, clinician confidence in diagnosing and managing dementia, and response to the intervention.
Results: Results included a substantial increase in screening for dementia, a modest increase in the proportion of patients who were diagnosed with dementia or mild cognitive impairment, and improved clinician confidence in diagnosing dementia. Although clinicians and medical assistants found the ROAM protocol easy to implement, there was substantial variability in adherence to the protocol among the 6 practices.
Conclusion: This study demonstrated the complex issues that must be addressed in implementing a dementia screening process in rural primary care. Further study is needed to develop effective strategies for overcoming the factors that impeded the full uptake of the protocol, including the logistic challenges in implementing practice change and clinicians’ attitudes toward dementia screening and diagnosis.
Dementia is common among older adults, with an estimated prevalence of 11% to 16% among people older than 70.1 Often, a primary care clinician (PCP) is the only clinician available to older patients with memory complaints, but numerous studies have found that as many as 50% of patients with dementia do not have a diagnosis of dementia documented in their medical chart.2–5 The subtlety of symptoms and time constraints in primary care practice make it challenging for PCPs to recognize and diagnose dementia.6,7
Although the US Preventive Services Task Force has not recommended screening for dementia,8 Alzheimer disease experts have identified a number of reasons why diagnosis is important, including ruling out treatable conditions that can cause cognitive impairment (eg, medication effects, cardiovascular conditions, or depression)9; offering treatments for cognitive and behavioral symptoms of dementia10; maintaining patients’ safety11; and support for the family.12 Given clinicians’ reliance on patients’ memories for symptom reporting and adherence to treatment recommendations, the identification of cognitive impairment in patients is essential.13–15
Several studies to improve the diagnosis of dementia conducted in academic and/or urban settings have had varying degrees of success.3,16,17 This is the first known study to test a dementia screen and diagnosis intervention in rural primary care. Improving dementia care is particularly challenging in rural areas where access to community resources, including medical specialists, are limited and primary care workloads are greater than in urban areas.18,19
Practice-based research networks offer a promising approach to improving primary care because clinicians and their staff are directly engaged in testing practice changes.20,21 Among other resources, practice-based research networks can provide direct assistance to practices engaged in the quality improvement studies through a “change facilitator.”22
The Rural Older Adult Memory (ROAM) study was conducted in the Oregon Rural Practice-based Research Network (ORPRN), a statewide network of 46 primary care practices in 35 communities that serve approximately 223,000 patients. Our aim was to evaluate the feasibility of the ROAM study for dementia screening and diagnosis in 6 rural practices that were members of ORPRN. We adapted the materials and procedures for the dementia-specific component of the Assessing Care of the Vulnerable Elders (ACOVE) model developed by geriatric experts at the University of California—Los Angeles and Rand Corp.17 ACOVE uses 5 clinic-based methods to improve practice: (1) efficient collection of condition-specific clinical data, (2) medical record prompts to encourage performance of essential care processes, (3) patient education materials and activation of the patient's role in follow-up, (4) physician decision support, and (5) physician education. Preliminary tests of this model found it to be feasible in urban and suburban practices but, to our knowledge, it has not been tested in rural practices. Furthermore, the ACOVE model has had limited testing of its dementia component as a single-condition intervention.
Methods
Study Setting
The Layton Aging and Alzheimer's Disease Center at Oregon Health and Science University worked in collaboration with ORPRN to implement the intervention. Eighteen clinicians and 26 medical assistants in 6 rural practices in western Oregon participated in the intervention. Using a convenience sample, we recruited practices of varying size, with the proportion of elderly patients in their patient profiles ranging from 6% to 25%, a mix of electronic medical records versus paper medical charts, and both private and publicly funded practices. Institutional review board approval was obtained and all study clinicians and medical assistants signed informed consent.
To facilitate implementation of the intervention, we engaged ORPRN`s Practice Enhancement and Research Coordinator (PERC). The PERC served as a liaison between the practices and the research team, assisted practices when difficulties with implementing the intervention arose, and kept an eye on adherence to the protocol and the project timeline.
Intervention
The intervention consisted of a memory screen (see Appendix) that was administered by a medical assistant and, for patients who screened positive for possible dementia, a memory evaluation was administered by the clinician. Patients were screened during a 3-month period with an additional 2 months allowed for memory evaluation by the clinicians of those patients with positive screens. Given the short time frame for this study we did not address the management of dementia for diagnosed patients; however, handout materials were provided for clinicians to give to patients or family members of persons diagnosed with mild cognitive impairment or dementia.
Implementation of Rural Older Adult Memory (ROAM) Study in 6 participating rural practices: intervention protocol. dx, diagnosis; mci, mild cognitive impairment.
ROAM, Rural Older Adult Memory; MMSE, mini-mental state examination; ADL, activities of daily living; IADL, instrumental activities of daily living; GDS, Geriatric Depression Scale; TSH, thyroid-stimulating hormone; CBC, complete blood count; MRI/CT, magnetic resonance imaging/computed tomography.
The “Memory Screen” included 4 indicators of possible dementia. First, patients were asked if they had noticed a change in their memory that concerned them. Second, if a family member or other informant was present, this person was asked if he or she had noticed a change in the patient's memory that concerned him or her. Although these 2 questions have not been formally tested for sensitivity and specificity, patient and family concerns about changes in memory are an important component of the diagnosis of mild cognitive impairment and dementia.23 The third item in the memory screen was 3-word recall.24 Before taking vital signs, the patient was asked to remember the words, “pony,” “quarter,” and “orange”; the patient's vital signs were recorded, and then the patient was asked to recall the 3 words. Failing to recall ≥2 words on the 3-word recall was considered positive. Finally, the memory screen form included a place for the medical assistant to check if he or she was concerned that the patient was cognitively impaired. We included medical assistant concern as a possible indicator because we theorized that medical assistants might be aware of a patient's memory problems, for example, if the patient appeared confused about their appointment schedule or had problems with medication instructions.
Memory Evaluation by Clinicians
The memory evaluation form (see Appendix) was provided to the clinicians for use with patients who screened positive. In addition to the single-page memory evaluation form, clinicians were provided with tools and instructions for completing the recommended tests. The memory evaluation included tests for 5 domains of cognitive function (memory, executive function, abstraction, construction ability, and verbal fluency)25: a mental status examination, abstraction and judgment questions, a clock drawing test,26 and a test of verbal fluency.27 We also provided an activities of daily living/independent activities of daily living scales,28 the “get up and go” test,29 the Geriatric Depression Scale,30 and the Caregiver Burden Assessment.31 These tests are consistent with quality indicators32 and aided clinicians in recognizing alternative diagnoses and family needs. Instructions for the neurologic examination and laboratory and imaging tests10 were provided in the training and included on the memory evaluation form. In addition, clinicians were provided print materials about prescribing for cognitive and psychiatric symptoms, a driving assessment, referral cards for the Alzheimer's Association, and other information sheets for caregivers.
Training
Study clinicians, medical assistants, nurses, and front office staff participated in a 2-hour training about recognition and diagnosis of dementia, principles of dementia care management, information about community resources, and the intervention protocol. The training was presented by the study geriatrician (EE), the director of the Alzheimer's Association Oregon Chapter, and the primary investigator (LB) via web conference (one-way PowerPoint presentation and 2-way audio). Each practice that received the web conference training had onsite support from a member of the ORPRN staff to ensure smooth operation of the video equipment. The training was audio taped and distributed to clinicians and staff for later review.
Intervention Protocol
During the 3-month period of the intervention, all patients aged 75 and older who were seen by a study clinician were eligible for screening. The medical assistant determined if the patient had a prior diagnosis of dementia, had been prescribed dementia medications, or was “too ill” to be screened; these were noted on the memory screen form and it was placed in the box of completed forms. If none of these were appropriate to describe a patient, the memory screen process was explained to the patient at the time he or she was escorted from the clinic waiting room to the examining room. If a family member was present, he or she was invited to accompany the patient for the first part of the visit. Patient refusal of the screen was recorded on the form and the form was placed in the collection box. If one or more of the 4 screen indicators was positive, the memory screen form and the memory evaluation form were placed on the patient's chart to alert the clinician about the need for a memory evaluation. The memory evaluation could be completed either during that visit or during a separately scheduled visit. For patients who screened positive, we collected data on work-ups conducted during the next 2 months after the end of the screening period.
Intervention Evaluation
Outcomes of interest included change in clinician confidence and comfort in diagnosing and managing dementia and changes in the incident diagnosis of dementia. In addition, we evaluated the adherence to the intervention protocol as well as patient, medical assistant, and clinician responses to the intervention.
Chart Review Before the Intervention
To compare the intervention with the practices before the intervention, records from patients aged 75 or older who were seen by study clinicians during a 2- to 4-week period in October 2006 were reviewed for incident dementia diagnosis during the next 4 months. The number of patient charts reviewed per clinic ranged from 24 to 67 (total of 310 charts).
Dementia Care Confidence Scale (DCCS)
The DCCS is an 8-item scale used to rate clinician confidence levels in assessing and diagnosing dementia, treating symptoms, managing care, differentiating delirium from dementia, differentiating depression from dementia, advising patients and family about community resources, and comfort with disclosing a diagnosis of dementia to a patient and family.33 We administered the DCCS as part of a written survey completed that was by study clinicians before and after the intervention.
Patient Satisfaction Surveys
Patients were asked by the medical assistant to complete a brief, written satisfaction survey immediately after they were screened. The patient survey asked if the patient had any concerns about the screening and if, in general, they thought it was a good idea for PCPs to assess older patients’ memory and thinking. Patients were also invited to explain any discomfort they experienced and if there was anything that could be done differently during the screening.
Feedback Sessions and Surveys of Clinicians and Practice Staff
We surveyed clinicians about their responses to the ROAM intervention and conducted feedback sessions with the clinicians and practice staff in each practice who had participated in the intervention. Sessions after the intervention were audio-taped and transcribed.
Data Analysis
We used descriptive statistics to report the uptake of the intervention. We compared the proportion of dementia-related diagnoses for eligible patients made during the intervention with the proportion of dementia-related diagnoses made for eligible patients in the period covered in the chart review before intervention. Wilcoxon signed rank tests were used to compare changes in clinician confidence with dementia diagnosis and management before and after the intervention. We used χ2 analysis to assess the effect of sex and age on the proportion of patients with positive screens, and used the Mantel-Haenszel statistic to test for any linear effect of age on positive screens. The clinician survey and transcripts of the group feedback session after the intervention were reviewed by the coauthors to gain insight into clinicians’ and medical assistants’ satisfaction with the intervention and to assess the feasibility of the protocol.
Results
Participation
Eighteen clinicians (11 family practice physicians, 1 internal medicine physician, 4 physician assistants, and 2 nurse practitioners) and 26 medical assistants participated in the study.
Intervention Process
Patient Screening
Seven hundred three patients aged 75 or older were seen by study clinicians during the intervention period (Figure 1). Medical assistants initiated a screen form for 495 patients of the 703 patients aged 75 or older who were seen by participating clinicians during the intervention period (70% of the eligible patients). Of the 427 patients who were given the 4-component screen, 198 (46%) screened positive. Ninety-seven screened patients (23%) were accompanied by a family member. Of these, 72 family members (74%) reported concerns about the patient's memory. As expected, older patients were more likely to screen positive (χ2 P = .001, with a highly significant linear trend; Mantel-Haenszel P < .0001). There was no significant difference in sex between the positive and negative screens.
Of the patients who screened positive, 73 (37%) had “work-up scheduled” checked on the screen form. For each of the 4 indicators of possible dementia, the proportion of patients who screened positive and who were scheduled for memory evaluation were as follows: three-word recall: 39%; patient report: 37%; family report: 43%; and screener (medical assistant) concern: 77%. Medical assistant concern was the only screen item that had a significant association with “work-up scheduled” checked (χ2 P < .0001). Of the 125 patients with “work-up scheduled” not checked, no reason for this was provided for 45 patients. For the others, the clinician wrote that the patient had refused a memory evaluation (n = 36), wrote that the memory evaluation was not necessary (n = 39), or gave other reasons for not scheduling a memory evaluation (n = 4). Twenty patients (4%) who were offered the screen refused and 36 (18%) patients with a positive screen refused a diagnostic evaluation.
Dementia Evaluations
Sixty-one patients received a clinical evaluation for dementia, which was conducted during a separate 30- to 40-minute visit.
Outcomes
Dementia Diagnosis
The patients included in the chart review before the intervention and those seen during the intervention were comparable with respect to age and sex. Of the 310 charts reviewed before the intervention, 3 patients (0.97%) had a recorded new diagnosis of dementia (n = 2) or mild cognitive impairment (n = 1). Among the 703 eligible patients who were seen during the intervention period, a total of 19 (2.7%) were newly diagnosed with dementia (n = 5) or mild cognitive impairment (n = 14) (difference between groups, P = .06). With respect to change in clinician confidence and comfort with diagnosing and managing dementia, all 18 clinicians completed the DCCS and there was a significant improvement in confidence in differentiating depression and dementia (P = .01). We also found trends of improvement in clinicians’ confidence in differentiating delirium and dementia, in treating the symptoms of dementia, and in clinicians’ comfort in disclosing the diagnosis to the patient (P =.06 to .10).
Response to Intervention
Patient Response to Screening
Three hundred twenty-five (76%) screened patients completed the survey to assess their response to screening. Ninety-eight percent of respondents reported no concerns or that they were “pleased to have [their] memory evaluated.” No patients reported that they were “a lot uncomfortable” with the memory screening. Ninety-one percent responded that, in general, memory evaluation for older patients was a good idea.
Clinician Response to Intervention: Survey Results
All study clinicians completed the survey after intervention and all rated the ROAM project as moderately or highly successful in increasing the identification of dementia, in increasing their knowledge about dementia diagnosis, and in their overall assessment of the intervention. Clinicians commented positively about knowledge gained and about the materials provided. Some wrote that they appreciated the opportunity for focused time to address a single issue and for communication with patients about dementia. A number of clinicians reported a desire for further training about the diagnostic protocol and interpretation of the diagnostic tests.
Feedback Group Results
The clinicians spoke highly of the continuing medical education training, the ease in which the screening process was implemented in their clinics, and the quality of the materials provided by the project. As in the written survey, some clinicians reported that they would have liked more training about the interpretation of cognitive tests used in the diagnostic work-up. A number of the clinicians and medical assistants commented about their surprise that the patients responded so positively to the screening. The medical assistants also reported that the screening process was easy to incorporate into their routine.
Individual Practice Differences
Despite the overall positive results from the intervention, substantial differences among the practices in adherence to the intervention protocol affected the outcomes, as shown in Table 1. To better understand these variations, we reviewed the transcripts of the feedback sessions and discussed the implementation of the intervention with ORPRN staff. We identified 2 primary factors that influenced these process variations:
Practice-Level Data for Intervention Protocol
1. Integrating the Intervention Protocol into Clinical Routines
Several practices seemed to have set up an effective process for identifying patients who were eligible for screening (practices A, C, D; see column 4 in Table 1) and/or for ensuring that patients who had been identified as being in need of a memory evaluation actually received one (practices A, C, D, E; see column 8 of Table 1). Practice F attributed their low proportion of screened patients to their being preoccupied with the implementation of a new electronic medical record. In addition, unlike the others, this practice did not create a centralized system (eg, using staff at the front desk) to place the screen forms on eligible patients’ charts. In practice B, a participating medical assistant unexpectedly died during the intervention period, understandably interrupting the clinical routine during the early part of the intervention.
A number of start-up issues surfaced at the other practices as well, although these were mainly addressed by the ORPRN research coordinator. For example, in one practice the clinicians and assistants initially presented the screening as an option, ie, they asked the patient if they would like to be screened (opt in) rather than presenting the screening as part of the routine with the opportunity for patients to refuse (opt out).
Another challenge in integrating the intervention into the practice routine was the allocation of time for training. Clinicians requested shortened training and resisted scheduling more than one training session. Nonetheless, a frequent comment after the intervention was the need for additional training.
2. Clinician Attitudes
The low proportion of patients who were scheduled for a memory evaluation among those with positive screens in 5 of the 6 practices suggests that the clinicians’ attitudes may have influenced adherence to this step in the process. Comments made by clinicians with a low percentage of evaluations support this: “I would have to say this: I wonder, really, is diagnosing Alzheimer's really that big a problem? I mean, how many [patients] do you come up with where you had no idea, you had no suspect [sic] that this person had a problem …. I’ve never seen [medications] be very efficient.”
“… it would be a lot easier to get enthused about a project that may have more medical relevance. Something that you might really get jazzed about.”
These comments are in contrast to those made by clinicians from practices with higher proportions of positive screens scheduled for diagnostic workup: “I thought the study was very useful because it identified patients that I was not aware of who had problems, whether it was Alzheimer's or depression or other memory problems.”
“I tend to think that the earlier [diagnosis] the better because it helps you think about your patient. It helps you think about their compliance, and helps you think down the road, what are we going to do, and it helps you get the family involved.”
In addition, in some cases, the clinicians apparently misapplied the results of the delayed 3-word recall: if the person passed this brief memory test but the patient reported concerns about their memory, the clinician often discounted the patient's concerns and opted to not schedule a diagnostic evaluation.
Discussion
This feasibility study was, to our knowledge, the first application of the ACOVE model for screening and diagnosing dementia in rural primary care. We found that the ROAM intervention was easily integrated into the routine patient process by the medical assistants and was well-received by patients. For clinicians, there was a more mixed response. Clinicians reported high satisfaction with the training, increased knowledge about dementia, and showed significant increases in confidence when diagnosing dementia; however, the overall uptake of the intervention was modest and variable across practices and the clinicians’ responses in assessments after intervention were mixed. The modest outcomes of the proportion of patients who were diagnosed with dementia may have been affected by these variations.
A major finding of this study was the reluctance of the clinicians to follow up on a positive dementia screen. Clinicians often determined that the symptoms did not warrant a dementia work-up. Although better training may have increased the proportion of patients who had a positive screen and who received a memory evaluation, a similar lack of follow-up on a positive dementia screen has been reported by other researchers16; this suggests that other factors may be at work. The lack of perceived benefit of diagnosing dementia by some clinicians undoubtedly influenced their decision whether or not to schedule a memory evaluation. Other possible reasons for lack of follow-up might include a lack of time, poor reimbursement, large patient volumes, the perceived complexity of the memory evaluation protocol, or a lack of clinician comfort and confidence in making a diagnosis; however, these were not discussed in the feedback sessions.
Much of what we learned in this study related to the intensely busy environment in which primary care clinicians practice. Our study offers the following insights:
Distance learning using web-based technologies seems to be a feasible method for training busy rural clinicians and medical assistants. We noted the tendency of the clinicians to resist lengthy training but, once the intervention was underway, clinicians actually expressed a desire for further training. The high proportion of patients with a positive screen who did not receive a dementia evaluation suggests that better training about the value of diagnosis may be warranted. Research is needed to further develop and test modes and content of training as a means of improving adherence to quality practice standards.
The medical assistants played a key role in this intervention, and their positive response suggests that greater attention be given to ways to engage them. Although we did include the medical assistant in our feedback sessions, we did not gather written input from them. The increased likelihood that clinicians would schedule a work-up for patients when the medical assistant noted a concern about the patient's memory was also intriguing, suggesting the high value these clinicians place on their assistants’ opinions. We believe, as others have recommended,34 that enhancing the role of medical assistants is a promising approach to improving care for older patients in rural communities. For example, medical assistants could be trained to assist in screening patients, administer cognitive and other tests used in diagnosis, and offer support to family caregivers and access to community resources.
Although we successfully screened 70% of the eligible patients, we recognize that our screening tool was not ideal. Several recent studies that used different dementia screening instruments reported higher sensitivity and specificity.3,16 We chose our tool to ensure that it could be feasibly administered by medical assistants in a busy clinic, but we would recommend using a screening instrument with higher specificity and validity for use in primary care, such as the Mini-Cog,35 for any future research. Another approach that merits consideration is the 2-stage screening process used by Boustani et al.3 Our screen was similar to the first-stage screen used in this study and had a similar proportion of positive screens (43% compared with 46%). With the addition of a second-stage, more sensitive 28-item screen, 13% in the study by Boustani et al3 were referred for diagnostic evaluation. Adding this screen would have increased the time demand on the medical assistant but could reduce the false-positive screens.
To ensure maximum participation and minimum disruption of practice routines, onsite assistance from research staff is essential. The ORPRN PERC was instrumental at the start of the intervention when the practices were becoming comfortable with the intervention protocol and were incorporating it into their clinical process. The research coordinator had good access to most of the clinicians and medical assistants when she visited the practices, and this facilitated the intervention process. The intervention uptake would probably have also been improved by delaying formal data collection for a week or two after the intervention began.
Limitations and Future Research
This study is one of the few attempts to test a way to improve the identification of persons with dementia in rural primary care. Its limitations are its small size, that it was not randomized, and its short time frame. In addition to implementing a larger, randomized, treatment-controlled intervention, in future research it will be important to identify a screening tool with better sensitivity and specificity than the one used in this study and to refine the processes for the diagnostic evaluation. The strengths of this study were that it was relatively easily integrated into the routines of the practices in which it was tested, that it resulted in high rates of screening, and that patients responded very positively. In those situations where there were procedural glitches needing to be ironed out before the intervention could move forward smoothly, the ORPRN PERC provided essential assistance.
Further research is needed to address the challenges of implementing practice change in rural practices as well as in addressing the barriers to improvement of the recognition and diagnosis of dementia. Improving rural practice is especially challenging because of limited resources within the practice and in the community in which they are located. As other researchers have emphasized,36 practice-based researchers must find the balance between stringent adherence to research protocol and the accommodation of variations in the practice's time, staffing resources, and clinician commitment.
Conclusion
Clearly changing practice related to dementia diagnosis has its challenges. The low rate of dementia work-up for patients who screened positive for possible dementia (found in this and other studies16) needs to be better understood and strategies to overcome this are needed. The lack of follow-through on positive dementia screens may reflect time pressures on clinicians and staff, concerns about reimbursement for visits, and/or uncertainty about the appropriate procedures for the diagnostic workup. Alternatively, lack of follow-through may be because of clinicians’ perceptions that dementia diagnosis is of limited benefit until more effective treatments are available. The mixed response of clinicians to this intervention should be viewed against the positive response of patients who overwhelmingly reported their appreciation of being screened and of the medical assistants who played a largely positive and supportive role in its implementation. This study demonstrated the complex issues that must be addressed in implementing dementia screening in rural primary care. The challenges of practice change in busy rural practices are substantial. Understanding and accommodating the resources that are available to primary care practices and the attitudes and practices of clinicians are essential if we hope to develop interventions that will be accepted in real world settings.
Acknowledgments
We would like to thank the clinicians, clinic staff, patients and family members who made this study possible.
Notes
This article was externally peer reviewed.
Funding: This study was supported by grant no. R03 HS016007-01 from the Agency for Health care Research and Quality; grant no. P30 AGO080 (Oregon Alzheimer's Disease Center) from National Institute on Aging; the Oregon Clinical and Translational Research Institute; and grant no. UL1 RR024140 from the National Center for Research Resources, a component of the National Institutes of Health and National Institutes of Health Roadmap for Medical Research.
Conflict of interest: none declared.
See Related Commentary on Page 429.
- Received for publication September 24, 2009.
- Revision received March 31, 2010.
- Accepted for publication April 13, 2010.
