Abstract
Purpose: The purpose of this study was to compare a modified version of the Mini-Mental State Examination (MMSE) with the standard MMSE and the Mini-Cog in patients ≥65 years old, stratified by education and literacy level.
Method: This cross-sectional exploratory study enrolled a convenience sample of 219 patients with a complaint of memory loss or a diagnosis of dementia from a geriatric outpatient clinic, nursing home, senior center, and university hospital. The MMSE was administered, and in addition to spelling and serial 7s backward, patients were asked to recite the days of the week backward with the intent to reduce educational bias. Scores on the modified MMSE were compared with scores of the MMSE and the Mini-Cog.
Results: Of the 219 patients, 157 were identified with cognitive impairment by the Mini-Cog. Using a cutoff of ≤23, the MMSE identified 118 patients and the modified MMSE identified 91 patients with cognitive impairment, and with a cutoff of ≤27 the MMSE identified 168 and the modified MMSE 149 patients. All cognitively intact subjects correctly recited the days of the week backward. Specificity of the modified MMSE was higher than the MMSE for most groups. The highest sensitivity and specificity (94% and 88%, respectively) as well as positive and negative predictive values (96% and 81%, respectively) were in patients with low levels of education for the modified MMSE using a cut off of ≤27.
Conclusion: Using the days of the week in the MMSE among illiterate and semiliterate participants and with education less than high school, and using a cutoff of 27 of 30, correlates better with Mini-Cog for dementia screening, with fewer false positives.
The US population is aging,1 and age is a risk factor for cognitive decline. The incidence of dementia doubles every 5 years after the age of 65 and reaches a prevalence of nearly 50% at age 85.2 Yet dementia is not recognized by 67% of primary care providers.3 The Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) criteria4 are used for diagnosis, but screening tests can identify patients at risk. Screening for the disease is important because of the long duration of the disease and high prevalence among the elderly. Health care providers have an opportunity to provide anticipatory guidance and help patients and families prepare for the financial burden of the disease and access community resources.
The Mini Mental State Examination (MMSE) is the most commonly used screening test5 and is useful in following disease progression over time. It has been used since 1975 as an integral instrument in screening for cognitive decline in the general population.6 The test evaluates cognitive level in several domains including reading, writing, and math. Its ability to accurately capture cognitive impairment is better in primary care than specialist settings.7 However, use MMSE has been questioned in subjects with lower levels of education,8⇓⇓–11 leading to adjustment of cut off points by education, with a cut off of 23 for high school–educated patients and 27 for college- educated patients.9,12 Fillenbaum et al13 showed lower specificity for various screening tests in minorities, which correlated with education. False-positive results misclassify normal subjects as cognitively impaired and may generate unwarranted anxiety among patients and families. Artificially increased disease burden may affect research on health disparities and dementia. Several studies have sought to reduce the effect of educational bias in screening instruments for dementia.14⇓⇓–17 Attempts to improve the accuracy of screening have included adapting existing tests, adjusting cut-points and scores, establishing group norms, and replacing culture-specific items that are less dependent on language, literacy, and other skills developed during formal education.8⇓⇓–11,14,18 The 3MS examination replaced reverse serial 7 counting with reverse 5 to 1 counting19 because of its difficulty.8 Tombaugh et al20 further adapted the 3MS examination by replacing reverse counting with reverse spelling. To our knowledge, no MMSE has been tested without a spelling subtest. Other tests that rely less on literacy and other skills related to formal education18,21 include the 7-minute screening battery,17 the animal naming test22 and the time and change test.15 However, these tests do not lend themselves to monitoring disease progression over time and are not used as much as the MMSE.
Many of our patients with low levels of literacy, who could not spell or count backward, would often not continue the test. We have been piloting the use of saying the days of the week backward in place of counting or spelling backward, but review of the literature showed no studies validating its use, and only one study used an alternative that was not dependent on education. Blessed et al14 included months of the year backward in their screening tests for assessment of concentration. Its use was not validated nor evaluated for education or literacy. This article describes our preliminary evaluation of a modified version of the MMSE (M-MMSE) with the addition of reciting the days of the week backward in the assessment of the concentration and calculation domain of cognitive status in elderly patients with varying levels of education and literacy.
Methods
Participants and Setting
This study was designed as a cross-sectional comparison of cognitive assessment strategies. We enrolled a convenience sample of people ≥65 years of age with memory complaints or a diagnosis of dementia who were seen in a geriatric outpatient center, family medicine clinic, and hospital of an academic medical system between January 2007 and April 2010; additional participants included residents of a nursing home and the local senior center.
Inclusion criteria were (1) age ≥65 years; (2) English speaking; (3) informed consent to participate provided by the subject or subject's informant; and (4) subjective memory loss or cognitive decline reported by the patient or their informants or documented in the electronic medical record. Exclusion criteria include d (1) delirium or active psychosis; (2) severe language or physical disabilities that affect the reading, drawing, or written portions of the test or the hearing instructions; and (3) history of head trauma within the past 3 months.
Patients meeting the eligibility criteria were enrolled when they signed the informed consent. Subjects with limited or no literacy had the consent read to them and either signed or marked it themselves while witnessed by their informant(s) or signed by their informant(s). A copy of the signed document outlining the study objectives and risks and providing investigator contact information was given to the participants. Race was documented by the interviewer and cross-referenced in the medical record, where it was established by patient report. Patients or caregivers were asked about years of schooling and ability to read. When the patient or caregiver were unsure of literacy, the patient's ability to read a newspaper or medication insert was identified as literate and the ability read single words or simple commands was considered semiliterate. The institutional review board at East Carolina University approved our study.
Interventions and Measures
We used the Mini-Cog as control test for assessing cognition. This instrument was chosen because it has high sensitivity and specificity (99% and 93%, respectively) in patients with memory problems as identified by an interview with the patient's informant, similar to our population.23 Without arithmetic and spelling, the Mini-Cog is less affected by literacy or education. Its brevity fits into the busy schedule of a primary care practice. It consists of the 3-item recall, the same as in the MMSE, which was not repeated, and a clock drawing test. Patients were asked to draw the numbers and hands on an existing circle showing the time of 11:10 on the face of an analog clock. Scoring of the Mini-Cog starts with the 3-item recall. Patients who could remember and recall all 3 items were considered normal on the Mini-Cog; those who did not remember any items were identified as having cognitive impairment. The results of the clock drawing test determine the cognitive impairment of subjects scoring 1 or 2 points on the 3-item recall. Those drawing the clock face with incorrect placement of or missing hands or numbers were classified as having cognitive impairment.
The Folstein MMSE was administered as recommended, using spelling world backward and counting backward by 7s from 100. We added a new question and asked patients to recite the days of the week backward (the M-MMSE) to test the potential of modifying the MMSE for people with low literacy and education.
The Folstein MMSE was purchased from Psychological Assessment Resources, Inc. (Lutz, FL) and administered by the authors and research assistants trained to administer screening tests; training of research assistants consisted of explanation of the test and supervision of at least 2 screening tests or until the principal investigator (IH) felt competency was achieved. The MMSE6 consists of answers that make up a total of 30 points. The concentration and calculation section of the standard version of the MMSE makes up 5 points: 1 point is scored for each correct serial subtraction of 7 from 100 up to a total of 5 consecutive instances (eg, 100, 93, 86, 79, 72); 1 point is also scored for each letter of the word world recited backward correctly up to the total 5 points. The better of these separate results, either spelling or counting backward, is included in the total of the standard MMSE test. Any incorrect or out of sequence part was subtracted respectively (eg, “drlow” would have received 3 points). Patients then were asked to sequentially recite the days of the week backward, starting with any day; 1 point was given for each correct answer, up to a total of 5 points. The results of the days of the week in reverse were recorded in place of counting or spelling backward and calculated as the M-MMSE. The remaining items6 of the screening test were the same between the MMSE and the M-MMSE and were not repeated to avoid a training effect. Each person then had an MMSE score containing the better of the counting or spelling backward and an M-MMSE score containing the days of the week backward.
The Geriatric Depression Scale24 was administered if the MMSE score was ≤23 to screen for depressive symptoms. Patients with a score of >5 (of 15) were given further evaluation, treatment, or both. These patients were not removed from the analysis; one meta-analysis shown that only 0.6% of dementia cases reversed with treatment of reversible causes.25
Statistical Analysis
Participants' total scores using the MMSE and the M-MMSE and the standard versus modified components were compared in reference to participant education level and standard MMSE cutoff scores. Scores also were compared by Mini-Cog classification rates of “normal” versus “cognitively impaired.” Education was documented subjectively as recalled number of total years of schooling. For purposes of this analysis, participants were categorized by education levels of high school graduate and beyond (≥12 years of schooling completed) and less than high school graduate (<12 completed years of schooling), with scores broken down into ≥24 (not cognitively impaired, “normal”) and ≤23 (cognitively impaired). Demographic data detailing ethnicity, sex, and age were cross-tabulated with results. Descriptive statistics were used to characterize the study sample with respect to age, sex, and education/literacy status. Mean scores from each cognitive assessment strategy (MMSE, M-MMSE, Mini-Cog) were computed and compared for the overall group, as well as within subgroups by education and literacy status, using the Student t test. Correlation between the M-MMSE and Mini-Cog was calculated with the Pearson coefficient. Using the Mini-Cog for comparison in identifying cognitive impairment, the sensitivity, specificity, and positive and negative predictive values of the MMSE and M-MMSE were compared using 2 different cut-points in the overall sample and in subgroups with limited education and literacy. While sensitivity and specificity establish validity, positive and negative predictive values describe utility in clinical settings.12 Statistical comparison of mean scores (t tests) was performed using SPSS version 16 (SPSS, Inc., Chicago, IL).
Results
This study enrolled 222 subjects with memory loss. Three did not perform the clock drawing test and were removed from the analysis (Figure 1). The mean ± participant age was 80 ± 7.0 years (range, 64–93 years), and almost half (n = 108; 49.3%) of the participants graduated from high school. Most (n = 145; 66.3%) described themselves as literate: 28 patients (12.8%) were illiterate and 46 (21.0%) were semiliterate. Almost half were African American (n = 107; 48.9%) and 157 (71.7%) were classified with cognitive impairment through Mini-Cog scoring.
Patient flow diagram.
Average total scores achieved with the MMSE were lower than with the M-MMSE (Table 1) in subjects identified with cognitive impairment by Mini-Cog. This pattern was consistent regardless of literacy, education level, sex, or race (data not shown). Table 1 shows the results of the modified subcomponent (weekdays) versus the calculation (serial 7) or spelling (world) section analyzed separately, with up to 5 possible points each. All normal subjects correctly identified the days of the week backward, whereas 79% and 39% of normal subjects made mistakes on the counting and spelling backward, respectively.
The M-MMSE classified fewer patients with cognitive impairment than Mini-Cog or MMSE. Table 2 shows subjects identified with cognitive impairment by Mini-Cog, stratified by differing cutoff points for the respective standard or modified MMSE. Mini-Cog identified 157 patients with cognitive impairment, 86 of whom had less than a high school education, and 59 of whom reported being illiterate or semiliterate. Using a cutoff of 23, the M-MMSE identified fewer patients with cognitive impairment. This trend attenuates with decreased education and literacy. Using a cutoff of 27 for the MMSE, which is used for patients who are college graduates, the standard MMSE identified more patients with cognitive impairment than Mini-Cog, and the M-MMSE showed mixed results across education and literacy levels (Table 2).
Among patients considered normal by Mini-Cog (n = 62), 13 patients scored ≤23 by MMSE; of those, 4 had a high school education or more and 5 described themselves as literate. Using M-MMSE, 4 patients scored ≤23; of those, 2 had a high school education or more and 2 described themselves as literate. Sensitivity was lower for the M-MMSE compared with MMSE; however, specificity and positive predictive value for the M-MMSE were higher in most subjects (Table 3).
By starting the concentration segment with the days of the week backward, we observed no abandoning of the screening process. Although we required all patients with an MMSE score of ≤23 undergo depression screening, many more (n = 171) completed the Geriatric Depression Scale. Of those with a positive screen (n = 48; 28%), all were assured further evaluation and treatment.
The MMSE and M-MMSE were highly correlated with the Mini-Cog in patients with low levels of education and low literacy (Pearson coefficient, 0.51; P < .001). This study showed that a greater number of patients identified as normal by Mini-Cog are identified with cognitive impairment using the MMSE than the M-MMSE. This holds true for patients with low levels of education and low literacy. Conversely, a cutoff of ≤23 (of 30) on the MMSE may falsely identify many patients with the disease. Moving the cutoff to ≤ 27and using the M-MMSE may be a more accurate cut off for identifying patients with low levels of education with cognitive impairment. Therefore, we calculated the specificity and sensitivity, shown in Table 3. Sensitivity of the M-MMSE is generally lower compared with the MMSE, with higher specificity. Positive predictive values are generally better for the M-MMSE than the MMSE, except for the M-MMSE at a cutoff of 27 in all subjects. The highest predictive value is in patients with low levels of education using the M-MMSE at a cutoff of 27.
Discussion
To our knowledge this is the first study to test a version of the MMSE that is less dependent on spelling or calculation in subjects with low literacy and levels of education. We found that among illiterate and semiliterate participants and those with less than a high school education, the M-MMSE (using a cutoff of 27 of 30) correlates better with Mini-Cog for cognitive impairment screening, with fewer false positives. Our efforts were focused on developing a method of accurate dementia screening for populations in the counties of Eastern North Carolina, where proportion estimates of level I literacy, corresponding to 0 to 8 years of schooling and the lowest level established by the National Adult Literacy Survey, can reach over 27%.26 A third of the participants of a 1993 National Adult Literacy Survey who performed at level I of the 3 levels of the survey were ≥65 years old.27 These numbers are even higher in African Americans, who comprise almost a third of the region's population28 and about half of our clinic population.
We expected the false positive rate for the M-MMSE to be lower than for the MMSE, considering that all normal subjects correctly recited the days of the week backward. However, only the low literacy group showed a lower false positive rate in the M-MMSE.
Strengths of our study include the comparison with the Mini-Cog, a validated test, and the variety of settings that included a hospital, outpatient clinic, nursing home, and senior center, making this a representative sample with broad applicability. Other strengths are an almost equal number of minority subjects and relatively good representation of subjects with low literacy and levels of education. Data regarding dementia among minority populations are largely absent, and the evaluation of screening instruments such as the MMSE were mostly based on white populations with an underrepresentation of minorities in Alzheimer disease research.29,30
Limitations of our study include a convenience sample that may not be representative of the general population. Education level defined by years of schooling was subject to recall bias, with no documentation or quality measure. Literacy, which is inherently difficult to measure, was similarly identified by subjective patient report of the ability to read a newspaper and/or simple directions. Some patients with normal Mini-Cog had abnormal MMSE and vice versa. While the diagnosis of dementia is based on the DSM-IV criteria4 and depends on decline over time, this could not be formally done in this cross-sectional pilot study. This study provides some evidence that the days of the week could be used in place of spelling or counting backward in the MMSE. With the current limitations, the efficacy of this approach cannot be shown conclusively. Further studies testing the M-MMSE using DSM-IV criteria are needed.
Conclusion
Our study shows that this modified screening instrument may lead to fewer participants being incorrectly categorized as cognitively impaired solely due to low literacy level. Taking the educational part (spelling and counting) out of the MMSE, using days of the week recited backward as the Modified-MMSE, and using a cut off 27 of 30 for low literacy, the M-MMSE detects cognitive impairment with fewer false positives.
Acknowledgments
The authors thank Andreea Doaga, MD, for her role in the inception and early conduct of the study; Hossein Movahed, MD, for help with the analysis; Charles Baker, MPH, for his contributions to previous versions of the manuscript; and Mary Beth Plane, PhD, for review of the manuscript.
Notes
This article was externally peer reviewed.
Funding: This study was supported by a Health Resources and Services Administration grant (1 K01 HP00159-01 Geriatric Academic Career Award (IH) that was awarded during the study. The Wooten Foundation provided $3000 for a student to help with gathering of data for the study.
Prior presentation: This study was presented as a poster at the annual meeting of the American Geriatrics Society, Orlando, FL, May 12–15, 2010.
Conflict of interest: none declared.
- Received for publication November 11, 2012.
- Revision received March 13, 2013.
- Accepted for publication March 25, 2013.
