Abstract
Background: Brief, global assessments such as the Montreal Cognitive Assessment (MoCA) are widely used in primary care for assessing cognition in older adults. Like other neuropsychological instruments, lower formal education can influence MoCA interpretation.
Methods: Data from 2 large studies of cognitive aging were used—Alzheimer’s Disease Neuroimaging Initiative (ADNI) and National Alzheimer’s Coordinating Center (NACC). Both use comprehensive examinations to determine cognitive status and have brain amyloid status for many participants. Mixed models were used to account for random variation due to data source.
Results: Cognitively intact participants with lower education (≤12 years) were more likely than those with higher education (>12 years) to be classified as potentially impaired using the MoCA cutoff of <26 (P < .01). Backwards selection revealed 4 MoCA items significantly associated with education (cube copy, serial subtraction, phonemic fluency, abstraction). Subtracting these items scores yielded an alternative MoCA score with a maximum of 24 and a cutoff of ≤19 for classifying participants with mild cognitive impairment. Using the alternative MoCA score and cutoff, among cognitively intact participants, both education groups were similarly likely to be classified as potentially impaired (P > .67).
Conclusions: The alternative MoCA score neutralized the effects of formal education. Although further research is needed, this alternative score offers a simple procedure for interpreting MoCAs administered to older adults with ≤12 years education. These educational effects also highlight that the MoCA is part of the assessment process—not a singular diagnostic test—and a comprehensive workup is necessary to accurately diagnose cognitive impairments.
- Clinical Medicine
- Cognition
- Cognitive Aging
- Geriatrics
- Montreal Cognitive Assessment
- Neuropsychology
- Psychometrics
Introduction
Brief neuropsychological instruments like the Montreal Cognitive Assessment (MoCA)1 are recommended for monitoring cognition of older adults.2 Like other brief assessments,3 educational attainment confounds the interpretation of MoCA results and a 1-point adjustment is recommended for examinees with ≤12 years of formal schooling. Despite this adjustment, the published cutoff score for potential cognitive impairment is still prone to false positives.4⇓⇓–7 At present, early detection of mild cognitive impairment (MCI) or dementia is essential for care and quality of life planning.8 The early detection of presymptomatic Alzheimer disease (AD) with biomarkers will likely become widespread when disease modifying treatments prove effective. Instruments like the MoCA will be at the front line of identifying individuals for biomarker workups, making educational confounds of the MoCA problematic now and in the foreseeable future. To address this, the present study used data from 2 large US cohorts to reexamine the effects of formal schooling on the MoCA and optimize scoring and interpretation.
The MoCA can be affected by factors other than education. Among cognitively intact older adults, lower MoCA scores are related to both subjective cognitive concerns9,10 and heightened depressive symptomatology,9 the latter creating the potential for false positives using the MoCA cutoff.11 Lower MoCA scores were related to polypharmacy in 1 study12 (cf. 9). In MCI, MoCA scores are further reduced in the presence of comorbid cerebrovascular disease and elevated brain amyloid burden, compared with either pathology alone.13,14 This relationship may be the same in cognitively intact older adults given that hypertension has been found to be related to slightly lower MoCA scores.15 Prior research has not found a relationship between elevated brain amyloid levels and MoCA score among cognitively intact older adults,16⇓–18 but it is not clear how that relates to educational attainment.
The present study addressed the practical use of the MoCA with older adults with lower formal education in 4 aims:
• Aim 1: we hypothesized that participants with ≤12 years of education would be at greater risk for false positives of probable cognitive impairment using the published MoCA cutoff score.
• Aim 2: we planned to identify the MoCA items sensitive to lower education in persons with nonelevated brain amyloid and compute an alternative MoCA score without those items. It was hypothesized that sensitive items would include, at least, cube copy, serial subtraction, and abstraction because these items have been associated with education in prior research.19–21
• Aim 3: we planned to assess the classification accuracy of this alternative MoCA score and derive from it a cutoff of potential cognitive impairment. Without knowing which items that Aim 2 would yield, no explicit hypothesis was made for Aim 3, but our expectation was that the alternative MoCA must classify cases accurately to have any practical value.
• Aim 4: it was identical to Aim 1, except that the alternative MoCA cutoff score from Aim 3 would be used instead of the published MoCA cutoff.
Methods
Data from the Alzheimer disease Neuroimaging Initiative (ADNI)22–24 and National Alzheimer disease Coordinating Center (NACC) Uniform Data Set (UDS)25,26 were used because both programs collect itemized MoCA data, brain amyloid status, and comprehensive physical, neurological, and neuropsychological diagnostic evaluations. We refer to these as ADNI and NACC, respectively.
Data used in the preparation of this article were obtained from the ADNI database (adni.loni.usc.edu). The ADNI was launched in 2003 as a public-private partnership, led by Principal Investigator Michael W. Weiner, MD. The primary goal of ADNI has been to test whether serial MRI, PET, other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of MCI and early AD. NACC UDS data are collected in a standardized way at ∼36 past and present Alzheimer’s Diseases Research Centers (ADRCs). The data include medical, neuropsychological, genetic, and other annual data from participants with dementia, MCI, and normal cognition.
For ADNI, data collected from September 2009 through 29 April 2021 were included in this study. For NACC, we used data collected from early 2015 through the March 2021 database lock. More detailed methods are described in Appendix materials.
Participants
The ADNI and NACC differ in inclusion criteria. For the ADNI, participants must be 55 to 90 years of age; must have a reliable informant, limited cerebrovascular risk factors, and ≥6 years of formal education; must be free of systematic illness, willing to complete repeated study visits, ≥1 lumbar puncture, and be fluent in English or Spanish. For the NACC, participants must be willing to undergo annual study visits and be fluent in English or Spanish. Otherwise, each ADRC recruits using its own criteria; some ADRCs require consent to donation of brain and autopsy at death. Internal Review Boards at each ADNI site and each NACC site approved study procedures. All participants gave informed consent before data collection.
Inclusion for the present study required an ADNI or NACC visit meeting the following criteria: > 59 years of age, a MoCA administration, a status of cognitively unimpaired or MCI (defined below), and an indicator of brain amyloid status (defined below). For the present study, the data used were from the earliest study visit in which a participant was older than 59, administered the MoCA, had amyloid status available, and were cognitively intact.
Cognitive Status
For the ADNI, site study physicians determined cognitive status, which was then reviewed by a Central Review Committee. Cognitively unimpaired participants performed in healthy ranges on neuropsychological assessment and were without significant impairment in daily life. For the NACC, a study clinician, formal consensus panel, or ad hoc group of clinicians determined cognitive status using established clinical guidelines. Only data from cognitively unimpaired participants were included in the present study, except for Aim 4, which included cognitively unimpaired and MCI participants.
Brain Amyloid Status
The ADNI and NACC differ in how amyloid status is captured. Whereas ADNI sites report raw florbetapir PET scan or CSF data, NACC uses local site standards and report yes/no indicators of elevated amyloid found on PET or in CSF. For the ADNI, raw values for PET or CSF were dichotomized using established cutoffs of elevated amyloid for each medium. In the present analyses participants were either elevated (Aβ+) or nonelevated (Aβ-) brain amyloid.
MoCA
MoCA scores range from 0 to 30, with lower values indicating greater cognitive impairment. Adjusted MoCA scores were computed by adding 1 point to the MoCA score of participants with ≤12 years of education. For the Aim 2 item analysis, the 6 Orientation items were summed for a single Orientation variable (range 0 to 6). The Trails, Cube, Clock Contour, Clock Numbers, Clock Hands, Tapping As, and Letter F items were treated as correct or incorrect. Naming (range 0 to 3), Registration (0 to 10), Digit Span (0 to 2), Serial 7 seconds (0 to 3), Sentence Repetition (0 to 2), Abstraction (0 to 2), Free Recall (0 to 5), and Orientation were treated as continuous. The published cutoff adjusted MoCA score of <26 for potential impairment was used was used to classify participants as either “likely normal” or “potentially impaired” = 1 (ie, <26).1
Covariates
For the analyses, education was treated as a dichotomous variable indicating either >12 or ≤12 years of education, referred to, respectively, as higher or lower education. Both the ADNI and NACC collect the 15-item Geriatric Depression Scale (GDS) as a measure of depressive symptomatology.27 The value for the GDS memory problems item was subtracted from the total GDS score and the memory problems item was used as a yes/no indicator of a subjective memory complaint. History of hypertension reported at the baseline study visit was used as a proxy for cerebrovascular disease risk.
Statistical Analyses
Covariates were compared between the higher and lower education groups. Cursory data visualizations indicated >1 point difference in average adjusted MoCA scores between education groups in NACC, but relatively similar means in the ADNI sample (not shown). Binomial generalized linear mixed models (GLMMs) were used to account for random variance due to data source. Significance level was set at 0.05 for all analyses.
For Aim 1 and Aim 4, MoCA classification was analyzed using both simple yes/no criteria with χ2 tests (such as how the MoCA cutoff might be applied in practice) and binomial GLMMs to control for covariates. For Aim 1, adjusted MoCA classification was the dependent variable and for Aim 4, the alternative MoCA classification (see below) was the dependent variable. For both Aims, education level was the predictor of interest.
For Aim 2, we used data from cognitively unimpaired Aβ- participants, because elevated brain amyloid status is associated with slight deficits on assessment.28 The dependent variable was education level and all MoCA items were included as predictors in a binomial GLMM. Backwards selection was used to eliminate nonsignificant items from the model. Scores for the MoCA items that significantly predicted education level after backwards selection were subtracted from the adjusted MoCA total to compute an alternative MoCA. (We also computed this using the raw MoCA score. Those results were similar to the results of Aim 1.) For Aim 3, the MCI indicator was the dependent variable in a binomial GLMM with the alternative MoCA score as the sole predictor. A cutoff score for potential impairment using the alternative MoCA was derived.
Results
Sample Characteristics
Between the ADNI and NACC, 139 participants with lower education and 1187 with higher education were cognitively intact (see Table 1). In the lower education group, there were significantly more non-White minorities (P < .001) and a higher proportion of participants with hypertension history (P < .001). The education groups had roughly equal proportions with elevated brain Aβ and did not differ on the other covariates. There were 1240 participants diagnosed with MCI who met the other inclusion criteria. In the MCI group, there were 167 participants with lower education (13.5%), which was slightly more than among the cognitively intact participants (10.5%), (χ2 = 5.44, P = .02; see Appendix Table 1).
GLMM Estimating Likelihood of Classification as Impaired Using MoCA
Aim 1
Participants with lower education (45.3%) were significantly more likely than those with higher education (32.8%) to be classified as potentially impaired using the published MoCA cutoff after score adjustment (χ2 = 8.6, P = .003). As shown in Table 2, compared with higher education, lower education was associated with a significantly greater odds of MoCA-indicated impairment (OR = 1.78; 95% CI, 1.20–2.64), when controlling for covariates. That is, compared with those with higher education, participants with lower education were nearly twice as likely to be misclassified as impaired using the MoCA cutoff of <26.
Aim 2
The backward elimination procedure yielded 4 MoCA items that were significant predictors of lower education (P > .14 for all dropped variables). The final model is shown in Table 3. A correct cube copy (OR = 0.56; 95% CI, 0.37–0.86) and generating more than 11 words on letter fluency (OR = 0.45; 95% CI, 0.28–0.72) both reduced the odds of having lower education. Each 1-point increase in serial subtraction (OR = 0.63; 95% CI, 0.44–0.92) and abstraction (OR = 0.45, 95% CI, 0.32–0.63) reduced the odds of having lower education. Shown in Figure 1 (Panels a–d) are estimated marginal probabilities of lower education for scores on these MoCA items. (See Appendix Table 2 for overview of eliminated variables.)
Estimated marginal probabilities of education ≤12 years for MoCA items significantly associated with formal education. Abbreviations: MoCA, Montreal Cognitive Assessment.
Table 1. Participant Characteristics
MoCA Items That Are Significant Predictors of Lower Education in a Backwards Selection Binomial GLMM
GLMM Examining Classification of MCI Using the Alternative MoCA Score
An alternative MoCA was computed by subtracting scores for the 4 items from the adjusted MoCA score. The maximum possible score of the alternative MoCA was 24. The mean was 20.14, S.D. = 2.09, with a range of 13 to 24.
Aim 3
The alternative MoCA score discriminated between MCI and intact cognition (see Table 4). Each 1-point increase on the alternative MoCA score significantly reduced the odds of MCI (OR = 0.62; 95% CI, 0.59–0.66]) when controlling for the covariates. Each covariate was significant (P < .001) except for age (P = .77), hypertension (P = .94), and education level (P = .068). The model was highly accurate, correctly classifying 82.9% of participants: area under the curve = 0.903, sensitivity = 0.86, and specificity = 0.79. Importantly, classification of cognitively normal participants was equally accurate for participants in lower (82%) and higher (81.6%) education groups.
This model was rerun with the alternative MoCA as the sole predictor of MCI. The results suggested a cutoff score of 19. The estimated probability of MCI for an alternative MoCA of 19 was equal to 0.488 for NACC sites, ∼0.023 greater than the suggested threshold of 0.465 (ie, NACC cutoff should have been 20). Because of the slight difference, 19 was settled on for parsimony, since ADNI had lower mean adjusted MoCA scores overall. The model correctly identified 80.2% of MCI and 64.8% of cognitively intact participants.
Aim 4
Using the alternative MoCA cutoff of 19, participants with lower education (36.7%) were statistically no more or less likely than those with higher education (35%) to be classified as potentially impaired (χ2 = 0.16, P = .69). As shown in Table 5, lower education was associated with no difference in the odds of MoCA-indicated impairment (OR = 1.09; 95% CI, 0.63–1.64), when controlling for covariates. Age, being male, and hypertension were related to increased risk of classification as impaired (P < .001) but subjective memory complaints (P = .45), amyloid status (P = .33), and depression (P = .06) were not significant predictors.
GLMM Estimating Likelihood of Classification as Impaired Using Alternative MoCA
Discussion
The present study examined how formal education can impact the interpretation of MoCA scores in cognitively intact older adults. In line with previous research, older adults with lower education were more likely to be classified as potentially impaired using the published MoCA cutoff score. The cube copy, serial subtraction, phonemic fluency, and abstraction items on the MoCA were significant predictors of educational attainment, as hypothesized. An alternative MoCA score computed without those items accurately detected MCI and, to a lesser extent, intact cognition. A cutoff for potential impairment on the alternative MoCA misclassified those with lower and higher formal education at the same rate. That is, an alternative MoCA score neutralized the classification bias against the lower education group.
In line with prior studies,4⇓⇓–7 Aim 1 showed how cognitively intact older adults with lower education are at a higher likelihood of being classified as potentially impaired using the published MoCA cutoff. Even the 1-point adjustment for ≤12 years education does not neutralize this risk. In practice, such educational confounds could be especially harmful if diagnostic—and treatment—decisions were made based on a MoCA score alone. For example, a diagnosis of MCI or dementia itself can cause psychological distress29 and unnecessary care following a misdiagnosis could also have untoward effects such as inappropriate medication use.30 This finding is a reminder that a single MoCA score alone is insufficient for diagnosing MCI or dementia, and part of a more comprehensive process to determine the presence and cause of cognitive impairments among older adults.
In Aim 2, as hypothesized,19–21 the most education-sensitive MoCA items included cube copy, serial subtraction, and abstraction, but also phonemic fluency. The lattermost is unexpected given that phonemic fluency tasks are associated with education31 as well as cerebrovascular disease32 and hypertension was significantly more prevalent in the lower education group. While other research has suggested gradating increasing score adjustments at levels of schooling lower than ≤12 years,33 the present study examined a novel alternative: Subtracting the scores of the most education-sensitive items from the MoCA total. We tested this approach with both the raw MoCA total and the MoCA total with the recommended 1-point adjustment for education. If the raw total had been used, the classification results of Aim 4 would have had the misclassification issues found in Aim 1. In other words, using the 1-point adjustment to the MoCA score seemed to neutralize the educational biases only after removing the items more sensitive to formal educational attainment. This finding also suggests that some emergent property of all the items taken together is sensitive to lower education, a feature not unique to the MoCA.34 Coupled with past research, the findings of Aim 2 also suggest a re-evaluation of the inclusion of abstraction items in the MoCA Basic,35 developed for use with individuals <5 years of formal education.
In Aim 3, the alternative MoCA cutoff was identified. It classified MCI (81.5%) and cognitively intact (63.6%) individuals at the same level as the adjusted MoCA published cutoff (MCI = 80.9% and intact = 65%). The accurate classification is likely due to the free recall score, as it has been shown to distinguish intact cognition from MCI due to AD.36 It also includes the orientation items which accurately distinguish AD dementia from MCI and intact cognition.36 Although diagnostic subgroup analyses were not done, the items retained after Aim 2 suggest that this alternative MoCA score might accurately detect cognitive impairment due to Parkinson’s37 but perhaps not cerebrovascular disease because items most sensitive to cerebrovascular disease were subtracted from the MoCA total.38 This should be explored in future studies or using retrospective data from any disease-specific cohorts that collect item-level MoCA data.
In Aim 4, the alternative cutoff entirely neutralized the educational bias (with the 1 point credited for lower education). Lower (34.5%) and higher (36.6%) education groups were misclassified at comparable levels using the alternative MoCA, even when accounting for pertinent covariates. This is further evidence of a practical application for the alternative MoCA score. A critical caveat is that, roughly 1 of 3 participants in each group was misclassified. These participants underwent extensive examinations and cognitive testing as part of their study visits, and a plurality had previous visits—all which is not usually a part of general practice. Thus, the use of this alternative MoCA in practice needs further systematic study.
Limitations
First, both the ADNI and NACC cohorts are highly selected and not fully representative of the older adult population. Only 12% of the participants were non-White. It is unclear how these findings will generalize to diverse racial and ethnic populations. In addition, as 1 of our reviewers pointed out, the decision to classify lower education as ≤12 years of schooling may affect generalizability because individuals who completed high school or equivalent (ie, years = 12) might meaningfully differ from those who did not (ie, years < 12). However, using data from separate cohorts that had significant differences in MoCA score distributions within education levels, provides compelling evidence for the applicability of these findings. Second, although we accounted for brain amyloid status in the analyses, we did not account for cerebrovascular pathology, which can have notable impacts on cognition. Hypertension was used as a proxy for cerebrovascular disease but there are myriad other morbidities than can disturb the vasculature of the brain. Nevertheless, hypertension was a significant predictor of classification for cognitively intact participants and thus helps clarify any conclusions drawn from the present findings. There are also limitations in using years of education as the primary variable in this analysis as it is widely appreciated that quality as well as quantity of education is an important variable for consideration when examining cognitive test biases. This may be especially true for many geographic and sociocultural populations where both years of education and quality of education may both be deficient.39 At present, however, using existing ADNI and NACC data we were unable to account for quality of education. Further work in this area is needed if the field is to fully understand and embrace the impact of education, as well as other sociodemographic factors such as race,40 on the cognitive test instruments used to inform diagnoses and ultimately treatment considerations.41
Conclusion
It is undeniable that the MoCA and other brief cognitive assessments play a vital role in monitoring cognition in older adults, both acutely and over time. Even so, potential educational biases inherent in such measures must be understood and considered when interpreting the results, before making an MCI or dementia diagnosis, and prescribing treatment. Our findings support the use of an alternative MoCA to rectify the disparity in classification accuracy across levels of education found when using the MoCA as published. It should also be noted again that the MoCA (original or alternative) is not a diagnostic instrument, rather 1 among many assessment tools that should be used to make the most accurate diagnosis possible of the individual. While extending this work to other measures is extremely important to advancing education-fair neurocognitive assessment, simple global cognitive tests like the MoCA are a mainstay for many primary care as well as general neurology clinics that serve the majority of aging population at risk for or with cognitive decline. As such, understanding the influence of educational-biases should remain a priority in the field.
Acknowledgments
The NACC database is funded by NIA/NIH Grant U24 AG072122. NACC data are contributed by the NIA-funded ADRCs: P30 AG019610 (PI Eric Reiman, MD), P30 AG013846 (PI Neil Kowall, MD), P50 AG008702 (PI Scott Small, MD), P50 AG025688 (PI Allan Levey, MD, PhD), P50 AG047266 (PI Todd Golde, MD, PhD), P30 AG010133 (PI Andrew Saykin, PsyD), P50 AG005146 (PI Marilyn Albert, PhD), P50 AG005134 (PI Bradley Hyman, MD, PhD), P50 AG016574 (PI Ronald Petersen, MD, PhD), P50 AG005138 (PI Mary Sano, PhD), P30 AG008051 (PI Thomas Wisniewski, MD), P30 AG013854 (PI Robert Vassar, PhD), P30 AG008017 (PI Jeffrey Kaye, MD), P30 AG010161 (PI David Bennett, MD), P50 AG047366 (PI Victor Henderson, MD, MS), P30 AG010129 (PI Charles DeCarli, MD), P50 AG016573 (PI Frank LaFerla, PhD), P50 AG005131 (PI James Brewer, MD, PhD), P50 AG023501 (PI Bruce Miller, MD), P30 AG035982 (PI Russell Swerdlow, MD), P30 AG028383 (PI Linda Van Eldik, PhD), P30 AG053760 (PI Henry Paulson, MD, PhD), P30 AG010124 (PI John Trojanowski, MD, PhD), P50 AG005133 (PI Oscar Lopez, MD), P50 AG005142 (PI Helena Chui, MD), P30 AG012300 (PI Roger Rosenberg, MD), P30 AG049638 (PI Suzanne Craft, PhD), P50 AG005136 (PI Thomas Grabowski, MD), P50 AG033514 (PI Sanjay Asthana, MD, FRCP), P50 AG005681 (PI John Morris, MD), P50 AG047270 (PI Stephen Strittmatter, MD, PhD).
Appendix
Notes
This article was externally peer reviewed.
Funding: There are no sources of funding to report.
Conflicts of interest: There are no conflicting or competing interests to declare.
To see this article online, please go to: http://jabfm.org/content/35/6/1043.full.
- Received for publication March 5, 2022.
- Revision received June 11, 2022.
- Accepted for publication June 14, 2022.
