Research ArticleOriginal Research

Brief Health Literacy Screening Items Predict Newest Vital Sign Scores

Vanessa Stagliano and Lorraine S. Wallace
The Journal of the American Board of Family Medicine September 2013, 26 (5) 558-565; DOI: https://doi.org/10.3122/jabfm.2013.05.130096

Abstract

Background: Numeracy is an important but understudied component of health literacy (HL). The purpose of this study was to examine the predictive ability of established general HL and numeracy screening items in estimating Newest Vital Sign (NVS) scores.

Methods: We studied 241 adults attending primary care clinics in the midwestern United States. Demographic items, HL screening questions, and the NVS were administered to patients. To determine the accuracy of HL and numeracy screening items, area under the receiver operating characteristic (AUROC) curves were determined for each screening item, using NVS scores as the reference standard.

Results: Patients' mean age was 46.1 ± 16.3 years; 71.0% were female, 53.4% were African American, 7.5% had less than a high school education, and 44.4% were insured by Medicaid/Medicare. The mean NVS score was 3.7 ± 2.0, with 17.8% classified as having inadequate HL/numeracy (NVS score of 0 or 1). The HL screening item, “How confident are you filling out medical forms by yourself?” was the best predictor of both limited (AUROC, 0.83; 95% confidence interval [CI], 0.76–0.89) and limited/marginal (AUROC, 0.79; 95% CI, 0.73–0.85) NVS scores. The numeracy screening item, “In general, how easy or hard do you find it to understand medical statistics?” was the best predictor of both limited (AUROC, 0.83; 95% CI, 0.76–0.89) and limited/marginal (AUROC = 0.78; 95% CI, 0.72–0.84) NVS scores.

Conclusion: Brief HL and numeracy screening items are useful for quickly estimating NVS scores among English-speaking primary care clinic populations.

More than one third of American adults have either basic or below basic health literacy (HL) skills.1 Over the past 2 decades, an accumulating evidence base has documented a strong and consistent link between limited HL and an array of poor health-related outcomes.2 Specifically, those with limited HL often lack medical-related knowledge,3 use fewer preventive services,4 are more likely to be hospitalized,5 and experience worse health-related outcomes.6

Numeracy is a key, but largely understudied, component of HL. This is important for at least 2 reasons. First, a large proportion of American adults—more than 50%—have basic or below basic quantitative numeracy skills.1 Second, adequate numeracy skills are required to successfully execute many health-related tasks, including calculating medication dosing intervals7⇓–9; interpreting medication and food labels10,11; deciphering charts (eg, growth, body mass index)12; and weighing the risks and benefits needed to make informed decisions related to health care.13,14

While valid and reliable HL assessment tools are available, several factors preclude their use in routine clinical practice. First, multi-item HL tools such as the Test of Functional Health Literacy in Adults (TOFHLA)15 and Rapid Estimate of Adult Literacy in Medicine (REALM)16 are too lengthy to administer in busy clinical settings.17 Second, specifically testing HL may cause patients to feel shame and/or embarrassment.18,19 For these reasons, brief, one-sentence items have been developed and tested to quickly identify in a non-threatening manner patients who are potentially at risk of having limited general HL and numeracy skills.20⇓⇓⇓⇓–25

To our knowledge, general HL screening items have been validated against both the short (S)-TOFHLA and REALM exclusively.20,23,24 Therefore, to address this gap in the literature, the purpose of this study was to examine the predictive ability of both established general HL20 and numeracy screening items25,26 in estimating Newest Vital Sign (NVS) scores. The NVS, an assessment tool for assessing both general HL and numeracy, requires that a patient interpret information presented on a nutrition label.27 In a previous study, patient comprehension of nutrition labels was found to be highly correlated with both general literacy and numeracy skills.28

Methods

Study Design, Setting, and Recruitment Process

Data for this study were pooled from 2 separate but similar investigations. Recruitment and interview processes for both studies were identical. The first study was nested within a larger patient education evaluation study conducted during the winter of 2012. The second study was nested within a comprehensive HL investigation with patient interviews completed during the summer of 2012. Interviews were conducted at 2 separate primary care clinics, both serving underserved populations in a large midwestern city. Patients received primary care services at one of the clinical sites; therefore, none of the patients participated in the study more than once. The overall combined response rate for both data collection periods was ≈80%. The most commonly cited reasons for refusing to participate were lack of time, lack of interest, or both. The Ohio State University Biomedical Institutional Review Board approved the research and informed consent procedures employed in both studies (nos. 2011H0349 and 2012E300).

Trained research assistants (RAs) (2 undergraduates and one medical student) approached patients immediately following their scheduled appointments. RAs explained the purpose of the study and informed the patient that his or her responses would be anonymous and that his or her name would be entered into a drawing for a $50 gift card to a local supermarket. Patients agreeing to participate were taken to an unoccupied examination room to complete a 5- to 6-minute interview. All recruited patients were older than 18 years of age, spoke English as their first or primary language, and had a scheduled appointment with a physician on the day of sampling. Patients who were acutely ill, had diminished decision-making capacity, or had poor visual acuity were excluded.

Interview Process

After receiving verbal consent from the patient, the RA administered HL and numeracy screening items. First, Chew et al's20 3 HL screening items were administered: (1) “How often do you have problems learning about your medical condition because of difficulty understanding written information?” (always, often, sometimes, occasionally, or never); (2) “How often do you have someone help you read hospital materials?” (always, often, sometimes, occasionally, or never); and (3) “How confident are you filling out medical forms by yourself?” (extremely, quite a bit, somewhat, a little bit, or not at all). Second, 2 subjective numeracy items were asked25,26: (1) “In general, how easy or hard do you find it to understand medical statistics?” (very easy, easy, hard, very hard); and (2) “How much do you agree or disagree with the following statement? In general, I depend on numbers and statistics to help me make decisions about my health” (strongly agree, somewhat agree, somewhat disagree, strongly disagree).

Next, the RA queried about sociodemographic items from the 2010 Behavioral Risk Factor Surveillance Survey,28 including sex, age, race/ethnicity, educational attainment, and health insurance coverage. Last, the RA administered the NVS.27 The NVS consists of 6 questions that require a combination of reading comprehension and manipulation of numerical data to interpret content presented within an ice cream container nutrition label. The likelihood of limited literacy/numeracy is based on the number of correct responses on the NVS: 0 to 1 (limited), 2 to 3 (marginal), and 4 to 6 (adequate). The NVS has been validated in previous studies.29⇓–31

Statistical Analysis

Data were analyzed using SAS software (version 9.2; SAS Inc, Cary, NC). A priori statistical significance was set at P < .05. Descriptive statistical tests were conducted to depict the sociodemographic characteristics of the sample and responses to HL and numeracy screening items and NVS questions. We compared the accuracy of individual HL and numeracy screening items with 2 comparison standards (limited and limited/marginal) based on NVS scores.

Next we computed the sensitivity and specificity of individual HL and numeracy screening items at different response thresholds. Areas under the receiver operating characteristics (ROC) curve (AUROCs), with accompanying 95% confidence intervals [CIs], were calculated while simultaneously controlling for age, sex, and educational attainment to compare the predictive ability of individual HL and numeracy items in estimating limited and limited/marginal NVS scores. The larger the AUROC (range, 0.0–1.0), the more accurate the test.

Results

A total of 241 patients, representing a wide range of ages, race, and formal education, completed study interviews (Table 1). The average age of patients was 46.1 years (standard deviation, 16.3), and 171 (71.0%) were women. There was a relatively even mix of patients identifying themselves as African American (53.4%) and white (46.8%). Eighteen patients had less than a high school education (7.5%), 55 (23.0%) were high school graduates, 87 (36.4%) had completed at least some college, and 79 (33.1%) were college graduates. Sixty-eight (28.2%) patients were insured through Medicaid.

View this table:
Table 1. Sociodemographic Characteristics of the Study Sample (n = 241)

Patient scores on the NVS averaged 3.73 ± 2.02. Distribution of NVS scores is presented in Figure 1. Sensitivities and specificities in detecting limited and limited/marginal NVS scores by response option are presented in Tables 2 and 3, respectively.

Figure 1.
Figure 1.

Distribution of Newest Vital Sign (NVS) scores.

View this table:
Table 2. Performance of Health Literacy and Numeracy Screening Items in Detecting Limited Newest Vital Sign Scores
View this table:
Table 3. Performance of Health Literacy and Numeracy Screening Items in Detecting Limited and Marginal Newest Vital Sign Scores

Figure 2 presents the ROC curves of HL and numeracy screening items identifying patients with limited NVS scores. All HL and numeracy screening items had similar AUROCs for predicting patients with limited NVS scores. Figure 3 presents the ROC curves of HL and numeracy screening items identifying patients with limited/marginal NVS scores. All HL and numeracy screening items had similar AUROCs for predicting those with limited/marginal NVS scores. The HL screening item “How confident are you filling out medical forms by yourself?” was the best predictor of both limited (AUROC, 0.83; 95% CI, 0.76–0.89) and limited/marginal (AUROC, 0.79; 95% CI, 0.73–0.85) NVS scores. The numeracy screening item “In general, how easy or hard do you find it to understand medical statistics?” was the best predictor of both limited (AUROC, 0.83; 95% CI, 0.76–0.89) and limited/marginal (AUROC, 0.78; 95% CI, 0.72–0.84) NVS scores.

Figure 2.
Figure 2.

Receiving operating characteristic (ROC) curves for health literacy (A) and numeracy screening (B) items in detecting limited Newest Vital Sign scores.

Figure 3.
Figure 3.

Receiving operating characteristic (ROC) curves for health literacy (A) and numeracy screening (B) items in detecting limited/marginal Newest Vital Sign scores.

Discussion

Because of the strong association of limited HL and numeracy skills with poor health-related outcomes and knowledge gaps, it would be useful for health care providers to be able to quickly identify patients at risk of limited HL and/or numeracy skills. Our findings build on previous studies that have shown brief screening items to be effective in identifying patients at risk of inadequate HL. In our sample, Chew et al's20 “confident with forms” item was the best predictor of identifying patients with both limited and limited/marginal NVS scores. Our results extend the findings of prior studies in which the “confident with forms” item was the best predictor of identifying patients at risk of low scores on both the S-TOFHLA and REALM.20,24 These findings are important because the NVS assesses a different skill set (numeracy) than either the S-TOFHLA (reading comprehension using the Cloze technique) or REALM (word recognition and pronunciation).

In addition, our findings also demonstrate that one-sentence numeracy questions are effective in identifying patients at risk of limited and/or marginal NVS scores. Both numeracy screening items (“In general, how easy or hard do you find it to understand medical statistics?” and “How much do you agree or disagree with the following statement? In general, I depend on numbers and statistics to help me make decisions about my health.”) were strong predictors of NVS scores in our study. Therefore, these HL and numeracy screening items could be quickly and easily administered in busy clinical settings by health care professionals with various levels of training.

Limitations

Our study findings should be considered within the context of several limitations. First, patients in this study were sampled from 2 primary care clinics located in a large midwestern city. As a result, our findings may not be representative of other more geographically diverse locations. The extent and direction of such bias, however, cannot be determined. Second, inclusion criteria required that English be the patient's primary language. Further study should be undertaken to explore the utility of these HL and numeracy screening items in predicting NVS scores in predominantly non-English-speaking populations as well. Third, this study included a convenience sample of patients who agreed to participate in the study. Fourth, nearly 20% of patients approached at both clinic sites did not agree to participate in the study. Fifth, the overall educational level of the study sample was relatively high.

Conclusions

Previously established HL and numeracy items were useful in identifying patients exhibiting limited and marginal NVS scores. Future work could focus on how to best implement effective strategies for improving physician-patient communication based on screening results, the use of calculators and other aides in addressing limited health numeracy, and the effectiveness of different modes of administration (verbal or written) of screening questions. In addition, studies should be undertaken to assess ways in which the health care system at large can mitigate both HL and numeracy demands placed on patients and those caring for them.

Notes

  • This article was externally peer reviewed.

  • Funding: none.

  • Conflict of interest: none declared.

  • Received for publication March 13, 2013.
  • Revision received June 3, 2013.
  • Accepted for publication June 7, 2013.

References

  1. 1.↵
    1. Kutner M,
    2. Greenberg E,
    3. Jin Y,
    4. Paulsen C
    . The health literacy of America's adults: results from the 2003 National Assessment of Adult Literacy. Washington, DC: National Center for Education Statistics, US Department of Education; 2006. Available from: http://nces.ed.gov/pubs2006/2006483.pdf. Accessed July 13, 2013.
  2. 2.↵
    1. Schwartzberg J,
    2. VanGeest J,
    3. Wang C
    , editors. Understanding health literacy: inspirations for medicine and public health. Chicago, IL: American Medical Association; 2004.
  3. 3.↵
    1. Shibuya A,
    2. Inoue R,
    3. Ohkubo T,
    4. et al
    . The relation between health literacy, hypertension knowledge, and blood pressure among middle-aged Japanese adults. Blood Press Monit 2011;16:224-30.
    OpenUrlPubMed
  4. 4.↵
    1. Pagán JA,
    2. Brown CJ,
    3. Asch DA,
    4. Armstrong K,
    5. Bastida E,
    6. Guerra C
    . Health literacy and breast cancer screening among Mexican American women in South Texas. J Cancer Educ 2012;27:132-7.
    OpenUrlCrossRefPubMed
  5. 5.↵
    1. Mitchell SE,
    2. Sadikova E,
    3. Jack BW,
    4. Paasche-Orlow MK
    . Health literacy and 30-day postdischarge hospital utilization. J Health Commun 2012;17(Suppl 3):325-38.
    OpenUrlPubMed
  6. 6.↵
    1. Sentell T,
    2. Baker KK,
    3. Onaka A,
    4. Braun K
    . Low health literacy and poor health status in Asian Americans and Pacific Islanders in Hawai'i. J Health Commun 2011;16(Suppl 3):279-94.
    OpenUrlCrossRefPubMed
  7. 7.↵
    1. Cavanaugh K,
    2. Huizinga MM,
    3. Wallston KA,
    4. Gebretsadik T,
    5. Shintani A,
    6. Davis D
    . Association of numeracy and diabetes control. Ann Intern Med 2008;148:737-46.
    OpenUrlCrossRefPubMed
  8. 8.↵
    1. Estrada CA,
    2. Martin-Hryniewicz M,
    3. Peek BT,
    4. Collins C,
    5. Byrd JC
    . Literacy and numeracy skills and anticoagulation control. Am J Med Sci 2004;328:88-93.
    OpenUrlCrossRefPubMed
  9. 9.↵
    1. Wallace LS,
    2. Keenum AJ,
    3. DeVoe JE,
    4. Bolon SK,
    5. Hansen JS
    . Women's understanding of different medication dosing instructions for a liquid medication. J Pediatr Health Care 2012;26:443-50.
    OpenUrlPubMed
  10. 10.↵
    1. Lokker N,
    2. Sanders L,
    3. Perrin EM,
    4. et al
    . Parental misinterpretations of over-the-counter pediatric cough and cold medication labels. Pediatrics 2009;123:1464-71.
    OpenUrlAbstract/FREE Full Text
  11. 11.↵
    1. Rothman RL,
    2. Housam R,
    3. Weiss H,
    4. et al
    . Patient understanding of food labels: the role of literacy and numeracy. Am J Prev Med 2006;31:391-8.
    OpenUrlCrossRefPubMed
  12. 12.↵
    1. Oettinger MD,
    2. Finkle JP,
    3. Esserman D,
    4. et al
    . Color-coding improves parental understanding of body mass index charting. Acad Pediatr 2009;9:330-8.
    OpenUrlCrossRefPubMed
  13. 13.↵
    1. Reyna VF,
    2. Nelson WL,
    3. Han PK,
    4. Dieckmann NF
    . How numeracy influences risk comprehension and medical decision making. Psychol Bull 2009;135:943-73.
    OpenUrlCrossRefPubMed
  14. 14.↵
    1. Schwartz LM,
    2. Woloshin S,
    3. Black WC,
    4. Welch HG
    . The role of numeracy in understanding the benefit of screening mammography. Ann Intern Med 2007;127:966-72.
    OpenUrl
  15. 15.↵
    1. Parker RM,
    2. Baker DW,
    3. Williams MV,
    4. Nurss JR
    . The test of functional health literacy in adults: a new instrument for measuring patients' literacy skills. J Gen Intern Med 1995;10:537-41.
    OpenUrlCrossRefPubMed
  16. 16.↵
    1. Davis TC,
    2. Long SW,
    3. Jackson RH,
    4. et al
    . Rapid estimate of adult literacy in medicine: s shortened screening instrument. Fam Med 1993;25:391-5.
    OpenUrlPubMed
  17. 17.↵
    1. Powers BJ,
    2. Trinh JV,
    3. Bosworth HB
    . Can this patient read and understand written health information? JAMA 2010;304:76-84.
    OpenUrlCrossRefPubMed
  18. 18.↵
    1. Parikh NS,
    2. Parker RM,
    3. Nurss JR,
    4. Baker DW,
    5. Williams MV
    . Shame and health literacy: the unspoken connection. Patient Educ Couns 1996;27:33-9.
    OpenUrlCrossRefPubMed
  19. 19.↵
    1. Wolf MS,
    2. Williams MV,
    3. Parker RM,
    4. Parikh NS,
    5. Nowlan AW,
    6. Baker DW
    . Patients' shame and attitudes toward discussing the results of literacy screening. J Health Commun 2007;12:721-32.
    OpenUrlCrossRefPubMed
  20. 20.↵
    1. Chew LD,
    2. Bradley KA,
    3. Boyko EJ
    . Brief questions to identify patients with inadequate health literacy. Fam Med 2004;36:588-94.
    OpenUrlPubMed
  21. 21.↵
    1. Fagerlin A,
    2. Zikmund-Fisher BJ,
    3. Ubel PA,
    4. Jankovic A,
    5. Derry HA,
    6. Smith DM
    . Measuring numeracy without a math test: development of the subjective numeracy scale. Med Decis Making 2007;27:672-80.
    OpenUrlAbstract/FREE Full Text
  22. 22.↵
    1. Lipkus IM,
    2. Samsa G,
    3. Rimer BK
    . General performance on a numeracy scale among highly educated samples. Med Decis Making 2001;21:37-44.
    OpenUrlAbstract/FREE Full Text
  23. 23.↵
    1. Sarkar U,
    2. Schillinger D,
    3. Lopez A,
    4. Sudore R
    . Validation of self-reported health literacy questions among diverse English and Spanish-speaking populations. J Gen Intern Med 2010;26:265-71.
    OpenUrlPubMed
  24. 24.↵
    1. Wallace LS,
    2. Rogers ES,
    3. Roskos SE,
    4. Holiday DB,
    5. Weiss BD
    . Screening items to identify patients with limited health literacy skills. J Gen Intern Med 2006;21:874-7.
    OpenUrlCrossRefPubMed
  25. 25.↵
    1. Woloshin S,
    2. Schwartz LM,
    3. Welch HG
    . Patients and medical statistics: interest, confidence and ability. J Gen Intern Med 2005;20:996-1000.
    OpenUrlCrossRefPubMed
  26. 26.↵
    National Cancer Institute, U.S. National Institutes of Health. Health Information National Trends Survey (HINTS). Available from: http://hints.cancer.gov/. Accessed March 12, 2013.
  27. 27.↵
    1. Weiss BD,
    2. Mays MZ,
    3. Martz W,
    4. et al
    . Quick assessment of literacy in primary care: the newest vital sign. Ann Fam Med 2005;3:514-22.
    OpenUrlAbstract/FREE Full Text
  28. 28.↵
    Centers for Disease Control and Prevention. Behavioral Risk Factor Surveillance System. Available from: http://www.cdc.gov/brfss/. Accessed March 12, 2013.
  29. 29.↵
    1. Osborn CY,
    2. Weiss BD,
    3. Davis TC,
    4. et al
    . Measuring adult literacy in health care: performance of the newest vital sign. Am J Health Behav 2007;31(Suppl 1):S36-46.
    OpenUrlCrossRefPubMed
  30. 30.↵
    1. Patel PJ,
    2. Joel S,
    3. Rovena G,
    4. et al
    . Testing the utility of the newest vital sign (NVS) health literacy assessment tool in older African-American patients. Patient Educ Couns 2011;85:505-7.
    OpenUrlPubMed
  31. 31.↵
    1. Shah LC,
    2. West P,
    3. Bremmeyr K,
    4. Savoy-Moore RT
    . Health literacy instrument in family medicine: the “Newest Vital Sign” ease of use and correlates. J Am Board Fam Med 2010;23:195-203.
    OpenUrlAbstract/FREE Full Text
Back to top

In this issue

Print
Download PDF
Article Alerts
Email Article
Citation Tools

Jump to section

Keywords