Inaccurate Risk Perceptions and Individualized Risk Estimates by Patients with Type 2 Diabetes
==============================================================================================

* Barry G. Saver
* Kathleen M. Mazor
* J. Lee Hargraves
* Marcela Hayes

## Abstract

*Background:* We evaluated how diabetic patients understand and respond to the presentation of personalized risk information.

*Methods:* This was a mixed methods study involving 56 patients with type 2 diabetes and at least 1 additional cardiovascular risk factor. We assessed participants' perceptions of diabetes-related risks; asked them to rank order 6 events (death, heart attack, stroke, blindness, amputation, and kidney failure) by likelihood of occurrence in a specified time frame; presented them with personalized risk estimates; and asked them to re-rank the risks. The final 18 participants were tested to verify understanding before re-ranking risks. Qualitative analysis of interview transcripts identified themes and concepts underlying participants' ways of perceiving and reacting to risk.

*Results:* While mortality was the most likely outcome for almost all participants, nearly all estimated it to be least likely; only 28% adjusted their mortality rankings to match model predictions. Some did not understand the risk information: only two thirds of those asked could rank risks according to the information presented. Risk perceptions were influenced by factors including “knowing myself,” powerful anecdotes, and belief that a “warning shot” would occur before death.

*Conclusions:* Personalized risk estimates, particularly about mortality, had limited salience. Some participants could not understand the information, despite presentation in ways suggested by previous research.

*   Communication
*   Decision Making
*   Risk
*   Type 2 Diabetes Mellitus

Patients with chronic health conditions face decisions about how to prioritize their efforts to maintain or improve health, typically with little information about their risks and how different actions can affect these risks. Multimorbidity makes this even more difficult; little research provides guidance for patients or their providers attempting to address multiple conditions simultaneously.1 Diabetes mellitus is a prototypical condition for this conundrum, affecting virtually all parts of the body and frequently being accompanied by comorbid conditions such as hypertension and dyslipidemia. Type 2 diabetes treatment has historically been very glucocentric,2,3 despite the fact that premature morbidity and mortality are more strongly affected by cardiovascular risk factors than glucose control.4

If patients with type 2 diabetes are to make informed choices about focusing their risk reduction efforts, access to accurate, personalized risk/benefit information is a logical first step. The UK Prospective Diabetes Study (UKPDS) Outcomes Model, based on observation of 3642 participants in the UKPDS, uses a system of equations derived from observation of the UKPDS population and based on time-varying risk factors to predict individual risks over up to 40 subsequent years of multiple outcomes for people with diabetes and the effects of risk reduction activities.5 There is limited published research on approaches to and effects of communication of personalized cardiovascular risk information; a review suggested that using percentages or frequencies, graphic representations, and short time frames may better induce behavior change.6 Learning how to help patients with type 2 diabetes understand risk information should also translate to interventions for other patients with multiple conditions and multiple possible risk reduction activities.

We report here findings of exploratory work examining the knowledge and attitudes of patients with type 2 diabetes and at least one other major cardiovascular risk factor as they relate diabetes-related risks and understanding of and reactions to presentation of personalized risk information produced by the UKPDS Outcomes Model.

## Methods

### Participant Recruitment

Participants were English- and Spanish-speaking adults with type 2 diabetes and at least one additional cardiovascular risk factor: hypertension, hyperlipidemia, body mass index >30 kg/m2 (all validated by chart review), or self-reported smoking. Participants were recruited from an family medicine clinic affiliated with an academic health center and a teaching community health center. We recruited subjects using (1) posters in the waiting room of the academic clinic and (2) mailed invitations to patients identified via electronic data from both clinics. With verbal consent, interested patients were screened by telephone to assess eligibility and obtain data needed to calculate personalized risk estimates using the UKPDS Outcomes Model7: length of time they had been diagnosed with diabetes and if/when they had been diagnosed with a heart attack, stroke, congestive heart failure, coronary artery disease, blindness due to diabetes, kidney failure, or amputation. With verbal consent, we obtained from participants' electronic health records hemoglobin A1c concentration, total and high-density lipoprotein cholesterol levels, in-clinic blood pressure readings, and weight. This project was approved by the University of Massachusetts Medical School's institutional review board.

### Generating Personalized Risk Estimates

The UKPDS Outcomes Model was developed from a population of patients with newly diagnosed diabetes and asks for risk factor levels at the time of diagnosis, information typically unavailable in practice (such as blood pressure, cholesterol, hemoglobin A1c, and weight), as well as current risk factor levels. We used the oldest risk factor values found in participants' electronic records within the time period they reported having diabetes. Individual risk estimates were generated with the UKPDS Outcomes Model7 using 10,000 loops to obtain stable estimates. Because short-term risks tend to be small and benefits of risk reduction activities accrue slowly over time, we chose to create risk projections over moderately long time frames since our goal was to develop an intervention based on presenting projections of benefits from risk reduction activities. In general, we calculated 20-year risk projections for participants <50 years old, 10-year projections for those aged 50 to 69 years, and 5-year projections for those ≥70 years old. These 3 time periods were based on our subjective judgments about what would be reasonable time frames for patients in those age groups that would have high enough event probabilities for risks and risk reduction estimates to be meaningful but mortality projections that are not so high as to indicate mortality as a near certainty.

### Study Visits

Interview activities are outlined in Table 1. After providing written, informed consent, participants supplied demographic information and completed brief assessments of literacy8,9 and numeracy.10 Based on our experiences with the first 38 participants, we modified our procedures, seeking better verification of participant understanding of the quantitative risk estimates. Therefore, the first 38 participants are labeled the “initial group” and the final 18 participants the “verification group.” Interviews were conducted in English or Spanish by one bilingual investigator (MH). All interviews were recorded and transcribed.

View this table:
[Table 1.](http://www.jabfm.org/content/27/4/510/T1)

Table 1. Overview of Study Procedures

Sample probe questions are shown in Appendix 1. Risk perceptions were a primary focus of the interviews. We did not expect most participants to have a quantitative understanding of the magnitude of their risks, so we asked them to sort 6 cards labeled “death,” “heart attack,” “stroke,” “blindness,” “kidney failure,” and “losing a foot or leg” based on how likely they thought each outcome was to happen to them.

Each participant then was presented his or her personalized risk estimates. We did not anticipate model predictions of extremely high mortality risks. We became concerned about presenting to participants estimates that might be overly distressing and modified our protocol, censoring very high projected mortality risks (initially, starting at approximately >75% and decreasing over time to >50%). For patients younger than 70, we also used estimates for shorter time frames to allow an estimate that was less potentially distressing to be presented. As a consequence of the censoring, only 39 participants viewed mortality projections; the 17 whose mortality risks were censored were not asked to rank mortality with their other risks. In addition, the UKPDS Outcomes Model does not produce risk estimates for recurrent events, so a participant with a history of a heart attack, for example, would not have been presented any risk estimate for that outcome.

Initial group participants were randomized to have risks presented as either a set of horizontal bar graphs (on one page) or a set of crowd charts (on separate pages), with a 10 × 10 array of items representing the relevant outcome and the probability of experiencing the outcome represented by shading the relevant number of figures, to study whether one of these evidence-supported representations6,11⇓⇓⇓⇓–16 seemed to be preferable. For mortality, we developed an array of pictures of 100 people representing a broad range of ages and racial/ethnic backgrounds. Other crowd charts used arrays of hearts, brains, eyes, kidneys, and feet. In both cases, the graphic information also was presented verbally as the number of people out of 100 expected to experience the outcome in the given time frame. After first reviewing one of the randomly chosen formats, we presented and reviewed the other risk representation. We asked participants whether they found the information easy or hard to understand, if it made sense, what they understood from it, and which format they preferred.

Next, we gave initial group participants a second set of 6 cards and asked them to rank their risks for these outcomes again, with their original ranking and our risk representations in view on the table. We asked them to discuss their thinking and probed about changes from their initial rankings and/or differences from the model predictions.

For the verification group, we sought clearer evidence that participants understood the computer-generated risk estimates. Participants were asked to rank the outcomes according to perceived risk, and then the bar charts were presented. If they did not seem to understand the risk information, we presented the crowd charts. If they stated they understood the risk estimates, we gave them a second set of cards and asked them to order the cards according to the computer-generated risk estimates, regardless of their personal feelings about their risks. If they could not do this accurately, we presented the crowd charts if we had not already done so and, in either case, reviewed the risk estimates. If they were able to order the cards according to the computer-generated estimates, they were given a third set of cards and, with both their original ranking and their ranking according to the computer-generated estimates in view, asked to rank the risks again based on their current thinking and then discuss their thinking.

### Analyses

Quantitative data were entered in Epi-Info (Centers for Disease Control and Prevention, Atlanta, GA) and analyzed using SAS version 9.2 (SAS Institute, Cary, NC). We used a general inductive approach17 to the qualitative analysis. Four team members (BGS, KMM, LH, and MH) reviewed the same 2 transcripts, discussed them, and developed an initial coding framework around personal health beliefs and current health-related behaviors. This framework was expanded and revised by reading, analyzing, and discussing 5 additional transcripts, at which point we felt our coding scheme was sufficiently developed and stable for independent coding. Adding 2 coders, we continued to inductively develop our coding structure through reading and coding transcripts and discussing coding and content. We developed a face sheet for recording agreement on the coding of main themes and major subthemes. After this, every fifth transcript was double-coded to check for consistency. Mean agreement between coders was 86%. Since we did not find substantive differences between English-language and Spanish-language interviews, we present here only quotations from English-language interviews.

## Results

### Participants

Participant characteristics are shown in Table 2. The initial and verification groups were quite comparable except the verification group had no participants who primarily spoke Spanish since we had targeted this group in our early recruitment to maximize representation.

View this table:
[Table 2.](http://www.jabfm.org/content/27/4/510/T2)

Table 2. Participant Characteristics, Including Comorbid Cardiovascular Risk Prevalence and Risk Factor Levels

### Risk Rankings

Figure 1 shows the mean rankings for each of the 6 outcomes, with 1 corresponding to the most likely and 6 to the least likely outcome, along with the mean probability estimates presented to participants. The greatest discrepancy between participants' initial rankings and the UKPDS-predicted rankings was found for death, which was estimated to be the most likely outcome for nearly all participants by the model but the least likely outcome by nearly all participants. After seeing their personalized risk projections, participants' rankings for death increased modestly but still remained markedly different from the model predictions. Participants tended to rank heart attack and stroke as more likely than blindness, amputation, and renal failure. While >90% of participants stated that they understood the risk data, only 12 of the 18 participants in the verification group were able to rank-order risk cards to match the model data. Members of the verification group who could not rank cards based on the presented probabilities had lower mean numeracy and Short Test of Functional Literacy in Adults scores than those who could (1.5 vs 4.0 [*P* = .02, unpooled *t* test] and 22.0 vs 32.5 [*P* = .08, pooled *t* test], respectively).

![Figure 1.](http://www.jabfm.org/https://www.jabfm.org/content/jabfp/27/4/510/F1.medium.gif)

[Figure 1.](http://www.jabfm.org/content/27/4/510/F1)

Figure 1. 
Mean outcome rankings and probabilities. MI, myocardial infarction; UKPDS, UK Prospective Diabetes Study.

Figure 2 shows initial rankings and change in rankings for the 39 participants who viewed mortality predictions. Of these, 29 participants (74%) initially ranked death as their least likely outcome. While 20 of 39 participants increased their ranking of death as an outcome after seeing their risk estimates, only 11 matched the model predictions for the ranking of mortality. The change in ranking and magnitude of the estimated mortality risk were not correlated (*r* = −0.03; *P* = not significant). Positive change in rank values for 3 participants means that they reranked death as less likely than their original ranking, contrary to the model predictions. Of the 12 participants in the verification group who were able to match the model predictions, only 2 reordered their risk rankings to exactly match the model predictions; 2 others produced similar, but not identical, rankings. The participants in the initial group were about equally split for preferred mode of representation (unrelated to order of presentation); 5 preferred seeing both and 2 stated neither was helpful.

![Figure 2.](http://www.jabfm.org/https://www.jabfm.org/content/jabfp/27/4/510/F2.medium.gif)

[Figure 2.](http://www.jabfm.org/content/27/4/510/F2)

Figure 2. 
Change in ranking of mortality risk, sorted by initial ranking. *The initial ranking was categorized as 1 = most likely to 6 = least likely.

### Participants' Understanding of Health Risks

#### Personal Risk Perception

Participants expressed a variety of ways in which they came to an understanding of their diabetes-related health risks, including self-knowledge, information from health care providers, family history as an indicator of vulnerability, and vicarious experience through others. About 75% of participants described “knowing myself” as an important way they came to understand their risks. This could include feeling the computer model was wrong about them, even if it was correct in general. “I think the model is incorrect because I know I am going to last longer than 10 years, because I do take care of myself” (participant B101); “…because I know my health better than I think some statistics show…” (participant B107); “Well it looked right [referring to our risk information]. It looked good, but … the picture I had is [better]” (participant F132); and “that's why I go by what my body experiences; besides the doctor, you are the one who knows how your body functions” (participant F148).

About 60% of participants reported that family history strongly influenced their perceptions of risk. For example, “I think like just family things, because a lot of my family had diabetes. And this is the way it went. Before my dad died he had a heart attack and a stroke and had his leg amputated, so…” (participant B109). Family history might increase perceived risk even with evidence of current good health: “I just had a stress test and they said my arteries were okay, but I think that that's right now one of my biggest, because it also runs in the family. My mother had a heart attack” (participant F153). But family history could also lead to a perception of decreased risk: “No, and I guess part of it is because in my family … there is no heart illnesses [sic] in my family at all on either side and so it's not something that I really worry about” (participant B123). Vicarious experience through people other than a blood relative could also lead to concern or a sense of vulnerability. One participant related, “And that may be more important to me because my husband had a heart attack about 4 years ago. So it's constantly on my mind even though he is not blood, but still I went through it with him” (participant B102).

#### Risk Controllability and Warning Shots

Virtually all participants endorsed the idea that they could reduce or avoid some risks. This could extend to a feeling of being immune from certain risks. Referring to his risks of heart attack, stroke, and kidney failure, one participant (B103) stated, “I think none of these could happen because of my diet.” Another stated, “I'm basically not worried about anything happening to me because of diabetes as long as I take care of myself” (participant F130). However, almost half of participants also expressed opinions that some risks—most often mortality—were not controllable. For example, “… when the man tells me it's time to punch my clock, it's time to punch your clock” (participant B127).

Over 70% of participants described a belief that a “warning shot” would occur before a more serious event such as death. For example, “I don't think there is a risk of death unless I have one of these 2 (heart attack or stroke)” (participant B120). Another participant stated, “And stroke because, again, of all the medical things, to me the kidney failure, the stroke, and a heart attack, which would be next, would lead up to my death. I don't … and, again, this is not being educated … I don't see diabetes killing me unless these other things fall into place” (participant F152). However, some participants acknowledged not paying attention even when they experienced a “warning shot”: “I mean I do have a lot of chest pains and I ignore them…. Many people tell me, ‘You are not supposed to ignore that. When you feel like that, go to the hospital no matter what’” (participant F148).

#### Information Sources and Sufficiency

Around 80% of participants reported that having information about diabetes and health risks was important to them, 60% identified health care providers as a source of risk information, and 40% expressed feeling they had sufficient information to make health decisions, though this did not exclude wanting to know more: “I mean I do have enough, but I think I need more information” (participant F148). Approximately 20% of participants explicitly stated feeling they did not have sufficient information about diabetes. One said, “It was just, ‘We've got to get your diet right.’ It's not, ‘This could happen to you if you keep this up; this could happen if you keep that up'” (participant F133).

#### Motivations for Behavior Change

The most common reasons for engaging in healthy behaviors, cited by around 60% of participants, related to staying healthy, improving quality of life, and feeling better as a result of engaging in some of the behaviors; for example, “Because once I do good, eat right and drink my water, exercise, I notice the tingling stops and I feel better about myself” (participant F133). The second most common reason, cited by almost half of participants, was a doctor's recommendation: “…your doctor says, ‘Do it 3 times a day,’ you do it 3 times a day” (participant F143). Feeling a sense of control over one's health was cited by almost 40% and needing to stay healthy for one's family by about 20%.

Specific events triggering behavior change were reported less commonly. “Warning shot” events and a specific instance of provider urging each were cited by about 20% of participants, 5% reported that receiving their diagnosis of diabetes had motivated behavior changes, and <4% stated that any biomedical information had triggered a behavior change. Less than 40% of participants felt the information we presented would motivate change, though almost 80% felt some or all the data did apply to them personally.

## Discussion

While our participants generally understood that cardiovascular events were more likely than events related to microvascular disease, nearly all underestimated their mortality risk, and many rejected the mortality estimates from the UKPDS Outcomes Model. Many participants believed that that self-awareness and occurrence of “warning shots” would give them time to change behaviors. Other information, particularly family history, seemed to be substantially more powerful in shaping beliefs than model estimates. In addition, participants viewed preventive behaviors as more effective and health risks as more controllable than they frequently are. Similar beliefs have been noted in other contexts, such as cancer prevention and screening.18

A substantial proportion of our participants did not understand personalized risk information when presented in 2 different formats suggested by previous literature6,11⇓⇓–14,16; some did not understand even the order of risks indicated by their personalized data. Few reported that specific biomedical information had motivated behavior change in the past. These findings do not encourage us to think that personalized, quantitative risk information will help type 2 diabetic patients adopt and maintain healthy behaviors.

Others have reported analogous findings for patient reactions to quantitative risk estimates. Goodyear-Smith et al19 reported that patients preferred relative over absolute cardiovascular risk reduction information, and a majority preferred a doctor's opinion to any risk information. A review of interventions using global coronary risk information found modest reductions in predicted risk with longitudinal exposure to risk information and/or counseling but not from single exposures.20 A Cochrane review of personalized risk information for screening decisions concluded there was weak evidence for a small effect; numerical risk estimates possibly decreased test uptake.16 A randomized controlled trial comparing personalized to standard risk factor education found no effects of either on risk reduction behaviors.21 One study found that presenting “lung age” to patients improved smoking cessation rates, but the effect was unrelated to whether or not the “lung age” was abnormal,22 suggesting that understanding the data was not the motivating factor. A randomized controlled trial among patients with diabetes found that more accurate risk perceptions observed 2 weeks after presentation of risk information disappeared by 12 weeks.23 A study comparing delivery of personalized to nonpersonalized risk estimates among people with prediabetes found no appreciable effect on risk perception or behavior.24

While some studies suggested that short-term risk figures may be preferred or better processed by patients,25,26 we chose a longer time frame because we hoped to develop an intervention using increases in life expectancy from projected risk reduction activities to motivate behavior change. Short-term predicted gains were negligible, and even with 10- and 20-year projections, estimated benefits often were only a few additional months of life expectancy, making them of questionable utility for motivating behavior change.

These interviews also suggest that anecdotal information with strong personal salience, such as personal and family history, influences risk perceptions for many patients, consistent with a number of other studies27 and some health behavior models.28 While providers typically rely on biomedical information, many patients may either not understand or not accept it. One study of women's perception of stroke risk concluded that it was primarily subjective, related to neither knowledge nor presence of risk factors.29 Providers who are aware of patients' beliefs and preferences may be better able to communicate effectively about risk, for instance, drawing on narrative information or anecdotes to illustrate possible outcomes.30⇓–32 Direct provider recommendation may also produce change, as reported by some participants in this study.

This study has a number of limitations. First, it is based on a modest sample of participants from 2 practice settings in one US city. Second, we prefaced presentation of the risk data with a statement that these were estimates with some uncertainty, although their rough magnitude should be correct. This may have made it easier for participants to discount the risk predictions, but we felt a disclaimer was necessary given limitations in the accuracy of individual risk predictions.33,34 Furthermore, a number of assumptions had to be made to use the UKPDS Outcomes Model since it was based on a cohort of people in the United Kingdom newly diagnosed with diabetes and asks for data about risk factor levels at the time of diagnosis that were not available to us and are frequently unavailable in practice; this made the estimates for our participants less accurate but would not have affected their understanding of or reactions to the risk information. Third, we modified our procedures over the course of this exploratory study in response to what we were observing, so all participants did not receive the same intervention. Fourth, we chose not to ask participants for absolute risk estimates because we wanted a risk estimation task we expected all participants could complete, limiting our ability to analyze changes in subjects' quantitative risk perceptions.

## Conclusions

Thus, our findings suggest that providing personalized, quantitative risk information is unlikely to help patients manage their diabetes-related risks. We identified a number of reasons for this, including difficulty understanding risk information, personal risk perceptions that overpowered information from model predictions, having a sense of immunity based on current risk reduction activities, and expectations that a “warning shot” will occur before a more serious event. The importance of anecdotes suggests that interventions that exploit the power of narratives deserve evaluation for helping patients make important behavior changes. Finally, the role of the physician's recommendation should not be overlooked, even if it does not fit the ideal of informed, shared decision making.

## Acknowledgments

We are grateful to Heather-Lyn Haley, PhD, and Pearl Houghteling for their assistance with the qualitative analyses. We also acknowledge the University of Oxford for providing a free research license to use the UKPDS Outcomes Model software.

## Appendix 1

#### Sample Probe Questions

*   What things are you are doing now for your health, related to diabetes and other related health conditions?

*   *For each, ask:* Why did you choose it? What do you think it is doing for you? What is the benefit to doing that?

*   I noticed you are taking some medications for \_\_\_|\\_\_\_|\\___|. Do you have any guess as to how it may be helping your health? Is that related to your diabetes?

*   What do you know about your risks of different health risks related to diabetes?

*   Where do you get information about this?

*   Have your doctors ever talked with you about likely you are to have different health problems, such as a heart attack or blindness, related to your diabetes?

*   *If yes, ask:* Do remember what they said? Did they compare how big different risks were? Did you find it helpful?

*   *If no, ask:* Do you think it would have been helpful if they had?

*   Does knowing about risks play a role in choosing what you do?

*   Do you feel you have enough information about diabetes and related health problems to make good decisions about what to do for your health?

*   *If no, ask:* What things do you feel you need more information about?

## Notes

*   This article was externally peer reviewed.

*   *Funding:* This work was funded by the Robert Wood Johnson Foundation (grant no. 63824).

*   *Conflict of interest:* none declared.

*   Received for publication February 7, 2014.
*   Revision received March 26, 2014.
*   Accepted for publication March 31, 2014.

## References

1.  1. Smith SM, Soubhi H, Fortin M, Hudon C, O'Dowd T. Interventions for improving outcomes in patients with multimorbidity in primary care and community settings. Cochrane Database Syst Rev 2012;(4):CD006560.
    
    

2.  2. Brown LC, Johnson JA, Majumdar SR, Tsuyuki RT, McAlister FA. Evidence of suboptimal management of cardiovascular risk in patients with type 2 diabetes mellitus and symptomatic atherosclerosis. CMAJ 2004;171:1189–92.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMToiMTcxLzEwLzExODkiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzI3LzQvNTEwLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

3.  3. Grant RW, Cagliero E, Murphy-Sheehy P, Singer DE, Nathan DM, Meigs JB. Comparison of hyperglycemia, hypertension, and hypercholesterolemia management in patients with type 2 diabetes. Am J Med 2002;112:603–9.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1016/S0002-9343(02)01103-8&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=12034408&link_type=MED&atom=%2Fjabfp%2F27%2F4%2F510.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000175917700001&link_type=ISI) 

4.  4. Hemmingsen B, Lund SS, Gluud C, et al. Targeting intensive glycaemic control versus targeting conventional glycaemic control for type 2 diabetes mellitus. Cochrane Database Syst Rev 2011;(6):CD008143.
    
    

5.  5. Clarke PM, Gray AM, Briggs A, et al. A model to estimate the lifetime health outcomes of patients with type 2 diabetes: the United Kingdom Prospective Diabetes Study (UKPDS) outcomes model (UKPDS no. 68). Diabetologia 2004;47:1747–59.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1007/s00125-004-1527-z&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=15517152&link_type=MED&atom=%2Fjabfp%2F27%2F4%2F510.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000225047900016&link_type=ISI) 

6.  6. Waldron CA, van der Weijden T, Ludt S, Gallacher J, Elwyn G. What are effective strategies to communicate cardiovascular risk information to patients? A systematic review. Patient Educ Couns 2011;82:169–81.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1016/j.pec.2010.04.014&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=20471766&link_type=MED&atom=%2Fjabfp%2F27%2F4%2F510.atom) 

7.  7. University of Oxford Diabetes Trials Unit and Health Economics Research Centre. UKPDS outcomes model user manual, version 1.3. Oxford: ISIS Innovation Ltd; 2010. Available from: [http://www.dtu.ox.ac.uk/outcomesmodel/UKPDSOutcomesManual.pdf](http://www.dtu.ox.ac.uk/outcomesmodel/UKPDSOutcomesManual.pdf). Accessed May 22, 2014.
    
    

8.  8. Baker DW, Williams MV, Parker RM, Gazmararian JA, Nurss J. Development of a brief test to measure functional health literacy. Patient Educ Couns 1999;38:33–42.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1016/S0738-3991(98)00116-5&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=14528569&link_type=MED&atom=%2Fjabfp%2F27%2F4%2F510.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000082751500005&link_type=ISI) 

9.  9. Nurss JR, Parker RM, Williams MV, Baker DW. Test of functional literacy in adults, 2nd ed. Snow Camp, NC: Peppercorn Books and Press; 2001.
    
    

10. 10. Lipkus IM, Samsa G, Rimer BK. General performance on a numeracy scale among highly educated samples. Med Decis Making 2001;21:37–44.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3BtZG0iO3M6NToicmVzaWQiO3M6NzoiMjEvMS8zNyI7czo0OiJhdG9tIjtzOjIwOiIvamFiZnAvMjcvNC81MTAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

11. 11. Garcia-Retamero R, Galesic M. Communicating treatment risk reduction to people with low numeracy skills: a cross-cultural comparison. Am J Public Health 2009;99:2196–202.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.2105/AJPH.2009.160234&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=19833983&link_type=MED&atom=%2Fjabfp%2F27%2F4%2F510.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000272503700019&link_type=ISI) 

12. 12. Nelson W, Reyna VF, Fagerlin A, Lipkus I, Peters E. Clinical implications of numeracy: theory and practice. Ann Behav Med 2008;35:261–74.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1007/s12160-008-9037-8&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=18677452&link_type=MED&atom=%2Fjabfp%2F27%2F4%2F510.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000258554500002&link_type=ISI) 

13. 13. Goodyear-Smith F, Arroll B, Chan L, Jackson R, Wells S, Kenealy T. Patients prefer pictures to numbers to express cardiovascular benefit from treatment. Ann Fam Med 2008;6:213–7.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6NzoiNi8zLzIxMyI7czo0OiJhdG9tIjtzOjIwOiIvamFiZnAvMjcvNC81MTAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

14. 14. Lipkus IM. Numeric, verbal, and visual formats of conveying health risks: suggested best practices and future recommendations. Med Decis Making 2007;27:696–713.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3BtZG0iO3M6NToicmVzaWQiO3M6ODoiMjcvNS82OTYiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzI3LzQvNTEwLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

15. 15. Edwards A, Elwyn G, Mulley A. Explaining risks: turning numerical data into meaningful pictures. BMJ 2002;324:827–30.
    
    [FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMjQvNzM0MS84MjciO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzI3LzQvNTEwLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

16. 16. Edwards AG, Evans R, Dundon J, Haigh S, Hood K, Elwyn GJ. Personalised risk communication for informed decision making about taking screening tests. Cochrane Database Syst Rev 2006;(4):CD001865.
    
    

17. 17. Thomas DR. A general inductive approach for analyzing qualitative evaluation data. Am J Eval 2006;27:237–46.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1177/1098214005283748&link_type=DOI) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000237660500007&link_type=ISI) 

18. 18. Mazor KM, Calvi J, Cowan R, et al. Media messages about cancer: what do people understand? J Health Commun 2010;15(Suppl 2):126–45.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1080/10810730.2010.499983&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=20845199&link_type=MED&atom=%2Fjabfp%2F27%2F4%2F510.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000281851600014&link_type=ISI) 

19. 19. Goodyear-Smith F, Kenealy T, Wells S, Arroll B, Horsburgh M. Patients' preferences for ways to communicate benefits of cardiovascular medication. Ann Fam Med 2011;9:121–7.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6NzoiOS8yLzEyMSI7czo0OiJhdG9tIjtzOjIwOiIvamFiZnAvMjcvNC81MTAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

20. 20. Sheridan SL, Viera AJ, Krantz MJ, et al. The effect of giving global coronary risk information to adults: a systematic review. Arch Intern Med 2010;170:230–9.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1001/archinternmed.2009.516&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=20142567&link_type=MED&atom=%2Fjabfp%2F27%2F4%2F510.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000274291100002&link_type=ISI) 

21. 21. Powers BJ, Danus S, Grubber JM, Olsen MK, Oddone EZ, Bosworth HB. The effectiveness of personalized coronary heart disease and stroke risk communication. Am Heart J 2011;161:673–80.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1016/j.ahj.2010.12.021&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=21473965&link_type=MED&atom=%2Fjabfp%2F27%2F4%2F510.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000289190500009&link_type=ISI) 

22. 22. Parkes G, Greenhalgh T, Griffin M, Dent R. Effect on smoking quit rate of telling patients their lung age: the Step2quit randomised controlled trial. BMJ 2008;336:598–600.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMzYvNzY0NC81OTgiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzI3LzQvNTEwLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

23. 23. Welschen LM, Bot SD, Kostense PJ, et al. Effects of cardiovascular disease risk communication for patients with type 2 diabetes on risk perception in a randomized controlled trial: the @RISK study. Diabetes Care 2012;35:2485–92.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZGlhY2FyZSI7czo1OiJyZXNpZCI7czoxMDoiMzUvMTIvMjQ4NSI7czo0OiJhdG9tIjtzOjIwOiIvamFiZnAvMjcvNC81MTAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

24. 24. Harle CA, Downs JS, Padman R. Effectiveness of personalized and interactive health risk calculators: a randomized trial. Med Decis Making 2012;32:594–605.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3BtZG0iO3M6NToicmVzaWQiO3M6ODoiMzIvNC81OTQiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzI3LzQvNTEwLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

25. 25. Asimakopoulou KG, Fox C, Spimpolo J, Marsh S, Skinner TC. The impact of different time frames of risk communication on type 2 diabetes patients' understanding and memory for risk of coronary heart disease and stroke. Diabet Med 2008;25:811–7.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1111/j.1464-5491.2008.02473.x&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=18644068&link_type=MED&atom=%2Fjabfp%2F27%2F4%2F510.atom) 

26. 26. Hill S, Spink J, Cadilhac D, et al. Absolute risk representation in cardiovascular disease prevention: comprehension and preferences of health care consumers and general practitioners involved in a focus group study. BMC Public Health 2010;10:108.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=20199692&link_type=MED&atom=%2Fjabfp%2F27%2F4%2F510.atom) 

27. 27. Walter FM, Emery J, Braithwaite D, Marteau TM. Lay understanding of familial risk of common chronic diseases: a systematic review and synthesis of qualitative research. Ann Fam Med 2004;2:583–94.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6NzoiMi82LzU4MyI7czo0OiJhdG9tIjtzOjIwOiIvamFiZnAvMjcvNC81MTAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

28. 28. 1.  Cameron LD, 
    2.  Leventhal H
    
    Leventhal H, Brissette I, Leventhal EA. The common-sense model of self-regulation of health and illness. In: Cameron LD, Leventhal H , eds. The self-regulation of health and illness behaviour. New York: Routledge; 2003:42–66.
    
    

29. 29. Dearborn JL, McCullough LD. Perception of risk and knowledge of risk factors in women at high risk for stroke. Stroke 2009;40:1181–6.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjk6IjQwLzQvMTE4MSI7czo0OiJhdG9tIjtzOjIwOiIvamFiZnAvMjcvNC81MTAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

30. 30. Mazor KM, Baril J, Dugan E, Spencer F, Burgwinkle P, Gurwitz JH. Patient education about anticoagulant medication: is narrative evidence or statistical evidence more effective? Patient Educ Couns 2007;69:145–57.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1016/j.pec.2007.08.010&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=17942268&link_type=MED&atom=%2Fjabfp%2F27%2F4%2F510.atom) 

31. 31. Kreuter MW, Holmes K, Alcaraz K, et al. Comparing narrative and informational videos to increase mammography in low-income African American women. Patient Educ Couns 2010;81(Suppl):S6–14.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1016/j.pec.2010.09.008&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=21071167&link_type=MED&atom=%2Fjabfp%2F27%2F4%2F510.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000286295700002&link_type=ISI) 

32. 32. McQueen A, Kreuter MW. Women's cognitive and affective reactions to breast cancer survivor stories: a structural equation analysis. Patient Educ Couns 2010;81(Suppl):S15–21.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1016/j.pec.2010.08.015&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=20850258&link_type=MED&atom=%2Fjabfp%2F27%2F4%2F510.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000286295700003&link_type=ISI) 

33. 33. Stern RH. The discordance of individual risk estimates and the reference class problem. Available from: [http://arxiv.org/pdf/1001.2499v1](http://arxiv.org/pdf/1001.2499v1). Accessed January 18, 2014.
    
    

34. 34. Saver BG, Hargraves JL, Mazor KM. Are population-based diabetes models useful for individual risk estimation? J Am Board Fam Med 2011;24:399–406.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMjQvNC8zOTkiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzI3LzQvNTEwLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==)