Abstract
Background: Black men are disproportionately impacted by prostate cancer. Guidelines agree that Black men should make informed decisions about whether to engage in prostate cancer screening. YouTube is widely used among Black men and impacts understanding of health conditions.
Objectives: Given that misleading online health information might be especially harmful to Black men, the objective of this study was to evaluate the quality of information regarding prostate cancer screening for Black men available on YouTube.
Methods: Four viewers watched the top 50 videos using the search term “Prostate Cancer Screening in Black Men.” Videos were scored using the previously validated DISCERN quality criteria for consumer health information and the Patient Education Materials Assessment Tool (PEMAT). Results were compared based on video characteristics like presenter perceived demographics and viewer engagement metrics.
Results: Inter-rater reliability testing showed consistency for the PEMAT (interclass correlation coefficient [ICC] = 0.69) and DISCERN (ICC= 0.85). Few videos (16%) met the DISCERN quality threshold (54.4/80), and 28% of videos met the PEMAT threshold (10.5/15). Less than half of videos addressed racial disparities in prostate cancer. There was no difference in quality based on perceived race of the presenter (DISCERN P = .06, PEMAT P = .43).
Conclusions: The overall quality of videos about prostate cancer screening in Black men is poor, including those with Black presenters. Clinicians should be aware of potential misinformation that Black patients receive from YouTube and the opportunity to improve the quality of available information about prostate cancer screening in Black men.
- Black Men
- Communications Media
- Early Detection of Cancer
- Family Medicine
- Health Behavior
- Health Care Disparities
- Mass Screening
- Patient Education
- Primary Health Care
- Prostate Cancer
- Prostate-Specific Antigen
- Social Media
- YouTube
Introduction
Prostate Cancer (PrCA) is the most common non-cutaneous cancer in American men and the second leading cause of cancer death after lung cancer in men.1,2 PrCA disproportionately impacts Black men, as shown by a mortality rate that is more than twice that in White men.3 Major guideline societies, including the US Preventive Services Task Force (USPSTF), agree that Black men are at increased risk of developing and dying from PrCA, and should make individualized informed decisions about whether to engage in PrCA screening.4⇓–6 Lack of knowledge about PrCA has been identified as a key barrier to Black men engaging in informed decisions about PrCA screening.7⇓⇓⇓–11
Black men are less likely than White men to have access to medical care and are more likely to avoid care due to previous negative experiences in the health care system.12⇓–14 Consequently, Black men are more likely to receive health information from nonmedical sources, which can contribute to inadequate knowledge regarding PrCA screening. Black men must receive accurate PrCA information from reliable sources to make informed decisions about screening, as recommended by the guidelines. This information also must be complete, given that PrCA screening has known risks and potential benefits.15,16
Approximately 50% of Black individuals seek health information online, and YouTube is the most commonly used social media platform among Black individuals.17⇓–19 Black Americans with low electronic health literacy have been found to have high perceived trust in the health information available on YouTube.20 The quality of general PrCA screening information available on YouTube is low, and the content is potentially misleading.21⇓–23 The quality of YouTube videos specific to PrCA screening in Black men has not been previously explored. Given that misleading online health information might be especially harmful to Black men, who are already disproportionately affected by PrCA, the objective of this study is to evaluate the information regarding PrCA screening for Black men available on YouTube videos.
Methods
Data collection
Internet searches were performed for YouTube videos using the term “prostate cancer screening in Black men.” A total of 50 videos were identified, meeting the following criteria: over 1000 views, between 1 and 10 minutes long, appears on the first search result screen. Videos shorter than 1 minute or longer than 10 minutes were excluded as it was deemed that these videos would either not contain sufficient information for analysis or be too long to sustain viewer interest.22 To account for the potential variability in video results depending on different user profiles, we followed a protocol for identifying videos to include. We included the top 10 videos from a Black male MD's personal computer, followed by the top 10 novel videos from a White female researcher's personal computer, followed by the top 10 novel videos from a Black male medical student's personal computer, followed by the top 10 novel videos from a public domain library computer, followed by the top novel 10 videos from the Black male MD's work computer. Four raters from the Medical University of South Carolina in Charleston, South Carolina, independently assessed the videos: a Black male MD, a White male MD, a Black male medical student, and a White female researcher. Videos were rated using the previously validated DISCERN criteria evaluating quality of information related to health care decision making and Patient Education Material Assessment Tool (PEMAT) for understandability and actionability of education materials.24,25 Table 1 shows the elements evaluated by each assessment tool. Study-specific constructs included the video's perceived target audience (all men or Black men), viewers' engagement metrics (number of views, comments, thumbs up or down), the presence or absence of commercial bias (ie, advertising products or services), and whether the videos addressed disparities (yes, no, or unclear). Presenter perceived race was classified as “other” in cases when the reviewer did not perceive the presenter to be White or Black or when videos did not visually display the speaker. The target audience was identified as Black if the presenter specifically stated that the information was directed toward Black men or if the information/statistics provided were specific to Black men. The number of views was adjusted for the length of time that a video had been posted, and so views/month was the variable included in the analysis. Thumbs up or down and the number of comments were not adjusted for time since the video had been posted because their numbers were too few and inconsistent to be meaningfully adjusted.
Validated Tools Used to Rate the Quality of YouTube Videos
Data analysis
Inter-rater reliability testing with a 2-way mixed model was performed for absolute agreement between the 4 raters for the 50 videos. This method assesses the similarity of video scoring among the raters rather than the consistency of individual raters.26 Average video characteristics were computed. Recommendation for shared decision making was part of the DISCERN criteria, measured on a scale from 1 to 5, and was reclassified into a yes or no variable at the 3.5 threshold to evaluate the percentage of videos addressing this specific point. Chi-squares and analysis of variance (ANOVA) were used to examine whether this specific point was related to video characteristics. Kendall's τ correlations were used to evaluate whether PEMAT or DISCERN scores were associated with the length of the video and/or viewer engagement metrics. T-test was used to evaluate the relationship between video quality and target audience (Black men vs all men). ANOVA was run to determine the effect of perceived presenter race (Black vs White) or commercial bias on the video ratings and on viewer engagement metrics. Fisher's exact tests were used to assess the relationship between perceived presenter race and target audience and whether disparities or shared decision making were discussed. Videos were subsequently divided into groups depending on whether they reached a quality threshold. The quality threshold was set at 68% (54.4/80) for the DISCERN and 70% (10.5/15) for the PEMAT, as per prior literature.25,27 Chi-squares were used to evaluate whether lower-quality videos differed from better quality ones according to target audience, presenter perceived race, perceived gender, discussion of disparities, and commercial bias.
Finally, videos were subclassified into temporal groups in relation to changes in recommendations from the USPSTF (before or on May 15, 2012, between May 15, 2012, and May 8, 2018, on or after May 8, 2018) and the American Urologic Association (AUA, before or after January 1, 2013). Video characteristics were analyzed again according to these categories. Since the DISCERN instrument specifically allocates points for shared decision making, an additional analysis excluding this question was run.
Results
We scanned through 190 videos to identify 50 that met the inclusion criteria. There were 6 videos excluded for being under 1 minute and 18 excluded for being longer than 10 minutes. An additional 116 videos were excluded for having fewer than 1000 views. We found 88% and 78% overlap between the top 50 videos using 3 different user profiles. The vast majority (96%) of videos included in the analysis came from hospitals, clinics, doctors, or professional and health organizations. Inter-rater reliability testing showed consistency among the 4 raters for PEMAT and DISCERN on the 50 videos (Table 2). Videos produced between September 2010 and October 2019 and their characteristics are shown in Table 3. A small proportion of videos supported shared decision making for screening or had a Black presenter, and less than half addressed disparities. Shared decision making was not related to viewer engagement metrics, perceived presenter race, target audience, or whether the video addressed disparities.
Interrater Reliability Test (Two-Way Mixed Model, Absolute Agreement) for the Three Raters Who Assessed the 50 YouTube Videos
Characteristics of the Evaluated YouTube Videos (n = 50)
The average DISCERN (quality of information for decision making) score was 40.24 (12.49 SD) and ranged from 21.50 to 70.00 out of 80, with 16.0% of the videos above the quality threshold of 54.4. The average PEMAT (understandability and actionability) score was 9.31 (1.78 SD) and ranged from 5.75 to 12.50 out of 15, with 28.0% above the quality threshold of 10.5. There was a significant positive correlation between PEMAT and numbers of views/month (Kendall's τ r = 0.308, P = .002). This was not observed for DISCERN (Kendall's τ r = 0.156, P = .110). The number of views per month was, in turn, positively correlated with the length of the video (Kendall's τ r = 0.252, P = .010). Perceived target audience was not related to the quality of the videos. DISCERN and PEMAT averages differed depending on presenter perceived race, with videos presented by a Black person scoring lower on average (33.59 and 8.79) than White (40.82 and 9.39 respectively). However, these differences did not reach statistical significance. Perceived presenter race and commercial bias were not related to viewer engagement metrics. The presenter's race and target audience were related, with 90.9% of videos targeting Black men being presented by a perceived Black presenter (P < .001). Videos above the quality threshold did not differ from those of lower quality in terms of target audience, presenter perceived race, and gender, discussion of disparities, or commercial bias.
DISCERN scores varied according to the year of publication of USPSTF recommendations. The quality of videos increased if published after the most recent change in recommendation (39.95 before May 2012, 37.85 between May 2012 and May 2018, 48.74 after May 2018, P = .049). This difference remained with removing the question about shared decision making (37.70 before May 2012, 35.43 between May 2012 and May 2018, 45.79 after May 2018, P = .042). PEMAT scores were all-around 9.3. The date of changes in AUA recommendations did not affect the video scores. Comparisons are summarized in Table 4.
Scores of the Evaluated YouTube Videos (n = 50)
Discussion
Our findings are consistent with previous studies reporting the overall poor informational quality of YouTube videos about prostate cancer.21⇓–23 Only 16% of videos met the quality threshold on the DISCERN, and 28% met the quality threshold for the PEMAT. Given the consistent finding of poor-quality prostate cancer screening videos over several studies, future interventions could work with information platforms such as YouTube to create a verification system to identify health information videos produced by reputable sources. Our interclass correlation coefficient indicated good agreement among raters and was higher than a previously reported study of YouTube videos.28 Our finding that only 20% of videos targeting Black men promoted shared decision making is significantly lower than a similar study looking at videos targeting all men, which found that about 50% promoted shared decision making.23 Of note, fewer than half of the videos targeting Black men discuss prostate cancer disparities, which is currently recommended as a best practice by the USPSTF and is essential information for Black men deciding about screening.4
It is notable that most videos targeting Black men had a Black presenter and that perceived presenter race was associated with lower, although not statistically significantly, rating scores. Previous work found that Black individuals may prefer to receive care and perceive higher quality information sharing from a racially concordant clinician.29⇓⇓–32 In the education world, Black students report higher satisfaction and demonstrate better academic achievement with a Black teacher.33⇓–35 Thus, this study suggests the need to improve the quality of information available on YouTube regarding prostate cancer screening in Black men, including that provided by Black presenters. An interesting finding was that DISCERN scores were significantly higher for videos created after the 2018 USPSTF guidelines calling for informed decision making about PSA screening. A previous study found that videos made before and after the 2012 USPSTF guideline change did not significantly impact the content or quality of videos.21 While this result may signify improved video quality in light of the 2018 guideline change; it also may reflect that the 2018 guidelines for informed decision making are more aligned with the DISCERN criteria, which includes shared decision making as part of its assessment. However, removing the question directly addressing shared decision making in the DISCERN analysis did not change this pattern.
One limitation of this study is that only YouTube was examined, and other websites or social media outlets were not evaluated. We selected YouTube based on data that it is the most widely used social network in the USA among Black individuals.19 Further, a number of studies evaluating prostate cancer screening information on YouTube have been published, which help contextualize our findings.21⇓–23 Future studies should evaluate the quality of information on other popular electronic platforms such as Facebook. An additional limitation is that YouTube employs a search algorithm that incorporates an individual's browser history, so there is some variability in the videos and order of videos displayed for different individuals. To accommodate for this, our video identification strategy involved searches from 3 different users on both public and private computers.
Further, most videos were produced before the 2018 USPSTF guideline changes. The default search settings in YouTube do not sort videos chronologically, and so the study team felt it important to emulate the default search strategy that we assumed most users would use. YouTube reports total times videos are viewed but does not report when the views occurred, so it was not possible to evaluate the percentage of views that occurred relative to the guideline change. We did analyze the number of views divided by months since publication to adjust as best as we could for the length of time the video was posted. We also do not know the search terms that Black men would most likely use to search for PrCA screening. We used the term “Black men” as we felt this was more inclusive of men born across the diaspora. Future studies should investigate whether Black men search for health information with race-specific terms and preferred terms. It is important to acknowledge that race is a sociopolitical construct most commonly ascribed through self-identification and that perceived racial identity is subject to bias. Previous work has illustrated that perceived racial identity is an important and independent variable in health care, impacting disparities and discrimination.36 This led to the inclusion of perceived presenter race as an important variable for evaluation. Finally, this study does not evaluate whether these videos led to a discussion with health care providers about PrCA screening or if videos impacted Black men's knowledge or screening behaviors. However, it is an initial step toward better understanding the information available to Black men about PrCA screening.
Current PrCA screening guidelines recommend informed decision making, especially for Black men who disproportionately die from PrCA. Given that Black men are less likely to access health care providers, they are especially susceptible to influence from sources of information like social media. Prior interventions have demonstrated that targeted YouTube videos focusing on men's health can have a wide reach.37 However, we found a lack of quality information regarding informed decision making about PrCA for Black men on YouTube. Most available information falls short in informing Black men of their increased risk of developing and dying from PrCA. Clinicians and medical societies should recognize the potential misinformation that Black men are receiving through YouTube. Current modalities for electronic information dissemination among Black men are already in existence but need to be better utilized. These include not only YouTube but also social media platforms like Facebook. Medical organizations should capitalize on the reach of these platforms by producing high-quality videos and implementing strategies to increase traffic to their sites so that the videos have far reach. This study elucidates the need for culturally tailored messaging as well as the need for quality information, including that provided by racially concordant informants, regarding PrCA disparities and prostate cancer screening in Black men.
Notes
Funding: None.
Conflict of interest: None.
To see this article online, please go to: http://jabfm.org/content/34/4/724.full.
- Received for publication December 9, 2020.
- Revision received March 23, 2021.
- Accepted for publication April 2, 2021.
