YouTube as a source of information on familial Mediterranean fever

TL;DR

This study evaluates the quality and reliability of YouTube videos about Familial Mediterranean Fever, finding that videos by physicians are more accurate and useful.

Contribution

The study provides a first assessment of YouTube content quality on FMF and highlights the role of physicians in improving reliable information.

Findings

62.3% of videos provided useful information, while 26.4% contained misleading content.

Videos uploaded by physicians were more likely to be high quality and contain accurate information.

39.6% of videos were rated as low quality, emphasizing the need for professional content creation.

Abstract

Familial Mediterranean fever (FMF) is the most common hereditary autoinflammatory disease. YouTube is a popular video-sharing platform that both patients and healthcare professionals access for medical information. This study aimed to assess the content, reliability, and quality of the YouTube videos related to FMF. To evaluate video quality and reliability, the Global Quality Scale (GQS) and DISCERN tool were used. Based on GQS scores, videos were categorized into high-, moderate-, and low-quality groups. Four groups were identified in terms of usefulness; useful information, misleading information, useful patient opinion, and misleading patient opinion. The video review was conducted on November 26, 2023. Fifty-three videos that met the inclusion criteria were included in our study. Among these videos, 33 (62.3%) were classified as useful information, 14 (26.4%) as misleading information, two (3.8%) as useful patient opinion, four (7.5%) as misleading patient opinion. Regarding quality, 21 (39.6%) videos were rated as low quality, 13 (24.5%) as moderate quality, and 19 (35.8%) as high quality. Videos uploaded by physicians were significantly more likely to contain useful information (p < 0.001), and demonstrated higher GQS and DISCERN scores compared with other groups (p = 0.003, p < 0.001, respectively). In addition to high-quality videos, there are also lower quality videos on YouTube, which may lead to the spread of misleading information. Therefore, physicians and professional organizations in the field of rheumatology need to make progress in publishing videos according to their quality by collaborating with YouTube and other video sharing sites. Furthermore, encouraging these organizations to share accurate and professionally produced videos on YouTube would benefit healthcare professionals, patients, and their families.

Figures

Full text

Introduction

Familial Mediterranean fever (FMF) is the most common hereditary autoinflammatory disease worldwide, characterized by short, recurrent and self-limiting attacks of fever and serositis (Ozdogan & Ugurlu, 2019; Tufan & Lachmann, 2020). FMF is particularly prevalent among Mediterranean ethnic groups such as Turks, Armenians, Jews, and Arabs (Ozdogan & Ugurlu, 2019; Özen, 2018). The prevalence in these populations ranges from 1:500 to 1:1,000, with the highest reported rate of 1:395 in the Central Anatolia region of Turkey (Tufan & Lachmann, 2020; Özen, 2018). Especially in the 20th century, the prevalence of FMF has started to increase all over the world due to migrations from endemic regions to the other countries (Tufan & Lachmann, 2020; Özen, 2018). FMF is an autosomal recessive disease. In 1997, the discovery of the Mediterranean fever (MEFV) gene, located on the short arm of the 16th chromosome (16p13.3), was a milestone in elucidating the etiopathogenesis of the disease (Ozdogan & Ugurlu, 2019; Tufan & Lachmann, 2020; Özen, 2018). The MEFV gene encodes a 781-amino acid protein called pyrin or marenostrin. Pyrin is an important regulator of inflammation. Mutations of MEFV gene cause pyrin dysfunction. As a result of this situation, uncontrolled interleukin (IL)-1 beta production occurs and FMF-related inflammatory attacks are triggered (Ozdogan & Ugurlu, 2019; Tufan & Lachmann, 2020). Colchicine is the mainstay of the therapy in FMF. In cases of colchicine resistance or intolerance, biological therapies must be considered, especially IL-1 antagonists (Ozdogan & Ugurlu, 2019; Tufan & Lachmann, 2020).

In today’s world, the Internet and social media are a part of daily life and video-sharing platforms have become popular sources of information (Drozd, Couvillon & Suarez, 2018; Madathil et al., 2015). Among these platforms, YouTube—launched in 2005 and now hosting over one billion users—receives more than two billion views per day. On average, a new video is uploaded every minute, and each user spends at least 15 min on the site daily (Drozd, Couvillon & Suarez, 2018; Madathil et al., 2015). Recent research has found that eight out of ten Internet users access their health-related information online. In particular, patients with chronic diseases increasingly rely on Internet-based resources to manage their conditions. Approximately 75% of such patients report that their treatment decisions are influenced by information obtained through online health searches (Madathil et al., 2015). Notably, YouTube enables users to upload medical content without any requirement for verification (Guler & Aydın, 2022). In addition, video contents are not revised by an official organization and misleading information can be spread dangerously (Gragnano et al., 2022). Because of this, healthcare providers and government agencies have expressed concern about the accuracy and quality of the information available on this platform (Madathil et al., 2015).

During the writing of our study, there was no article in the literature examining only FMF-related YouTube videos, except for one study that included all autoinflammatory diseases. In that study, a total of 105 YouTube videos were analyzed, of which only 16 were related to FMF (Sasse et al., 2023). Afterwards, another study focusing on FMF and Youtube videos was published (Coşkun et al., 2024). However, the methods and findings of that study differed from ours. Therefore, the present study was designed to analyze the content, reliability, and quality of YouTube videos concerning FMF based on our own data.

Materials and Methods

YouTube (http://www.youtube.com) was searched using the keyword “familial Mediterranean fever” (FMF) on November 26, 2023. The browser search history was deleted prior to the study to prevent results from being influenced by previous search results. Youtube’s default “relevance” mode was used as a filter, as most viewers do. The first 100 videos were included in the study. All videos and their URLs were recorded for analysis, as YouTube search results are subject to change over time. Only English-language videos were included in the study. Non-English, duplicate, irrelevant, advertisement, and audio-free videos were excluded. All videos were independently evaluated and analyzed by two rheumatologists (CO and ABKD), based on the inclusion and exclusion criteria. The Cohen’s kappa coefficient was used to determine inter-rater agreement. In cases of disagreement between CO and ABKD, a third rheumatologist (SCS) reviewed the video, and a consensus was reached.

The following characteristics of each YouTube video were recorded; upload date (to determine the duration on YouTube), video length (in seconds), and the total number of views, likes, and comments. The number of dislikes was also required to calculate the like ratio ([(likes × 100)/(likes + dislikes)]) and the video power index (VPI: [(like ratio × view ratio)/100]) values used to evaluate the video popularity (Sasse et al., 2023). However, in early October 2021, YouTube announced the removal of public dislike counts from the platform. Therefore, the like ratio and VPI values could not be calculated, as the number of dislikes was unavailable. The view ratio for each video was calculated using the following formula: [views/day] (Sasse et al., 2023).

Videos were categorized into four groups based on the upload source: physician, patient, health related websites, and other (non-medical independent user, non-medical media organization).

They were also classified according to their target audience as healthcare professionals, patients, and non-classified.

All included videos were evaluated for usefulness and classified into four mutually exclusive categories (Tolu et al., 2018):

1.Useful information: These videos contained medically useful and accurate information for learning about FMF. 2.Misleading information: These videos contained clinically incorrect or unproven information based on currently available scientific evidence. If the video contained partially useful and partially misleading information, it was classified as misleading. 3.Useful patient opinion: These videos contained a patient’s personal experience and/or feelings that were useful with regards to the FMF. 4.Misleading patient opinion: These videos contained a patient’s personal experience and/or feelings which were not useful or with misleading opinions with regards to the FMF.

The quality of each video was assessed using the Global Quality scale (GQS), which was also used in previous similar studies (Sasse et al., 2023; Coşkun et al., 2024; Tolu et al., 2018; Bernard et al., 2007). The GQS is a five-point scale, with higher scores indicating better video quality. Scores of 4–5 points were considered high quality, 3 points moderate quality, 1–2 points low quality.

Video reliability was evaluated using the modified DISCERN tool, which has also been used in several similar studies (Sasse et al., 2023; Tolu et al., 2018; Singh, Singh & Singh, 2012). Each question is answered as “yes” (scored 1 point) or “no” (scored 0 points), with total scores ranging from 0 to 5. Higher scores represent greater reliability. The questions of the GQS and modified DISCERN tool are presented in Table 1.

Table 1: Assessment tools for reliability and quality.

Statistical analysis

All the data were analyzed statistically using the Statistical Package for the Social Sciences (SPSS) version 22 (SPSS Inc., Chicago, IL, USA). Prior to analysis, the Shapiro–Wilk test was used to assess the normality of data distribution. Non-normally distributed continuous variables were expressed as median (minimum–maximum) values, while categorical variables were expressed as counts and percentages. The Kruskal-Wallis test was used to detect statistically significant differences among more than two independent groups. In cases where a significant difference was identifie in the Kruskal-Wallis test, pairwase comparisons were evaluated using the Mann–Whitney U Test with Bonferroni correction to determine the groups that led to significant differences. The χ^2^ test was used to evaluate categorical variables. The relationship between variables were analyzed using Spearman’s correlation coefficients. The inter-rater agreement was assessed using the Cohen’s kappa coefficient. A p value of < 0.05 was considered statistically significant.

Results

A total of 100 videos were initially screened. Of these, six duplicate videos, 19 videos presented in languages other than English (nine in Turkish, six in Arabic, two in German, and two in French), 17 irrelevant videos, three advertisement videos, and two videos without audio were excluded from the study. Then, 53 videos that met the determined criteria were selected for further analysis. The features, usefulness categories, quality stratification, upload sources, and target audiences of these videos are summarized in Table 2.

Table 2: Baseline characteristics, usefulness categories, and quality stratification of the analyzed videos.

The videos were categorized into three groups based on their GQS scores. Except for the number of days on YouTube, all other video features were significantly higher in the high-quality group compared with the moderate- and/or low-quality groups. The moderate-quality group had significantly higher scores than the low-quality group only in terms of the DISCERN score. To determine which groups the statistical differences originated from, pairwise comparisons were conducted using the Mann–Whitney U test with Bonferroni correction, and a p-value of < 0.017 was considered statistically significant. When a significant difference was detected, the corresponding letter was added to the upper right of the p-value (Table 3).

Table 3: Comparison of the video features according to the their quality stratification.

When the videos were analyzed in four groups based on their usefulness, all video characteristics—except for the number of days on YouTube—showed statistically significant differences among the groups. To determine which groups contributed to these differences, pairwise comparisons were performed using the Mann–Whitney U test with Bonferroni correction, and a p-value of < 0.0125 was considered statistically significant. When a significant difference was observed, the corresponding letter was added to the upper right of the p-value. The findings are summarized in Table 4.

No statistically significant differences were observed between the useful information and useful patient opinion groups in terms of main variables (p > 0.05, Table 4). However, the median values of number of views, likes and comments were numerically higher in the useful patient opinion group. Videos uploaded by physicians were found to be significantly more useful than those uploaded by other sources (patients, health related websites, and other) (Table 4).

In terms of upload source, GQS scores were found to be significantly higher in the physician group than in the patient, health-related websites, and other groups [4 (2–5), 1.5 (1–3), 3 (1–4), and 1 (1–4), respectively; p = 0.003]. The same statistical evaluation was determined for DISCERN scores [3 (2–5), 1.5 (1–3), 2 (1–5), and 1 (1–2), respectively; p <0.001]. Additionally, video durations were significantly longer in the physican group [479 (107–3028), 231.5 (199–3090), 147 (33–733), and 148 (39–2685), respectively].

Table 4: Analyses of video characteristics by categories of usefulness.

A significant moderate positive correlation was detected between GQS and DISCERN scores. Except for the number of days on YouTube, significant positive correlations were found between other video features (video duration, number of views, view ratio, number of likes, and comments) and GQS/DISCERN scores (Table 5).

Table 5: Analysis of the correlation between GQS, DISCERN and video features.

Cohen’s kappa coefficient for inter-rater agreement was 0.829 (p < 0.001) for the GQS score, 0.848 (p < 0.001) for the DISCERN score, and 1.000 (p < 0.001) for video usefulness.

Discussion

Today, the internet is widely used, and many people turn to video-sharing platforms such as YouTube to obtain information on various topics, particularly health-related issues. However, the diversity of authorship and the absence of peer-review process on this platform have led to the spread of false and misleading information (Delli et al., 2016; Kunze, 2020) because any YouTube user, regardless of their health knowledge, experience, or professionalism, can upload a video to this platform. For this reason, YouTube videos related to rheumatological diseases have been evaluated in terms of their usefulness, reliability, quality and accuracy. These studies have shown that, there are videos containing useful and quality information, as well as videos containing missing and incorrect information (Delli et al., 2016; Ng, Lim & Fong, 2020; Onder & Zengin, 2021; Onder & Zengin, 2023; Elangovan, Kwan & Fong, 2021).

FMF is a rare disease, and it can be difficult to reach specialists on this condition. For this reason, patients and individuals seeking information about FMF often turn to the internet, particularly to video-sharing platforms. The findings of the two studies conducted to date have shown differing results regarding the evaluation of FMF-related videos (Sasse et al., 2023; Coşkun et al., 2024). Therefore, we aimed to present and compare our own data.

In our study, the majority of non-English videos were in Turkish and Arabic. This finding can be explained by the widespread use of these languages in geographical regions where FMF is prevalent (Ozdogan & Ugurlu, 2019; Tufan & Lachmann, 2020; Özen, 2018).

The majority of the videos were uploaded by physicians (50.9%), followed by health related websites, other sources and patient-sourced videos (24.5%, 13.2%, 11.3%, respectively). The primary target audience of the videos was patients (45.3%), followed by healthcare professionals (39.6%) and non-classified audience (15.1%). This distribution is consistent with findings from previous studies (Sasse et al., 2023; Coşkun et al., 2024; Ng, Lim & Fong, 2020; Onder & Zengin, 2021; Onder & Zengin, 2023; Elangovan, Kwan & Fong, 2021; Pamukcu & Izci Duran, 2021; Koo, Kim & Jun, 2021). These data give the impression that most of the sources are reliable. Considering the target audience, it is thought that it may be suitable for informational use by patients or their relatives and for educational purposes by healthcare professionals. However, whether these videos are indeed suitable for such purposes should be further evaluated in conjunction with the other findings of our study.

When the videos were evaluated collectively, the median GQS score was 3 (1–5). Previous studies have reported similar (Ng, Lim & Fong, 2020; Pamukcu & Izci Duran, 2021; Zengin & Onder, 2021), lower (Sasse et al., 2023), or higher (Coşkun et al., 2024; Onder & Zengin, 2021; Onder & Zengin, 2023) values. In one study assessing YouTube videos on autoinflammatory diseases, the GQS score for the FMF group was as low as 2 (1–5) (Sasse et al., 2023), whereas another study focusing solely on FMF reported a considerably higher score of 5 (1–5) (Coşkun et al., 2024). The first study included only 16 FMF-related videos, which may explain the low average score. Furthermore, these differences may be attributable to factors such as the inclusion criteria, timing of the search, and variability among raters in both studies. Although this finding clearly indicates a discrepancy, we believe that the GQS distribution observed in our study is more accurate and representative. One study reported a very low GQS value, while another reported a very high value. The GQS value obtained in our study lies between these two extremes and is therefore considered more consistent and compatible with the existing literature.

According to GQS scores, videos were classified as high (35.8%), moderate (24.5%), and low quality (39.6%). While these proportions are consistent with some prior studies (Zengin & Onder, 2021; Kocyigit, Akaltun & Sahin, 2020), others reported different distributions (Onder & Zengin, 2021; Chang, Lee & Park, 2022; Kocyigit & Akyol, 2021). In another study examining FMF-related YouTube videos (Coşkun et al., 2024), the proportion of high-quality videos was remarkably high (72.9%), whereas medium- and low-quality videos accounted for 16.9% and 10.2%, respectively. Although the videos analyzed in both studies were related to the same disease, this article showed the least similarity to our findings among the studies available in the literature. Although this raises the possibility that different videos were analyzed due to variations in evaluation methods or search keywords, the fact that the proportions differ so greatly remains quite thought-provoking.

When GQS groups were evaluated in terms of DISCERN scores; the high-quality group had significantly higher DISCERN scores than both the intermediate- and low-quality groups. Similarly, the intermediate-quality group had higher DISCERN scores than the low-quality group. This finding is consistent with previous studies (Coşkun et al., 2024; Pamukcu & Izci Duran, 2021; Zengin & Onder, 2021). In addition, we found a significant positive correlation between GQS and DISCERN scores, indicating that as the quality of the videos increases, their reliability also increases.

The number of views, view ratio, and number of likes were also significantly higher in the high-quality group compared to the intermediate- and low-quality groups. These higher values may suggest that the quality of a video is better. When evaluated in terms of video duration and number of comments, the high-quality group also showed higher numerical values than both the intermediate- and low-quality groups. However, statistical significance was observed only between the high- and low-quality groups. Previous studies have reported both similar and different findings (Coşkun et al., 2024; Pamukcu & Izci Duran, 2021; Zengin & Onder, 2021; Kocyigit, Akaltun & Sahin, 2020; Kocyigit & Akyol, 2021). Based on this information, if a video is longer and has a higher number of comments, it may indicate better quality. Of course, the higher numerical values discussed in this paragraph do not necessarily guarantee that video quality will always be high.

As of early October 2021, YouTube stopped publicly sharing the number of dislikes, so the like ratio and VPI values could not be calculated in our study. However, in a previous study related to FMF and YouTube videos (Coşkun et al., 2024), the number of dislikes was assumed to be zero for all evaluated videos. Accordingly, in the formula used to calculate the like ratio and VPI values, the dislike value was set to zero. However, we believe that performing such an evaluation without access to the actual dislike numbers may lead to inaccurate results. For instance, in the aforementioned study, no statistically significant difference was found between the low-, medium-, and high-quality groups in terms of either like ratio or VPI values. Nevertheless, if these evaluations had been conducted using the real dislike data, the results might have been different. Therefore, we consider that the discussion based on these data is open to interpretation and that its validity should be questioned.

The median values for the number of views, likes, and comments were numerically higher in the useful patient opinion group, although the difference was not statistically significant. There were studies in which this situation was the same (Tolu et al., 2018) and similar (Ng, Lim & Fong, 2020; Kocyigit & Akyol, 2021). Three possible explanations may account for this observation. First, patients often seek to hear about the experiences of others with the same disease and to identify similarities and differences with their own situation. Second, videos created by healthcare professionals may be less appealing to patients because they often contain excessive medical information. Finally, videos with a high number of views may also have more likes and comments.

GQS and DISCERN scores were significantly higher in the physician group compared to other video sources (patient, health-related websites, and other). Similar findings have been reported in previous studies (Tolu et al., 2018; Onder & Zengin, 2021; Onder & Zengin, 2023; Elangovan, Kwan & Fong, 2021). Although this finding is not surprising, it is noteworthy that it was confirmed through scientific comparison. Based on this evidence, physician-sourced videos can be recommended to individuals seeking information about FMF on YouTube. We recommend that authorities and YouTube administrators take initiatives to increase the number of videos produced by physicians. Moreover, highlighting these videos in search results, or even marking them as “recommended”, could help disseminate accurate and high-quality information.

A significant positive correlation was observed between GQS and DISCERN scores, consistent with some previous studies (Pamukcu & Izci Duran, 2021; Chang, Lee & Park, 2022). Based on this, it can be interpreted that videos with high quality scores may be more reliable. The DISCERN scoring system evaluates whether the information and sources used are reliable, balanced, and free from bias; therefore, high DISCERN scores can serve as an indicator of high quality.

Except for number of days on YouTube, significant positive correlations were found between other video features (video duration, number of views, view ratio, number of likes, and number of comments) and GQS/DISCERN scores. Although no previous study has correlated all these parameters simultaneously with GQS and DISCERN scores, some studies reported similar findings (Coşkun et al., 2024; Pamukcu & Izci Duran, 2021; Chang, Lee & Park, 2022).

Longer videos tended to have higher GQS and DISCERN scores, which can be interpreted as longer videos providing more comprehensive information. We found that videos in the physician group were longer and had higher GQS and DISCERN scores than those in the other groups. Based on these findings, physician-uploaded videos can be recommended as a first choice. Additionally, higher numbers of views, likes, and comments may also serve as useful indicators when selecting videos.

Limitations

Our study has several limitations, similar to previous YouTube studies. First, it is a cross-sectional study, evaluating only a snapshot at a specific point in time. YouTube is a dynamic video-sharing platform, with content that is constantly changing and expanding. Second, video evaluations are inherently subjective, although a sufficient Kappa score was achieved to indicate inter-rater agreement. Third, only English-language videos were included. Fourth, because YouTube no longer shares dislike counts, like ratio and VPI values could not be calculated in this study. Finally, in our study, only YouTube videos were evaluated, videos on other social media platforms were not analyzed.

Conclusion

In addition to high-quality videos, most of which are uploaded by physicians, there are also lower-quality videos on YouTube, which may lead to the spread of misleading information. Enhancing the accuracy and reliability of online health information, and having it monitored by health authorities when necessary, would benefit many patients—particularly those with limited access to healthcare—and individuals seeking health information. For this reason, physicians and professional organizations in the field of rheumatology need to make progress in publishing videos according to their quality by collaborating with YouTube and other video sharing sites. Additionally, these organizations could share reliable videos targeting healthcare professionals, patients, and their families. We believe that it would be an important step for health authorities to implement programs guiding the public in accessing accurate information, in addition to digital health education programs for patients.

Supplemental Information

10.7717/peerj.20963/supp-1Supplemental Information 1Raw data