English-language videos on YouTube as a source of information Human Papilloma Virus vaccination

Objective: To assess the quality of information presented in YouTube videos about human papillomavirus vaccination. Method: The descriptive study was conducted at Konya Meram Education and Research Hospital, On October 15, 2019 when the YouTube website was search using the terms ‘human papillomavirus’, ‘HPV vaccine’ and ‘Gardasil’. The vide-os were recorded to a playlist by two individual gynaecologists to prevent any change in the listed videos. The videos were categorised into 3 groups; useful in-formation group A, misleading information group B and insufficient information group C. The quality of the videos was scored using global quality scale from 1 = poor quality to 5 = excellent quality. DISCERN scale was used for reliability. A 10-point scale was used to evaluate comprehensiveness of the videos. Data was analysed using SPSS 20. Results: Of the 200 videos assessed, 179(89.5%) were analysed. There were 17(9.5%) videos in group A, 38(21.2%) in group B and 124(69.3%) in group C. Mean global quality scale score was 3.94±1.39 group A, 1.84±0.59 group B and 3.13±0.94 group C (p<0.001). Mean reliability values were 4.18±1.13group A, 1.66±0.66 group B and 3.03±0.87 group C (p<0.001). Comprehensiveness scores were 6.94±2.49 group A, 1.53±0.95 group B and 4.87±1.72 group C (p<0.001). Conclusion: Professional organisations, university channels and doctors should provide accurate, unbiased and evidence-based information on YouTube for community awareness.


Introduction
Human papillomaviruses (HPV) are sexually-transmitted viruses having more than 200 types that infect only humans. Depending on the type, HPV infects various body sites and causes anogenital and oropharyngeal diseases ranging from benign warts to cervical, genital and head and neck cancers in both males and females 1 .
Since the discovery of HPV vaccines, HPV-related diseases have become preventable. There are 9-valent, quadrivalent or bivalent HPV vaccines available against cancers caused by HPV infection. The Advisory Committee on Immunisation Practices (ACIP) and the Centres for Disease Control (CDC) recommend vaccination for individuals aged 11-12 years before HPV exposure, and catch-up vaccination is recommended until the age of 26 years with 9-valent HPV vaccine 2 . If adolescent immunity is weak or the HPV vaccine series is started at age ≥15 years, three doses of HPV vaccine are recommended. Besides, adults can take the vaccine until 45 years of age after consultation with healthcare providers (HVPs) 3,4 . Although there are many HPV vaccination programmes worldwide, a lack of knowledge about HPV vaccines is reported, and initiation and completion of vaccine rates have been suboptimal, with security concerns and lack of awareness being the most common factors behind low HPV vaccine rates 5 . With the wide-spread use of the internet all over the world, people's reach to information has become easier. Today, the increasing use of the internet and access to information online have led to the change of the old methods used to solve people's health issues 6 . However, misleading information leads to disruption in the immunisation of society and individuals, and can lead to critical health problems 7 .
There are some studies assessing the quality, accuracy and comprehensiveness of YouTube website (www.youtube.com; YouTube LLC, San Bruno, CA) videos about HPV vaccine in English language [8][9][10] , but incorrect or insufficient information available on YouTube about HPV vaccine has not been analysed. The current study was planned to fill the gap by evaluating information related to HPV vaccine available in the shape of YouTube videos.

Materials and Methods
The descriptive study was conducted at Konya Meram Education and Research Hospital, On October 15, 2019 when the YouTube website was search using the terms 'human papillomavirus', 'HPV vaccine' and 'Gardasil'. Approval was not required from the ethics review board as no human or animal intervention was involved. Studies have shown that the first 3 pages in the search results happen to be the most watched ones among internet users 11 . The first 200 videos were recorded to a playlist by two individual gynaecologists to prevent the change of the listed videos because search results may progressively change on YouTube. Video searches were performed without registration in the browsers' private browsing mode to avoid bias. Two blinded reviewers excluded repetitious, irrele-vant videos and those that were not in English language or had no audio.
The videos were evaluated on the basis of input from independent doctors who did not participate in the study. The videos were categorised into 3 groups; useful information group A, misleading information group B and insufficient infor-mation group C. The categorisation was done in line with literature 12 .
For each video analysed, data included the total number of views, the total duration of the content, and the length of time for which the video had been available online. Viewer interaction with the video was assessed by daily view rate which is calculated as total views for the video divided by the number of days on YouTube, number of 'likes', 'dislikes' and comments [12][13][14] .
The source of the videos was classified into 5 categories: verified government and news agencies; private or public university channels, professional health organisations, non-profit physician and physician groups; stand-alone health information websites without any connection; medical advertisements/for-profit companies; and individuals.
Videos were also segregated in terms of target audience, like females, males or gender-neutral, as well as according to the speaker type, as physician, non-physician health provider or layperson in the video and voiceover.
DISCERN tool was used for reliability assessment 15 , while the quality of the videos was scored using global quality scale (GQS) from 1 = poor quality to 5 = excellent to evaluate the flow and facility of use of the information provided in the video 17 . Besides, a 10-point scale was used to evaluate comprehensiveness of the videos in line with a framework designed using CDC, ACIP and the American College of Obstetricians and Gynaecologists (ACOG) guidelines 2, 16 . (Table 1).
Data was analysed using SPSS 20. Descriptive data was expressed as mean and standard deviation (range) for continuous variables. Categorical variables were expressed as frequencies and percentages. Kolmogorov-Smirnov test was used to analyse normality of quantitative data. Analysis of variance (ANOVA) test was used for comparison of groups with respect to nonnormally distributed continuous variables. P<0.05 was considered statistically significant.
Views per day, length of time of the videos on YouTube, likes, dislikes and comments were not significantly different among the groups(p>0.05). The groups were significantly different when in terms of video length (p=0.027) (   (Table 3).

Discussion
The current descriptive study evaluated the quality and reliability of videos on YouTube to increase awareness about HPV vaccination. Despite the information shared and frequently updated by major health organisations related to the HPV vaccine, it was found that the information on YouTube, which patients can easily access, was not sufficient. Although high quality useful information was available, the rate of misleading and insufficient information was much higher.
It is very important in today's era to make available accurate information on the internet, which is the most frequently used platform accessed for information by people around the world. Approximately 59.6% of the global population are using the internet every day 18     Many studies investigating the quality of the videos on YouTube have found that high-quality information content is often created by university channels, professional organisations, non-profit physicians and physician groups 21,22,24 . When the current study evaluated the videos in terms of uploading sources, surprisingly there was no significant difference among the groups. The results are comparable with earlier studies 25,26 . Radonjic et al. reported that the training videos uploaded by non-physicians were significantly more popular 26 . Further, Adhikari et al. reported that the most popular videos were based on personal experiences, although most of the videos were uploaded by professional associations in the shape of news reports and lessons 24 .
In the current study, 179 videos related to cervical cancer were analysed and the quality of the videos discussing different aspects of cervical cancer was inadequate, and the least uploaded videos were personal videos, and no personal video fell under the useful information category. Personal videos mostly included the experiences of patients with cervical cancer and irrelevant side effects of the HPV vaccines. The rate of personal videos compared to other studies was found to be quite low 21,23,27 . This may be because patients do not want to use their personal information about their illness. Although the proportion of videos created by reliable sources was high, the low proportion of videos containing useful information may be due to several factors, such as the sample size which is larger compared to other studies and HPV vaccine and instructions that are updated frequently [8][9][10] .
In the analysis of audience interaction parameters, the study found that there was no difference among the groups in terms of total views, daily views, duration of videos, likes, dislikes and comments. Although there was no significant difference, the number of daily views in the useful information group was higher than the other groups. Although some studies 24,27 have significant differences in terms of video quality with these parameters, most studies are similar to the current results. Esen et al. 26 and Kocyigit et al. 24 reported similar results in this domain. Esen et al. also found that the number of daily views was higher in the misleading information group 26 . Ku et al. 22 assessed the quality of YouTube videos on male infertility and found that the total number of days, likes and dislikes, and total views did not correlate with video quality. Therefore, internet users should take these parameters less into account when evaluating videos on YouTube. When the groups were compared in terms of video length, there was a significant difference. Especially useful information videos were longer, and in the binary comparison, the insufficient information group was longer than the misleading information group (Table 3).
The World Health Organisation (WHO) recommended HPV vaccination in 2009 to prevent cervical cancer cases 28 . HPV virus is a sexually-transmitted infection agent that can cause life-threatening cancers in older ages in both genders. Several studies have shown that vaccination in both men and women is more effective at reducing HPV infection 29 . The overall burden of HPVrelated cancers and pre-cancerous lesions among men is less than that of cervical cancer in women. How-ever, the overall benefit of vaccinating men outweighs potential risks due to the added population benefits resulting from herd immunity and documented safety of HPV vaccines 29 . Therefore, it is important to recommend HPV vaccine for both genders. In the current study, HPV vaccine recommendation rates in both genders were found to be 88.2% in the useful information group, 75.8% in the inadequate information group and 59.5% in the misleading information group.
The WHO / United Nations Children's Fund (UNICEF) has confirmed that vaccine hesitation in many parts of the world has increased annually between 2014 and 2016 30 .
Since vaccine instability remains a global problem, some content against HPV vaccination was also noted in the current study. The concerns about HPV vaccine have shown that it can lead to sexual activity among children, or that it is unnecessary or not safe 31 . Therefore, healthcare providers should communicate the effectiveness and importance of HPV vaccine with more accurate information.
The current study has its limitations. Given that YouTube is constantly evolving, selecting the top 200 videos at the same time may not accurately reflect what the users view in real. Also, since only English videos were analysed, the results may not represent the entire audience. Moreover, there is no way to determine the demographic characteristics of the audience in the analysis. Finally, a cached browser was used to eliminate the impact of web history on results before searching, but YouTube's 'relevance' function can change search results.

Conclusion
While YouTube has the potential to provide easy and often free access to data, it is vulnerable to unreliable information getting uploaded on the platform. Although there is a lot of evidence of the benefits of immunisation, prejudices against all kinds of vaccinations have been increasing in recent years. With respect to HPV vaccination, YouTube can provide useful information as well as largely misleading and insufficient information. Professional organisations, university channels and doctors should provide accurate, unbiased and evidencebased information to raise community awareness through informative and educational videos to YouTube.