Research Article - The International Tinnitus Journal (2024) Volume 28, Issue 1

Readability and quality assessment of patient education websites related to eustachian tube balloon dilatation

¹Department of Otolaryngology, Barking Havering Redbridge NHS Trust, Queen’s Hospital, Rom Valley Way, Romford, UK

²Department of Otolaryngology, United Lincolnshire Hospital NHS Trust, Lincoln County Hospital, Greetwell Road, Lincoln, Lincolnshire, LN2 5QY, UK

Send correspondence to:
Charles Dibor
Department of Otolaryngology, Barking Havering Redbridge NHS Trust, Queen’s Hospital, Romford, London, England
E-mail: c.dibor1@nhs.net

Paper submitted on December 12, 2023; and Accepted on January 10, 2024

Citation: Charles Dibor. Readability and quality assessment of patient education websites related to Eustachian tube balloon dilatation. Int Tinnitus J. 2024;28(1):001-006

Abstract

Introduction: Eustachian Tube Dysfunction (ETD) occurs as a result of failure of the Eustachian Tube (ET) to equilibrate pressures between the middle ear and the atmosphere, via the nasopharynx. This leads to a plethora of symptoms including otalgia, temporary hearing loss, aural fullness and tinnitus. Eustachian Tube Balloon Dilatation (ETBD) has been proven, through numerous studies, to be a safe method of treating obstructive ETD. Method: 20 websites were located using Google and inclusion/exclusion criteria. The DISCERN website quality assessment tool was used together with Flesh Reading Ease score and Simple Measures of Gobbledygook (SMOG) index to assess readability. Results : Based in the DISCERN quality assessment tool, 55% of the selected websites (11/20) were rated as being of poor quality, 40% (8/20) were rated as being of average or fair quality and only 1 sites out of the 20 was graded to be of good quality. The majority of websites -16 out of 20 (80%) - were rated as “difficult to read”. 1/20 websites (5%) received an average reading grade. The remaining 3 websites (15%) were rated as “easy to read”. 18/20 websites (90%) had readability ratings below the average reading age of 9 years while the remaining 2 websites (10%) had a reading age of 9 years or greater. Discussion: Most of the ETBD websites scored poorly on the topic areas of the DISCERN assessment tool related to informed consent and discussing complications and these are clearly of great importance, irrespective of the country where the surgery is performed. Conclusion: Websites for patient information should be deigned to support a good doctor-patient relationship and the process of informed consent. Authors of Eustachian tube balloon dilatation patient information websites need to keep improving websites through the use of multiple readability indexes and tools and consideration of the DISCERN framework together with other design and usage factors that have a bearing on quality and readability but may not be captured by standardised assessment tools. A Multi-disciplinary teams approach involving IT staff and patients who can offer feedback may lead to better results.

Introduction

Eustachian Tube Dysfunction (ETD) occurs as a result of failure of the Eustachian Tube (ET) to equilibrate pressures between the middle ear and the atmosphere, via the nasopharynx. This leads to a plethora of symptoms including otalgia, temporary hearing loss, aural fullness and tinnitus [1,2]. The ET also serves a number of other functions including clearance of middle ear secretions via its mucociliary mechanism, protection of the middle ear from reflux of nasopharyngeal pathogens and gastric contents as well as modulation of excessive voice and breath sounds. Specific symptoms will vary between patients and will also depend on whether the ETD is deemed to be “obstructive” or “patulous” (a tube that is too patent) [3-6].

ETD is believed to be prevalent in 0.9% of the general population [2]. The exact mechanism of ETD is unknown. However, various theories for the causes of benign ETD have been proposed and include negative middle ear pressure leading to tympanic membrane retraction, middle ear fluid accumulation, obstruction of the ET orifice due to adenoid hypertrophy or microbial overload of the nasopharynx [2]. Specific complications of ETD include chronic otitis media, tympanic membrane retraction and development of a cholesteatoma. Since there is no single test that will confirm the existence of ETD, in addition to a thorough history and examination, many Ear, Nose and Throat (ENT) surgeons will rely on the use of ETD Questionnaires (such as ETDQ-7) and the findings from audiograms and tympanograms in order to support a diagnosis of ETD [4,5]. Principles of ETD diagnosis have been outlined by a European consensus statement. It is suggested that ETD is present when the aforementioned symptoms are experienced for a period of 3 months or longer together with evidence of tympanic membrane retraction, with or without negative middle ear pressure, as confirmed on a tympanogram [3].

Guidelines produced by the American Academy of Otolaryngology–Head and Neck Surgery state that it is insufficient to rely on “patient reported symptoms alone” when trying to establish a diagnosis of ETD and advise that ETDQ7 questionnaires need to be supported with findings from nasal endoscopy and otoscopy examination as well as comprehensive audiometry and tympanometry. Not only does this help to confirm the diagnosis but also helps to exclude other differentials with similar presentations. That being said, the academy and other authors concede that ETD may still be present even when these tests are reported as normal, highlighting the importance of a thorough history and focussed clinical judgement [3,7-10].

Conventional ETD treatments include intranasal steroid sprays, intranasal decongestants and antihistamine therapy. Surgical procedures used in the past include insertion of ventilation tubes (grommets), widening of the ET by surgical debridement or laser tuboplasty and, in the case of patulous ETD, trans-tympanic membrane catheter insertion into the ET [2-6,11]. However, most of these treatments have only provided temporary benefit and refractory ETD can be a source of frustration for patients and ENT surgeons [3-6,11].

An alternative surgical treatment for obstructive with longer-term results is Eustachian Tube Balloon Dilatation (ETBD). First introduced in 2010, this technique involves inserting an empty balloon supported by a rigid plastic tube endoscopically via the nose and guiding it via the ET orifice into the cartilaginous part of the ET whereupon it is inflated under controlled conditions using set guidelines to cause ET dilatation. The inflated balloon causes longitudinal and circumferential crushing of what is believed to be either inflamed mucosa or sub mucosal adenoid-like lymphoid hyperplasia within the cartilaginous ET lumen [1-6]. Justification for the use of ETBD is not exactly clear-cut for all cases of ETD since diagnosis can be difficult to determine.

Since its inception over 10 years ago, ETBD has been proven, through numerous studies, to be a safe method of treating obstructive ETD. Trial results have shown that ETBD patients demonstrate clinically superior responses to this type of treatment compared to groups of patients treated by medical management alone; these improvements in baseline symptoms are sustained over a 12-26 month follow-up period [2-6]. According to the literature, ETBD complications are uncommon. Some of those reported during trials included failure of normalisation of post treatment tympanograms or ETD questionnaire scores (the common measurement of ETBD “success”), induction of patulous ET, and accidental creation of a false passage during the procedure [4]. Other studies have reported minor epistaxis, self-resolving subcutaneous emphysema, acute otitis media (the incidence of this was found to decrease when post-operative antibiotics were used and minor mucosal lacerations) [6].

In the current climate, patients often search for materials related to their medical condition via the internet, which occurs as a consequence of limited access to GPs or outpatient clinic appointments due to long waiting times [12,13]. Additionally, patients and non-ENT healthcare workers may be poorly informed regarding less commonly performed ENT procedures like ETBD that are only performed in certain hospital units. This can lead to confusion and a demand for sources of information away from the hospital environment [14,15]. It is imperative that good websites are available that are designed specifically to meet the needs of patients and convey vital, good quality information in a way that is easy to read and understand.

The average reading age of adults in the UK has been estimated at 9 years old according to research from the National Institute for Health and Care Research (NICE) and National Literacy Trust data estimates that up to 9 million adults in the UK are functionally illiterate and many struggles to understand basic patient educational material. This has been shown to have a detrimental impact on health outcomes and gives further support to a need for producing clear material at the right reading level that can be understood by as many people as possible [16,17]. Therefore, this study will investigate the quality and readability of ETBD web based patient educational materials [18-20].

Methods

Search strategy

The most popular UK search engine by market share is Google with Chrome and Safari being the most frequently used UK browsers on PC and Apple Mac computers, respectively [21-23]. They tend to generate similar search results according to research [24,25]. ““Eustachian tube balloon dilation”” was used as the search term on Google.

Inclusion and Exclusion criteria

Inclusion criteria: websites about “Eustachian tube balloon dilation” for patients.

Exclusion criteria: non-English language websites, journal articles for health professionals, those requiring logins.

Quality assessment of websites using the DISCERN instrument

The DISCERN website quality assessment tool was used [26,27]. The 16 DISCERN questions fall into 3 categories:

1. Reliability of website information (questions 1-8) - answers are graded from 1-5 to indicate a strong “no” (1 out of 5) or a strong “yes” (5 out of 5).

2. Treatment description (questions 9- 15) - answers are graded from 1-5

3. Overall website quality (question 16) -here the website is ranked from low to high quality in the mind of the assessor based on how many important quality characteristics are present or missing and given a score of 1 – 5, respectively. A website can be given a maximum DISCERN score of 80 [28,29] Table 1 and Table 2.

Table 1. Illustrates the questions used in the DISCERN instrument [30]

Table 2. Interpretation of DISCERN scores [31,32]

Readability assessment using Flesch Reading Ease Score and Simple Measures of Gobbledygook Index

Readability of a text is a measure of how easy it is for people of different abilities to read and grasp its meaning based on the language and structure used and the way the content is organised and presented [17]. Multiple readability tools exist. Variation exists between the score generated by different readability assessment tools and there can even be variation among the results obtained when the same tool is used on the same piece of writing at different times [19,22]. For this reason it is prudent to use at least two tools when assessing a piece of work. The two readability tools that were used in this study were the Simple Measure of Gobbledygook (SMOG) Index and the FRE Score.

Readability measuring tools vary in the UK and SMOG has been shown to produce more consistent scores than the other tools, is simpler to use and has more recent validation criteria [19,22,23]. Although traditionally SMOG was obtained through manual counting of words, new online calculators can generate results just with a user entering the website address. This eliminates the risk of human error and makes results consistent, irrespective of which person is applying the formula. The SMOG formula generates a number approximating to the years of education required to understand the article with figure of 10 suggesting a reader would need to be 10 years or older to understand the article. Higher SMOG index numbers suggest a website is more difficult to read. The SMOG indices for the websites were calculated using a readability calculator website [24].

The Flesch Reading Ease (FRE) readability score calculation tool is well recognised in the UK [17-21]. This differs from the Flesch Kincaid Grade Level in the US where the educational grading systems are different [18]. The FRE Score is built into Microsoft Word and can be calculated relatively easily using this programme and works on a manual “copy and paste” system where by text form a website being assessed for readability is copied and pasted directly into word where a readability score can be calculated. While easy to use, this has the problem of inter-rate variability if different portions of website text are selected. There are also copyright issues and the burden of time required if multiple websites are being assessed for readability.

A method that overcomes these problems is to use an online readability score calculator that delivers a website readability score once a website address has been entered. For this reason the same website used to generate the SMOG index was also used to produce a FRE readability score simultaneously [24] Table 3.

Table 3. Interpretation of FRE Scores [33,34]

Results

Overall website features

34 websites were viewed. 10 (29%) were journal articles and 4 (12%) were YouTube videos. 20(59%) met the “patient educational material” inclusion criteria so these were selected. 2 of the 20 websites (10%) were from the UK, 17 (85%) of the websites were from the US, and the remaining website (5%) was from New Zealand.

Quality

The DISCERN scores across the 20 websites ranged from 28-58. The mean DISCERN score was 39 with a standard deviation of +/- 7.5 and a median score of 37.

Based in the DISCERN quality assessment tool, 55% of the selected websites (11/20) were rated as being of poor quality, 40% (8/20) were rated as being of average or fair quality and only 1 sites out of the 20 was graded to be of good quality.

Readability

FRE Score

The majority of websites -16 out of 20 (80%) - were rated as “difficult to read”. 1/20 websites (5%) received an average reading grade. The remaining 3 websites (15%) were rated as “easy to read” Table 4.

Table 4. Summarised the results for the two groups of Readability scores

SMOG Index

18/20 websites (90%) had readability ratings below the average reading age of 9 years while the remaining 2 websites (10%) had a reading age of 9 years or greater.

Correlation between reading scores

The Pearson rank correlation coefficient gave a score of r = -0.73 showing a strong negative correlation between Flesch and SMOG index readability scores.

Discussion

Most of the websites found in this study (90%) were rated by the SMOG Index as having a readability score below the average reading age of 9. The Flesch Reading Ease score rated 80% of the websites as being “difficult to read.” 5% of the websites were of “average reading level” and 10% were “easy to read.” Furthermore, the DISCERN scores rated the quality of these websites as poor (55%) or average (45%) with only 1 out of 20 given a “good” score. This means online material may still be of poor quality even if the reading level is below average and there isn’t a clear relationship between the quality scores and the readability scores. Including a larger number of websites may have made trends between these factors more apparent.

Websites have been shown to positively influence patients’ decision-making [35,36]. The websites sampled here were from 3 different countries (United Kingdom, United States and New Zealand). One consideration is the impact different countries and cultures have on website requirements. Website criteria deemed to be important may be considered superfluous to requirements somewhere else [37-39]. Most of the ETBD websites scored poorly on the topic areas of the DISCERN assessment tool related to informed consent and discussing complications and these are clearly of great importance, irrespective of the country where the surgery is performed. DISCERN has also been shown to give consistent results in different countries. For that reason, shortcomings revealed in this study cannot be dismissed. A balance has to be achieved between too much information and neglecting to mention key facts that may create uncertainty for patients such as whether a procedure has alternatives, which complications are most common and what can be done in the event of certain complication occurring.

Ongoing improvement in the quality and accessibility of patient information websites about Eustachian tube balloon dilatation Good website design will require not only due consideration of DISCERN and readability instruments and further research with larger sample numbers, but also the stellar coordinated efforts of multi-disciplinary teams consisting of healthcare professions, patients who can give feedback on what changes are require and what designs are most helpful and the support of financial and IT teams to correct technical problems as they arise [28,40].

Limitations

Only 20 websites were considered in this study; increasing the number of websites from 20 to 100 would have provided more insights about the links between quality and readability. Because quality was assessed only through the lens of a DISCERN instrument score factors like the design [41], clarity and lay out of ease of use of the site were not considered. It would have been useful to include other scoring systems for website quality that incorporated these features and expanding the number of sites samples would provide more insight into these factors.

It would have been helpful to use more than one online website readability score calculator to check for consistency of scores. Exploring the correlation between each of those score would also have given more information about which readability assessment methods are better matched.

Another important step with obtaining more readability score would be to correlate the DISCERN website quality score with the various readability scores and see which ones were poorly or strongly correlated. This is likely to be more beneficial not only with a larger sample of websites but also with multiple readability scores. This would give greater weight to any findings or recommendations from such calculations that some readability methods may be better correlated to the DISCERN quality score than others [42].

Although we focussed on websites for this study, patients use a variety of approaches to gaining information about operations that include talking to family and friends who have had similar surgeries, the use of YouTube videos and social media to gather information and the traditional patient information leaflets. If a patient has had similar surgery in the past, they may approach information gathering in a different way. There is also the impact of media stories about surgeries. None of these factors where considered in this study and one possibility for future study would be to combine the approach of information gathered here on a larger scale with research about alternative sources of patient information.

We restricted the search term to “eustachian tube balloon dilatation”. It may have been more prudent to include broader terms such as “Eustachian tube surgery”, “balloon dilatation for ears” and surgery for “eustachian tube dysfunction”. Including broader terms like these may give a better representation of the terms [43] patients may use in their search efforts and thus provide a more representative sample for assessment of quality and readability.

Conclusion

Websites for patient information should be deigned to support a good doctor-patient relationship and the process of informed consent. Authors of eustachian tube balloon dilatation patient information websites need to keep improving websites through the use of multiple readability indexes and tools and consideration of the DISCERN framework together with other design and usage factors that have a bearing on quality and readability but may not be captured by standardised assessment tools. A Multi-disciplinary teams approach involving IT staff and patients who can offer feedback may lead to better results.

References

1. Ars B, Dirckx J. Eustachian tube function.  Otolaryngol Clin North Am. 2016;49(5):1121-33.
2. Huisman JM, Verdam FJ, Stegeman I, de Ru JA. Treatment of Eustachian tube dysfunction with balloon dilation: a systematic review. Laryngoscope. 2018;128(1):237-47.
3. Froehlich MH, Le PT, Nguyen SA, McRackan TR, Rizk HG, Meyer TA. Eustachian tube balloon dilation: a systematic review and meta-analysis of treatment outcomes. Otolaryngol Head Neck Surg. 2020;163(5):870-82.
4. Anand V, Poe D, Dean M, Roberts W, Stolovitzky P, Hoffmann K, et al. Balloon dilation of the Eustachian tube: 12-month follow-up of the randomized controlled trial treatment group. Otolaryngol Head Neck Surg. 2019;160(4):687-94.
5. Cutler JL, Meyer TA, Nguyen SA, O’Malley EM, Thackeray L, Slater PW. Long-term outcomes of balloon dilation for persistent eustachian tube dysfunction. Otol Neurotol. 2019;40(10):1322-5.
6. Hwang SY, Kok S, Walton J. Balloon dilation for eustachian tube dysfunction: systematic review. J Lar Otol Lond. 2016 Jul;130(S4):S2-6.
7. Van Roeyen S, Van de Heyning P, Van Rompaey V. Responsiveness of the 7-item Eustachian tube dysfunction questionnaire. J Int Adv Otol. 2016;12(1):106-8.
8. McCoul ED, Anand VK, Christos PJ. Validating the clinical assessment of eustachian tube dysfunction: the Eustachian Tube Dysfunction Questionnaire (ETDQ-7). Laryngoscope. 2012;122(5):1137-41.
9. Tucci DL, McCoul ED, Rosenfeld RM, Tunkel DE, Batra PS, Chandrasekhar SS, et al. Clinical consensus statement: balloon dilation of the eustachian tube. Otolaryngol Head Neck Surg. 2019;161(1):6-17.
10. Schröder S, Lehmann M, Sauzet O, Ebmeyer J, Sudhoff H. A novel diagnostic tool for chronic obstructive eustachian tube dysfunction—the eustachian tube score. Laryngoscope. 2015;125(3):703-8.
11. Oh SJ, Lee IW, Goh EK, Kong SK. Trans-tympanic catheter insertion for treatment of patulous eustachian tube.        Am J Otolaryngol. 2015;36(6):748-52.
12. Turner A, Morris R, Rakhra D, Stevenson F, McDonagh L, Hamilton F, et al. Unintended consequences of online consultations: a qualitative study in UK primary care. Br J Gen Pract. 2022;72(715):e128-37.
13. Leach B, Parkinson S, Gkousis E, Abel G, Atherton H, Campbell J, et al. Digital facilitation to support patient access to web-based primary care services: Scoping literature review. J Med Internet Res. 2022;24(7):e33911  .
14. Chandrashekharayya SH, Kavitha MM, Handi P, Khavasi P, Doddmani SS, Riyas M. To study the level of awareness about complications of chronic suppurative otitis media (CSOM) in CSOM patients. J Clin Diagnostic Res. 2014;8(2):59.
15. McKearney TC, McKearney RM. The quality and accuracy of internet information on the subject of ear tubes. Int J Pediatr Otorhinolaryngol. 2013;77(6):894-7.
16. S. Taylor. Literacy in the workplace: how literate are your emaployees.
17. Gursul D. Health information: are you getting your message across. Nurs Tim. 2022.
18. Kong K, Hu A. Readability assessment of online tracheostomy care resources. Otolaryngol Head Neck Surg. 2015;152(2):272-8.
19. Ferster AP, Hu A. Evaluating the quality and readability of Internet information sources regarding the treatment of swallowing disorders. Ear Nose Throat J. 2017;96(3):128-38.
20. Ting K, Hu A. Evaluating the quality and readability of thyroplasty information on the internet. J Voice. 2014;28(3):378-81.
21. Inclusive writing: Accessibility and readability -definitions and guidance. University of Bristol.
22. Flesch Reading Ease and the Flesch Kincaid Grade Level. Readable.
23. San Giorgi MR, de Groot OS, Dikkers FG. Quality and readability assessment of websites related to recurrent respiratory papillomatosis. Laryngoscope. 2017;127(10):2293-7.
24. United Kingdom Most Used Search Engines in June 2023. Similarweb
25. British Library and the University of Oxford. The DISCERN Instrument.
26. S. Raath. Google Chrome vs. Safari: Which browser is better for iPhone and Mac?. Express VPN.
27. Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. 1999;53(2):105
28. Safari vs. Chrome: Which Is Better and How Do They Compare?. J. Gray.
29. Desktop Browser Market Share Worldwide. Statcounter.
30. Most used search engines by brand in the UK as of June 2023. Statista.
31. Wang LW, Miller MJ, Schmitt MR, Wen FK. Assessing readability formula differences with written health information materials: application, results, and recommendations. Res Social Adm Pharm. 2013;9(5):503-16.
32. Acar A, ISISAG KU. Readability and comprehensibility in translation using reading ease and grade indices. Int J Comp Lit Translat Stud. 2017;5(2):47-53.
33. Grose EM, Holmes CP, Aravinthan KA, Wu V, Lee JM. Readability and quality assessment of internet-based patient education materials related to nasal septoplasty. Otolaryngol Head Neck Surg. 2021;50(1):1-8.
34. Zhou S, Jeong H, Green PA. How consistent are the best-known readability equations in estimating the readability of design standards?. IEEE Trans Prof Commun. 2017;60(1):97-111.
35. Fitzsimmons PR, Michael BD, Hulley JL, Scott GO. A readability assessment of online Parkinson's disease information. J R Coll Physicians Edinb. 2010;40(4):292-6.
36. ‘Readability Test Quick and easy way to test the readability of your work. WebFX.
37. Daraz L, Morrow AS, Ponce OJ, Beuschel B, Farah MH, Katabi A, et al. Can patients trust online health information? A meta-narrative systematic review addressing the quality of health information on the internet. J Gen Intern Med. 2019;34:1884-91.
38. Rothenfluh F, Schulz PJ. Content, quality, and assessment tools of physician-rating websites in 12 countries: quantitative analysis. J Med Internet Res. 2018;20(6):e212.
39. Cermák R. Culturally sensitive website elements and features: A cross-national comparison of websites from selected countries. Acta Inform Prag. 2020;9(2):132-53.
40. Ramos JE. Preoperative education needs in ear, nose, & throat clinic: a patient perspective . Walden University.
41. Steenkamp JB, Geyskens I. How country characteristics affect the perceived value of web sites. J Mark. 2006;70(3):136-50.
42. Vaughan L, Zhang Y. Equal representation by search engines? A comparison of websites across countries and domains. J Comput Mediat. 2007;12(3):888-909.
43. Miniukovich A, De Angeli A, Sulpizio S, Venuti P. Design guidelines for web readability. Designing Interactive Systems. 2017:285-296.