Original Article

High tibial osteotomy: A review of the readability and quality of patient information on the internet

Introduction

Knee osteoarthritis (OA) is the most common joint disorder, representing a significant health burden. ¹ The lifetime risk of developing knee OA is estimated at 47% in women and 40% in men. ² Symptoms of knee OA include pain, stiffness and activity limitations.

Initial management of knee OA is non-operative, with simple analgesics and lifestyle adaptations including weight loss and activity modification to relieve symptoms. If symptoms progress despite optimisation of non-operative measures, surgical options are considered. High tibial osteotomy (HTO) offers an alternative treatment to arthroplasty in active patients with malalignment and isolated compartment OA. HTO is a joint preserving procedure which allows return to high impact activities and sports with no restrictions, ³ whilst delaying or preventing any subsequent requirement for arthroplasty.

Patient’s utilise many types of information when making a decision about elective surgery, including online information, leaflets, and face-to-face consultations. ⁴ Information retention following a face-to-face consultation is low. ⁵ A substantial proportion of patients therefore utilise the internet to research any potential surgery following their initial consultation. ^4,6 The internet is largely unregulated and there are concerns regarding the quality of information provided. ^7,8 Information presented to patients must be of high quality and appropriate readability to promote patient engagement with decision-making and prevent unnecessary anxiety, distress and mis-information.

Readability of text refers to the levels of reading difficulty based on sentence length and syllable count. Readability reflects the comprehension level a person must have to understand the health information presented. It is recommended that the readability of health care literature should be of US school grade level 5 or below equating to age 11 years. ^9,10 The usefulness of online material may also be limited if the information presented is of poor quality. This can misinform the patient and provide a bias picture of the treatment proposed. Previous studies have found information online related to orthopaedics to be of low quality. ^11-13

The patient’s ability to access information and interpret the information can impact the decisions they make about treatment going forward. ¹⁴ Patient’s age and socioeconomic status can affect engagement with online information. ¹⁵ It is important to enable shared decision making around healthcare decisions. In doing so patient’s engage more actively in follow up, which is vitally important following HTO. ¹⁶ To support this shared patient decision making, information provided to patients needs to be clear, accessible, focussed and evidenced based as well as including risks and likely outcomes from each intervention. ¹⁷

In this study, we aim to examine the information that is available on the internet for patients undergoing or considering HTO, for both quality and readability. This is the first study, to our knowledge, which looks at online information relating to HTO.

Materials and Methods

A review of the online literature available to patient’s relating to HTO was conducted and the webpage data analysed.

Two independent investigators (MC + MH) used Google®, Bing® and Yahoo® search engines to search the terms “high tibial osteotomy” and “tibial osteotomy” separately (April 2020). The first three pages of results for each search engine were reviewed. This strategy was chosen as internet users rarely go beyond the first three pages of results. ¹⁸ Any duplicates were removed from subsequent analysis. Inclusion criteria included webpages related to HTO including, articles giving information that is relevant to patients undergoing or considering HTO (indications, techniques, recovery time and outcomes). Exclusion criteria included language other than in English, scholarly articles, advertisements for products and hospitals, personal experience or blogs, paediatric resources, video resources, password protected sites, articles not related to HTO and material solely aimed at medical professionals. A third investigator (IB) resolved any disagreement regarding the inclusion of web pages. The website title and address were recorded on a spreadsheet prior to full data extraction. Raw text was extracted from the webpages and stored. Videos and images were not examined ().

Close

Figure 1. Website inclusion process. A total of 90 webpages were screened with 42 duplicates removed. 24 webpages were removed based on exclusion criteria with 24 webpages remaining for analysis

The readability of each webpage was calculated using the Flesch Reading Ease Scale (FRES), Flesch-Kincaid Grade level (FKGL) and the Simple Measure of Gobbledygook formula (SMOG). A widely available free online tool was used to calculate these scores (https://readabilityformulas.com). Titles, subtitles, references and advertising text were excluded from the analysis, including only body text and bullet points in the analysis.

The FRES and the FKGL provide quantitative measures of readability. The FRES is calculated from the formula 206.835 – 1.015 x (average number of words per sentence) – 84.6 x (average number of syllables per word). It scores the text from 0-100, with a high score representing easier to read text. The FKGL is calculated from 0.39 x (words/sentences) + 11.8 x (syllables/words) – 15.59. The FKGL estimates the US school grade reading ability required to understand the text.

The SMOG readability formula estimates the number of years of education a person requires to understand a piece of written information. In brief, this is calculated from the square root of the number of words with more than three syllables in a sample of 10 sentences at the beginning, middle and end of the text, and adding 3.

DISCERN tool

The DISCERN tool is a validated and reliable instrument for assessing the quality of written healthcare information related to treatment choices. ¹⁷ The DISCERN questionnaire comprises of 16 questions analysing reliability and quality of information. Each question is scored on a Likert scale from 1-5. A score of 1 corresponds to a “no” answer, 3 to “partially” and 5 to “definite yes”. Section one, questions 1 to 8, focuses on the reliability of the source and section two, questions 9 to 15, focuses on the quality of the information provided on treatment choices. Question 16 is an overall judgement of the information. The maximum achievable score is 80.

Example questions examining reliability of the source include:

“Is it (the source) clear what sources of information were used to compile the publication (other than the author or producer)?

Is it (the source) balanced and unbiased?”

Example questions examining quality of the source include:

“Does it describe how each treatment works?

Does it describe how the treatment choices affect overall quality of life?”

Two independent investigators scored each webpage using the DISCERN tool. A disagreement in >2 points for the final score led to the webpage being scored by a third independent investigator. If the third reviewer was within 2 points of one of the original reviews, the mean score was taken of these two. If the third reviewer was >2 points from each of the two previous scores, a mean of the three scores was used.

JAMA benchmarking

The JAMA benchmarks were established by the Journal of the American Medical Association (JAMA). ¹⁹ This provides a score for a webpage from 0-4. The score is generated by scoring one point if each of the following are accounted for:

1. Authorship (authors/contributors and their affiliations and relevant credentials are provided)

2. Attribution (there are clear references and sources for all content included)

3. Disclosure (website ownership should be disclosed with sponsorship, commercial funding/support, conflicts of interest)

4. Currency (dates that the content was posted should be available and dates of any updates).

Health On the Net certification

The Health on the Net Foundation Code of Conduct (HONcode), intends to hold adult health care literature web site developers to basic ethical standards in the presentation of information and help ensure readers always know the source and purpose of what they are reading. ²⁰ Website developers can apply to for a HONcode seal, if they can demonstrate they meet the eight principles of the HONcode. A HONcode seal is a marker of ethical editorial process, similar to JAMA benchmarking, but is not an endorsement of accuracy or reliability of information.

Statistical analysis

The data were checked for normal distribution with the D’Agostino and Pearson normality test. Data are presented as median (inter-quartile range) for non-normally distributed data and mean (standard deviation) for normally distributed data. All analyses were completed on GraphPad Prism version 8.4.2 (San Diego, California, USA).

Results

One hundred and eighty webpages were initially screened. After excluding duplicates between search engines and applying exclusion criteria, 24 webpages were included in the analysis.

Readability

The mean FRES was 53.2 (standard deviation: 9.1), FKGL 10.7 (SD: 1.8), SMOG 10.4 (SD: 1.5). Only 4 webpages (16.6%) had a FRES greater than 60 and only 1 webpage (4%) had a FKGL of less than 8. The mean SMOG was 10.43 (SD: 1.50). None of the webpages had a recommended reading SMOG or FKGL of 5 or below.

Quality

The mean total DISCERN score was 42.0 (SD: 12.3). Total DISCERN scores ranged from 23 to 63. Only 2 webpages (8.3%) had a DISCERN score of greater than 60, with 10/24 (41.7%) webpages scoring less than 40. The full breakdown of scores are displayed in . Websites tended to score higher on descriptions of the treatment, the risks and the advantages. Websites scored poorly on the whole for having clear aims to the publication, clear sources of information, providing additional resources, describing what would happen if no treatment was given and providing support for shared decision making.

Close

Figure 2. Pie chart to show break down of DISCERN scores. Two webpages scored greater than 60 points on the DISCERN questionnaire (yellow), 5 webpages scored 50-59 points (grey), 7 webpages scored 40-49 points (orange) and 10 webpages scored less than 40 points (blue).

No websites met all four of the JAMA benchmarks. One website met three of the criteria, 8 met two of the criteria, 7 met one criterion and 8 webpages did not meet any of the criteria. The individual question results can be seen in Table 1. Two webpages had a health on the net seal of approval (2/24 8.3%).

Discussion

In this study, we find the readability and quality of the information available online for patients to be of a low standard. To our knowledge, this is the first study to assess online patient information for HTO analysing both readability and quality measures. The prospect of any surgery can be daunting to patients. Many will use the internet to research the procedure prior to surgery, therefore ensuring information of good readability and quality online is of high importance.

The mean FRES in our study was 53.2 and the mean FKGL was 10.7, which is significantly higher than the US recommended FKGL of 5 or below. ^9,10 A readability study of information available online for ACL injuries analysing 35 web pages, found a FRES of 47.4 and FKGL 11.77 ¹¹ and a study of patient information materials available on the American Association of Orthopaedic Surgery (AAOS) website found a FKGL of 10.43, with less than 2% of the resources having a readability of less than 6^th grade. ¹³ A study analysing online patient materials for hip and knee joint arthroplasty reports a mean FKGL of 12, with only four articles (3%) having a FKGL of less than 8^th Grade. ²¹ This is similar to our findings with only 4% of websites reporting a reading level below 8^th grade standard. Our readability scores showed a slightly easier level of comprehension than those reported in a study looking at the readability of stem cell therapy in the knee. The authors report a FRES of 35.9 and FKGL of 14. This is likely to be explained as the authors did not exclude academic articles. The results of the non-academic webpages are more consistent with our findings, with FRES ranging from 38.5-52.0. ¹² We excluded academic articles in our study to better represent patient experience online.

US guidelines recommend that health care literature should have readability of US grade reading level 5 or below to promote understanding amongst the general public. ^9,10 The mean SMOG reported in our study was 10.4. This is significantly higher than the fifth-grade target recommended. However, this value was lower than those reported in similar studies of readability in other medical conditions including Parkinson’s disease ²² (mean SMOG 14.6), congestive heart failure ²³ (mean SMOG 11.39) and bariatric surgery (mean SMOG 12.43). ²⁴ Use of fewer words per sentence and fewer syllable words would help make the readability of the information easier online.

The DISCERN questionnaire has been used to examine online information available on stem cell injections in the knee. Ng et al reported a mean DISCERN score of 49.5 for all webpages, including academic articles. ¹² Our mean DISCERN score of 42 is lower than this as we did not include scholarly articles. Our mean DISCERN score is similar to other procedural based studies which excluded academic articles, for example endoscopic retrograde cholangiopancreatography (mean score 42) ²⁵ and caesarean section (mean score 43.6). ²⁶ Only two webpages included in our study had HONCode approval (8%) which is similar to a previous orthopaedic readability study on joint arthroplasty (12%). ²⁷

To help improve readability and quality of information relating to HTO online to promote the information delivered to patients we make the following recommendations. To improve the quality of information, websites should include clear authorship and potential conflicts of interests as well as a clear date of when the information is published. Websites generally performed well when describing HTO and the risks but seldom discussed alternative treatment options or what would happen without HTO. Additionally, including references and the source of the information presented and signposting to further information would help patients read further if they wished. To improve readability, we would encourage website authors to check their content on free online readability checkers, such as those used in this work, to understand the current grade reading age (FKGL or SMOG). If the authors recognise that this level is above the recommendation of 5, then rephrasing and simplifying the content would be recommended. Use of short sentences, and avoiding jargon, will help ensure patients do not misinterpret the information, which could lead to distress or anxiety around HTO. Addressing these issues would help promote patient education and better inform shared decision making outside the consulting room.

Our study has a number of strengths. We examined readability using three methods and quality using the DISCERN questionnaire, JAMA benchmarks and the presence of a HONCode seal. We excluded academic articles to better represent websites which patients are likely to access. There are limitations to this study. Firstly, search engines are dynamic and constantly changing. Therefore, sites may change going forward as more information about HTO becomes available. The accuracy of information presented was not assessed in this study. Furthermore, there are limitations to all current readability measures, with significant miscorrelation between formulae, though by using three separate measures, this mitigates the potential bias of using a single measure.

Conclusions

The online information available to patients considering HTO is generally of low readability and quality. A greater awareness of readability and quality, and how online content creators can use freely-available validated tools and measures to actively quantify these, would encourage improvements, which will help ensure patients’ can access more appropriate information to inform shared decision making. Clinicians should also be aware of issues around readability and quality when signposting patients to online resources.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Competing interests

There are no conflicts of interest to declare.

Ethical approval

Not applicable.

Authors’ contributions

MC designed the study, collected data, performed data analysis and drafted the manuscript. IB collected data and drafted the manuscript. MH collected data and drafted the manuscript. RDS designed the study, collected data and drafted the manuscript. AL coordinated the study and helped to draft the manuscript. All authors read and approved the final manuscript.

The authors claim that no part of this paper is copied from other sources.

References

1. Zhang Y, Jordan JM. Epidemiology of osteoarthritis. Clin Geriatr Med 2010; 26(3):355-69. doi: 10.1016/j.cger.2010.03.001 [Crossref]
2. Lawrence RC, Felson DT, Helmick CG, Arnold LM, Choi H, Deyo RA. Estimates of the prevalence of arthritis and other rheumatic conditions in the United States. Part II. Arthritis Rheum 2008; 58(1):26-35. doi: 10.1002/art.23176 [Crossref]
3. Ekhtiari S, Haldane CE, de Sa D, Simunovic N, Musahl V, Ayeni OR. Return to work and sport following high tibial osteotomy: a systematic review. J Bone Joint Surg Am 2016; 98(18):1568-77. doi: 10.2106/jbjs.16.00036 [Crossref]
4. Atlas A, Milanese S, Grimmer K, Barras S, Stephens JH. Sources of information used by patients prior to elective surgery: a scoping review. BMJ Open 2019; 9(8):e023080. doi: 10.1136/bmjopen-2018-023080 [Crossref]
5. Watson PW, McKinstry B. A systematic review of interventions to improve recall of medical advice in healthcare consultations. J R Soc Med 2009; 102(6):235-43. doi: 10.1258/jrsm.2009.090013 [Crossref]
6. Atci IB, Yilmaz H, Kocaman U, Samanci MY. An evaluation of internet use by neurosurgery patients prior to lumbar disc surgery and of information available on internet. Clin Neurol Neurosurg 2017; 158:56-9. doi: 10.1016/j.clineuro.2017.04.019 [Crossref]
7. Vance K, Howe W, Dellavalle RP. Social internet sites as a source of public health information. Dermatol Clin 2009; 27(2):133-6, vi. doi: 10.1016/j.det.2008.11.010 [Crossref]
8. Cassidy JT, Baker JF. Orthopaedic patient information on the World Wide Web: an essential review. J Bone Joint Surg Am 2016; 98(4):325-38. doi: 10.2106/jbjs.n.01189 [Crossref]
9. The Joint Commission. The Joint Commission: Advancing Effective Communication, Cultural Competence, and Patient- and Family-Centered Care: A Roadmap for Hospitals. Oakbrook Terrace, IL: The Joint Commission; 2010. p. 94.
10. U.S. Department of Health and Human Services (HHS). Saying it Clearly. Washington, DC: HHS; 2000.
11. Gosselin MM, Mulcahey MK, Feller E, Hulstyn MJ. Examining internet resources on gender differences in ACL injuries: what patients are reading. Knee 2013; 20(3):196-202. doi: 10.1016/j.knee.2012.11.006 [Crossref]
12. Ng MK, Mont MA, Piuzzi NS. Analysis of readability, quality, and content of online information available for “stem cell” injections for knee osteoarthritis. J Arthroplasty 2020; 35(3):647-51. doi: 10.1016/j.arth.2019.10.013 [Crossref]
13. Sabharwal S, Badarudeen S, Unes Kunju S. Readability of online patient education materials from the AAOS web site. Clin Orthop Relat Res 2008; 466(5):1245-50. doi: 10.1007/s11999-008-0193-8 [Crossref]
14. Australian Commission on Safety and Quality in Health Care (ACSQHC). Consumers, the Health System and Health Literacy: Taking Action to Improve Safety and Quality. ACSQHC; 2013.
15. Wong C, Harrison C, Britt H, Henderson J. Patient use of the internet for health information. Aust Fam Physician 2014; 43(12):875-7.
16. Page AE. Safety in surgery: the role of shared decision-making. Patient Saf Surg 2015; 9:24. doi: 10.1186/s13037-015-0068-3 [Crossref]
17. Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health 1999; 53(2):105-11. doi: 10.1136/jech.53.2.105 [Crossref]
18. Hirsch M, Aggarwal S, Barker C, Davis CJ, Duffy JMN. Googling endometriosis: a systematic review of information available on the internet. Am J Obstet Gynecol 2017; 216(5):451-8. doi: 10.1016/j.ajog.2016.11.1007 [Crossref]
19. Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the internet: caveant lector et viewor--Let the reader and viewer beware. JAMA 1997; 277(15):1244-5.
20. Health on the Net. Health on the Net. Available from: https://www.hon.ch/en/. Accessed September 8, 2020. Published 2020.
21. Doinn TÓ, Broderick JM, Abdelhalim MM, Quinlan JF. Readability of patient educational materials in Hip and Knee arthroplasty: has a decade made a difference?. J Arthroplasty 2020; 35(11):3076-83. doi: 10.1016/j.arth.2020.05.076 [Crossref]
22. Fitzsimmons PR, Michael BD, Hulley JL, Scott GO. A readability assessment of online Parkinson’s disease information. J R Coll Physicians Edinb 2010; 40(4):292-6. doi: 10.4997/jrcpe.2010.401 [Crossref]
23. Kher A, Johnson S, Griffith R. Readability assessment of online patient education material on congestive heart failure. Adv Prev Med 2017; 2017:9780317. doi: 10.1155/2017/9780317 [Crossref]
24. Meleo-Erwin Z, Basch C, Fera J, Ethan D, Garcia P. Readability of online patient-based information on bariatric surgery. Health Promot Perspect 2019; 9(2):156-60. doi: 10.15171/hpp.2019.22 [Crossref]
25. Kaicker J, Dang W, Mondal T. Assessing the quality and reliability of health information on ERCP using the DISCERN instrument. Health Care Curr Rev 2013; 1(1):104. doi: 10.4172/2375-4273.1000104 [Crossref]
26. Fioretti BT, Reiter M, Betrán AP, Torloni MR. Googling caesarean section: a survey on the quality of the information available on the internet. BJOG 2015; 122(5):731-9. doi: 10.1111/1471-0528.13081 [Crossref]
27. Davaris MT, Dowsey MM, Bunzli S, Choong PF. Arthroplasty information on the internet. Bone Jt Open 2020; 1(4):64-73. doi: 10.1302/2633-1462.14.bjo-2020-0006 [Crossref]