Original Article

Psychometric properties and validation of short health literacy screening (SHLS) to identify low health literacy

Introduction

Health literacy, as a key determinant of health, is crucial for improving healthcare services and reducing health disparities.¹ Health literacy is defined by the World Health Organization (WHO) as “cognitive and social skills, which determine the motivation and ability of individuals to gain access to understand and use information in ways which promote and maintain good health”.² It is more than just reading and writing; it involves several abilities that enable people to interact more actively in their community and take a greater role in their own health-related actions and decision-making.³ Inadequate or low health literacy has become a global public health concern.⁴ Poor health literacy has been related to non-adherence to medications, higher rate of hospitalization, reduced use of preventive interventions, greater health care costs, higher mortality rates in the elderly, as well as worse overall health status. Low literacy also perpetuates existing disparities and follows a socioeconomic gradient.⁵ Along with the growing attention to health literacy at both national and international scales, there has been an increasing need to develop reliable and valid tools for the measurement of health literacy.⁶ The evaluation of health literacy in individuals seeking care, and even healthy people, may contribute to improvement of care delivery by healthcare facilities.⁷

There are some validated tools to evaluate health literacy, including Rapid Estimate of Adult Literacy (REALM)⁸ and Short Test of Functional Health Literacy in Adults (STOFHLA).^9,10 However, available instruments are either too time-consuming and could make patients feel embarrassed.¹¹ In addition, the majority of standardized instruments were designed and developed for English-speaking individuals, potentially limiting their utility in multicultural nations. It has been recognized that numerous health literacy tools do not account for the cultural and contextual factors affecting health literacy.¹² Recognizing patients’ cultural backgrounds and values helps to improve the validity of research tools. Surveys that are brief, simply worded, easy to understand, and presented by professionals who speak the local language will facilitate the trust and confidence of the target population.¹² In busy clinical settings, short and easy-to-use screening tool could help physicians identify people with limited health literacy, who may need special support and communication methods. Large-scale research to comprehend the impacts of health literacy and the efficacy of interventions would also be more feasible with these screening tools.¹³ Brief health literacy screening questions were developed by Chew et al¹⁴ to assess perceived difficulties in understanding, reading, and reporting medical information. It provides a quick method for detecting patients with low health literacy, making it suitable for clinical use.

Although this screening tool has been translated into several languages,^15-18 no study has investigated its psychometric properties among the Iranian people. To our knowledge, there has been no implementation of a screening questionnaire for evaluating health literacy in Iran, which encompasses the majority of health literacy components and aligns with the values of Iranian culture. Considering the critical role of health literacy in public health, it is beneficial to screen health literacy levels among the Iranian population. This endeavor will serve as a valuable tool for evaluating whether the social and health ambitions of Iran aimed at sustainable health promotion are successfully met. Therefore, the aim or our study was to assess the psychometric properties and performance of Chew screening questions, along with a fourth question developing based on our context, as short health literacy screening (SHLS), to identify individuals with inadequate and limited health literacy based on Health Literacy for Iranian Adults (HELIA) questionnaire in a large sample of inpatients, outpatients and healthy individuals in two large cities in Iran.

Methods

Study Population

A cross-sectional study was conducted to investigate the psychometric properties of health literacy screening questions between April 2023 and September 2023. For a comprehensive evaluation of the health literacy of Iranian people, we conducted our research in two large cities in Iran: (1) Mashhad, (2) Shiraz. In each city, we collected our study participants from three types of people: (1) inpatients, (2) outpatients, and (3) healthy individuals.

• Inpatient individuals were collected from internal medicine and surgery departments of three types of hospitals: (1) public (governmental), (2) semi-private, and (3) private hospitals. Public hospitals were referral and governmental. Semi-private hospitals were non-referral, and part of their budget was funded by the government. Private hospitals were non-referral, and their budget was provided by the private sector. The required sample size from inpatients was obtained by the convenience sampling method on randomly selected days of the week.

• Outpatients were collected from two types of clinics (with different socio-economic status): (1) public (governmental), and (2) private clinics. Public clinics that were governmental and referral or non-referral (two types of public clinics were considered). Private clinics were non-referral, and their budget was provided by the private sector. The required sample size from outpatients was obtained by the convenience sampling method on randomly selected days of the week.

• Healthy people were selected from non-first-degree healthy companions of hospitalized patients who had no history of hospitalization in the last year. The required sample size from healthy individuals was obtained by the convenience sampling method on randomly selected days of the week, on the same days as the inpatients were collected.

We conducted this study to determine whether the diagnostic sensitivity of the SHLS for detecting low health literacy is at least acceptable. Based on conventional screening benchmarks—where values < 0.70 are commonly regarded as sub-optimal for screening tools and ≥ 0.80-0.85 as desirable—we set the null hypothesis at p₀ = 0.70 (minimum acceptable sensitivity) and the alternative at p₁ = 0.85 (anticipated sensitivity). With a two-sided alpha = 0.05 and 80% power, using Bujang and Adnan¹⁹ method, the required sample size was n = 82 for testing sensitivity against p₀. Because the target population comprised inpatients, outpatients, and healthy individuals, and we planned stratified analyses to ensure each subgroup met the same minimum operating characteristics, we applied n = 82 per stratum (total n = 246 per city). To maintain interpretability across subgroups, we aimed for approximately equal recruitment within each stratum. With two cities, the planned sample was n = 492, and allowing ~10% attrition/non-evaluable data, the final target was n = 540.

To be eligible, participants had to be Iranian, able to speak Persian, and between 18 and 65 years old. Study participants who claimed to be illiterate were excluded from the study. In addition, we excluded individuals who were too ill to complete the questionnaire, had diminished decision-making capacity, severely impaired vision, hearing problems, overt psychiatric disorders, and severe cognitive impairments.

Study Protocol

The study purpose was explained to all participants by trained research assistants, and they were informed that their responses would be anonymous. After obtaining informed consent, they completed the questionnaire, which included questions about demographic and socioeconomic characteristics, followed by four screening questions and the HELIA questionnaire, under the supervision of research assistants. The ethics committees of Mashhad and Shiraz Universities of Medical Sciences approved the study (Ethical codes: IR.MUMS.MEDICAL.REC.1401.702 and IR.SUMS.REC.1402.069, respectively).

Demographics

Data collected included single-item questions for age, gender, city (Mashhad, Shiraz), job status (employed, unemployed (or housekeeper)), educational status (under the diploma, diploma, or higher), and participants’ status (inpatient, outpatient, or healthy).

Health Literacy Screening Questions

SHLS is a four-item assessment tool which included Chew et al¹⁴ three screening items: “How often do you have someone (like a family member, friend, hospital/clinic worker or caregiver) help you read hospital materials?” (Help Read), “How often do you have problems learning about your medical condition because of difficulty understanding written information?” (Problems Reading), and “How confident are you filling out forms by yourself?” (Confident with Forms). These questions were translated into Persian. The forward-backward translation approach was used for translation. Accordingly, two independent English language professionals translated the English questionnaire into Persian. Afterwards, two professionals who had not received the original English questionnaire translated the Persian version back into English. A research team comprising clinical and language experts then compared it with the original version to identify any areas or inconsistencies where the meaning may have changed. The wording of the questions was slightly modified to accommodate language differences and enhance participants’ understanding. Also, we added the fourth question, “How well do you understand the medical prescriptions your doctor told you?” (Comprehension of Prescriptions), based on the experts’ opinions. Responses were scored on a Likert scale from 0 to 4. The response choices for “Help Read” and “Problems Reading” questions included always, often, sometimes, occasionally, and never, and for “Confident with Forms” and “Comprehension of Prescriptions” included not at all, a little bit, somewhat, quite a bit, and extremely. The total score ranges from 0 to 16 points, and higher scores represent a higher level of health literacy.

Validity and Reliability Assessment

Content validity was assessed by a group of ten experts with backgrounds in epidemiology, biostatistics, medical education, community medicine, and public health, all have conducted research in the field of health literacy. The content validity index (CVI) was used to evaluate the screening questions’ relevancy based on a four-point scale (1 = not relevant, 2 = needs essential revision, 3 = relevant but needs revision, and 4 = very relevant). The CVI was calculated by dividing the number of respondents who scored three or four by the total number of them. A score below 0.70 indicates the item is unacceptable and should be removed.

To assess the face validity, the questionnaire was given to relevant experts (with backgrounds in epidemiology, biostatistics, medical education, community medicine, and public health) for evaluation of each item for relevancy, clarity, difficulty, grammar, and vocabulary as a quick overall validity of the items, and they noted no specific problems.

Internal consistency reliability was measured by Cronbach’s alpha, with an alpha value of 0.7 or higher considered acceptable. To evaluate the test-retest reliability, at least 40 participants completed the questionnaire twice within a two-week interval. Then, the Spearman-Brown correlation coefficient was computed. It was interpreted as follows: 0.40 to 0.69 represented a moderate correlation, 0.70 to 0.89 and 0.90 to 1 were considered strong and very strong correlations, respectively.

Comparison Standard

To evaluate the performance of SHLS, we used two comparison standards: inadequate health literacy and the summation of inadequate and marginal as limited health literacy, as defined by the HELIA questionnaire. The HELIA, developed by Montazeri et al,²⁰ is the first native questionnaire for the measurement of health literacy among the Iranian population. The psychometric properties of this instrument have been demonstrated in several studies.^21-23 The HELIA is a 33-item reading assessment tool that contains five subscales (dimensions): reading (4 items), access to information (6 items), understanding (7 items), appraisal (4 items), and decision-making/behavioral intention (12 items). The total score ranges from 0 to 100 points. According to established cutoff scores, participants were divided into four categories: inadequate (0-50), marginal (50.1-66), adequate (66.1-84), and excellent health literacy (84.1-100). Also, two categories of inadequate and marginal were considered as limited (0-66), as well as adequate and excellent levels as sufficient health literacy (66.1-100).

Statistical Analysis

The accuracy of SHLS was compared with two comparison standards: inadequate and limited health literacy, as assessed by HELIA. Receiver operating characteristic (ROC) curves, which illustrate the sensitivity versus (1-specificity), were used to determine the optimal trade-off between sensitivity and specificity compared with a reference standard. We calculated the areas under the receiver operating characteristic curve (AUROC) with 95% confidence intervals (CIs) to compare the overall performance of the screening items. The optimal screening of inadequate health literacy would have an AUROC of 1.0, while an AUROC below the null value of 0.5 indicates no information is provided. To determine which question(s) or different combinations of them would be optimal, we selected an individual question with the greatest AUROC and compared it with AUROCs of all other questions or combinations of them, considering the AUROC correlations from the same population. We calculated specificity, sensitivity, and negative and positive likelihood ratios (LRs) with 95% CI for different responses to screening questions. Youden’s J index ([sensitivity + specificity] - 1) was also calculated, which indicates overall effectiveness in terms of both sensitivity and specificity. Closer values to 1 indicate greater effectiveness. Furthermore, the Hanley test was used to compare the AUROCs. The effect size using Cohen’s d was also calculated for each question. Mann-Whitney U test was used for assessing the hypothesis of no difference across groups. We did not employ any imputation method in our analysis, since our dataset had no missing values. The P value < 0.05 was considered statistically significant. MedCalc version 22.009 (MedCalc Software, Mariakerke, Belgium) was used in this study.

Results

Validity and Reliability Assessment

The panel of experts included ten specialists evaluated face validity and content validity using CVI. For face validity assessment, experts’ opinions were collected and used to modify SHLS. None of the items were removed. The CVI scores for “Help Read”, “Problems Reading”, “Confident with Forms”, and “Comprehension of Prescriptions” were acceptable (0.92, 0.78, 0.85, and 0.85, respectively).

Reliability was evaluated by calculating Cronbach’s alpha and the Spearman-Brown correlation coefficient. Cronbach’s alpha coefficients were more than 0.7 for all the factors (0.713), indicating that all items had acceptable internal consistency. The Spearman-Brown correlation coefficient was used to assess the test-retest reliability. It was calculated as 0.827, 0.928, 0.581, and 0.712 for the aforementioned screening questions, respectively (P value < 0.001 for all).

Participants’ Characteristics

A total of 547 individuals were included in our study. Of them, 62% (339) were female, and most of them were between 18 and 45 years (415, 75.9%). The questionnaires were collected from outpatients (32.7%, 179), inpatients (32.5%, 178), and healthy individuals (34.7%, 190). The prevalence of inadequate and limited health literacy among study participants, based on the HELIA questionnaire, was 10.2% (56) and 35.1% (192), respectively (Table 1).

Detecting Inadequate and Limited Health Literacy

The AUROCs with 95% CI for each of the screening questions to detect people with inadequate and limited health literacy are shown in Table 2. All the questions had statistically significant discriminatory ability. Compared to other screening questions, the “Comprehension of Prescriptions” item showed higher AUROC, with 0.690 (0.620-0.759) for inadequate health literacy and 0.666 (0.619-0.713) for limited health literacy. Figure 1 illustrates the ROC curves for each of the screening questions in detecting inadequate and limited health literacy. For both inadequate and limited health literacy, the Hanley test showed a significant difference in the AUROC between the “Help Read” and “Comprehension of Prescriptions” questions.

Close

Figure 1.

Receiver operating characteristic (ROC) curves of four screening questions to identify inadequate (A) and limited (B) health literacy

Figure 1.

Receiver operating characteristic (ROC) curves of four screening questions to identify inadequate (A) and limited (B) health literacy

To determine whether the combining of screening questions may enhance their performance, we performed analyses evaluating the screening performance of different combinations of questions. The AUROCs of various combinations of questions for detecting inadequate and limited health literacy are presented in Table 3. The AUROC of “Confident with Forms” & “Comprehension of Prescriptions” and all four questions were higher than the best-performing individual question (“Comprehension of Prescriptions”) for inadequate and limited health Literacy, respectively. Screening with “Confident with Forms” & “Comprehension of Prescriptions” had an AUROC of 0.705 (0.638-0.773) for inadequate health literacy, and screening with all four questions had an AUROC of 0.683 (0.636-0.729) for limited health literacy. The results indicated that these two combinations could alternatively be used as fair or reasonable screening tests. All the combinations had statistically significant discriminatory ability. ROC curves for different combinations of screening questions for identifying inadequate and limited health literacy are shown in Figure 2. For both inadequate and limited health literacy, based on the Hanley test, the comparisons of “Help Read & Problems Reading” with “All Four Questions,” “Confident with Forms & Comprehension of Prescriptions,” and “Problems Reading & Comprehension of Prescriptions” were significant. Similarly, the comparisons involving “Help Read & Confident with forms” with “All Four Questions,” “Confident with Forms & Comprehension of Prescriptions,” and “Problems Reading & Comprehension of Prescriptions” were also significant.

Close

Figure 2.

Receiver operating characteristic (ROC) curves of different combinations of screening questions to identify inadequate (A) and limited (B) health literacy

Figure 2.

Receiver operating characteristic (ROC) curves of different combinations of screening questions to identify inadequate (A) and limited (B) health literacy

Sensitivities, specificities, Youden’s J indices, and negative and positive LRs with 95% CI for different responses of the screening questions are displayed in Tables 4 and 5. According to Youden’s J Index, “Somewhat” and “Quite a bit” were the optimal cutoff responses for detecting inadequate and limited health literacy, respectively. However, Youden’s J indices should be interpreted cautiously as the values for sensitivity and specificity are equally weighted. When using screening questions, sensitivity and specificity are not always regarded as equally important. It seems that sensitivity to be more valued than specificity.^24,25 Therefore, we considered the response “Quite a bit” as the optimal screening threshold for “Comprehension of Prescriptions” to identify people with inadequate and limited health literacy. For inadequate health literacy, the LR + for this response was 1.45 (1.27-1.65), and the LR- was 0.35 (0.18-0.67). The response of “Quite a bit” or less confident increased the odds of inadequate health literacy by 1.45-fold, while a more confident response reduced the odds by 0.35-fold. The effect sizes for inadequate health literacy ranged from 0.340 for the “help read” question to 0.565 for the “comprehension of prescriptions” question. For limited health literacy, the LR + for this response was 1.53 (1.35-1.73), and the LR- was 0.42 (0.31-0.57). A response of “Quite a bit” or less confident increased the odds of limited health literacy by 1.53-fold, while a more confident response reduced the odds by 0.42-fold. The effect sizes for limited health literacy ranged from 0.323 for the “help read” question to 0.539 for the “comprehension of prescriptions” question. The Mann–Whitney U test results were significant for all questions in both the inadequate and limited health literacy groups. This indicated that, for both categories, question scores differed significantly from those with higher health literacy.

Discussion

Our study, the first to evaluate the psychometric properties of the Persian translation of the health literacy screening questions, showed that SHLS had acceptable psychometric properties. Chouinard et al¹⁸ evaluated the psychometric properties of French translation of Chew screening questions among patients with chronic disorders in a primary care setting. The intraclass correlation coefficient, assessing test-retest reliability, was 0.69 (0.45-0.83), and Cronbach’s alpha for internal consistency was 0.77. In another study, Cronbach’s alpha for Chew screening questions, administered by research assistants, was 0.79 and 0.71 among hospital and clinic patients, respectively.⁷ These screening questions have also been translated into Spanish,¹⁶ Arabic,¹⁵ and Turkish,¹⁷ but their test-retest reliability or internal consistency were not assessed. No data were found that evaluated the content validity and face validity of these health literacy screening questions.

Our investigation is the first to validate the health literacy screening questions for identifying individuals with inadequate and limited health literacy using the HELIA questionnaire. Our study demonstrated that the “Comprehension of Prescriptions” question showed a better performance than the other three questions in detecting inadequate and limited health literacy. This question was slightly more accurate in identifying respondents with inadequate health literacy, as indicated by a higher AUROC than limited health literacy. The combination of “Confident with Forms & Comprehension of Prescriptions” had higher performance than the mentioned individual question for inadequate health literacy. Additionally, all four items together had a higher AUROC than “Comprehension of Prescriptions” for identifying limited health literacy, indicating that the tool as a whole is more sensitive than its components.

Chew et al¹³ designed a 16-item self-reported health literacy screening questionnaire and then condensed it into a shortened version of three questions. They later revealed that the individual “Confidence with Forms” question was the best predictive of inadequate health literacy out of three questions. However, this item performed less well in those with inadequate/marginal health literacy.¹⁴ Other studies have also shown that it performed best in detecting inadequate health literacy.^16,26-28 However, in a sample of patients at a university-based vascular surgery clinic²⁹ and those in a Veterans Affairs (VA) preoperative clinic,¹³ the question “Help Read” showed better performance in detecting inadequate health literacy. In contrast to our results, Chew et al¹⁴ found that no combination of three questions had better performance in comparison to the “Confidence with Forms” question. These different results may be related to factors that have not yet been clarified, including the demographics of study populations and the context of healthcare systems. It is important to conduct further research to understand the reasons behind these differences. In another study evaluating the performance of Chew screening questionnaire among English and Spanish-speaking individuals with type 2 diabetes, it has been shown that the summative scale performed as well as the single question “Confident with Forms”.²⁸ Haun et al¹¹ investigated the efficacy of these questions along with the fourth question “How often do you have a problem understanding what is told to you about your medical condition?” to assess difficulties with auditory health information in a VA ambulatory care setting. Consistent with our findings, the combination of all questions had a greater AUROC than either of the individual questions for identifying inadequate and inadequate/marginal health literacy.

To establish the optimal cut point for a screening instrument, several factors need to be considered, including test accuracy, testing expenses, the prevalence of low health literacy, and the benefits of detecting true positives and false positives. If a screening test is intended to identify individuals lacking adequate health literacy skills, one would select a cutoff response with high sensitivity and low LR- thereby those with negative test results are most probably to have adequate health literacy. But if the aim is to detect people with poor health literacy, a cutoff response with high specificity and high LR + would be used thereby those with positive test results are most probably to have poor health literacy. However, based on the prevalence of insufficient health literacy in the study population, a positive or negative test could have different implications.¹⁴ We recommended using the “Quite a bit” response to “How well do you understand the medical prescriptions your doctor told you?” as the optimal cutoff value for detecting inadequate and limited health literacy. This cut-point, with reasonable specificity, identified 79.69% of individuals with limited and 85.71% of people with inadequate health literacy in our study population. The optimal cutoff values for these questions need to be determined in other clinical settings with different prevalence of low health literacy¹³ and depending on whether specificity or sensitivity should be emphasized.²⁷

To achieve successful health-related outcomes, health literacy assessment must be incorporated into clinical settings.³⁰ The SHLS developed in this study can be used in primary care settings, where health care professionals often face challenges in establishing an effective relationship with patients from diverse educational and cultural backgrounds. It can also be useful for the management of patients with chronic disorders to improve their compliance and prognosis. As a validated, easy-to-use screening tool, SHLS can help healthcare providers identify individuals with limited health literacy who may need targeted support and communication strategies. SHLS can also be implemented in electronic medical record systems and serve as a helpful guide for health care providers when encountering patients, especially in busy clinical settings.

Our research featured a number of notable strengths. We chose a random sample of people from two of the biggest cities in the east and center of Iran with different cultural context. In addition, study participants were selected from inpatient and outpatient settings with different socioeconomic status as well as healthy people to achieve a more representative results. Due to the large sample size, we were able to measure the effectiveness of screening questions much more precisely. We also added a fourth question regarding the comprehension of medical prescriptions to better adaptation to our healthcare system context.

In relation to literacy levels, cultural beliefs regarding health and illness have a significant effect on the individuals’ ability to comprehend and adhere to a physician’s advises.³¹ It is argued that cultural minority patients are more affected by low health literacy than those from the dominant culture, due to interactions between literacy, language barriers, and biased experiences.³² Moreover, cultural beliefs affect educational levels and the ability to obtain healthcare information.³⁰ The extent of cultural diversity among various populations and the affected factors may contribute to disparities in health literacy levels and assessment tool performances across different studies. The main advantage of the SHLS is that it is brief, requiring only a few minutes to complete, and is a useful tool for screening limited health literacy among Iranian populations. Given the cultural diversity in our country and its impact on health literacy, the SHLS can be utilized as a screening tool, and based on its results, more culture-based methods can be used to improve the quality of health care services.

Despite these strengths, this study has some drawbacks. As a limitation for the screening tool, the limited number of questions in SHLS (four items), make it difficult to conduct factor analysis. Demographic characteristics including age and educational status were not evaluated alone or combined with SHLS to identify people with inadequate and limited health literacy. We used self-reported or subjective questionnaire in this study and response bias remains a possibility. While we incorporated participants from three distinct groups in two of Iran’s main cities, larger multicentric studies are necessary to enhance the generalizability for the entire Iranian population.

Future research could focus on how screening results be used for improving physician-patient communication and adding calculators, demographic variables, and other tools to address low health literacy. The effectiveness of separate ways of administrating screening questions (written or verbal) could also be explored. People with positive screening for low health literacy may benefit from interventions using special support and communication methods to help them navigate the healthcare system effectively.

Conclusion

Considering our results, the single question “How well do you understand the medical prescriptions your doctor told you?” and the summative scale of four screening questions as SHLS could discriminate people with inadequate and limited health literacy. Our study suggests the application and further evaluation of SHLS in epidemiological and clinical research among different populations.

Competing Interests

None to declare.

Ethical Approval

The ethics committees of Mashhad and Shiraz Universities of Medical Sciences approved this study (Ethical codes: IR.MUMS.MEDICAL.REC.1401.702 and IR.SUMS.REC.1402.069, respectively). Informed verbal consent was obtained from study participants before completing the questionnaire.

References

1. Ghanei Gheshlagh R, Mahmoodi H, Pelikan JM, Afkhamzadeh A, Ebadi A. Psychometric evaluation of the Persian version of the European Health Literacy Instrument (P-HLS-EU-Q47). Health Promot Int 2023; 38(5):daad135. doi: 10.1093/heapro/daad135 [Crossref] [ Google Scholar]
2. World Health Organization (WHO). Track 2: Health Literacy and Health Behaviour. Available from: https://www.who.int/teams/health-promotion/enhanced-wellbeing/seventh-global-conference/health-literacy. Accessed November 14, 2023.
3. Miller TA. Health literacy and adherence to medical treatment in chronic and acute illness: a meta-analysis. Patient Educ Couns 2016; 99(7):1079-86. doi: 10.1016/j.pec.2016.01.020 [Crossref] [ Google Scholar]
4. Liu H, Zeng H, Shen Y, Zhang F, Sharma M, Lai W. Assessment tools for health literacy among the general population: a systematic review. Int J Environ Res Public Health 2018; 15(8):1711. doi: 10.3390/ijerph15081711 [Crossref] [ Google Scholar]
5. Baccolini V, Rosso A, Di Paolo C, Isonne C, Salerno C, Migliara G. What is the prevalence of low health literacy in European Union member states? A systematic review and meta-analysis. J Gen Intern Med 2021; 36(3):753-61. doi: 10.1007/s11606-020-06407-8 [Crossref] [ Google Scholar]
6. Miri MR, Malaki Moghadam H, Eftekhari H, Yousefi A, Norozi E. Developing and validating the functional, communicative, and critical health literacy questionnaire among the Iranian general population. Oman Med J 2020; 35(2):e106. doi: 10.5001/omj.2020.24 [Crossref] [ Google Scholar]
7. Wallston KA, Cawthon C, McNaughton CD, Rothman RL, Osborn CY, Kripalani S. Psychometric properties of the brief health literacy screen in clinical practice. J Gen Intern Med 2014; 29(1):119-26. doi: 10.1007/s11606-013-2568-0 [Crossref] [ Google Scholar]
8. Davis TC, Crouch MA, Long SW, Jackson RH, Bates P, George RB. Rapid assessment of literacy levels of adult primary care patients. Fam Med 1991; 23(6):433-5. [ Google Scholar]
9. Parker RM, Baker DW, Williams MV, Nurss JR. The test of functional health literacy in adults: a new instrument for measuring patients’ literacy skills. J Gen Intern Med 1995; 10(10):537-41. doi: 10.1007/bf02640361 [Crossref] [ Google Scholar]
10. Lee SY, Bender DE, Ruiz RE, Cho YI. Development of an easy-to-use Spanish health literacy test. Health Serv Res 2006; 41(4 Pt 1):1392-412. doi: 10.1111/j.1475-6773.2006.00532.x [Crossref] [ Google Scholar]
11. Haun J, Noland-Dodd V, Varnes J, Graham-Pole J, Rienzo B, Donaldson P. Testing the BRIEF health literacy screening tool. Fed Pract 2009; 26(12):24-31. [ Google Scholar]
12. Nair SC, Satish KP, Sreedharan J, Ibrahim H. Assessing health literacy in the eastern and middle-eastern cultures. BMC Public Health 2016; 16:831. doi: 10.1186/s12889-016-3488-9 [Crossref] [ Google Scholar]
13. Chew LD, Bradley KA, Boyko EJ. Brief questions to identify patients with inadequate health literacy. Fam Med 2004; 36(8):588-94. [ Google Scholar]
14. Chew LD, Griffin JM, Partin MR, Noorbaloochi S, Grill JP, Snyder A. Validation of screening questions for limited health literacy in a large VA outpatient population. J Gen Intern Med 2008; 23(5):561-6. doi: 10.1007/s11606-008-0520-5 [Crossref] [ Google Scholar]
15. Fadda M, Kanj M, Kabakian-Khasholian T, Johannes Schulz P. Validation of three Arabic health literacy assessment tools in Lebanon. Health Promot Int 2018; 33(2):261-7. doi: 10.1093/heapro/daw079 [Crossref] [ Google Scholar]
16. Singh R, Coyne LS, Wallace LS. Brief screening items to identify Spanish-speaking adults with limited health literacy and numeracy skills. BMC Health Serv Res 2015; 15:374. doi: 10.1186/s12913-015-1046-2 [Crossref] [ Google Scholar]
17. Eyüboğlu E, Schulz PJ. Validation of Turkish health literacy measures. Health Promot Int 2016; 31(2):355-62. doi: 10.1093/heapro/dau111 [Crossref] [ Google Scholar]
18. Chouinard MC, Lambert M, Lavoie M, Lambert SD, Hudon É, Dumont-Samson O. Measuring health literacy in primary healthcare: adaptation and validation of a French-language version of the brief health literacy screening among patients with chronic conditions seen in primary care. Int J Environ Res Public Health 2022; 19(13):7669. doi: 10.3390/ijerph19137669 [Crossref] [ Google Scholar]
19. Bujang MA, Adnan TH. Requirements for minimum sample size for sensitivity and specificity analysis. J Clin Diagn Res 2016; 10(10):YE01-6. doi: 10.7860/jcdr/2016/18129.8744 [Crossref] [ Google Scholar]
20. Montazeri AL, Tavousi M, Rakhshani F, Azin SA, Jahangiri K, Ebadi M, et al. Health Literacy for Iranian Adults (HELIA): development and psychometric properties. Payesh 2014;13(5):589-99. [Persian].
21. Haeri-Mehrizi A, Tavousi M, Rafieifar S, Soleymanian A, Sarbandi F, Ardestani MS, et al. Health Literacy for Iranian Adults (HELIA): the confirmatory factor analysis. Payesh 2016;15(3):251-7. [Persian].
22. Zareban I, Izadirad H, Araban M. Psychometric evaluation of health literacy for adults (HELIA) in urban area of Balochistan. Payesh 2016;15(6):669-76. [Persian].
23. Tavousi M, Haeri-Mehrizi A, Rakhshani F, Rafiefar S, Soleymanian A, Sarbandi F. Development and validation of a short and easy-to-use instrument for measuring health literacy: the Health Literacy Instrument for Adults (HELIA). BMC Public Health 2020; 20(1):656. doi: 10.1186/s12889-020-08787-2 [Crossref] [ Google Scholar]
24. Lalkhen AG, McCluskey A. Clinical tests: sensitivity and specificity. Continuing Education in Anaesthesia, Critical Care & Pain 2008; 8(6):221-3. doi: 10.1093/bjaceaccp/mkn041 [Crossref] [ Google Scholar]
25. Barnard-Brak L, Brewer A, Chesnut S, Richman D, Schaeffer AM. The sensitivity and specificity of the social communication questionnaire for autism spectrum with respect to age. Autism Res 2016; 9(8):838-45. doi: 10.1002/aur.1584 [Crossref] [ Google Scholar]
26. Stagliano V, Wallace LS. Brief health literacy screening items predict newest vital sign scores. J Am Board Fam Med 2013; 26(5):558-65. doi: 10.3122/jabfm.2013.05.130096 [Crossref] [ Google Scholar]
27. Wallace LS, Rogers ES, Roskos SE, Holiday DB, Weiss BD. Brief report: screening items to identify patients with limited health literacy skills. J Gen Intern Med 2006; 21(8):874-7. doi: 10.1111/j.1525-1497.2006.00532.x [Crossref] [ Google Scholar]
28. Sarkar U, Schillinger D, López A, Sudore R. Validation of self-reported health literacy questions among diverse English and Spanish-speaking populations. J Gen Intern Med 2011; 26(3):265-71. doi: 10.1007/s11606-010-1552-1 [Crossref] [ Google Scholar]
29. Wallace LS, Cassada DC, Rogers ES, Freeman MB, Grandas OH, Stevens SL. Can screening items identify surgery patients at risk of limited health literacy?. J Surg Res 2007; 140(2):208-13. doi: 10.1016/j.jss.2007.01.029 [Crossref] [ Google Scholar]
30. Ingram RR. Using Campinha-Bacote’s process of cultural competence model to examine the relationship between health literacy and cultural competence. J Adv Nurs 2012; 68(3):695-704. doi: 10.1111/j.1365-2648.2011.05822.x [Crossref] [ Google Scholar]
31. Shaw SJ, Huebner C, Armin J, Orzech K, Vivian J. The role of culture in health literacy and chronic disease screening and management. J Immigr Minor Health 2009; 11(6):460-7. doi: 10.1007/s10903-008-9135-5 [Crossref] [ Google Scholar]
32. Singleton K, Krause EM. Understanding cultural and linguistic barriers to health literacy. Online J Issues Nurs 2009; 14(3):4. doi: 10.3912/OJIN.Vol14No03Man04 [Crossref] [ Google Scholar]