The data of this cross-sectional-descriptive study were collected from 712 individuals who were older than 18 years old, who were at least primary school graduates, who volunteered to participate in the study, who registered in a popular social network such as WhatsApp, by using the Snowball Method online between April and May 2022. Exclusion criteria were having a serious mental disease, post-traumatic stress disorder, or panic attacks. Voluntary participants had the right to back down from the study at any time. The general rule for determining the sample size in factor analysis is 10 participants and a 20% loss probability for each item [15]. Since the original scale item was 51 items in this study, at least 610 participants were needed in the sample size. The sample was randomly divided into two groups for factor analysis. A total of 712 individuals were included in the study for exploratory (n=350) and confirmatory (n=362) factor analysis [16, 17]. In the first phase of this study, which consisted of two stages, the scale was translated and culturally adapted; in the second stage, the psychometric evaluation of the scale was performed (Fig. 1).

The study data were collected by using The Sociodemographic Variable Form and COVID-19-HLS.

The Sociodemographic Information Form was formed by researchers based on literature, and it contained questions about socio-demographic characteristics such as age, gender, education level, and COVID 19 status [9, 10, 14]. COVID-19-HLS was developed by Sanaeinasab et al. (2022) to find out the health literacy levels regarding COVID 19 [18]. The scale consists of 51 items. Each item of the original scale is graded with a 4-point Likert-type scale ranging from “never” (1) to “always” (4). The scale has five factors. Negative items in the scale are recorded and computed by dividing the total score by the number of items. A total of 1 to 4 points can be obtained from all sub-dimensions and the total scale. An increase in the total score obtained from the scale shows a higher level of HL.

The COVID-19-HLS was translated from English into Turkish by bilingual linguists and the authors independently. The translation of the scale was checked by a bilingual team of linguists consisting of seven experts. Later, it was independently back-translated from Turkish into English by a bilingual linguist. Conceptual and linguistic suitability was checked by three different experts.

2.3.1. Content Validity

The finalized COVID-19-HLS was sent to seven public health nurse specialists at different universities in Turkey for content validity, with both Turkish and English forms, and the experts who participated made minor corrections for cultural and language differences. Davis Technique was used for content validity of the scale. Each expert was asked to rate each item on a 4-point scale (1=not appropriate, 2= needs to be changed, 3= needs minor correction, 4=completely appropriate). When the Content Validity Index (CVI) is evaluated, a value greater than 0.80 suggests that the item is sufficient in terms of content validity (16). In the analyses performed, it was found that the CVI scores for all items ranged between 0.8 and 1.0. Due to content validity, no item was excluded from the scale.

Fig. (1). COVID-19-HLS flow diagram.

The study data were collected by using The Snowball Method. According to this method, the first data source was the students studying with the nursing faculty and their families. After completing the questionnaire, nursing students were asked to send it to their family members. On the first page of the questionnaire, there is a text explaining the purpose of the study to the individuals. The next section includes the study criteria. Pages with study questions are followed for individuals who meet all of the study criteria.

Permission was obtained by e-mail from the authors who developed the scale in accordance with ethical principles. Also, ethical approval was obtained from the Ethics Committee of Atatürk University Faculty of Medicine (B.30.2.ATA. 0.01.00/261).

The Statistical Package for the Social Sciences (SPSS) version 20.0 (SPSS v.20.0, IBM Corp., Chicago, IL, USA) and AMOS version 20.0 were used to analyze the study data. Language and content validity, Item-Total Score Correlation, Test-Retest Reliability Methods, Explanatory and Confirmatory, Cronbach’s Alpha Coefficient, Split-Half Reliability Analysis, Factor Analysis, and criterion-concurrent validity were performed to the validity-reliability analysis.

The internal consistency coefficient of the four-factor scale was found to be 0.941, and internal consistency coefficients of the factors were 0.922 for the analysis factor, 0.890 for the behaviour factor, 0.846 for the information seeking factor, and 0.746 for the application factor. The Spearman-Brown Correlation Coefficient and Guttmann Split-Half Value were found to be 0.836 and 0.841, respectively. The Spearman-Brown Correlation Coefficients of the factors were found to be 0.861, 0.850, 0.781, and 0.685, respectively. Hoteling’s T² value was 1615.931, p<0.001. The difference between the means of COVID-19-HLS items was found to be significant. In addition, the reliability of the data obtained was determined by the intra-class correlation coefficient. These values were found to be 0.919 for the analysis factor, 0.887 for the behaviour factor, 0.830 for the information seeking factor, and 0.742 for the application factor (Table 3).

Varimax Rotational EFA results showed a four-factor structure, unlike the original five-factor scale with more than one eigenvalue [22, 23]. The factor loads obtained for each item were >.30, as suggested in the literature. Factors explained 74.09% of the total variance. This was reported to be 47.3% in the original study of the scale. For multi-factor scales, the total variance is >40%, which is accepted as the lower limit value [11, 23]. This shows that the total variance explained by the factors is acceptable. The model appropriate for the four-factor structure of the scale was tested with CFA. It is a validation method specifically used in the application of measurement instruments developed in other cultures and samples [11, 24]. A series of goodness-of-fit measures were used to assess the model fit. When the values of the CFA fit indices were examined, it was determined that all values were within acceptable limits and they showed perfect fit [11, 19, 25]. It was also found that the factor loads of COVID-19-HLS varied between 0.45 and 0.74. These results confirmed the four-factor structure. The DFA results of this study could not be compared with those of the original study, as the original study did not include the DFA results [18].

Fig. (2). COVID-19-HLS path diagram.

In this study, four reliability methods, Internal Consistency, Split-Half Reliability, Test-Retest Reliability, and Intraclass Correlation Coefficient Methods, were used [22, 23]. The results obtained were high for the scale’s internal consistency, Split-Half Reliability, Test-Retest Reliability, and Intraclass Correlation Coefficient in terms of the overall scale and its factors. In this study, it was observed that the Cronbach Alpha Value of the total scale was 0.94, which was 0.89 in the original version of COVID-19-HLS [18]. These results showed that COVID-19-HLS and its factors had a high level of reliability for the Turkish society. In addition, the Split-Half Reliability Analysis in the present study showed that a strong and significant relationship was between the two halves of the scale and the scale had a high level of reliability for the Turkish population.

In general, the correlation value obtained from the test-retest in scale development and adaptation should be ≥ 0.70 [22]. In this study, the test-retest correlation value was found as 0.88 for the total scale and a positive and strong correlation was found between the test-retest scores. Test-retest correlation value of the original scale was reported to be 0.93 for the total scale [17]. It can be said that test-retest reliability results of both the original and Turkish versions of the scale are high. In addition, no significant difference was found between the average scores obtained from the two applications. This may be attributed to the comprehensibility of the scale. Hoteling’s T² Test, which found out even if there is a significant difference between the scale items in terms of means, whether the difficulty levels of the questions are equal, whether the answers given by the participants to the items are similar, and shows the significance of all items for the scale, showed that the difference between the COVID-19-HLS item scores was significant in this study [21]. A comparison could not be made since this analysis was not available in the original scale study.

In validity and reliability studies, floor and ceiling effects show the measurability of the scale factors [26]. It can be seen that the COVID-19-HLS total and factor scores do not show a floor or ceiling effect and these effects of the scale are within the desired limits.

One of the strengths of this study is that in addition to standard scale adaptation analyses, test-retest application, EFA and CFA were also performed. The fact that the scale is easy to understand shows that it can be easily applied and interpreted, which provides convenience to researchers. Another strong aspect of the study is that it represents the general population of Turkey in terms of socio-demographic factors such as gender, age, and educational level. The fact that the study was conducted on an online platform, electronic devices were dependent on the Internet and the participants had to have internet skills, and the data collection process started with nursing students and their families are the limitations of the study.