All published articles of this journal are available on ScienceDirect.
Content Validity of the AHRQ Health Care Professional Survey on Informed Consent: A Methodological Study
Abstract
Objective:
To assess the content validity of the Agency for Health Research and Quality (AHRQ) healthcare professional survey on informed consent.
Background:
The AHRQ has developed the healthcare professional survey on informed consent to assess the impact of the informed consent quality improvement training module on the healthcare professional knowledge, practices, and attitudes about informed consent.
Methods:
A qualitative study was carried out as part of a doctorate student dissertation study to assess the healthcare providers' and patients' perspectives and experiences on the informed consent process utilizing a descriptive, cross-sectional study design. The researchers have contacted 13 experts by email, asking for their voluntary support to validate the survey. The researchers adopted five domains of content validity measures, including relevancy, sufficiency, simplicity, clarity, and ambiguity. A four-point Likert scale was adopted and communicated to the experts to guide their scoring criteria. Preliminary pilot testing was done to assess the psychometric properties of the newly modified tool.
Results:
Seven experts have responded and shared their feedback either through email or hard copies, representing a response rate of 53.8%. Only one survey item scored less than 0.78 on I-CVI and was dropped from the survey. One subscale, “the Informed consent process overall effectiveness,” was dropped from the study as it falls below the acceptable level of 0.9. All edits requested by the experts' panel were done. The psychometric properties were then tested, and further enhancement of the tool was done to reach the acceptable Cronbach's alpha level.
Conclusion:
The AHRQ initiated the first stage of survey item development, and this study continued the efforts by validating the content of the survey and testing its content validity. The final poll was judged to have excellent content validity, good psychometric properties, and 41 items.
1. INTRODUCTION
Patients' informed consent to medical/surgical treatment is a complex process that evolved to protect the patients and ensure their rights. The legal history of modern informed consent was advanced through courts of battery cases at the beginning of the twentieth century—the known battery case of Schloendorff v. Society of New York Hospitals in 1914 was the most critical to advance the patient's rights to self-determination [1]. In this case, Ms. Schloendorff sued the hospital of New York for the actions of the hospital surgeons who removed a lump from her body without consent. Unfortunately, the surgical procedure of Ms. Scholendroff’s was complicated by gangrene in one arm that necessitated the amputation of some fingers [1]. This case established the foundational structure for the patient’s self-determination act and the principle of organization respondeat superior.
After that, many battery cases were held and accordingly supported the development of the informed consent modern processes. Informed consent is a deliberate interaction between a healthcare provider and patient to make the autonomous informed treatment choice that includes, on a minimum, the proposed treatment, benefits, risks, complications, alternatives, and the likelihood of success. It represents a contractual agreement between the patient, the healthcare organization, and the healthcare providers and supports the patient's comprehension of the treatment plan before granting their treatment consent [2, 3].
While millions of patients are attending healthcare settings to receive treatments worldwide, and the frequent application of informed consent during their healthcare encounters, the quality of the informed consent process was reported to vary considerably [4-6]. A systematic review was conducted to evaluate patients' understandability of informed consent elements within the clinical settings and research in 2009 and concluded a lack of patients' comprehension of informed consent [4]. Patients’ from five different hospitals in Croatia were surveyed, and more than a quarter stated that their' treatments were decided by physicians alone, approximately half of them did not express their consent in writing, and the majority perceived signing the informed consent form as a decorum of healthcare [5].
Moreover, studies from Iran and Croatia found that at least one-third of the hospitalized patients' did not comprehend and understand informed consent [5, 6]. The informed consent process was considered a medically driven process that ignores the patient's right to make an informed decision [7]. In Pakistan, researchers studied the patients' perception of the informed consent process and found that half of the patients' were not informed of the possible complications, one-quarter of patients were not informed about the nature of the surgical treatment and medical treatment alternatives, and more than one-third of the patients' did not value the process [8]. Furthermore, a study from Iran about patients' satisfaction with the informed consent process revealed patient dissatisfaction [9]. These findings revoke the informed consent process guiding principle of making an informed decision [2].
In addition to that, research findings showed that physicians usually do not inform their patients of their underlying conditions, treatment options, treatment alternatives, risks, complications, and expected benefits [10]. Surgery residents reported their incompetence to consent patients because of their unfamiliarity with all procedure-specific knowledge, complications, and risks [11]. Physicians reported multiple factors were reported that affect the process of informed consent, such as the hospital policies and procedures, time constraints, financial pressures, different medical opinions, patients' level of comprehension, patient understandability, disclosure of risks to patients, family members’ engagement, and norms of the community [12].
The nurses' role in the informed consent process is not well-demarcated, and nurses were reporting multiple roles in the process to support their patients. A qualitative study was conducted at a large teaching hospital in the United States and noted no structured or clear role of nurses [13]. The informed consent process among Iranian nurses was investigated, and it found that nurses held the physician's responsibility to grant consent from the patients [14]. Moreover, nurses' contribution to the informed consent process was affected by the nurses' communication, hospital policies and procedures, and patients' comprehension [15]. Nurses can be better prepared to handle their roles by establishing a clear position [13], training them on communication skills, legal and regulatory requirements, and enhancing their clinical knowledge [15].
The way of achieving the patient’s informed consent depends on the physician and patient information sharing activity that varies based on the physician's adopted informed consent methodology. The Joint Commission interpreted the hospital standards of informed consent broadly and did not provide a specific, informed consent methodology. It left it for the hospitals to establish their policies and procedures [16]. This means that informed consent policies and practices are not standardized and differ across hospitals [16, 17]. Some hospitals may apply generic informed consent forms, while others have procedure-specific documents. The generic consent forms usually do not detail, in writing, the proposed procedure benefits, risks, complications, and alternative procedures, while the procedure-specific informed consent provides all the needed details. Some hospitals support their patients with complimentary patient educational materials, while others do not. These findings and variations in practices inform us of the need to study the current process and plan for its improvement.
Research on the informed consent process has informed us of improvement. Healthcare organizations have reported more than 40 informed consent-related sentinel events between 2010 and 2016 in the United States, which urged the Joint Commission to release a quick safety advisory guidance about informed consent [18]. Poor patient-healthcare provider communication contributed to the informed consent process problems [18, 19]. The Joint Commission has called to improve the process by enforcing the adoption of informed consent as a process, not a document to sign. It emphasized the need for healthcare organizations to establish their informed consent policies and procedures, train their staff on effective communication and the informed consenting process, simplify the informed consent forms, and adopt appropriate and culturally sensitive communication tools [18].
The Agency for Health Research and Quality (AHRQ] has developed the healthcare professional survey on informed consent to assess the impact of the informed consent quality improvement training module on the healthcare professionals (HCPs] informed consent knowledge, practices, and attitudes [20]. However, applying the quality improvement framework may support the organizational local improvement activities such as the standardization of informed consent forms, the generation of educational material, and the use of high-quality decision aids. However, it may not support the systematic investigation of the problem of informed consent processes. Thus, understanding the healthcare providers' knowledge, practices, and attitudes may be the most critical initial step to improving informed consent.
There are no internationally psychometrically validated and accepted tools to assess the healthcare providers' knowledge, practices, and attitudes about informed consent. The AHRQ healthcare professional survey on informed consent is composed of 53 questions distributed as one question asking about the HCPs role, eight questions assessing the HCPs knowledge of the hospital's current informed consent policy, thirteen questions about the HCPs informed consent practices, thirteen questions about the HCPs working unit informed consent practices, nine questions assessing the HCPs attitudes toward patient rights of informed consent, two questions about the informed consent process effectiveness, one question about the efficacy of the teach-back process and six demographical questions [20]. However, this survey was developed for quality improvement, and its content was not validated, nor its psychometric properties were tested [21].
Measuring the tool validity resembles the first fundamental step of tool development to ensure the tool measures what it is intended to measure [22, 23]. The measurement protocol should focus on validating the tool contents and its psychometric properties. Nurse researchers commonly use it to validate their newly developed instruments [24]. Content validity can be defined as ”the extent to which a subject's responses to the items of a test may be a representative sample of his/her responses to a real or hypothetical universe of situations which together constitute the area of concern to the person interpreting the test” [25]. In other words, the content validity should reflect the adequacy of the tool sampled items to cover the constructs domain and its definitions. Accordingly, the researchers have focused on validating the contents of the AHRQ healthcare providers' survey of informed consent. This study aimed to assess the content validity of the HCP survey on informed consent.
2. METHODS
Study Design
This methodological study is part of a doctorate student dissertation study that was carried out to assess the healthcare providers' and patients' perspectives and experiences towards the informed consent process utilizing a descriptive, cross-sectional study design. The researchers' initial search revealed the lack of a unified research-driven survey on the informed consent process that addresses the HCPs' knowledge, practices, and attitudes but found the AHRQ has developed a quality improvement training and tool on informed consent [20]. The researchers have contacted the AHRQ and got the appropriate permission to use the tool and conduct a content validity study [26].
A committee of four experts has examined the AHRQ healthcare professional survey on the informed consent tool and recommended conducting a content validity study. The researchers have then asked the experts panel to voluntarily review the content of the HCP survey on informed consent and validate it against the overall survey objectives, the purpose of development, and its underlying concepts. The experts' panel was chosen based on the experts' experiences, expertise, research interest, and clinical practice domain. Also, the experts' panel was selected to represent the target participant groups: physicians and nurses.
The researchers have contacted 13 experts by email, asking for their voluntary support to validate the survey. The researchers adopted five domains of relevancy, sufficiency, simplicity, clarity, and ambiguity measures. A four-point Likert scale was adopted and communicated to the experts to guide their scoring. Please refer to Table 1. A gentle reminder was sent to the expert who did not submit their feedback nor apologize within seven days of the original email and for a maximum of three times, each email in one week apart. This study was done from January to April 2021.
| Content Validity Measures | Four point Likert Scale | |||
| Relevancy | Not Relevant | Somewhat Relevant | Quite Relevant | Very Relevant |
| Sufficiency | Not Sufficient | Somewhat Sufficient | Quite Sufficient | Very Sufficient |
| Simplicity | Not Simple | Somewhat Simple | Quite Simple | Very Simple |
| Clarity | Not Clear | Somewhat Clear | Quite Clear | Very Clear |
| Ambiguity | Doubtful | Somewhat Ambiguous | No doubt but need minor revision | Meaning is Clear |
There are two methods of calculating the content validity index: the universal agreement and item average. The universal agreement requires the consensus agreement of all experts (S-CVI/UA], while the item average is calculated by averaging the experts' scores of the survey items (S-CVI/Ave] [25, 27]. Regardless of its form, it is required by the researchers dichotomize the experts' responses into two dichotomies to calculate the content validity index. Accordingly, the researchers combined the responses of three and four together and one and two. These scores combination yielded two dichotomized responses of relevant, not relevant, sufficient, not sufficient, simple, not simple, straightforward, not clear, and ambiguous and not ambiguous. After that, the researchers calculated the item content validity (I-CVI] index by dividing the number of experts who scored their responses as three or four over the total number of experts' responses. The researchers adopted an I-CVI index of 0.78 as the minimum item content validity acceptable value, reflecting a modified kappa value of 0.85. Also, the researchers adopted a scale content validity index (S-CVI] value of 0.9 and more as the minimum acceptable value to accept the scale [25].
3. RESULTS
Seven experts responded and shared their feedback either through email or hard copies, representing a response rate of 53.8%. The respondents' experts were four nurse researchers, one consultant physician, and two nurses working in quality administrative positions. Five experts held a Ph.D. degree or equivalent, equivalent, and two had a Master's degree. Also, four experts were male, and three were females. No clarifications or justification for not validating the tool was communicated by the non-responder experts.
All I-CVI on the relevancy scale scored less than 0.78 were dropped, and it was only one survey item. All suggested experts' feedback improved the items' relevancy, sufficiency, simplicity, clarity, and ambiguity were amended. Also, the researchers calculated the scale and subscales' universal and average agreement scores across the content validity measures adopted by the researchers. The researchers' adopted a scale content validity index (S-CVI) of at least 0.9 as the minimum acceptable value [25].
| Scale | Average Agreement | Universal Agreement | Interpretation |
| Relevancy Scale | |||
| Role in the informed consent | 1.00 | 1.00 | Acceptable |
| Current informed consent policy | 0.95 | 0.88 | Acceptable |
| Current informed consent process | 0.99 | 0.92 | Acceptable |
| Current informed consent practices | 0.99 | 0.92 | Acceptable |
| Informed consent process overall effectiveness | 0.86 | zero | Eliminated |
| Teach back self-efficacy | 1.00 | 1.00 | Acceptable |
| Attitudes about informed consent | 1.00 | 1.00 | Acceptable |
| Overall | 0.98 | 0.89 | Acceptable |
| Sufficiency Scale | |||
| Role in the informed consent | 1.00 | 1.00 | Acceptable |
| Current informed consent policy | 0.96 | 0.75 | Acceptable |
| Current informed consent process | 0.98 | 0.85 | Acceptable |
| Current informed consent practices | 0.98 | 0.85 | Acceptable |
| Informed consent process overall effectiveness | 0.93 | 0.50 | Acceptable |
| Teach back self-efficacy | 1.00 | 1.00 | Acceptable |
| Attitudes about informed consent | 1.00 | 1.00 | Acceptable |
| Overall | 0.98 | 0.85 | Acceptable |
| Simplicity Scale | |||
| Role in the informed consent | 1.00 | 1.00 | Acceptable |
| Current informed consent policy | 0.93 | 0.75 | Acceptable |
| Current informed consent process | 0.92 | 0.62 | Acceptable |
| Current informed consent practices | 0.91 | 0.62 | Acceptable |
| Informed consent process overall effectiveness | 0.93 | 0.50 | Acceptable |
| Teach back self-efficacy | 1.00 | 1.00 | Acceptable |
| Attitudes about informed consent | 0.98 | 0.89 | Acceptable |
| Overall | 0.94 | 0.70 | Acceptable |
| Clarity Scale | |||
| Role in the informed consent | 1 | 1 | Acceptable |
| Current informed consent policy | 0.91 | 0.75 | Acceptable |
| Current informed consent process | 0.92 | 0.69 | Acceptable |
| Current informed consent practices | 0.91 | 0.69 | Acceptable |
| Informed consent process overall effectiveness | 0.93 | 0.50 | Acceptable |
| Teach back self-efficacy | 1 | 1 | Acceptable |
| Attitudes about informed consent | 0.98 | 0.89 | Acceptable |
| Overall | 0.93 | 0.74 | Acceptable |
| Ambiguity Scale | |||
| Role in the informed consent | 1.00 | 1.00 | Acceptable |
| Current informed consent policy | 0.93 | 0.75 | Acceptable |
| Current informed consent process | 0.93 | 0.77 | Acceptable |
| Current informed consent practices | 0.93 | 0.77 | Acceptable |
| Informed consent process overall effectiveness | 0.93 | 0.50 | Acceptable |
| Teach back self-efficacy | 1.00 | 1.00 | Acceptable |
| Attitudes about informed consent | 0.98 | 0.89 | Acceptable |
| Overall | 0.95 | 0.79 | Acceptable |
Accordingly, one subscale, “the Informed consent process overall effectiveness,” was dropped from the survey as it falls below 0.9 acceptable level; please refer to Table 2. Based on the content validity analysis and the minimal edits requested by the experts' panel, there was no need to conduct another round of experts' panel feedback [25]. After deleting items with I-CVI of less than 0.78 and accepting S-CVI of 0.9 and more, the final survey was judged to have excellent content validity and was composed of 46 items. Please refer to Table 2.
3.1. Pilot Study
After validating the HCP survey on informed consent, doing the required experts' panel edits, and getting the needed permission and ethical approval from the participating hospitals, the researchers did pilot testing. The researchers have randomly distributed the newly modified survey to 10% of the original study sample size for reliability testing purposes. The researchers have distributed five surveys per participating hospital, either in hard copy or electronic format, to the study participants who met the original inclusion and exclusion criteria. The researchers adopted an electronic version to manage the risk of COVID-19 transmission, noting that the pilot testing was done during the second wave of infection in Jordan and based on study participants' preferences. The researchers' have ensured the representatives of the distributed surveys by personally inviting participants from different hospital units and professions. The researcher did a debriefing session to ensure the study participants understood the words and expressions used in these instruments. A total of 32 responses were analysed. Please refer to Table 3.
| Scale | Cronbach’s alpha |
| Healthcare Providers Knowledge | 0.939 |
| Healthcare Providers Unit Practices | 0.909 |
| Healthcare Providers Own Practices | 0.945 |
| Healthcare Provider Attitude | 0.746 |
| Overall Survey | 0.805 |
Cronbach's alpha was generated to test the survey reliability. The HCPs knowledge scale Cronbach's alpha was 0.939, the HCPs unit practices scale Cronbach's alpha was 0.909, the HCPs own practices scale Cronbach's alpha was 0.207, and the HCPs attitude scale Cronbach's alpha was 0.537. One item, “offer choices including nothing,” was deleted from the HCPs practices scale, improving its Cronbach's alpha from 0.207 to 0.945. Also, four items were deleted from the HCPs' attitude scale (Clinicians are responsible for ensuring that patients understand all their options before making a decision; Lack of a patient understanding of the benefits, harm, and risks of treatments is a serious patient safety problem; Clinicians are responsible for ensuring that patients understand all their options before making a decision.; The informed consent process is worth the time it takes] to improve its reliability and increase it to 0.746. The overall survey Cronbach's alpha was 0.903. The average time to complete the study was 15 minutes. The final survey was composed of 41 items.
4. DISCUSSION
Informed consent is an essential element of the varnished modern healthcare system that focuses on delivering patient-centered care. Studying HPCs' perception of their informed consent knowledge, attitudes, and practices is important to equip, be aware, and prepare the HPCs to deliver patient-centered care and protect the patient's rights to informed decisions and choices. However, there was no internationally recognized and accepted tool. Previous studies on informed consent did not address the concept holistically or reciprocally measured. Also, it lacks standardized and psychometrically validated measures of informed consent knowledge, attitudes, and practices. This study was conducted to assess the content validity of the AHRQ healthcare professional survey on informed consent, a tool that was initially developed for quality improvement purposes, and presented the content validity indices.
Many studies were conducted investigating various issues related to informed consent [10-12, 14, 15]. However, almost all studies involving the knowledge, attitudes or practices of the HCPs did not use an internationally accepted and psychometrically validated tool. The validation of the AHRQ tool may It helps to understand the knowledge base of the HCPs, how they behave, and their held attitudes. It will identify the needs, barriers, and opportunities for improvement that needs to be in place to improve the informed consent process. The validation of the AHRQ tool may facilitate its international adoption and accordingly test its validity and reliability across healthcare settings and cultures.
Moreover, it will facilitate the comparison of 'HCPs' perceptions and deepen our understanding of how the HCPs perceived the informed consent process. Also, this may support the researchers, healthcare policymakers and educators' roles in enhancing the healthcare providers' knowledge and improving their practices and training. This research study can enhance the quality of the informed consent process and highlights opportunities for improvement that the healthcare system can focus on as a priority in its quality improvement initiatives. It can also guide physicians and nurses to enhance the process, facilitate patient engagement, manage patient expectations, and allow patients to practice their rights for autonomy effectively.
In an effort to assess the surgical residents' knowledge and attitudes toward the informed consent process in Pakistan, researchers have administered a self-developed questionnaire to 300 physicians and received a response from 231, divided into 199 junior and 32 senior surgeons. The researchers concluded that junior surgery residents lacked knowledge and attitudes compared to seniors. The researchers recommended training surgery residents on the informed consent process and adopt a structured informed consent template [28-30]. Measuring the HCPs knowledge, attitudes and practices through a structured and validated tool such as the AHRQ healthcare professional survey on informed consent should facilitate establishing targeted training programs that meet the needs of the HCPs. An example of comprehensive training program is the AHRQ and Joint commission training on informed consent [20].
While content validity is the standard validity procedure required to appraise newly developed tools [23], the validation process is lengthy, and instrument developers usually do not report it [25, 28]. In this study, we contacted 13 experts in the field to support the validation of this tool, but only seven experts have responded, representing a response rate of 54%. Those experts were from diverse background and experience and were from the academic and clinical fields. This representation of experts enhanced the depth and the power of this study. On the other hand, the researchers emailed the non-respondents experts three times, one week apart to increase the response rate or understand the reasons behind not conducting this content analysis feedback but could not attain a justification. However, this study was conducted during the pandemic crisis of COVID-19 in which researchers might be sick or busy with other more priority stuff such as patients care or online education.
Two approaches of validation have been in use which were the universal agreement and the average agreement. In the current study, the researchers have adopted the average agreement and followed Polite et al. recommendations of an acceptable I-CVI index of 0.78, reflecting a modified kappa value of 0.85 and a scale content validity index (S-CVI] of at least 0.9. Accordingly, only one item was dropped as its I-CVI was 0.57, and one scale, “the Informed consent process overall effectiveness', was eliminated from the survey as it scored on the average agreement 0.86. Accordingly, there was no need to conduct another round of 'experts' panel feedback [25]. The above process was followed by pilot testing of the amended survey items based on the validation results, strengthening the survey's psychometric properties. A couple of survey items were dropped from the attitude and practice scales to improve their internal reliability and consistency. The overall scale and subscale Cronbach's alphas were more than 0.7, which is satisfactory [28]. Further large-scale testing is needed to ensure the reliability of the overall scale and subscales.
The current study's limitations include the subjective feedback of the content experts, subject to research bias that may unintentionally reflect the experts' feedback. While this survey was developed for quality improvement purposes that meet the United States' knowledge, practices, and attitude toward informed consent, the experts were asked to suggest changes, amendments, and other survey items that reflect the international practices. This may help minimize the limitation of appropriate and relevant content validity domains. Accordingly, minimal edits were requested by the experts' panel, and there was no need to conduct another round of expert panel feedback [25]. The final survey was judged to have excellent content validity and was composed of 41 items. This may indicate the adequacy of the content validity domain.
CONCLUSION
This methodological study of content validity represents the systematic application of processes to ensure measuring what we are intended to measure. The AHRQ initiated the first stage of survey item development, and this study continued the efforts by validating the content of the survey and testing its content validity. Despite the content, validity takes a long time, but the steps are worth the consumed time. The validated tool was then followed by preliminary pilot testing, strengthening the tool and ensuring its reliability. The final survey was composed of 41 items that may need further testing internationally and with a bigger sample size.
LIST OF ABBREVIATIONS
| AHRO | = Agency for Health Research and Quality |
| HCP | = Health Care Professionals |
| S-CVI | = Scale Content Validity Index |
ETHICS APPROVAL AND CONSENT TO PARTICIPATE
The institutional review board of the University of Jordan has approved this study.
HUMAN AND ANIMAL RIGHTS
No animals were used as the basis of this study. This research was conducted according to the Declaration of Helsinki principles.
CONSENT FOR PUBLICATION
Informed consent has been received from all participants.
STANDARD OF REPORTING
STROBE guidelines were followed.
AVAILABILITY OF DATA AND MATERIAL
Not applicable.
FUNDING
None.
CONFLICT OF INTEREST
The author(s) confirm that this article content has no conflict of interest.
ACKNOWLEDGEMENTS
Declared none.
