Comparison of Perception Differences among Nurses based on Harm Assessment Education in the Patient Safety Classification System

: Background: Accurate harm assessment is critical in the patient safety event management system, but few studies have been published to support the need for training in harm assessment of patient safety events for nurses. Objective: The purpose of this study was to see if there was a difference in the degree of agreement between before and after nurses received training on a patient safety event harm assessment guideline. Methods: After participating in online harm assessment education, 65 subjects completed the self-report questionnaire. Data for the general characteristics of the study subjects and the frequency of respondents for each scenario were analyzed using descriptive statistics. Fleiss' kappa was calculated by estimating the inter-rater agreement among respondents for each scenario. Results: The agreement value of subjects by Fleiss' Kappa value improved from k = 0.23 before education to k = 0.31 after education, according to the harm assessment. There was no change at k = 0.30 before and after education for the harm period. Implication for Nursing & Conclusion: This study’s findings suggest that harm assessment agreement among nurses could be increased through harm assessment education. As a result, case-based education on harm assessment must be expanded, as well as related programs for practical education via patient safety event casebooks.


INTRODUCTION
The Institute of Medicine emphasized the importance of patient safety and patient safety reporting in its 1999 report "To Err Is Human: Building a Safer Health System" and the World Health Organization (WHO) Board of Directors announced in 2003 the need to develop standard terminology and classification systems, harm assessment, and reporting-learning systems as systematic factors for patient safety [1,2].According to reports on patient's safety, the reporting system Formats, a standardized PSE reporting format [5].In addition, the WHO developed a 5-point scale for harm assessment [6], the National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) developed a tool for considering the duration and permanence of harm, with a kappa value of 0.74 [7].The Institute for Healthcare Improvement developed a Global Trigger Tools with a mean kappa value of 0.65 [8,9].The reliability of the reporter's judgment in the type and severity of PSEs is the most important factor in the harm score.If the reliability of the harm assessment is low, comparing PSE data and determining direction will be difficult.
The clinical backgrounds of clinicians, nurses, and pharmacists who report PSEs in the clinical setting differ, which is the cause of deviation in harm assessment.Nurses were reported to be the most active in reporting events, owing to the fact that they witnessed PSEs the most frequently during the nursing process, and reported lower harm assessments for medication errors than clinicians [10].Second, researchers emphasize the importance of education and training in understanding harm scores [11].Finally, it has been demonstrated that the reliability of PSEs can be affected by how they are reported and that bias can be reduced because web-based reporting can review integrated data and direction rather than paper-type reporting [12].Because nurses report more PSEs than clinicians [13], nurses' behaviors, attitudes, and perceptions are especially important when reporting PSEs [3].It has also been reported that nurses' relationships with doctors, workload, and working environment, such as colleagues, all have an impact on data quality [14].
According to the study of inter-rater agreement in harm assessment for nurses, the AHRQ Common Format Harm Scale version 1.1 and 1.2 (AHRQ's version 1.1 or 1.2) showed a moderate degree of agreement, with k=0.45, but agreement for each ward in the same institution was found to be very low [14].Another study for nurses using the AHRQ's version 1.2 found low agreement between raters, with k = 0.21 [15].The low consistency in nurses' harm assessment can be interpreted as a subject in the interpretation of each patient safety event scenario of the AHRQ's version 1.2 used in the study, but it also suggests the need for education so that harm assessment scores can be assigned accurately [3].
Similarly, according to the findings of a study that examined PSEs reported in the emergency room, emergency room residents require education on the classification of patient safety events and harm assessment [16].However, few studies have been published to support the need for training in harm assessment of PSEs for nurses or clinicians who are major PSE reporters in the clinical setting.As a result, the goal of this study was to see if there was a difference in consistency before and after developing guidelines for PSE harm assessment education for nurses in clinical settings.

Research Design
This is a single-group pre-and post-design experimental study comparing before and after education by developing a "Harm assessment guideline" educational material to improve the consistency of harm assessment in nurse's PSE reports.

Study Population
The nurses who participated in the previous study's survey were evaluated using the developed "Harm assessment guideline" with the cooperation of the national cancer center specializing in cancer with 555 beds in Gyeonggi-do, which conducted the harm and harm period evaluation among nurses in 2018.Following education on the harm assessment of PSEs, a survey on nine scenarios of PSEs, was conducted, and the results before and after education were compared and analyzed.
The study was conducted after receiving Institutional Review Board (IRB) approval in February 2020, but due to the unexpected COVID-19 situation, collective education was not possible, so the research method was changed to online education and an online survey.
Following the creation of an educational video by the investigator, the investigator commissioned an online survey specialist to have the subjects watch the educational video and then participate in the survey.Many of the 133 subjects who participated in the 2018 survey no longer worked at the institution due to job changes or leave, and some refused to participate in the study.As a result, the results of 65 research subjects who completed their education and participated in the survey were analyzed after they voluntarily agreed to participate in the study.

Guidelines for Harm Assessment and Educational Videos
A survey of clinical nurses was conducted using the AHRQ's version 1.2 to report PSEs [15], indicating that harm and harm period evaluation were dispersed when harm such as "wrongly labeled samples" or "thoracic drainage" was not obvious in the scenario.With these findings in mind, the "Harm assessment guideline" was created by referring to data developed elsewhere to assist in assessing the harm and harm period of nurses.
We developed "Harm assessment guidelines" by referring to existing guidelines [16 -18] related to harm assessment.The "Harm assessment guidelines" began with an overview of the harm WHO's harm classification system and scale, as well as the NCC MERP and AHRQ harm classification system and scale.The AHRQ's version 1.2 was then described.Since it is necessary to confirm whether there has been a deviation from standard treatment when classifying preventable harm assessment, the considerations when assessing the patient's outcome were explained.Finally, the harm assessment guidelines in PSEs, as well as the PSEs classification and harm system were explained and the harm assessment was conducted using actual patient safety event examples.Fig. (1) depicts the contents of the "patient safety event classification" presented in this study.
As for the expert review of the developed guidelines, it was impossible to hold face-to-face meetings due to the COVID-19 situation, so the contents were revised and supplemented by listening to patient safety experts' reviews and opinions through non-face-to-face methods such as video conferencing and e-mail.

Patient Safety Event Scenarios are Chosen and Reviewed for Harm Assessment
The patient safety scenario of the questionnaire was developed in this study with reference to AHRQ's version 1.2 [4], with a focus on domestic patient safety cases.Several emails were sent to the study manager for use approval prior to AHRQ's version 1.2 in this study, but no response was received, so the contents are described in the related table .The AHRQ's version 1.2 is intended to evaluate harm and harm duration for each of the nine patient safety scenarios.Each scenario includes (1) incorrect administration route, (2) lacerations of body parts, (3) allergic reactions to contrast agents, (4) abdominal infections, (5) mislabeled samples, (6) chest tube drainage, ( 8) overdose, and ( 9) incorrect time administration; in this study, actual cases related to this were selected by referring to the Korean Association of Hospital Nurses "Patient Safety Incident Cases and Prevention" [19].
After selecting the scenario, three experts and one university professor working in relation to patient safety work at a medical institution reviewed each topic and whether the selected scenario was appropriate, and then conducted a study.
As the general characteristics of the subject, questions such as gender, length of service at current workplace, current department, job satisfaction in current department, completion of patient safety education in current department, patient safety education method, and content of patient safety education were included in the questionnaire.
There were also 13 questions to determine patient safety incident participation in the current department, patient safety incident experience in the current department, patient safety incident type, patient safety incident report, and experienced patient safety incident type.

Definition of Harm and Harm Duration
The level of harm and harm duration were assessed for domestic patient safety incidents using the criteria presented by NCC MERP [20] and classified as follows; first, the level of harm is near miss (A): an environment that can cause a hazardous event, such as unorganized medical equipment, near miss (B): an incident occurred but did not reach the patient, and no harm event (C): an incident occurred, but no harm to the patient and no additional monitoring required; no harm event (D): if an incident has occurred and no harm will occur to the patient, or additional intervention is required to prevent harm, adverse event (E): when an incident occurs that causes temporary harm to the patient and requires additional treatment or intervention, adverse event (F): An accident that causes temporary harm to the patient and necessitates short or longterm hospitalization, and a sentinel event: an incident that causes near-death or permanent harm to the patient, or the patient died, or the event contributed to the patient's death.When an incident occurs and temporary harm to the patient lasts more than one year, it is classified as permanent, temporary harm to the patient and lasts less than one year, and unknown.

Statistical Analysis
SPSS 26.0 was used to analyze the collected data (SPSS Inc, Chicago, IL, USA).First, the general characteristics of the study subjects were calculated, as well as the frequency of respondents for each scenario before and after education.The Fleiss' kappa value was then calculated by estimating the interrater agreement for each scenario among respondents before and after education.The agreement rates among respondents were: 0.81-1.00for perfect agreement, 0.61-0.8for significant agreement, 0.41-0.60 for moderate agreement, 0.21-0.40for slight agreement, 0.01-0.20 for slight agreement and a score of 0 or less for almost no agreement.

Ethical Consideration
Before proceeding with the research, approval was obtained from the IRB of the institution to which the research subjects belonged.Following that, the purpose of the study was explained to the hospital's nursing headquarters, and the study was carried out with cooperation.
The researcher explained the purpose and method of the study to the research subjects who participated in the 2018 study through an explanation for the online education and survey.After collecting e-mail addresses for online education and re-survey, the relevant web address was sent if the subject voluntarily expressed their intention to participate.Prior to the start of online education, participants could only participate in the education and survey if they agreed voluntarily.Furthermore, to protect personal information, the collected email addresses of the subjects were discarded immediately after the online training and survey were completed.

General Characteristics of Research Subjects
All of the subjects in this study were female, and the general ward was the most common work department, accounting for 52.3% of the subjects.When asked how long they had been at their current job, 33.8% said 5 years or more and less than 10 years, 58.8% said more than 10 years, and the majority of subjects said more than 5 years.
In terms of job satisfaction, "satisfied" was the highest at 70.8%.It was discovered that all of the subjects received patient safety education at their current workplace, with 86.3% receiving "certificated brochure training" and 70.8% receiving "theoretical lectures."Patient safety education content, was 90.8% "incident reporting procedures," 84.6% "understanding patient safety," and 81.5% "time and method identification."At 64.6%, patient safety education time was greater than one hour but less than four hours (Table 1).

Patient Safety Incident Experience of Research Subjects
The majority of study subjects (92.3%) had experience with PSEs, with "no harm safety event" 83.3% and "near miss" 66.7% having the highest duplicate responses to experienced PSEs.When asked if they had any experience reporting PSEs, 83.1% said they had, with the most common types of patient safety incidents being "fall" 83.3% and "medication" 66.7% (Table 2).

Agreement of Harm and Harm Duration after Education
Tables 3 and 4 show the results of the study subjects" evaluations of harm and duration of harm for each scenario.The items that showed changes before and after training in the distribution of respondents by scenario were the 'Medication given via wrong route' scenario.Before education, it was 46.6% of "mild harm" and 24.8% of "moderate harm," but after education, it was 52.3% of "mild harm" and 27.7% of "no harm."Following that, in the "Wrong site surgery" scenario, 71.4% of "Sentinel Event" and 27.1% of "Severe Harm" were evaluated as 89.2% of "Sentinel Event" after education.Finally, in the "Medication given at the wrong time" scenario, the "Near Miss" rate before education was 9.8% but increased to 16.9% after education.In contrast to the harm assessment, there were no significant differences in the harm period evaluation before and after education (Tables 3 & 4).
The Fleiss' kappa value of respondents' agreement showed that the harm assessment improved from k = 0.23 before education to k = 0.31 after education.There was little change in the harm period from k=0.29 before education to k = 0.30 after education (Table 5).

DISCUSSION
Because patient safety is dependent on information about PSEs reported in the patient safety incident reporting system, the harm of the incident, and the factors that caused the occurrence, harm assessment of patient safety incident reporters standardization is critical.In particular, since the Patient Safety Act was enacted in Korea in 2016, patient safety incidents are managed with a focus on reported incidents, and in order to promote qualitative improvement, with voluntary reporting of each patient safety incident, a standardized evaluation of harm and factors is very necessary.As a result, the accuracy of the classification system of PSEs reported in the patient safety system is critical, but the findings of previous studies [3,4,15] show that there is a significant difference in nurses' perception of PSEs.
The AHRQ Common Format Harm Scale evaluation tool used in this study was created as part of a project to standardize the assessment of harm in PSEs in the United States.The assessment tool has created a scenario for determining the extent of harm and the duration of the harm.In the harm assessment criteria, a 'near miss' is a concept that defines situations and events that did not reach the patient but may cause 'harm' in the future, whereas a 'no harm event' and an 'adverse event' are concepts that are divided by 'harm' among the events that did reach the patient.The harm period is a concept that divides whether the harm occurs for more than one year.
The inter-rater harm confidence value for the AHRQ's version 1.1 PSE scenario for 921 institutional quality, risk, and safety managers at UHC PSN (Patient Safety Net ® ) was k=0.51, and reliability measurements for version 1.2 with an updated version 1.1 were conducted on 13,280 UHC PSN managers, with a moderate reliability value of k=0.47 between inter-rater [4].
The same tool AHRQ'S versions 1.1 and 1.2 measured inter-measurer reliability values for the risk of PSE scenarios for nurses working in cancer centers, with version 1.1 k=0.45 and version 1.2 k=0.48, indicating moderate reliability [3].In another study, doctors, nurses, pharmacists, and other healthcare professionals who are the primary reporters of PSEs in the clinical field were evaluated for nine PSE scenarios, and it was found that the nurse group had a higher reliability difference in patient safety case assessment than the other healthcare professionals [10].
In this study, general nurses working at advanced general hospitals and the national cancer center in Korea were reported as having k=0.21 as a result of evaluating the harm of PSEs between nurses using a questionnaire tool developed based on domestic PSEs [15].This study also revealed that k=0.23 before and k=0.31 after education, which was lower than the previous study [3,4].Even though the subject was a patient safety manager of a medical institution, in the case of a study conducted under the supervision of UHC in the United States, there was a significant difference in the perception of patient safety events between domestic studies that targeted general nurses.
The reasons for perception among healthcare professionals, including nurses, doctors, and pharmacists, are due to the diversity of each healthcare professional's educational background [13], differences in understanding of harm items [10], and whether the format in which PSEs are reported is paper-based or web-based [12].Finally, it was reported that unclear guidelines for the definition and knowledge structure of each harm item in harm assessment may lead to a biased view of harm assessment [3,21].Furthermore, when harm assessment is conducted using standardized evaluation tools such as the AHRQ'S version 1.1, the low consistency value between inter-rater is inconsistent with the understanding of harm items as in other studies, and the selection of risk scores among evaluators is very subjective, the necessity of education was suggested as a way to reduce this [3,22].
Until now, no studies have reported the results of training medical personnel to increase the consistency of evaluators of harm assessment in reporting actual PSEs, but assessment guidelines have been developed [16].In this study, an educational video was created in order to improve the consistency of nurses' harm assessment by referring to existing guidelines, and then the consistency value was compared after education.
After explaining the developed harm classification system and scale using the example of PSEs in the educational video, nurses who participated in the education were organized to conduct harm assessment of PSEs based on the actual educational contents.Due to the COVID-19 situation, online education was conducted, but nurses who participated in the education were asked to participate in a harm assessment survey after watching educational videos.
The study found that the Fleiss' kappa value, which represents agreement among raters in harm assessment before education, was 0.23 but improved to 0.31 after education.According to the findings of this study, nurses who play a significant role in patient safety incident reporting can perform well as nurses by providing feedback through continuous monitoring of patient safety reports and practical education on harm assessment.It stated that an educational program that actively assists in achieving this goal is required.Furthermore, it is thought necessary to create a tool for standardizing harm assessment at the level of a multidisciplinary society.
There are several limitations in generalizing the results of this study.First, as the study was conducted only for nurses working in one medical institution, there are limitations in its application to other ethnicities or countries.Second, this study was not able to conduct a control comparison due to the same group comparison before and after education on harm assessment, and all subjects were female nurses who were in charge of nursing cancer patients as the institution where this study was conducted was a National Cancer Center.Furthermore, when the research was divided into before and after education, the research subjects were asked to read and evaluate each scenario immediately without separate education to explain the definition and standards of harm and harm period, it is believed that differences in education and experience between individuals could have been greatly reflected.Depending on their work experience other than online video education in this study, the survey results of the research subjects who participated in the education may have resulted from the accumulation of experience related to reporting related patient safety incidents, etc.Finally, it may have influenced the results of this study due to the difference in the survey method before and after education and the number of subjects in the second survey decreased significantly due to those who first participated in the survey who resigned from their jobs or refused to participate in the survey that conducts education, and future studies require to minimize the gap between studies.

IMPLICATION FOR NURSING
The finding of this study has shown that harm assessment education was effective among ways to increase the consistency of harm assessment of PSEs among nurses, which is important information for patient risk management of institutions.Based on this, it can be used as a basic material for developing standardized educational materials that can help nurses assess the harm of PSEs when reporting PSEs in the future and educational methods using them.

CONCLUSION AND RECOMMENDATIONS
In this study, accurate harm assessment of medical staff is critical in the patient safety incident reporting system, but there is a perception gap due to a lack of education.A study was conducted to develop guidelines for education and to understand the difference in the effectiveness of education in the degree of consistency in harm assessment before and after education using them in order to increase the consistency of the most active nurse's harm assessment in patient safety event reporting in the clinical field.
According to the findings of this study, harm assessment education in the patient safety classification system can increase the consistency rate in harm assessment among nurses.As a result, it is suggested that a related education program be developed that can actively assist patients in performing their safety obligations as nurses through the expansion of offline field education using harm assessment guidelines and casebooks of patient safety incidents.

HUMAN AND ANIMAL RIGHTS
This study did not involve any animals.All humans were used in accordance with the ethical standards of the committee in charge of human research with the Bioethics and Safety Act and with the Helsinki Declaration of 1975, as revised in 2013.

CONSENT FOR PUBLICATION
After explaining the purpose of the study and assuring the subject about the privacy and confidentiality of the obtained data, informed consent was obtained from the participants in this study.
the research, the Institutional Review Board (IRB) of the institution to which the research subjects belonged obtained ethical approval (IRB No. NCC 2020-0049).Prior to the start of online education, participants could only participate in the education and survey if they agreed voluntarily.