RESEARCH ARTICLE
Comparison of Perception Differences among Nurses based on Harm Assessment Education in the Patient Safety Classification System
Kwangmi Lee1, Eunhee Shin2, *
Article Information
Identifiers and Pagination:
Year: 2023Volume: 17
E-location ID: e187443462308170
Publisher ID: e187443462308170
DOI: 10.2174/18744346-v17-230927-2023-5
Article History:
Received Date: 09/01/2023Revision Received Date: 30/05/2023
Acceptance Date: 21/07/2023
Electronic publication date: 04/10/2023
Collection year: 2023
open-access license: This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International Public License (CC-BY 4.0), a copy of which is available at: https://creativecommons.org/licenses/by/4.0/legalcode. This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Abstract
Background:
Accurate harm assessment is critical in the patient safety event management system, but few studies have been published to support the need for training in harm assessment of patient safety events for nurses.
Objective:
The purpose of this study was to see if there was a difference in the degree of agreement between before and after nurses received training on a patient safety event harm assessment guideline.
Methods:
After participating in online harm assessment education, 65 subjects completed the self-report questionnaire. Data for the general characteristics of the study subjects and the frequency of respondents for each scenario were analyzed using descriptive statistics. Fleiss' kappa was calculated by estimating the inter-rater agreement among respondents for each scenario.
Results:
The agreement value of subjects by Fleiss' Kappa value improved from k = 0.23 before education to k = 0.31 after education, according to the harm assessment. There was no change at k = 0.30 before and after education for the harm period.
Implication for Nursing & Conclusion:
This study’s findings suggest that harm assessment agreement among nurses could be increased through harm assessment education. As a result, case-based education on harm assessment must be expanded, as well as related programs for practical education via patient safety event casebooks.
1. INTRODUCTION
The Institute of Medicine emphasized the importance of patient safety and patient safety reporting in its 1999 report “To Err Is Human: Building a Safer Health System” and the World Health Organization (WHO) Board of Directors announced in 2003 the need to develop standard terminology and classification systems, harm assessment, and reporting-learning systems as systematic factors for patient safety [1, 2]. According to reports on patient’s safety, the reporting system assists each medical institution in collecting data on patient safety events (PSEs) severity of the event, and the impact on the patient [3], and health care professionals or patient safety experts rely on information about the event that occurred, the degree of harm, and information on contributing factors [4]. However, while the patient safety reporting system provides critical information for analyzing actual or potential PSEs, low-quality data limits its utility. Accurate harm assessment is critical in the PSE management system because it is an essential step in the study of PSEs and provides an opportunity to lead patient risk management.
The Patient Safety Organization developed the Common Formats, a standardized PSE reporting format [5]. In addition, the WHO developed a 5-point scale for harm assessment [6], the National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) developed a tool for considering the duration and permanence of harm, with a kappa value of 0.74 [7]. The Institute for Healthcare Improvement developed a Global Trigger Tools with a mean kappa value of 0.65 [8, 9]. The reliability of the reporter's judgment in the type and severity of PSEs is the most important factor in the harm score. If the reliability of the harm assessment is low, comparing PSE data and determining direction will be difficult.
The clinical backgrounds of clinicians, nurses, and pharmacists who report PSEs in the clinical setting differ, which is the cause of deviation in harm assessment. Nurses were reported to be the most active in reporting events, owing to the fact that they witnessed PSEs the most frequently during the nursing process, and reported lower harm assessments for medication errors than clinicians [10]. Second, researchers emphasize the importance of education and training in understanding harm scores [11]. Finally, it has been demonstrated that the reliability of PSEs can be affected by how they are reported and that bias can be reduced because web-based reporting can review integrated data and direction rather than paper-type reporting [12]. Because nurses report more PSEs than clinicians [13], nurses' behaviors, attitudes, and perceptions are especially important when reporting PSEs [3]. It has also been reported that nurses' relationships with doctors, workload, and working environment, such as colleagues, all have an impact on data quality [14].
According to the study of inter-rater agreement in harm assessment for nurses, the AHRQ Common Format Harm Scale version 1.1 and 1.2 (AHRQ’s version 1.1 or 1.2) showed a moderate degree of agreement, with k=0.45, but agreement for each ward in the same institution was found to be very low [14]. Another study for nurses using the AHRQ’s version 1.2 found low agreement between raters, with k = 0.21 [15].
The low consistency in nurses’ harm assessment can be interpreted as a subject in the interpretation of each patient safety event scenario of the AHRQ’s version 1.2 used in the study, but it also suggests the need for education so that harm assessment scores can be assigned accurately [3].
Similarly, according to the findings of a study that examined PSEs reported in the emergency room, emergency room residents require education on the classification of patient safety events and harm assessment [16]. However, few studies have been published to support the need for training in harm assessment of PSEs for nurses or clinicians who are major PSE reporters in the clinical setting. As a result, the goal of this study was to see if there was a difference in consistency before and after developing guidelines for PSE harm assessment education for nurses in clinical settings.
2. MATERIALS AND METHODS
2.1. Research Design
This is a single-group pre- and post-design experimental study comparing before and after education by developing a “Harm assessment guideline” educational material to improve the consistency of harm assessment in nurse's PSE reports.
2.2. Study Population
The nurses who participated in the previous study’s survey were evaluated using the developed “Harm assessment guideline” with the cooperation of the national cancer center specializing in cancer with 555 beds in Gyeonggi-do, which conducted the harm and harm period evaluation among nurses in 2018. Following education on the harm assessment of PSEs, a survey on nine scenarios of PSEs, was conducted, and the results before and after education were compared and analyzed.
The study was conducted after receiving Institutional Review Board (IRB) approval in February 2020, but due to the unexpected COVID-19 situation, collective education was not possible, so the research method was changed to online education and an online survey.
Following the creation of an educational video by the investigator, the investigator commissioned an online survey specialist to have the subjects watch the educational video and then participate in the survey. Many of the 133 subjects who participated in the 2018 survey no longer worked at the institution due to job changes or leave, and some refused to participate in the study. As a result, the results of 65 research subjects who completed their education and participated in the survey were analyzed after they voluntarily agreed to participate in the study.
2.3. Guidelines for Harm Assessment and Educational Videos
A survey of clinical nurses was conducted using the AHRQ’s version 1.2 to report PSEs [15], indicating that harm and harm period evaluation were dispersed when harm such as “wrongly labeled samples” or “thoracic drainage” was not obvious in the scenario. With these findings in mind, the “Harm assessment guideline” was created by referring to data developed elsewhere to assist in assessing the harm and harm period of nurses.
We developed “Harm assessment guidelines” by referring to existing guidelines [16-18] related to harm assessment. The “Harm assessment guidelines” began with an overview of the harm WHO’s harm classification system and scale, as well as the NCC MERP and AHRQ harm classification system and scale. The AHRQ’s version 1.2 was then described. Since it is necessary to confirm whether there has been a deviation from standard treatment when classifying preventable harm assessment, the considerations when assessing the patient’s outcome were explained. Finally, the harm assessment guidelines in PSEs, as well as the PSEs classification and harm system were explained and the harm assessment was conducted using actual patient safety event examples. Fig. (1) depicts the contents of the “patient safety event classification” presented in this study.
As for the expert review of the developed guidelines, it was impossible to hold face-to-face meetings due to the COVID-19 situation, so the contents were revised and supplemented by listening to patient safety experts’ reviews and opinions through non-face-to-face methods such as video conferencing and e-mail.
 |
Fig. (1). Algorithm for classifying patient safety events. |
2.4. Research Tool
2.4.1. Patient Safety Event Scenarios are Chosen and Reviewed for Harm Assessment
The patient safety scenario of the questionnaire was developed in this study with reference to AHRQ’s version 1.2 [4], with a focus on domestic patient safety cases. Several emails were sent to the study manager for use approval prior to AHRQ’s version 1.2 in this study, but no response was received, so the contents are described in the related table.
The AHRQ’s version 1.2 is intended to evaluate harm and harm duration for each of the nine patient safety scenarios. Each scenario includes (1) incorrect administration route, (2) lacerations of body parts, (3) allergic reactions to contrast agents, (4) abdominal infections, (5) mislabeled samples, (6) chest tube drainage, (8) overdose, and (9) incorrect time administration; in this study, actual cases related to this were selected by referring to the Korean Association of Hospital Nurses “Patient Safety Incident Cases and Prevention” [19].
After selecting the scenario, three experts and one university professor working in relation to patient safety work at a medical institution reviewed each topic and whether the selected scenario was appropriate, and then conducted a study.
As the general characteristics of the subject, questions such as gender, length of service at current workplace, current department, job satisfaction in current department, completion of patient safety education in current department, patient safety education method, and content of patient safety education were included in the questionnaire.
There were also 13 questions to determine patient safety incident participation in the current department, patient safety incident experience in the current department, patient safety incident type, patient safety incident report, and experienced patient safety incident type.
2.4.2. Definition of Harm and Harm Duration
The level of harm and harm duration were assessed for domestic patient safety incidents using the criteria presented by NCC MERP [20] and classified as follows; first, the level of harm is near miss (A): an environment that can cause a hazardous event, such as unorganized medical equipment, near miss (B): an incident occurred but did not reach the patient, and no harm event (C): an incident occurred, but no harm to the patient and no additional monitoring required; no harm event (D): if an incident has occurred and no harm will occur to the patient, or additional intervention is required to prevent harm, adverse event (E): when an incident occurs that causes temporary harm to the patient and requires additional treatment or intervention, adverse event (F): An accident that causes temporary harm to the patient and necessitates short or long-term hospitalization, and a sentinel event: an incident that causes near-death or permanent harm to the patient, or the patient died, or the event contributed to the patient’s death. When an incident occurs and temporary harm to the patient lasts more than one year, it is classified as permanent, temporary harm to the patient and lasts less than one year, and unknown.
2.5. Statistical Analysis
SPSS 26.0 was used to analyze the collected data (SPSS Inc, Chicago, IL, USA). First, the general characteristics of the study subjects were calculated, as well as the frequency of respondents for each scenario before and after education. The Fleiss' kappa value was then calculated by estimating the interrater agreement for each scenario among respondents before and after education. The agreement rates among respondents were: 0.81–1.00 for perfect agreement, 0.61–0.8 for significant agreement, 0.41–0.60 for moderate agreement, 0.21–0.40 for slight agreement, 0.01–0.20 for slight agreement and a score of 0 or less for almost no agreement.
2.6. Ethical Consideration
Before proceeding with the research, approval was obtained from the IRB of the institution to which the research subjects belonged. Following that, the purpose of the study was explained to the hospital’s nursing headquarters, and the study was carried out with cooperation.
The researcher explained the purpose and method of the study to the research subjects who participated in the 2018 study through an explanation for the online education and survey. After collecting e-mail addresses for online education and re-survey, the relevant web address was sent if the subject voluntarily expressed their intention to participate. Prior to the start of online education, participants could only participate in the education and survey if they agreed voluntarily. Furthermore, to protect personal information, the collected e-mail addresses of the subjects were discarded immediately after the online training and survey were completed.
3. RESULTS
3.1. General Characteristics of Research Subjects
All of the subjects in this study were female, and the general ward was the most common work department, accounting for 52.3% of the subjects. When asked how long they had been at their current job, 33.8% said 5 years or more and less than 10 years, 58.8% said more than 10 years, and the majority of subjects said more than 5 years.
In terms of job satisfaction, “satisfied” was the highest at 70.8%. It was discovered that all of the subjects received patient safety education at their current workplace, with 86.3% receiving “certificated brochure training” and 70.8% receiving “theoretical lectures.” Patient safety education content, was 90.8% “incident reporting procedures,” 84.6% “understanding patient safety,” and 81.5% “time and method identification.” At 64.6%, patient safety education time was greater than one hour but less than four hours (Table 1).
3.2. Patient Safety Incident Experience of Research Subjects
The majority of study subjects (92.3%) had experience with PSEs, with “no harm safety event” 83.3% and “near miss” 66.7% having the highest duplicate responses to experienced PSEs. When asked if they had any experience reporting PSEs, 83.1% said they had, with the most common types of patient safety incidents being “fall” 83.3% and “medication” 66.7% (Table 2).
| Variables | Categories | n (%) |
|---|---|---|
| Gender | Male | 0(0.0) |
| Female | 65(100.0) | |
| Current department | General ward | 34(52.3) |
| Special ward (ICU, ER, etc) | 11(16.9) | |
| Outpatient department | 16(24.6) | |
| Others+ | 4(6.2) | |
| Working period at current job (year) | <1 | 0(0.0) |
| ≥1–<5 | 5(7.7) | |
| ≥5–<10 | 22(33.8) | |
| ≥10 | 38(58.8) | |
| Job satisfaction at current job | Very satisfied | 2(3.1) |
| Satisfied | 46(70.8) | |
| Unsatisfied | 17(26.2) | |
| Very unsatisfied | 0(0.0) | |
| Whether or not getting a patient safety training at current job | Yes | 65(100.0) |
| No | 0(0.0) | |
| Method of a patient safety training (duplication check) | Theoretical lectures | 46(70.8) |
| Case-based discussion training | 5(7.7) | |
| Certificated brochure training | 56(86.3) | |
| Department conveying training | 37(56.9) | |
| Others++ | 2(3.1) | |
| Contents of a patient safety training (duplication check) | Understanding patient safety | 55(84.6) |
| Time and method of patient identification | 53(81.5) | |
| Grade and criteria of patient safety incident reporting | 51(78.5) | |
| Incident reporting procedures | 59(90.8) | |
| Inpatient care management | 23(35.4) | |
| Activating near miss reporting | 24(36.9) | |
| Others | 0(0.0) | |
| Time of a patient safety training (hour) | <1 | 19(29.2) |
| ≥1–<4 | 42(64.6) | |
| ≥4–<8 | 3(4.6) | |
| ≥8 | 1(1.5) | |
| Missing | 2(0.7) | |
| Whether or not participating events§ of a patient safety at current job | Yes | 32(49.2) |
| No | 33(50.8) |
| Categories | n (%) |
|---|---|
| Experience of patient safety accident at current job | - |
| Yes | 60(92.3) |
| No | 5(7.7) |
| Type of patient safety accident (duplication check) | - |
| Near miss | 40(66.7) |
| No harm safety event | 50(83.3) |
| Mild/Moderate/Severe safety event | 28(46.7) |
| Sentinel event | 13(21.7) |
| Reporting on patient safety accidents | - |
| Yes | 54(83.1) |
| No | 6(9.2)) |
| Missing | 5(7.7) |
| Types of patient safety accidents experienced (duplication check) | - |
| Surgery | 4(6.7) |
| Delivery | 0(0.0) |
| Treatment procedure | 8(13.3) |
| Anesthesia | 0(0.0) |
| Clinical examination | 7(11.7) |
| Blood transfusion | 0(0.0) |
| Medication | 40(66.7) |
| Infection | 1(1.7) |
| Computerized disorder | 0(0.0) |
| Medical equipment/Medical device | 4(6.7) |
| Hospital meal | 3(5.0) |
| Fall | 50(83.3) |
| Treatment material contamination /failure | 3(5.0) |
| Suicide/Self-harm | 7(11.7) |
| Other | 1(1.7) |
3.3. Agreement of Harm and Harm Duration after Education
Tables 3 and 4 show the results of the study subjects” evaluations of harm and duration of harm for each scenario. The items that showed changes before and after training in the distribution of respondents by scenario were the 'Medication given via wrong route' scenario. Before education, it was 46.6% of “mild harm” and 24.8% of “moderate harm,” but after education, it was 52.3% of “mild harm” and 27.7% of “no harm.”
| Scenario | Frequency of Respondents (%) | |||||||
|---|---|---|---|---|---|---|---|---|
| - |
Near Miss (A) |
Near Miss (B) |
No Harm (C) |
Mild Harm (D) |
Moderate Harm (E) |
Severe Harm (F) |
Sentinel Event | |
| 1. Medication given via wrong route | Before | 3 (4.6) | 2 (3.1) | 4 (6.2) | 31 (47.7) | 15 (23.1) | 5 (7.7) | 5 (7.7) |
| After | 1 (1.5) | 0 (0.0) | 18 (27.7) | 34 (52.3) | 11 (16.9) | 1 (1.5) | 0 (0.0) | |
| 2. Body part laceration during surgery | Before | 0 (0.0) | 0 (0.0) | 0 (0.0) | 0 (0.8) | 19 (29.2) | 31 (47.7) | 15 (23.1) |
| After | 0 (0.0) | 0 (0.0) | 0 (0.0) | 0 (0.0) | 18 (27.7) | 40 (61.5) | 7 (10.8) | |
| 3. Contrast allergy | Before | 0 (0.0) | 0 (0.0) | 1 (1.5) | 2 (3.1) | 25 (38.5) | 33 (50.8) | 4 (6.2) |
| After | 1 (1.5) | 0 (0.0) | 0 (0.0) | 6 (9.2) | 30 (46.2) | 28 (43.1) | 0 (0.0) | |
| 4. Abdominal site infection | Before | 0 (0.0) | 1 (1.5) | 0 (0.0) | 3 (4.6) | 3 (4.6) | 35 (53.8) | 23 (35.4) |
| After | 2 (3.1) | 0 (0.0) | 0 (0.0) | 0 (0.0) | 3 (4.6) | 41 (63.1) | 19 (29.2) | |
| 5. Mislabeled specimen | Before | 0 (0.0) | 0 (0.0) | 1 (1.5) | 0 (0.0) | 29 (44.6) | 17 (26.2) | 18 (27.7) |
| After | 0 (0.0) | 0 (0.0) | 0 (0.0) | 5 (7.7) | 35 (53.8) | 10 (15.4) | 15 (23.1) | |
| 6. Wrong site surgery | Before | 0 (0.0) | 1 (1.5) | 0 (0.0) | 0 (0.0) | 1 (1.5) | 14 (21.5) | 49 (75.4) |
| After | 0 (0.0) | 0 (0.0) | 0 (0.0) | 0 (0.0) | 3 (4.6) | 4 (6.2) | 58 (89.2) | |
| 7. Chest tube drain | Before | 0 (0.0) | 3 (4.6) | 11 (16.9) | 31 (47.7) | 17 (26.2) | 1 (1.5) | 2 (3.1) |
| After | 1 (1.5) | 3 (4.6) | 21 (32.3) | 27 (41.5) | 13 (20.0) | 0 (0.0) | 0 (0.0) | |
| 8. Medication overdose | Before | 2 (3.1) | 1 (1.5) | 3 (4.6) | 34 (52.3) | 23 (35.4) | 0 (0.0) | 2 (3.1) |
| After | 0 (0.0) | 1 (1.5) | 1 (1.5) | 41 (63.1) | 21 (32.3) | 1 (1.5) | 0 (0.0) | |
| 9. Medication given at the wrong time | Before | 2 (3.1) | 6 (9.2) | 26 (40.0) | 27 (41.5) | 3 (4.6) | 1 (1.5) | 0 (0.0) |
| After | 2 (3.1) | 11 (16.9) | 33 (50.8) | 18 (27.7) | 1 (1.5) | 0 (0.0) | 0 (0.0) | |
| - | Frequency of Respondents (%) | |||
|---|---|---|---|---|
| - | Permanent | Temporary | Unknown | |
| 1. Medication given via wrong route | Before* | 5 (7.7) | 29 (44.6) | 29 (44.6) |
| After | 0 (0.0) | 36 (55.4) | 29 (44.6) | |
| 2. Body part laceration during surgery | Before | 30 (46.2) | 15 (23.1) | 20 (30.7) |
| After | 22 (33.8) | 27 (41.5) | 16 (24.6) | |
| 3. Contrast allergy | Before | 1 (1.5) | 53 (81.5) | 11 (16.9) |
| After | 0 (0.0) | 54 (83.1) | 11 (16.9) | |
| 4. Abdominal site infection | Before | 25 (38.5) | 14 (21.5) | 26 (40.0) |
| After | 25 (38.5) | 10 (15.4) | 30 (46.2) | |
| 5. Mislabeled specimen | Before | 6 (9.2) | 34 (52.3) | 25 (38.5) |
| After | 7 (10.8) | 38 (58.8) | 20 (30.8) | |
| 6. Wrong site surgery | Before | 61 (93.9) | 1 (1.5) | 3 (4.6) |
| After | 64 (98.5) | 0 (0.0) | 1 (1.5) | |
| 7. Chest tube drain | Before | 1 (1.5) | 58 (89.2) | 6 (9.2) |
| After | 0 (0.0) | 57 (87.7) | 8 (12.3) | |
| 8. Medication overdose | Before | 2 (3.1) | 41 (63.1) | 22 (33.9) |
| After | 0 (0.0) | 41 (63.1) | 24 (36.9) | |
| 9. Medication given at the wrong time | Before | 0 (0.0) | 36 (55.4) | 29 (44.6) |
| After | 0 (0.0) | 33 (50.8) | 32 (49.2) | |
** Adapted from the work of T. Williams et al. The reliability of AHRQ Common Format Harm Scales in rating patient safety events. J Patient Saf 2015 Mar;11(1):52-9.
Following that, in the “Wrong site surgery” scenario, 71.4% of “Sentinel Event” and 27.1% of “Severe Harm” were evaluated as 89.2% of “Sentinel Event” after education. Finally, in the “Medication given at the wrong time” scenario, the “Near Miss” rate before education was 9.8% but increased to 16.9% after education. In contrast to the harm assessment, there were no significant differences in the harm period evaluation before and after education (Tables 3 & 4).
The Fleiss' kappa value of respondents’ agreement showed that the harm assessment improved from k = 0.23 before education to k = 0.31 after education. There was little change in the harm period from k=0.29 before education to k = 0.30 after education (Table 5).
| - | Overall Kappa Value* | |
|---|---|---|
| Before Education | After Education | |
| Harm scale | .23 | .31 |
| Harm duration | .29 | .30 |
4. DISCUSSION
Because patient safety is dependent on information about PSEs reported in the patient safety incident reporting system, the harm of the incident, and the factors that caused the occurrence, harm assessment of patient safety incident reporters standardization is critical. In particular, since the Patient Safety Act was enacted in Korea in 2016, patient safety incidents are managed with a focus on reported incidents, and in order to promote qualitative improvement, with voluntary reporting of each patient safety incident, a standardized evaluation of harm and factors is very necessary. As a result, the accuracy of the classification system of PSEs reported in the patient safety system is critical, but the findings of previous studies [3, 4, 15] show that there is a significant difference in nurses' perception of PSEs.
The AHRQ Common Format Harm Scale evaluation tool used in this study was created as part of a project to standardize the assessment of harm in PSEs in the United States. The assessment tool has created a scenario for determining the extent of harm and the duration of the harm. In the harm assessment criteria, a ‘near miss’ is a concept that defines situations and events that did not reach the patient but may cause 'harm' in the future, whereas a 'no harm event' and an 'adverse event' are concepts that are divided by 'harm' among the events that did reach the patient. The harm period is a concept that divides whether the harm occurs for more than one year.
The inter-rater harm confidence value for the AHRQ’s version 1.1 PSE scenario for 921 institutional quality, risk, and safety managers at UHC PSN (Patient Safety Net®) was k=0.51, and reliability measurements for version 1.2 with an updated version 1.1 were conducted on 13,280 UHC PSN managers, with a moderate reliability value of k=0.47 between inter-rater [4].
The same tool AHRQ’S versions 1.1 and 1.2 measured inter-measurer reliability values for the risk of PSE scenarios for nurses working in cancer centers, with version 1.1 k=0.45 and version 1.2 k=0.48, indicating moderate reliability [3]. In another study, doctors, nurses, pharmacists, and other healthcare professionals who are the primary reporters of PSEs in the clinical field were evaluated for nine PSE scenarios, and it was found that the nurse group had a higher reliability difference in patient safety case assessment than the other healthcare professionals [10].
In this study, general nurses working at advanced general hospitals and the national cancer center in Korea were reported as having k=0.21 as a result of evaluating the harm of PSEs between nurses using a questionnaire tool developed based on domestic PSEs [15]. This study also revealed that k=0.23 before and k=0.31 after education, which was lower than the previous study [3, 4]. Even though the subject was a patient safety manager of a medical institution, in the case of a study conducted under the supervision of UHC in the United States, there was a significant difference in the perception of patient safety events between domestic studies that targeted general nurses.
The reasons for perception among healthcare professionals, including nurses, doctors, and pharmacists, are due to the diversity of each healthcare professional's educational background [13], differences in understanding of harm items [10], and whether the format in which PSEs are reported is paper-based or web-based [12]. Finally, it was reported that unclear guidelines for the definition and knowledge structure of each harm item in harm assessment may lead to a biased view of harm assessment [3, 21].
Furthermore, when harm assessment is conducted using standardized evaluation tools such as the AHRQ’S version 1.1, the low consistency value between inter-rater is inconsistent with the understanding of harm items as in other studies, and the selection of risk scores among evaluators is very subjective, the necessity of education was suggested as a way to reduce this [3, 22].
Until now, no studies have reported the results of training medical personnel to increase the consistency of evaluators of harm assessment in reporting actual PSEs, but assessment guidelines have been developed [16]. In this study, an educational video was created in order to improve the consistency of nurses' harm assessment by referring to existing guidelines, and then the consistency value was compared after education.
After explaining the developed harm classification system and scale using the example of PSEs in the educational video, nurses who participated in the education were organized to conduct harm assessment of PSEs based on the actual educational contents. Due to the COVID-19 situation, online education was conducted, but nurses who participated in the education were asked to participate in a harm assessment survey after watching educational videos.
The study found that the Fleiss' kappa value, which represents agreement among raters in harm assessment before education, was 0.23 but improved to 0.31 after education. According to the findings of this study, nurses who play a significant role in patient safety incident reporting can perform well as nurses by providing feedback through continuous monitoring of patient safety reports and practical education on harm assessment. It stated that an educational program that actively assists in achieving this goal is required. Furthermore, it is thought necessary to create a tool for standardizing harm assessment at the level of a multidisciplinary society.
There are several limitations in generalizing the results of this study. First, as the study was conducted only for nurses working in one medical institution, there are limitations in its application to other ethnicities or countries. Second, this study was not able to conduct a control comparison due to the same group comparison before and after education on harm assessment, and all subjects were female nurses who were in charge of nursing cancer patients as the institution where this study was conducted was a National Cancer Center. Furthermore, when the research was divided into before and after education, the research subjects were asked to read and evaluate each scenario immediately without separate education to explain the definition and standards of harm and harm period, it is believed that differences in education and experience between individuals could have been greatly reflected. Depending on their work experience other than online video education in this study, the survey results of the research subjects who participated in the education may have resulted from the accumulation of experience related to reporting related patient safety incidents, etc. Finally, it may have influenced the results of this study due to the difference in the survey method before and after education and the number of subjects in the second survey decreased significantly due to those who first participated in the survey who resigned from their jobs or refused to participate in the survey that conducts education, and future studies require to minimize the gap between studies.
5. IMPLICATION FOR NURSING
The finding of this study has shown that harm assessment education was effective among ways to increase the consistency of harm assessment of PSEs among nurses, which is important information for patient risk management of institutions. Based on this, it can be used as a basic material for developing standardized educational materials that can help nurses assess the harm of PSEs when reporting PSEs in the future and educational methods using them.
CONCLUSION AND RECOMMENDATIONS
In this study, accurate harm assessment of medical staff is critical in the patient safety incident reporting system, but there is a perception gap due to a lack of education. A study was conducted to develop guidelines for education and to understand the difference in the effectiveness of education in the degree of consistency in harm assessment before and after education using them in order to increase the consistency of the most active nurse's harm assessment in patient safety event reporting in the clinical field.
According to the findings of this study, harm assessment education in the patient safety classification system can increase the consistency rate in harm assessment among nurses. As a result, it is suggested that a related education program be developed that can actively assist patients in performing their safety obligations as nurses through the expansion of offline field education using harm assessment guidelines and casebooks of patient safety incidents.
LIST OF ABBREVIATIONS
| WHO | = World health organization |
| PSEs | = Patient Safety Events |
| NCC MERP | = National coordinating council for medication error reporting and prevention |
ETHICS APPROVAL AND CONSENT TO PARTICIPATE
Before proceeding with the research, the Institutional Review Board (IRB) of the institution to which the research subjects belonged obtained ethical approval (IRB No. NCC 2020-0049). Prior to the start of online education, participants could only participate in the education and survey if they agreed voluntarily.
HUMAN AND ANIMAL RIGHTS
This study did not involve any animals. All humans were used in accordance with the ethical standards of the committee in charge of human research with the Bioethics and Safety Act and with the Helsinki Declaration of 1975, as revised in 2013.
CONSENT FOR PUBLICATION
After explaining the purpose of the study and assuring the subject about the privacy and confidentiality of the obtained data, informed consent was obtained from the participants in this study.
AVAILABILITY OF DATA AND MATERIALS
The data sets used in the current study are available upon reasonable request from the corresponding author [F.S].
FUNDING
This study was funded by the MOE (Ministry of Education) through the NRF (National Research Foundation of Korea) (NRF2018R1D1A3B07049955).
CONFLICT OF INTEREST
There are no conflicts of interest reported by the authors. This article’s content and writing are solely the responsibility of the authors.
ACKNOWLEDGEMENTS
The contributions of all participants in this study are greatly appreciated. We would like to thank Editage (www.editage.co.kr) for English language editing.
REFERENCES
| [1] | Institute of Medicine. Patient safety: Achieving a new standard of care. Washington, DC: National Academy Press 2003. |
| [2] | World Health Organization. World Alliance for Patient Safety: forward programme 2005. Geneva: World Health Organization 2004. |
| [3] | Abbasi T, Adornetto-Garcia D, Johnston PA, Segovia JH, Summers B. Accuracy of harm scores entered into an event reporting system. J Nurs Adm 2015; 45(4): 218-25. |
| [4] | Williams T, Szekendi M, Pavkovic S, Clevenger W, Cerese J. The reliability of AHRQ common format harm scales in rating patient safety events. J Patient Saf 2015; 11(1): 52-9. |
| [5] | Clancy CM. Common formats allow uniform collection and reporting of patient safety data by patient safety organizations. Am J Med Qual 2010; 25(1): 73-5. |
| [6] | WHO. Conceptual framework for the international classification for patient safety version 1.1: final technical report January. 2009. |
| [7] | Forrey RA, Pedersen CA, Schneider PJ. Interrater agreement with a standard scheme for classifying medication errors. Am J Health Syst Pharm 2007; 64(2): 175-81. |
| [8] | Classen DC, Resar R, Griffin F, et al. Global trigger tool shows that adverse events in hospitals may be ten times greater than previously measured. Health Aff 2011; 30(4): 581-9. |
| [9] | Hanskamp-Sebregts M, Zegers M, Vincent C, van Gurp PJ, de Vet HCW, Wollersheim H. Measurement of patient safety: A systematic review of the reliability and validity of adverse event detection with record review. BMJ Open 2016; 6(8): e011078. |
| [10] | Williams SD, Ashcroft DM. Medication errors: How reliable are the severity ratings reported to the national reporting and learning system? Int J Qual Health Care 2009; 21(5): 316-20. |
| [11] | Mayo AM, Duncan D. Nurse perceptions of medication errors: What we need to know for patient safety. J Nurs Care Qual 2004; 19(3): 209-17. |
| [12] | Liang C, Gong Y. Predicting harm scores from patient safety event reports. Stud Health Technol Inform 2017; 245: 1075-9. |
| [13] | Evans SM, Berry JG, Smith BJ, et al. Attitudes and barriers to incident reporting: A collaborative hospital study. Qual Saf Health Care 2006; 15(1): 39-43. |
| [14] | Patrician PA, Brosch LR. Medication error reporting and the work environment in a military setting. J Nurs Care Qual 2009; 24(4): 277-86. |
| [15] | Lee K, Yoon K, Yoon B, Shin E. Differences in the perception of harm assessment among nurses in the patient safety classification system. PLoS One 2020; 15(12): e0243583. |
| [16] | AHRQ. Users guide version 1.2 AHRQ Common Formats for Patient Safety Organization. 2013. |
| [17] | HPI. SEC & SSER Patient safety measurement system for healthcare. 2011. |
| [18] | ASHRM. Serious safety events: A focus on harm classification: Deviation in care as link. 2014. |
| [19] | Chae G, Noa W, Park M, Jung E, Cho E, Jung M. Prevention and risk management for patient safety. In: Hospital Nurses Association. 2014; pp. 26-105. |
| [20] | NCC MERP Index for categorizing medication errors algorithm [Internet]: NCC MERP; c1996-2001. Available from: https://www.nccmerp.org/types-medication-errors (revised 2001 Feb 20) |
| [21] | Liang C, Gong Y. Knowledge representation in patient safety reporting: An ontological approach. J Data Inf Sci 2017; 1(2): 75-91. |
| [22] | Feeser VR, Jackson A, Senn R, et al. Patient safety event reporting and opportunities for emergency medicine resident education. West J Emerg Med 2020; 21(4): 900-5. |
