Use of Online Machine Translation for Nursing Literature: A Questionnaire-Based Survey

AIM

Our aim was to explore how Japanese nursing professionals use online MT and perceive its usability in reading English articles and to discuss what should be considered for better utilisation of online MT lessening the language barrier. More specifically, we attempted to investigate the following research questions: How do Japanese nurses use online MT and perceive its usability? What factors are associated with their online MT use and the perceived usability?

Study Participants and Data Collection

A cross-sectional exploratory survey was conducted with 250 Japanese nursing participants, using a self-administered questionnaire. Our objective was to select a study population of native Japanese speakers with the necessary nursing experience to understand the clinical contexts of technical papers. Assistants and research associates^ at nursing universities were assumed to be an appropriate choice because they had already had professional experience both as clinical nurses and as teachers, usually with more experience in clinical settings than in educational or research environments. Nevertheless, it was anticipated that these people would have some idea of how the research might be utilised.

In Japan, there were 200 4-year nursing programs as of February, 2012, and the number of assistants and research associates across Japan was assumed to be over 2,000. Two to four individuals were randomly selected from the website of each university, for a total of 500 candidates. The actual participants were recruited by mail. Each candidate was sent the questionnaire and a cover letter asking for his/her participation, and 498 of them received it. The survey was completed by each participant and anonymously returned by mail. In total, 250 individuals responded to the survey, a response rate of 50.2%. The data were collected from March to April of 2012.

Measures

The questionnaire included background questions, previous and current online MT usage, and perceived usability of online MT (Tables 1 and 2). The number of technical articles read within approximately the previous 3 months was asked and scored on a five-point Likert scale (1: 0, 2: 1–5, 3: 6–10, 4: 11–20, and 5: 21 or more). The frequency of language barrier perception was also asked about and scored on a five-point Likert scale (1: never, 2: rarely, 3: occasionally, 4: often, and 5: always). For the question regarding previous online MT use, it could be answered Yes or No. The frequency of online MT use was rated on a three-point Likert scale (1: rarely, 2: occasionally, and 3: always/almost always). The perceived usability of online MT was measured by answers on a five-point Likert scale (1: not usable at all, 2: not very usable, 3: neither usable nor unusable, 4: usable to some degree, and 5: very usable). The question of participants’ experience using Internet search engines that offer online MT had the following choices: 1) Yahoo, 2) Google, 3) Bing, 4) Excite, 5) Nifty, 6) Livedoor, 7) OCN, 8) Sonet, 9) other, and 10) do not remember the names of the search engine(s).

Respondents who answered that they had never used online MT were asked to select a reason from among the following: “I do not need it”, “I do not think it is usable”, “I didn’t know there were online MT services”, and “any other reason(s)” (to be specified). In addition, those who had used online MT, but stopped using it could answer “I do not use it anymore”, optionally selecting a reason from “I do not need it”, “ I do not think it is useful”, or “any other reason(s)” (to be specified).

Data Analysis

The data were analysed using the Statistical Package for the Social Sciences (SPSS), version 20. Descriptive statistics, the t-test, analysis of variance (ANOVA), the chi-square test, and Spearman’s correlation were used. Raw data were analysed after calculating the means, standard deviations (SDs), distributions, and percentages for questionnaire items. The chi-square test was used to identify the differences in the number of English articles read and perceived language barrier according to the respondents’ demographic data, where the number of English articles read was measured based on the binary responses “0–5” and “6 or more”, and perceived language barrier was measured based on the binary responses “less” and “more”. Also, the chi-square test was used to determine the differences in perceived usability, measured based on the binary responses “not usable” and “usable”, according to the participants’ characteristics. The ttest and ANOVA were used to detect the differences in the frequency of online MT system use according to participant characteristics. A post hoc examination was performed, using a multiple comparison procedure (Tukey’s HSD test). The relationships between professional experience and the outcome variables were examined using a Spearman’s correlation coefficient.

Ethical Considerations

The Ethical Review Board of the University of Tokyo approved this study. Participants were informed that all information would be treated confidentially, and that they could contact the researcher if they had any questions.

Participants’ Characteristics

The respondents were predominantly women, three-fourths of whom were research associates, with a most frequently reported age group of 30s (Table 1). Most respondents had a master’s degree. Approximately 45% of the respondents reported their English language proficiencies, based on their scores or grades from the following tests: the Test of English for International Communication (TOEIC), the Test of English as a Foreign Language (TOEFL), the Society for Testing English Proficiency (STEP) test, and the United Nations Association’s Test of English (UNATE). The English proficiency levels were operationally defined and divided into two categories: above and below intermediate (Table 1). Above intermediate refers to a score of 600 or above on the TOEIC, 2^nd grade on the STEP test, or a grade of B or above on the UNATE, while scores and grades below these thresholds were referred to as below intermediate [25]. The mean number of years of clinical experience was more than twice the mean number of years of teaching (Table 1).

More than 50% of the participants responded that they had read one to five technical articles in English language within approximately the previous 3 months (Table 1), and more than 20% reported having read no article in English. Although they amounted to less than 4%, a few nurses had read more than 21 articles in English. More than 80% said they often or always feel a language barrier when reading nursing articles in English, and, overall, as many as 98% of the respondents perceived a language barrier (Table 1).

Previous and Current Use of Online MTs and Perceived Usability

More than 70% of respondents indicated that they had indeed used MTs (Table 2). Among the 65 individuals who said they had never used an online MT, 9 (9%) indicated “I do not need it”, 36 (53%) reported “I do not think it is useful”, and 12 (18%) responded “I never knew there were online MT services”. Eight (20%) respondents specified other reasons (e.g., “I cannot learn to read English with MT”, “Reading English by myself is somewhat faster and more efficient for me”, “I would rather use a commercially available MT software package”, “I never read articles written in any foreign languages”, “I do not have any opportunities to be involved in research activities”, and “I am lazy about using online MT”). As for the frequency of using online MT, occasional users were predominant (Table 2). Additionally, 31 (18%) respondents who had previously used online MTs said that they did not use them anymore.Among the reasons why they stopped using MTs, 3 answered that they “do not need it”, and 28 answered that online MT was “not useful”.

Several online MT systems were used by the nursing professionals in this study. The respondents chose Internet search engines that offer online translation tools within Japan, and Yahoo (n = 106), Google (n = 100), and Excite (n = 102) 

were remarkably popular compared with Nifty (n = 14), Bing (n = 17), Livedoor (n = 16), OCN (n = 16), So-net (n = 13), and others (n = 19). Eleven respondents answered that they did not remember the names of the Internet engine(s) they had used.

Nearly half of all respondents evaluated online MT as “usable to some degree”, followed by “not very usable”, “neither usable nor unusable”, and “very usable”. The percentage that answered “not usable at all” was the lowest (Table 2).

Differences in the Number of English Articles Read and Perceived Language Barrier According to Participant Characteristics

Significant differences in the number of English articles read were observed according to job title and the level of English proficiency (Table 3). For each of these, participants at higher levels tended to read more articles in English. Also, the amount of teaching experience was weakly correlated with the number of articles read. There were significant differences in the perceived language barrier according to academic degree and English proficiency level. Participants with lower academic degrees and lower English proficiencies were more likely to feel the language barrier when reading articles written in English.

Differences in the Frequency of Use and the Perceived Usability of Online MTs According to Participant Characteristics

There was a significant difference in the frequency of online MT use according to the perceived language barrier (Table 4). The results of the post-hoc multi-comparison procedure (Tukey HSD test) revealed that the difference was significant between the “always” and “never–occasionally” categories of the frequency of online MT use (mean difference = .552, SE = .187, P = .011 [P < .05], CI = .11 –1.00). There were no statistically significant differences in perceived usability according to respondent characteristics.

Participants’ Online MT Use and its Perceived Usability

A large number of participants indicted that they had previously used online MTs when reading technical papers. It is perhaps only natural that people would enjoy the convenience of automatic translations on websites in this era of the Internet. However, this result implies that many nurses require help when reading technical English language papers in their own fields. Accordingly, it indicates that online MT is much in demand among nursing professionals, highlighting the necessity of language support for them. Also, the comments written on the questionnaire indicated that they tended to use online MT services to grasp the basic meaning of the original text, similar to the results of a previous study [15]. In addition, some participants used multiple online MT systems for the same text, to identify the most useful translation (which is an advantage of the free accessibility of online translators). Some reported that they use online MT systems in combination with a dictionary of technical terminology, indicating that the respondents do not accept translations from online MT systems without question. Also, from the fact that 43% of the respondents used online MT “occasionally” when reading technical literature, it is assumed that they did not regard it as adequate for full-time use, although their perception was that this tool was somewhat helpful.

More than half of our respondents perceived that online MT systems are useful, and hence the general tendency was toward relatively positive impressions. In a previous study, university students rated online MT performance, and more than 30% of the respondents judged it favourably [21]. More of our subjects reported positive perceptions, and in one previous study, online MT received a far better evaluation [23]. The reasons for these differences among study populations are not known but could involve the factors used in the study samples, respondent needs, and the kinds of source texts.

Factors Affecting the Frequency of Online MT Use and its Perceived Usability

The perceived language barrier was associated with frequency of use. Respondents who felt more of a language barrier were likely to use online MT systems more often. This may indicate that MT is relatively advantageous for users with higher language proficiency. Because Japanese users with extremely poor English language skills cannot judge MT performance by looking at a source text written in English, it is difficult for them to utilise MT, and thus they will use it less often. Hence, this result appears to be consistent with previous studies that have indicated the importance of English language ability for the effective use of MT [15, 24]. The perceived language barrier was associated with the participants’ academic degree (Table 3). In Japan, nurses have more opportunities to read English in graduate programs. In such programs, nurses might become accustomed to reading research articles or presenting their own research projects, eventually improving their English skills. In fact, some graduate nursing programs in Japan offer a course in reading English [26].

In contrast, no significant correlations were found between perceived usability and participant background information. Factors that could be associated with perceived usability include motivation for use, type of use, and the expected effectiveness of online MT systems, in addition to the demographic attributes of participants. To obtain a beneficial outcome for users, factors related to usability, such as willingness to use, and specific ways and expected benefits of using online MT services, should be investigated in future studies.

Practical and Educational Implications

The comments written on the questionnaire by the participants included their expectations for and suggestions to developers and providers to improve the systems for the benefit of nursing and healthcare professionals. Specifically, they wanted better translations of technical terms. Their awareness of language difficulties when reading and writing English was also often mentioned in connection with desired improvements in MT performance. Their feedback should be helpful to MT providers for improving translations in nursing fields, as well as beneficial to nursing professionals. Furthermore, our respondents pointed out the important issue of transmitting nursing information from Japan in English. They were concerned with producing internationally interactive nursing personnel resources. Thus, online MT systems are expected to produce better translations not only from English to Japanese, but also from Japanese to English.

English education for nursing professionals must evolve. In general, Japanese users of MTs have some knowledge of the English language, but are uncertain of their proficiency [15]. In our study, the percentage of respondents who felt more of a language barrier when reading English articles was significantly high, indicating the need to improve English education. English education using MT systems may be helpful. However, a negative aspect of MT in foreign language education is that students may accept an output without verifying its correctness, and stop trying to improve their own language skills [24]. Previous studies have indicated some effectiveness in the use of MTs in English language education. It has been claimed that the use of MT lessens students’ aversion to reading and writing English sentences, and helps to train them in logical thinking [27]. It will be important to provide nursing professionals with opportunities to receive training in effective ways of using online MT systems. Also, Japanese nurses need to have training to learn how to specifically read scientific papers in English because this type of document has its own rules for readers. Knowing these rules can save time and reduce inconvenience. These educational opportunities should be provided during post-graduate/continuing professional training (rather than solely as a part of their primary education), in the form of integrated and organisational programs that have yet to be established in the nursing workplaces of Japan. The understanding and collaboration of managerial level nurses will be required to make this continuing education a reality, and we will need to investigate their opinions and attitudes regarding education in the English language and reading scientific papers as part of the occupational development of nursing professionals.

Limitations

Our sample was limited to two types of nursing faculty members, therefore, their academic degrees may have been higher than those typically held by nurses in clinical settings. Nursing professionals from a broader range of settings and levels will be necessary for a deeper understanding of general tendencies. In addition, the sample might have contained a disproportionate number of respondents that were already interested in MT services and/or users of online MT systems.

The English language test scores and grades reported by our respondents did not appear to accurately represent their English reading skills. The participants’ English language proficiencies were judged solely in terms of scores and grades on general language tests. Japanese English language examinations usually measure overall skills, and half of the score is based on listening and/or speaking skills. Therefore, the total score on one of these examinations could be lower if the examinee’s listening or speaking skills are poor, even if his/her reading proficiency is higher. In the present study, the participants’ reading and writing skills should have been directly assessed, and analyses based on these assessments are needed. Also, recall bias could have been introduced when the participants were asked about the number of articles read within approximately the previous 3 months.