Use of Online Machine Translation for Nursing Literature: A Questionnaire-Based Survey
RESEARCH ARTICLE

Use of Online Machine Translation for Nursing Literature: A Questionnaire-Based Survey

The Open Nursing Journal 01 Feb 2013 RESEARCH ARTICLE DOI: 10.2174/1874434601307010022

Abstract

Background: The language barrier is a significant obstacle for nurses who are not native English speakers to obtain information from international journals. Freely accessible online machine translation (MT) offers a possible solution to this problem.

Aim: To explore how Japanese nursing professionals use online MT and perceive its usability in reading English articles and to discuss what should be considered for better utilisation of online MT lessening the language barrier.

Method: In total, 250 randomly selected assistants and research associates at nursing colleges across Japan answered a questionnaire examining the current use of online MT and perceived usability among Japanese nurses, along with the number of articles read in English and the perceived language barrier. The items were rated on Likert scales, and t-test, ANOVA, chi-square test, and Spearman’s correlation were used for analyses.

Results: Of the participants, 73.8% had used online MT. More than half of them felt it was usable. The language barrier was strongly felt, and academic degrees and English proficiency level were associated factors. The perceived language barrier was related to the frequency of online MT use. No associated factor was found for the perceived usability of online MT.

Conclusion: Language proficiency is an important factor for optimum utilisation of MT. A need for education in the English language, reading scientific papers, and online MT training was indicated. Cooperation with developers and providers of MT for the improvement of their systems is required.

Keywords: Japanese nurses, language barrier, nursing literature, online machine translation, usability, questionnaire.

INTRODUCTION

Utilising research is critical for nurses in any field [1-3]. Because evidence-based practise (EBP) is increasingly being encouraged in many countries, nurses who are not native English speakers need to read internationally published research papers written in the English language [4]. The problems created by language differences can become a serious barrier to success for nurses in non-English-speaking countries, such as Japan [4-6]. English reading skills are perceived as highly important [3], and the need to read articles written in the English language has been recognised among Japanese nurses [7-9]. Nevertheless, no association between the reading of English technical literature and the language barrier has been examined.

It is more important for Japanese nurses to obtain nursing information quickly, to provide quality care, than it is to spend time mastering the English language [10]. The use of a technology such as machine translation (MT) may be an important step toward achieving this goal [10]. MT is a system in which text in one language is automaticallytranslated into another language [11], and has long been used as an aid to worldwide multilingual communication. The MT concept originated in the United States, as early as the 1930s. During the 1950s, the development of computers led to the practical realisation of this concept [12, 13]. The emergence of the Internet allowed MT systems to be made available via various websites, and they have become popular and increasingly convenient. Online MT systems are freely accessible, and although many of their translations are beset with shortcomings, and leave much room for improvement, they are generally valuable and practical [14, 15]. Among the Japanese, it has been indicated that people use MT systems to avoid the task of looking up unknown words one by one in the dictionary, and/or because they only want a rough idea of what the source texts contain, rather than detailed translations [15]. Thus the current function of MT can be described as ancillary in nature, and is limited to this type of usage.

Many studies have been conducted on evaluating MT performance with various language pairs as well as on MT users. Among online MT systems, Google Translate (one of the major systems) has generally been reported to outperform the others [14, 16, 17]. Evaluation methods and processes, as well as the usability of translations, have been investigated for healthcare documents such as medical records, health information, and patient educational materials, indicating conditional usability [18, 19]. Previous studies on MT users have indicated various points of view regarding the use of online MT and wide-ranging perceptions of usability by general users [20-24]. Some user feedback on MT translations via the Internet has demonstrated a wide range of opinions, varying from gratitude to harsh criticism of the output [20]. Surveys on MT use by general Internet users (which includes a large portion of online MT users) have been conducted in Japan and different ways of using this tool have been suggested [22]. Also, Japanese English learners have reported the perceived usefulness of online MT in a large number of subjects [23]; moreover, students whose level of English proficiency was intermediate or higher were able to assess the quality of the MT systems on websites more critically [24]. However, few studies have been conducted to examine the usability of these systems by medical and healthcare professionals. Furthermore, no specifically nursing-oriented investigation of this type has been conducted, in spite of the obvious need for English language reading capabilities to enhance the practise in non-English-speaking countries such as Japan. It is currently unknown how Japanese nursing professionals, whose native tongue is one of the minor languages in academia, perceive online MT tools and their usability. Therefore, a knowledge base must be developed to investigate the practicality of online MT in relation to the perceived language barrier presented by internationally published nursing literature.

AIM

Our aim was to explore how Japanese nursing professionals use online MT and perceive its usability in reading English articles and to discuss what should be considered for better utilisation of online MT lessening the language barrier. More specifically, we attempted to investigate the following research questions: How do Japanese nurses use online MT and perceive its usability? What factors are associated with their online MT use and the perceived usability?

METHODS

Study Participants and Data Collection

A cross-sectional exploratory survey was conducted with 250 Japanese nursing participants, using a self-administered questionnaire. Our objective was to select a study population of native Japanese speakers with the necessary nursing experience to understand the clinical contexts of technical papers. Assistants and research associates at nursing universities were assumed to be an appropriate choice because they had already had professional experience both as clinical nurses and as teachers, usually with more experience in clinical settings than in educational or research environments. Nevertheless, it was anticipated that these people would have some idea of how the research might be utilised.

In Japan, there were 200 4-year nursing programs as of February, 2012, and the number of assistants and research associates across Japan was assumed to be over 2,000. Two to four individuals were randomly selected from the website of each university, for a total of 500 candidates. The actual participants were recruited by mail. Each candidate was sent the questionnaire and a cover letter asking for his/her participation, and 498 of them received it. The survey was completed by each participant and anonymously returned by mail. In total, 250 individuals responded to the survey, a response rate of 50.2%. The data were collected from March to April of 2012.

Measures

The questionnaire included background questions, previous and current online MT usage, and perceived usability of online MT (Tables 1 and 2). The number of technical articles read within approximately the previous 3 months was asked and scored on a five-point Likert scale (1: 0, 2: 1–5, 3: 6–10, 4: 11–20, and 5: 21 or more). The frequency of language barrier perception was also asked about and scored on a five-point Likert scale (1: never, 2: rarely, 3: occasionally, 4: often, and 5: always). For the question regarding previous online MT use, it could be answered Yes or No. The frequency of online MT use was rated on a three-point Likert scale (1: rarely, 2: occasionally, and 3: always/almost always). The perceived usability of online MT was measured by answers on a five-point Likert scale (1: not usable at all, 2: not very usable, 3: neither usable nor unusable, 4: usable to some degree, and 5: very usable). The question of participants’ experience using Internet search engines that offer online MT had the following choices: 1) Yahoo, 2) Google, 3) Bing, 4) Excite, 5) Nifty, 6) Livedoor, 7) OCN, 8) Sonet, 9) other, and 10) do not remember the names of the search engine(s).

Respondents who answered that they had never used online MT were asked to select a reason from among the following: “I do not need it”, “I do not think it is usable”, “I didn’t know there were online MT services”, and “any other reason(s)” (to be specified). In addition, those who had used online MT, but stopped using it could answer “I do not use it anymore”, optionally selecting a reason from “I do not need it”, “ I do not think it is useful”, or “any other reason(s)” (to be specified).

Data Analysis

The data were analysed using the Statistical Package for the Social Sciences (SPSS), version 20. Descriptive statistics, the t-test, analysis of variance (ANOVA), the chi-square test, and Spearman’s correlation were used. Raw data were analysed after calculating the means, standard deviations (SDs), distributions, and percentages for questionnaire items. The chi-square test was used to identify the differences in the number of English articles read and perceived language barrier according to the respondents’ demographic data, where the number of English articles read was measured based on the binary responses “0–5” and “6 or more”, and perceived language barrier was measured based on the binary responses “less” and “more”. Also, the chi-square test was used to determine the differences in perceived usability, measured based on the binary responses “not usable” and “usable”, according to the participants’ characteristics. The ttest and ANOVA were used to detect the differences in the frequency of online MT system use according to participant characteristics. A post hoc examination was performed, using a multiple comparison procedure (Tukey’s HSD test). The relationships between professional experience and the outcome variables were examined using a Spearman’s correlation coefficient.

Ethical Considerations

The Ethical Review Board of the University of Tokyo approved this study. Participants were informed that all information would be treated confidentially, and that they could contact the researcher if they had any questions.

RESULTS

Participants’ Characteristics

The respondents were predominantly women, three-fourths of whom were research associates, with a most frequently reported age group of 30s (Table 1). Most respondents had a master’s degree. Approximately 45% of the respondents reported their English language proficiencies, based on their scores or grades from the following tests: the Test of English for International Communication (TOEIC), the Test of English as a Foreign Language (TOEFL), the Society for Testing English Proficiency (STEP) test, and the United Nations Association’s Test of English (UNATE). The English proficiency levels were operationally defined and divided into two categories: above and below intermediate (Table 1). Above intermediate refers to a score of 600 or above on the TOEIC, 2nd grade on the STEP test, or a grade of B or above on the UNATE, while scores and grades below these thresholds were referred to as below intermediate [25]. The mean number of years of clinical experience was more than twice the mean number of years of teaching (Table 1).

Table 1.

Participant Background Information (nr = No Response)

Variables n %
Sex Men 22 8.8
Women 226 90.4
nr 2 .8
Title Assistant 59 23.6
Research Associate 189 75.6
nr 2 .8
Age group 20s 19 7.6
30s 135 54
40s 81 32.4
50s 13 5.2
60s 2 .4
Academic degree Diploma 4 1.6
Associate 6 2.4
Bachelor 47 18.8
Master 171 68.4
Doctoral 20 8.0
nr 2 .8
English language
proficiency (n = 113)
≥ intermediate 54 47.79
≤ intermediate 59 52.21
N Mean SD
Professional
experiences (years)
Clinical 246 8.84 5.26
Teaching 248 4.32 3.67
Answers Value %
Number of articles read
in English within
approximately 3 months
(n = 247)
0 57 23.1
1–5 143 57.9
6–10 28 11.3
11–20 10 4.0
21 ≤ 9 3.6
Frequency of
perceived language
barrier (n = 245)
Never 1 .4
Rarely 2 .8
Occasionally 42 17.1
Often 74 30.2
Always 126 51.4

More than 50% of the participants responded that they had read one to five technical articles in English language within approximately the previous 3 months (Table 1), and more than 20% reported having read no article in English. Although they amounted to less than 4%, a few nurses had read more than 21 articles in English. More than 80% said they often or always feel a language barrier when reading nursing articles in English, and, overall, as many as 98% of the respondents perceived a language barrier (Table 1).

Previous and Current Use of Online MTs and Perceived Usability

More than 70% of respondents indicated that they had indeed used MTs (Table 2). Among the 65 individuals who said they had never used an online MT, 9 (9%) indicated “I do not need it”, 36 (53%) reported “I do not think it is useful”, and 12 (18%) responded “I never knew there were online MT services”. Eight (20%) respondents specified other reasons (e.g., “I cannot learn to read English with MT”, “Reading English by myself is somewhat faster and more efficient for me”, “I would rather use a commercially available MT software package”, “I never read articles written in any foreign languages”, “I do not have any opportunities to be involved in research activities”, and “I am lazy about using online MT”). As for the frequency of using online MT, occasional users were predominant (Table 2). Additionally, 31 (18%) respondents who had previously used online MTs said that they did not use them anymore.Among the reasons why they stopped using MTs, 3 answered that they “do not need it”, and 28 answered that online MT was “not useful”.

Table 2.

Use of Online MT and Perceived Usability

Variables Answers n %
Previous online
MT use
(n = 248)
Yes 183 73.8
No 65 26.2
Frequency of
online MT use
(n =143)
Rarely 49 34.3
Occasionally 62 43.4
Always/almost always 32 22.4
Perceived usability
(n =182)
Not usable at all 3 1.6
Not usable very much 51 28.0
Neither usable nor unusable 25 13.7
Usable to some degree 91 50.0
Very usable 12 6.6

Several online MT systems were used by the nursing professionals in this study. The respondents chose Internet search engines that offer online translation tools within Japan, and Yahoo (n = 106), Google (n = 100), and Excite (n = 102) 

were remarkably popular compared with Nifty (n = 14), Bing (n = 17), Livedoor (n = 16), OCN (n = 16), So-net (n = 13), and others (= 19). Eleven respondents answered that they did not remember the names of the Internet engine(s) they had used.

Nearly half of all respondents evaluated online MT as “usable to some degree”, followed by “not very usable”, “neither usable nor unusable”, and “very usable”. The percentage that answered “not usable at all” was the lowest (Table 2).

Differences in the Number of English Articles Read and Perceived Language Barrier According to Participant Characteristics

Significant differences in the number of English articles read were observed according to job title and the level of English proficiency (Table 3). For each of these, participants at higher levels tended to read more articles in English. Also, the amount of teaching experience was weakly correlated with the number of articles read. There were significant differences in the perceived language barrier according to academic degree and English proficiency level. Participants with lower academic degrees and lower English proficiencies were more likely to feel the language barrier when reading articles written in English.

Table 3.

Frequency of Online MT Use and Perceived Usability According to Participant Characteristics

Variables Numbers of Articles Read in English Perceived Language Barrier
n 0-5 (%) ≥ 6 (%) χ2 P n Less (%) More (%) χ2 P
Title Assistant 59 55 (93.2) 4 (6.8) 7.713 .005 55 6 (10.9) 49 (89.1) 2.484 .115
Research Associate 186 143 (76.9) 43 (23.1) 188 38 (20.2) 150 (79.8)
Age group ≤ 30s 152 127 (83.6) 25 (16.4) 1.709 .191 149 24 (16.1) 125 (83.9) 1.295 .255
≥ 40s 95 73 (76.8) 22 (23.1) 96 21 (21.9) 75 (78.1)
Academic degree ≤ bachelors 56 50 (89.3) 6 (10.7) 3.359 .067 58 4 (7.5) 49 (92.5) 5.097 .024
≥ masters 189 148 (78.3) 41 (21.7) 190 40 (21.1) 150 (78.9)
English level ≤ intermediate 59 53 (89.9) 6 (10.1) 6.026 .014 56 8 (14.3) 48 (85.7) 4.629 .031
≥ intermediate 53 38 (71.7) 15 (28.3) 53 17 (31.5) 37 (68.5)
n r* P** n r* P**
Professional experience Clinical 243 -.045 .485 241 .14 .029
Teaching 245 .186 .004 243 -.112 .018

Differences in the Frequency of Use and the Perceived Usability of Online MTs According to Participant Characteristics

There was a significant difference in the frequency of online MT use according to the perceived language barrier (Table 4). The results of the post-hoc multi-comparison procedure (Tukey HSD test) revealed that the difference was significant between the “always” and “never–occasionally” categories of the frequency of online MT use (mean difference = .552, SE = .187, P = .011 [P < .05], CI = .11 –1.00). There were no statistically significant differences in perceived usability according to respondent characteristics.

Table 4.

Frequency of Online MT Use and Perceived Usability According to Participant Characteristics

Variables Numbers of Articles Read in English Perceived Language Barrier
n 0-5 (%) ≥ 6 (%) χ2 P n Less (%) More (%) χ2 P
Title Assistant 59 55 (93.2) 4 (6.8) 7.713 .005 55 6 (10.9) 49 (89.1) 2.484 .115
Research Associate 186 143 (76.9) 43 (23.1) 188 38 (20.2) 150 (79.8)
Age group ≤ 30s 152 127 (83.6) 25 (16.4) 1.709 .191 149 24 (16.1) 125 (83.9) 1.295 .255
≥ 40s 95 73 (76.8) 22 (23.1) 96 21 (21.9) 75 (78.1)
Academic degree ≤ bachelors 56 50 (89.3) 6 (10.7) 3.359 .067 58 4 (7.5) 49 (92.5) 5.097 .024
≥ masters 189 148 (78.3) 41 (21.7) 190 40 (21.1) 150 (78.9)
English level ≤ intermediate 59 53 (89.9) 6 (10.1) 6.026 .014 56 8 (14.3) 48 (85.7) 4.629 .031
≥ intermediate 53 38 (71.7) 15 (28.3) 53 17 (31.5) 37 (68.5)
n r* P** n r* P**
Professional experience Clinical 243 -.045 .485 241 .14 .029
Teaching 245 .186 .004 243 -.112 .018

DISSCUSSION

Participants’ Online MT Use and its Perceived Usability

A large number of participants indicted that they had previously used online MTs when reading technical papers. It is perhaps only natural that people would enjoy the convenience of automatic translations on websites in this era of the Internet. However, this result implies that many nurses require help when reading technical English language papers in their own fields. Accordingly, it indicates that online MT is much in demand among nursing professionals, highlighting the necessity of language support for them. Also, the comments written on the questionnaire indicated that they tended to use online MT services to grasp the basic meaning of the original text, similar to the results of a previous study [15]. In addition, some participants used multiple online MT systems for the same text, to identify the most useful translation (which is an advantage of the free accessibility of online translators). Some reported that they use online MT systems in combination with a dictionary of technical terminology, indicating that the respondents do not accept translations from online MT systems without question. Also, from the fact that 43% of the respondents used online MT “occasionally” when reading technical literature, it is assumed that they did not regard it as adequate for full-time use, although their perception was that this tool was somewhat helpful.

More than half of our respondents perceived that online MT systems are useful, and hence the general tendency was toward relatively positive impressions. In a previous study, university students rated online MT performance, and more than 30% of the respondents judged it favourably [21]. More of our subjects reported positive perceptions, and in one previous study, online MT received a far better evaluation [23]. The reasons for these differences among study populations are not known but could involve the factors used in the study samples, respondent needs, and the kinds of source texts.

Factors Affecting the Frequency of Online MT Use and its Perceived Usability

The perceived language barrier was associated with frequency of use. Respondents who felt more of a language barrier were likely to use online MT systems more often. This may indicate that MT is relatively advantageous for users with higher language proficiency. Because Japanese users with extremely poor English language skills cannot judge MT performance by looking at a source text written in English, it is difficult for them to utilise MT, and thus they will use it less often. Hence, this result appears to be consistent with previous studies that have indicated the importance of English language ability for the effective use of MT [15, 24]. The perceived language barrier was associated with the participants’ academic degree (Table 3). In Japan, nurses have more opportunities to read English in graduate programs. In such programs, nurses might become accustomed to reading research articles or presenting their own research projects, eventually improving their English skills. In fact, some graduate nursing programs in Japan offer a course in reading English [26].

In contrast, no significant correlations were found between perceived usability and participant background information. Factors that could be associated with perceived usability include motivation for use, type of use, and the expected effectiveness of online MT systems, in addition to the demographic attributes of participants. To obtain a beneficial outcome for users, factors related to usability, such as willingness to use, and specific ways and expected benefits of using online MT services, should be investigated in future studies.

Practical and Educational Implications

The comments written on the questionnaire by the participants included their expectations for and suggestions to developers and providers to improve the systems for the benefit of nursing and healthcare professionals. Specifically, they wanted better translations of technical terms. Their awareness of language difficulties when reading and writing English was also often mentioned in connection with desired improvements in MT performance. Their feedback should be helpful to MT providers for improving translations in nursing fields, as well as beneficial to nursing professionals. Furthermore, our respondents pointed out the important issue of transmitting nursing information from Japan in English. They were concerned with producing internationally interactive nursing personnel resources. Thus, online MT systems are expected to produce better translations not only from English to Japanese, but also from Japanese to English.

English education for nursing professionals must evolve. In general, Japanese users of MTs have some knowledge of the English language, but are uncertain of their proficiency [15]. In our study, the percentage of respondents who felt more of a language barrier when reading English articles was significantly high, indicating the need to improve English education. English education using MT systems may be helpful. However, a negative aspect of MT in foreign language education is that students may accept an output without verifying its correctness, and stop trying to improve their own language skills [24]. Previous studies have indicated some effectiveness in the use of MTs in English language education. It has been claimed that the use of MT lessens students’ aversion to reading and writing English sentences, and helps to train them in logical thinking [27]. It will be important to provide nursing professionals with opportunities to receive training in effective ways of using online MT systems. Also, Japanese nurses need to have training to learn how to specifically read scientific papers in English because this type of document has its own rules for readers. Knowing these rules can save time and reduce inconvenience. These educational opportunities should be provided during post-graduate/continuing professional training (rather than solely as a part of their primary education), in the form of integrated and organisational programs that have yet to be established in the nursing workplaces of Japan. The understanding and collaboration of managerial level nurses will be required to make this continuing education a reality, and we will need to investigate their opinions and attitudes regarding education in the English language and reading scientific papers as part of the occupational development of nursing professionals.

Limitations

Our sample was limited to two types of nursing faculty members, therefore, their academic degrees may have been higher than those typically held by nurses in clinical settings. Nursing professionals from a broader range of settings and levels will be necessary for a deeper understanding of general tendencies. In addition, the sample might have contained a disproportionate number of respondents that were already interested in MT services and/or users of online MT systems.

The English language test scores and grades reported by our respondents did not appear to accurately represent their English reading skills. The participants’ English language proficiencies were judged solely in terms of scores and grades on general language tests. Japanese English language examinations usually measure overall skills, and half of the score is based on listening and/or speaking skills. Therefore, the total score on one of these examinations could be lower if the examinee’s listening or speaking skills are poor, even if his/her reading proficiency is higher. In the present study, the participants’ reading and writing skills should have been directly assessed, and analyses based on these assessments are needed. Also, recall bias could have been introduced when the participants were asked about the number of articles read within approximately the previous 3 months.

CONCLUSION

Online MT had been used by a sizeable percentage of the study participants as an aid to reading international nursing journals. The level of the perceived language barrier was related to the likelihood of using online MTs. For optimal use of online MT, it is necessary to improve users’ language skills. We should consider the English language education of nurses, as well as the possibility of using online MTs in English education, and providing training in reading scientific papers and the use of MT tools. It will also be a role of our work to promote user cooperation with the developers and providers of MT systems to improve MT dictionaries with regard to nursing-specific terms. Future studies should include a wider range of nurses, assessing English proficiency precisely, an investigation of the usability of online MT considering possible influencing factors (such as willingness to use the systems and specific ways and expected benefits of using this technology), investigations of the need for research education on using international journals, and promoting postgraduate English education according to the perceptions of nurses at the managerial level.

Issues related to reading articles published in English should be seriously considered, to be resolved not only in Japan, but also in many other non-English-speaking countries. The present study will provide incentive for further exploration of possible linguistic support for nurses via advanced Internet-based technology, leading to more effective use of MT tools, and eventually more opportunities for nurses to access internationally published nursing information and obtain updated knowledge for better care delivery.

CONFLICT OF INTEREST

The authors confirm that this article content has no conflicts of interest.

ACKNOWLEDGMENTS

This study was funded by the Graduate School of Medicine, the University of Tokyo.

REFERENCES

1
Strickland RJ, O'Leary-Kelley C. Clinical nurse educators' perceptions of research utilization: barriers and facilitators to change. J Nurses Staff Dev 2009; 25: 164-7.
2
Forsman H, Rudman A, Gustavsson P, Ehrenberg A, Wallin L. Nurses' research utilization two years after graduation--a national survey of associated individual, organizational, and educational factors. Implement Sci 2012; 7: 46.
3
Anazawa R. Review of the book Lotta to Hana no tanoshii kihon kango eigo [Lotta and Hana's Joyful Basic Nursing English]. Shukan Igakkai Shimbun 2012; 2975: 7.
4
Anazawa R, Ishikawa H, Park MJ, Kiuchi T. Online machine translation use with nursing literature: evaluation method and usability. Comp Inform Nurs 2012; 5: 635.
5
Suwanraj M. Current practice, perceived barriers, and perceived facilitators of Thai nurses on using evidence-based practice on pain assessment and pain management in older adults. PhD thesis Iowa City(Iowa) University of Iowa 2010.
6
Anazawa R. Kangoshi no eigo bunken riyou to eigo gakusyu English literature utilization and English learning for Japanese nurses. Jpn J Child Nurs 2011; 34: 508-9.
7
Anazawa R, Ishikawa H, Kiuchi T. The perceived need for Japanese nursing faculty members to learn English issues related to career development. J Contin Educ Nurs 2012; 43: 188-91.
8
Motooka N, Kawasaki H. An investigation on the language attitudes change of nursing students from the viewpoint of ESP (English for Specific Purposes). Bull Hiroshima Prefect Coll Health Welfare 1999; 4: 25-33.
9
Onjoji Y. Needs analysis of English in nursing. J Med Engl Educ 2005; 4: 22-3.
10
Kojima M, Akiyama T. Efficiency of computer information retrieval and translation for nurses. Nurs Inform 1994; 1: 41-7.
11
Kirchhoff K, Turner AM, Axelrod A, Saavedra F. Application of statistical machine translation to public health information a feasibility study. J Am Med Inform Assoc 2011; 18: 473-8.
12
Taguchi S. Machine translation, present status and the difficulties. Office Automation 1985; 6: 4-8.
13
Miyahira T, Watanabe H, Tazoe E, Kamiyama T, Takeda K. Internet Kikai Honyaku no Sekai The World of Internet Machine Translation. Tokyo Mainichi Communications 2004; 00: 0-0.
14
Kit C, Tak MW. Comparative evaluation of online machine translation systems with legal texts. Law Library J 2008; 100: 299-321.
15
Araki K. Shizen gengo koto hajime. Introduction to Natural Language Processing Tokyo: Morikita Publishing 2004; 00: 0-0.
16
Naskar SK, Toral A, Gaspari F, Way A. A framework for diagnostic evaluation of MT based on linguistic checkpoints. Available from http://wwwmt-archiveinfo/MTS-2011-Naskarpdf [Cited 21 January 2012] 2011; 00: 529-36.
17
Toral A, Gaspari F, Naskar SK, et al. Comparative evaluation of research vs. online MT systems In Proceedings of the 15th Conference of the European Association for machine translation Leuven Belgium 2011; pp: 13-20.
18
Zeng-Treitler Q, Kim H, Rosemblat G, Keselman A. Can multilingual machine translation help make medical record content more comprehensible to patients?. Stud Health Technol Inform 2010; 160(Pt 1): 73-.
19
Khanna RR, Karliner LS, Eck M, Vittinghoff E, Koenig CJ, Fang MC. Performance of an online translation tool when applied to patient educational material. J Hosp Med 2011; 6: 519-25.
20
Yang J, Lange ED, Farwell D, Gerber L, Hovy E. SYSTRAN on AltaVista a user study on real-time machine translation on the Internet. In Proceedings of the 3rd conference of the Association for Machine Translation in the Americas Machine translation and the information soup Langhor e PA; USA. Berlin Springer 1998, pp: 275-85.
21
Gaspari F. The role of online MT in webpage translation [PhD thesis]. Manchester, UK University of Manchester 2007.
22
Waheeed A. AAMT Working Group 2. Ei nichi/nichi ei kikai honyaku ni kansuru anketo kekka houkoku [Report of the survey on the English to Japanese/Japanese to English machine translation] AAMT J 2011; 50: 52-60.
23
Matsui J, Magnusson D. Six Pre-edit techniques for enhancing Japanese to English machine translations. Interpret Transl Stud 2011; 11: 173-87.
24
Osuka N, Mano C. The negative effects of using translation sites on college students' language learning. J Facult Int Stud Bunkyo Uni 2003; 13: 133-44.
25
Waheeed A. Kobe City University of Foreign Studies. TOEIC/TOEFL/IEL TS/STEP Score Converting System [internet] Available from http://wwwkobe-cufsacjp/about/contribution/license/files/toeic_to efl_scorepdf [Cited 2012 July 28]
26
Kawagoe K. ESP education for graduate students of nursing science. Jpn Assoc Coll English Teachers (JACET) 2003; 42: 154-5.
27
Ando S. Kikai honyaku (MT) and search engine wo eigo gakusyu ni katsuyo suru [Utilizing machine translation (MT) and search engines in English language learning]. Inform Process Soc Jpn 1997; 97: 9-15.