The aim of this study was to discover the impact on student support activity that taking part in the OQME pilot had on those placement areas that participated. Objectives were to discover if, and how, student support policies and practices were developed in these placement areas as a result of the pilot activity.

This study was a longitudinal design. After taking part in the data base pilot (May-June 2005), evaluative data were collected in a questionnaire, from which issues of student support will be reported. Further follow-up to assess the longer term impact on student support was undertaken by telephone interview in summer 2006.

A questionnaire was chosen as an appropriate method for collecting data in the initial phase of the study for pragmatic reasons; namely, the ease and simplicity it gave, with a minimum of burden to placement area staff who had been involved with the time-consuming OQME pilot process. The questionnaire contained 23 open ended questions designed by the researchers, allowing for the maximum of flexibility in responses from participants. Telephone interviews were chosen for the follow-up one year later to obtain an in-depth picture of how enduring were any changes to student support practices, and again for pragmatic reasons of ease of access to busy clinical staff who would be unlikely to attend timetabled interviews on a University campus away from their clinical bases: telephone methods have been noted to improve participation rates, and to be particularly useful in accessing participants in dispersed geographic locations [24], and were thus ideal for our purpose in recruiting across three rural counties in South West England. A schedule of questions was developed by the researchers, aimed at eliciting participants’ views concerning whether student support policies and procedures were altered by taking part in the OQME pilot and if these developments were enduring one year later (see Table 2). These interviews lasted approximately 20 minutes, and were transcribed and analyzed.

In spring 2005, nine clinical placement providers and three HEIs in the South West Peninsula of England were invited to take part in an exercise piloting a Microsoft Windows ACCESS electronic database for the OQME standards in their prototype format. Placement providers were chosen as representative of the diversity of local placement areas. The clinical areas were a Cardio Respiratory Unit, an Orthopaedic Ward and a Radiography Department from a Healthcare NHS Trust, an Intermediate Care Unit, a Speech and Language Therapy Department and a Health Centre from a Primary Care Trust, and two hospices and a Macmillan nurse (although she did not complete due to IT problems) from a Hospice and Palliative Care service. Three education institutions participated in the self-assessment of the HEI standards and the joint standards with placement provider units. These were HEI 1 with Healthcare NHS Trust placements in their pre-registration nursing programme, HEI 2, a College with a Speech and Language Therapy Department and Primary Care Trust placements in their pre-registration Allied Health Profession Programme, and HEI 3, a University with Hospice placements in their Post-qualifying Partnership Palliative Care Module. In each setting, a senior member of staff with a responsibility for education and student support participated in the study by completing the ACCESS data base and emailing it to the Health Authority for collation [23], completing the questionnaire and later taking part in the telephone interviews. As two of this study’s authors (GRW and VH) from one HEI participated in data base and questionnaire completion they were not interviewed by telephone for the longitudinal follow-up.

The questionnaire arm of the study audited the pilot and so did not require formal ethical approval. The telephone arm of the study was submitted for approval by the NHS Combined Research Ethics Committee (COREC, a body charged with ethical approval of research taking place in more than one NHS site in England) but was also deemed by them to be audit and their approval to undertake the study was given on that basis. Informed consent was secured verbally after the study aims and objectives and issues of data collection and analysis were explained. Participants were assured that their confidentiality and anonymity would be respected, and that they had the right to withdraw at any time without prejudice. Telephone interviews were undertaken by a research fellow and a research assistant at the University’s Centre of Excellence in Professional Placement Learning (CEPPL) ensuring that the principal investigator (GRW) was sufficiently distanced from the research and participants to avoid potential biases.

Data from the telephone interviews were collected as responses to the open-ended questions and overarching themes produced as a simplification of these data. Data from the telephone interviews were analysed using Miles and Huberman’s [25] framework. This took place in three forms: data reduction, data display and drawing and verifying conclusions. Data reduction took place throughout the analysis, and involved summarizing, coding (labeling and categorizing data) and memoing (theorizing and writing-up ideas about codes and their relationships). Data display involved organizing and compressing data at all stages of the analysis; an essential activity given that qualitative data is voluminous. Thirdly, conclusions were drawn and verified: these were not finalized until all the data are collected, although ideas developed throughout the analysis [26].

It is essential that qualitative research is rigorous, trustworthy and credible [26, 27]. In order to ensure that this study met these criteria, as well as the data analysis steps described above, three researchers (GRW, LC and DW) analysed the questionnaire findings and the interview transcripts independently and compared analyses. Any differences were resolved through discussion of key themes and their interpretation. Findings were compared and contrasted to get a picture of the extent and nature of changes and developments that occurred in the participating placement provider organisations as a result of taking part in the pilot, with questionnaire data used to give an initial picture of the impact and telephone interview data used to assess the longitudinal impact.

Findings in the form of themes generated from the questionnaire element relating to student support are listed in Table 3. Taking part in the pilot was noted as giving the opportunity to review or reflect upon, and improve practices in regard to student support and the resources available with which to do this. There were also benefits accruing from interdisciplinary working, in the sense of sharing and collaborating with other professions and organisations.

Anticipating the main long-term benefits of OQME, respondents believed that it would enhance the placement experience for students, provide evidence for the quality of placements, and improve the relationships between HEIs and placements because of the new requirement to share and agree the joint standards of the OQME document. One potential gain noted was that OQME would enable sharing of best practice.

Overall, participants enjoyed being part of the OQME data base pilot, believing that it gave them the opportunity to focus on improving students’ placement experience, as well as an opportunity for collaborative working. Two quotes illustrate participants’ thinking concerning how taking part in the pilot had impacted on student support. Both indicate that a major gain for clinical placement areas involved in OQME was the opportunity to discuss, collect and collate existing information, policies and practice concerning student support:

‘It was a delight to see how much good practice there was and for us to be able to congratulate ourselves on all the positive work we are achieving with student support.’

Another participant noted that the OQME process was: ‘firmly rooted in what we are doing and need to do to improve and enhance placement quality and opportunities’, showing a similar focus on identifying good practice and the benefit this might have for students.

Analysis of the data revealed four themes: evidencing, sharing and communication, changing practice, and operational aspects of OQME. Each of these themes is presented below, illustrated by quotes from the data (summarized in Table 4).

Participants were generally positive about the implementation of OQME in their placement areas. One of the strongest themes to emerge from the data illustrating this positive perception was how OQME enabled staff in placement areas to ‘evidence’ concerning what student support activities had been carried out. OQME therefore provided a structure for assembling evidence of such procedures. For example, Participant 1 reflected on the procedures in place prior to OQME, and how the new system enabled clarification of evidence:

‘It was very piecemeal and… [staff] knew what needed to be done but having that… clear process to work [with]... made them think about what they were doing and how to evidence it how we were [previously] evidencing things was very piecemeal.’ (Participant 1).

Several participants highlighted the capacity of OQME to enable and to emphasize the importance of documenting and evidencing procedures within their placement areas. Further, one participant stated the usefulness of increased documentation in providing evidence of student support where the level of such support can be disputed:

‘…It covered my back really…if students don’t feel supported…and we can show that they actually are…we are doing all that is required of us…’ (Participant 3).

Therefore, although it was agreed by participants that the OQME pilot did not change the policies that were already in place in the pilot placement areas, it did provide a framework that allowed existing processes to be measured. Furthermore, by being able to observe these processes in a structured way, participants were able to feel confident about the efficacy of their current practices concerning student support:

‘…Probably the most valuable thing from my point of view looking at it now, was the issue of having some sort of measurable [sic] tool that looked at what we’ve already got … in place at the moment…at that stage there was no real direction, and actually it was reassuring to find that a lot of the things that… they were looking for were actually already in place and weren’t actually too difficult to find. It was probably reassuring that there was already good practice going on, really…’ (Participant 7).

In some instances, however, although OQME did not have a direct impact on policy, there was some impact with regard to student support practice, which was stimulated by the evidencing processes discussed above. Therefore, by engaging with a tool that enabled comprehensive evidencing of procedures and processes in place, any gaps in this evidence were highlighted as were subsequent changes in practice required.

The second theme regarding positive aspects of involvement in the pilot was the development of sharing and communication of practices both between placement areas and HEIs and also within the placement areas. By creating closer links between placement providers and universities, lines of communication were opened which further enhanced opportunities for sharing existing best practice captured by OQME:

‘…They were certainly starting to work more closely with some of the universities…as some of them had been a little bit distant. But, they were getting more involvement and more feedback through universities, as a result trying, actually trying to do the work for the OQME they realized that they needed to communicate more to get this kind of thing happening…’ (Participant 1).

Another participant noted that: ‘It was very useful…to do the joint standards and the discussion …for the future.’ (Participant 4).

Therefore, the implementation of OQME both encouraged greater communication to take place and helped participants to realize the importance of such communication and sharing. Similarly, the process of ‘evidencing’ work (discussed above) in itself necessitated cooperative working. For example, one participant found great value in a benchmarking exercise undertaken by herself and a colleague from another profession:

‘…We actually found it a really helpful exercise and because we did it jointly… [we were] able to do a lot of unifying of paperwork together…and [it] actually helped us to share the idea that we had [about our] best practice. So that was really helpful… it was very helpful as a tool, and to share best practice…’ (Participant 6).

Therefore, taking part in the OQME pilot and undertaking procedures involved in its operation provided, for the majority of participants, a rationale for both internal and external cooperation and dissemination.

The two themes presented above show how, in general, engaging in the OQME pilot underlined the need for the comprehensive documentation of evidence, for improved communication in terms of student support practice, and for dissemination within and outside placement areas. In some cases this need emphasized that a change in practice was necessary in order to fulfill the requirements of OQME. As discussed above, although participants did not believe that any policy changes took place as a result of the use of OQME, certain practices were adapted or changed as a result: ‘…it’s about strengthening induction materials and individual units sort of learning from best practice, if you like…picking up on that and trying to improve what they had…so that’s process or procedure but in terms of policy, no…’ (Participant 2).

Further, as in the example below, in some cases new resources were created as a direct result of participation in the pilot. For example, the production of an action plan (below), initiated due to the use of OQME, instigated a change in practice with the use of a checklist benefiting both supervising staff and their placement students.

‘I suppose there’s just the longer term impact really because I think the useful thing about being involved was, um, actually being able to get an action plan for the department that’s been something we’ve just been able to use in our annual service plan which has been really useful… one of the things we did from the action plan was develop a general…checklist just for the first day for the supervising clinicians to use…’ (Participant 4).

Another participant acknowledged that since the commencement of the OQME pilot the tutorial time of ‘return to practice’ students had been increased, therefore showing a direct change in practice in response to OQME:

‘…Probably the main thing I think really - especially from the return to practice side of it… was we’ve increased the tutorial time…probably over and above what the University was originally saying’. (Participant 7).

Participants spoke about a number of technical issues related to the implementation and operation of OQME in their placement areas. These issues ranged from organisational ones concerning implementing the OQME pilot process and the resources involved, to the IT skills and workload of staff utilizing the tool. Most of these technical issues were met and dealt with in the initial stages of the application of the tool, for example:

‘…The biggest problems were technical... [these] are now being sorted out. Since then the whole computer system has been upgraded. That was half the problem: the actual resource itself was running on a much higher spec[ification] than… what we had at the time. We produced a lot of work towards the end, because we were waiting for the upgrade before we could actually do it’. (Participant 1).

Another participant stated difficulties with the set up of the system, although once installed found it to be user friendly despite initial anxiety regarding her level of necessary IT skills:

‘I did take it on rather, sort of, in trepidation, because I didn’t think my IT skills were wonderful for doing something like that, but other than the initial hiccup we had with just getting the system set up which was an NHS and organisational problem the system in place could use the data base, it was really very easy to use and user friendly’. (Participant 4).

Although it appeared that, overall, implementation difficulties were relatively easily resolved, participants did comment that using the tool to document the evidence that they found so useful was a repetitive process:

‘…I think the computer system was…quite…complex really…a bit repetitive…yeah I think some of the questions on it were very repetitive.’ (Participant 3).

Further, despite the benefits of the introduction of the tool that were stated by participants, both the implementation and day-to-day operation of OQME necessarily added pressure to normal workloads:

‘I think the IT challenges usually were a headache because we were left with very, very little time to do it, and that was a pressure because obviously that’s additional to the normal workload and…there’ll be lots of other pressures then.’ (Participant 4).

In summary, it is clear that a high degree of consensus existed between the findings from the questionnaire administered immediately after the OQME pilot in summer 2005 and the more in-depth telephone interviews carried out a year later in 2006. In the questionnaire, taking part in the OQME pilot was noted to give placement areas the opportunity to review, or reflect upon, and to improve student support practices and the resources available, and this was still in evidence a year later where the pilot was described as giving the opportunity to provide evidence of aspects of student support practice, sharing and communicating and changing or developing aspects of that practice. There were also benefits accruing from interdisciplinary working, in the sense of sharing and collaborating with other professions and organisations was mentioned in the questionnaire findings, and this was also a key theme in the telephone interviews. In summer 2005, respondents anticipated long-term OQME benefits, believing that it would enhance the placement experience for students; provide evidence for the quality of placements; and improve the relationships between HEIs and placements because of the new requirement to share and agree the joint standards of the OQME document. One significant potential gain noted was that OQME would enable sharing of best practice. All of these areas emerged from the telephone interviews in summer 2006.