UK multisite evaluation of the impact of clinical educators in EDs from a learner’s perspective

Background In England, demand for emergency care is increasing while there is also a staffing shortage. The Royal College of Emergency Medicine (RCEM) suggested that appointment of senior doctors as clinical educators (CEs) would enable support and development of learners in EDs and improve retention and well-being. This study aimed to evaluate the impact of CEs in ED on learners. Methods CEs were placed in 54 NHS Acute Trust EDs for a pilot beginning July 2018 and ending October 2020. Learners from multiple disciplines working at 54 NHS Acute Trust EDs where CEs were deployed were invited to complete an online survey designed to identify the impact of CEs in July of 2019, as part of an interim service evaluation. Results Respondents numbered 493 from 49 of 54 study sites, including 286 (58%) medical (non-consultant) and 72 (14.6%) all other nursing, allied health professionals. 9 out of 10 learners reported having experienced a change to their learning as a result of the deployment of CEs in their department. 49.9% (246/493) reported that CEs had a positive impact on their well-being. 95% (340/358) reported an improved accessibility to undertaking clinical based assessments. 78% (281/358) perceived that access to CEs increased likelihood of passing assessments. Of those responding, 80.9% (399/493) reported they would remain/return to the same ED with a CE, and 92.5% (456/493) responded that they would prefer to go to a Trust with a CE. Conclusions According to survey respondents, deployment of CEs across NHS Trusts has resulted in improvement and increased accessibility of learning and assessment opportunities for learners within ED. The impact of CEs on well-being is uncertain with half reporting improvement and the remaining half unsure. Further evaluation within the project will continue to explore the service benefit and workforce impact of the CEED intervention.


INTRODUCTION
EDs have seen a rise in demand for services by patients and members of the public and crowding globally. 1 Exacerbating this challenge in the UK are issues of recruitment into emergency medicine (EM) training posts and workforce retention thereafter. 2 In 2012, The Royal College of Emergency Medicine UK (RCEM) highlighted a number of concerns to the General Medical Council (GMC) related to EM training: continuing service pressures, which reduced the amount of time trainers can dedicate to delivering training; rota gaps, which increase the pressure on doctors in training to work more out-of-hours shifts; a lack of senior supervision for junior doctors in training; and a lack of resources, leading to ineffective simulation training.
The GMC in turn published a review of training within a test group of seven EDs, which identified concerns about the amount and quality of supervision received by EM Trainees. 3 Previous assessments of training by EM trainees have reported disillusionment with the specialty of EM with high rates of burnout reported, concerns over intensity of the workload and the quality of training (GMC National Training Survey and Emergency Medicine Training Association surveys). 4 All of this suggests a need to develop within ED multiprofessional teams a culture that supports shop-floor, integrated learning. 2 Shop-floor training is an important part of EM education in the UK and beyond, and its relevance in the USA has been highlighted. 5

Key messages
What is already known on this subject ► There is a rise in demand for services in the ED. ► The intense working environment has been recognised as a leading cause of medical staff dissatisfaction, attrition and premature career burnout. Attrition rate in EM in the UK is high. ► There is a lack of evidence on the deployment, impact and effect of having a CE, but some suggestion that a learning environment would help recruitment and retention.
What this study adds ► This study reports on a 54 site (England, UK) online questionnaire evaluating the impact of deploying CEs-consultants with protected time to provide training and support to the emergency medicine medical trainees and learners from other healthcare professions in ED. ► The findings suggest that CE deployment has had a reported improvement in accessibility to learning opportunities for learners. The pilot's impact on well-being however is uncertain.
In October 2017, RCEM, Health Education England (HEE), NHS England and NHS Improvement published 'Securing the Future Workforce for Emergency Departments in England'. 2 This workforce strategy recommended a range of interventions to ensure a sustainable workforce, capable of meeting the needs of an increasing patient population, presenting with ever-more varied and complex health and care needs. One such recommendation involved the development of a novel clinical educator (CE) strategy, to support multiprofessional clinical staff working in ED. The CE strategy would look to address these issues through an intervention that could enable dedicated training time within the EDs most in need of shop-floor educational support (figure 1).
In the fall of 2018, a pilot programme, the CE in EDs (CEED) was initiated in 55 EDs in England. The programme part-funds an EM consultant to serve in the role of a dedicated CE during clinical shifts. It was expected that learner groups would be multiprofessional and representative of the ED clinical team and that 90% of training would be delivered on the shop-floor. It was anticipated that the presence of defined CEs in the ED might realise system benefits including: improved knowledge and understanding of EM and emergency care in general; increased contact time between educators and learners leading to an improved sense of value, well-being and job satisfaction; opportunities to undertake workplace-based assessments and supervision of cases, and of particular skills, for example, ultrasound and conscious sedation, etc, with reduced stress associated with assessments; increased opportunities to address the individual educational needs of the learners; and improved identification of learners' unique and team-wide training needs.
A test of concept and linked evaluation were deemed necessary to justify any future development, integration or commissioning of CE roles. This paper reports on the results of a survey of learners at the interim point in the study on the impact of CEs. From January 2018, a partnership including Health Education England (HEE), National Health Service (NHS) Improvement (NHSI), NHS England (NHSE) and Royal College of Emergency Medicine (RCEM) tasked all heads of school of emergency medicine (EM) in England to identify and rank acute Trust EDs according to their need for educational support. The identification of sites within which to test the concept was determined by heads of schools of EM and the RCEM Training Standards Committee (TSC). Data from the 2017 GMC survey, Acute Care Common Stem/Higher Speciality Training in EM surveys, local education surveys, HEE quality visits, care quality commission visits, resignation rates and local intelligence was used to provide a rationale for allocation of rankings in each region. A total of 72 Trusts were initially identified as potential pilot sites and confirmed by the TSC and HEE. HEE funding was secured to support the release of (the equivalent of) 162 programmed activities (PAs) of clinical educator (CE) time, divided across pilot sites. A conservative estimate was that each CE might have responsibility for the shop-floor education of 5-20 clinicians. approximately 1000 in total. Of those Trusts that expressed an interest, 55 were able to match HEE funding and sought to identify consultants to fill the CE role. Each CE post was match funded in a 50:50 ratio by HEE and the participant acute Trust. This was a condition of involvement and was consistent across all study sites. Pilot sites joined the project between October and December 2018.

THE CEED INTERVENTION
CEED commenced from October 2018, with an intention to conclude data capture in October 2020 and present pilot findings in January 2021. 169 CE posts were recruited across 54 sites, one site failed to recruit. The number of CEs and the number of PA per CE were agreed locally based on the numbers of consultants who applied for posts and the number of PAs that Trusts were willing to support.
CEED was developed to test the service benefit of a CE role, the purpose being to provide dedicated or 'ring fenced' time for education on a weekly basis for a minimum of 4 hours and a maximum of 20. The role was initially made available to EM Consultant Doctors (FRCEM holders), and later expanded to include Member Royal College of Paediatrics and Child Health (MRCPCH) qualifications. RCEM TSC suggested that the development of innovative new CE roles might support retention and well-being of multiprofessional clinical teams in the ED. An independent evaluation of the project was commissioned and awarded to Aston University (Academic Practice Unit), supported by RCEM.

METHOD
The pilot programme ran from July 2018 to October 2020. At the interim phase of the study 11 July 2019, a 15-question survey was designed to independently evaluate learner perspectives of having a CE in the ED. At this point in the study, all CEs were consultants in EM, with a minimum of 1 year experience at consultant level. The survey was developed, piloted internally and approved by academics from Aston University (including academic nurses and pharmacists), clinical members of RCEM (consultants in ED) and the HEE programme team. This survey was designed using JISC online surveys (formerly known as Bristol Online Survey). 6 The survey link was sent via an invite from HEE to each of the 54 active CEED NHS Trust ED sites. Site study leads were asked to distribute the link to their learners (any ED non-consultant medical and all other ED nursing, allied health professionals) in the ED during 11 July and 31 August 2019. Two reminder emails were sent to sites during the data capture period. At this point in the study, five sites did not provide any data returns, which prompted direct discussion with the site leads, without resolution during this period.
The questions were a series of categorical Likert score questions, with a focus on learners' experiences, opinions and recommendations relating to CEs on their learning, training and access to assessments. The impact on the well-being of the learners as well as details of the types of activities they received as part of the CE pilot were also explored. A summary of the questions is provided in table 1.
The online survey data were collected, and analysed via: descriptive statistics using Microsoft Excel 2013, the export report from online survey and IBM SPSS V.23. The free-text responses were analysed via thematic analysis. An initial framework was established by the academic authors (lead CH), verified by the wider study team and summary key findings agreed in open discussion prior to inclusion in this manuscript.

Patient and public involvement
No patient involved.

RESULTS
The survey was completed by 493 respondents across 49 NHS Trusts representing 91% (49/54) of the Trusts that took part in the CE pilot. The number of responses ranged from 0 to 53 responses per site. Multiple healthcare professionals completed the survey, with medical learners (trainees and non-trainees) making up 77.5% of respondents (see table 2).

Change of learning due to CE pilot
Most respondents reported improvement in learning: 48.1% (237) reported excellent improvement, and 42.2% (208) reported some improvement. A percentage of 8.9 (44) reported no change, 0.4% (2) reported that learning worsened and 0.4% (2) reported that it had considerably worsened.
Examples of responses per scoring rating is shown in figure 1.

Learners' experience and evaluation of the CE on site during the pilot
Learners reported the following based on their experience of a CE: 80.9% (399/493) reported they would remain/return to the same ED with a CE and 92.5% (456/493) responded that they would prefer to go to a Trust with a CE. With regards to their well-being, 49.9% (246/593) have reported that their wellbeing had improved as a result of having a CE on site. Further results of questions relating to learner experience and evaluation of the effect of having a CE on site are summarised in online supplemental information S1 and figure 2, respectively. Free type response explanations of the impact of CE on learner's well-being is included in online supplemental information S2.

Activities during the pilot
Respondents reported that the most common form of teaching was shop-floor teaching (including in situ simulation) (89.7%, 442/493) (table 3). The most common types of workplace-based assessments reported by learners as completed were case-based discussions (79.4%), followed by Mini-Clinical Evaluation Exercise (65.6%) (table 3). The longer type of assessments such as the Extended Supervised Learning Events were among the least reported (20.1%) (table 4).

Teaching or assessment from other senior staff
A percentage of 23.5 (116/493) reported that they received teaching or help with assessment by a staff member other than a CE deployed into the ED as part of the pilot.  8 which focused on what learners would want from their ED clinical teachers. The multisite focus group across five academic centres in Canada reported that learners considered the following attributes as important from their clinical teachers: 'takes time to teach'; 'gives them feedback'; 'tailors teaching to the learners'; 'uses teachable moments' and has 'a good teacher attitude'. No follow-up studies were published as to how this was implemented into practice. 7 Our survey did not cover the attributes of appointed CEs; however, the respondents reported that CE access had a positive impact on learning. The single site USA study showed that implementing a rotation of an ED resident to teach medical school students, and other medical trainees in the ED improved patient flow, procedure performance and undergraduate medical learning experience. However, this study used only Likert-score based quantitative findings which were only reported without reporting explanations behind the context. 8 There are limitations to this present study at the interim point. The principal limitation is that the full-staff denominator is unknown. Only 49/53 sites with CEs participated. There is also a potentially skewed response to the survey presented in this paper, with 53 respondents coming from one of the study sites, which may lead to bias. Due to the transitory nature of trainees as well as the rotations of staff, it is difficult to estimate with accuracy the number of learners per department. A second limitation was that despite there being opportunities for those surveyed to provide free-text answers, on analysis, there was insufficient information provided by the responses to fully analyse qualitative elements of  the respondents' views and perceptions. Further studies of a qualitative nature will be undertaken during the remainder of the pilot evaluation term to elicit more in-depth information on the impact of the programme on training as well as well-being.

DISCUSSION
In conclusion, most learners in the 54 NHS Trusts involved in the CEED study reported improvement in clinical learning opportunities within the ED at this interim point in the pilot. Impact on well-being is less clear. Further evaluation within the pilot will realise further evidence and data in relation to the impact of CEs on the recruitment, retention and well-being of the multiprofessional ED workforce.
Contributors MH, EP and MA contributed towards the conception and design of the study, drafting, revising and reviewing the manuscript for final approval. CH contributed towards the conception and design of the study, acquisition, analysis and interpretation of the data, drafting, revising and reviewing the manuscript for final approval. BK, MC and WH contributed towards the conception and design of the study, and reviewing the final manuscript for final approval. GR and AK contributed towards the conception and design of the study and the acquisition of the data and reviewing the manuscript for final approval. DT contributed towards the conception and design of the study, analysis and interpretation of the data, revising and reviewing the manuscript for final approval.
Funding This study was funded by Health Education England Commissioned study.
Competing interests None declared.

Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.

Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.