The Secrets of a Successful Simulation Program

What’s not to love about simulation?

As a method for training EMS providers and keeping their skills sharp throughout their careers, it has a lot of benefits: It allows replication and standardization of patient maladies both common and rare. It permits unlimited practice even on seldom-seen conditions. It keeps real patients safe from learners and their errors. And it’s shown to improve knowledge and skill performance across the medical fields. 

That’s all easy enough to grasp, and paramedic programs throughout the land have accordingly invested in wide ranges of simulation equipment, including task trainers, manikins of varying complexity, other live-patient simulators, and computer-based and virtual-reality programs. 

Virtually all accredited paramedic educational programs can use at least some of these tools. In fact, the 2015 SUPER (Simulation Use in Paramedic Education Research) study discovered 100% of the respondents to its survey had or had access to task trainers and simple manikins. That number was 99% for intermediate manikins and even 91% for advanced, fully programmable manikins and 83% for live simulated patients.¹ 

And that’s where the findings got interesting. Because many of those programs weren’t always using what they could. 

Apart from the task trainers, used by 97% of programs that had or could get them, a lot of tools sit idle. Simple manikins were available to all responding programs but only used by 92%. Intermediate manikins were available to 99% but only used by 93%. Advanced manikins, available to 91%, were used by just 71%. 

That dropoff persisted among less-ubiquitous items: Live simulated patients were accessible to 83% of programs, used by 66%. Computer-based simulation was available to 71%, used by 31%. Only 19% of programs reported access to virtual reality training, and just 4% said they used it.

There are some sensible whys behind all that. Programs were asked whether they had or had access to the specified tools; those that had such tools on hand used them more than those that could merely obtain them. Further reasons involved lack of training and personnel to support simulation-related activities. 

The overall picture captured by the SUPER study, then, was a frustrating one of a resource that’s common, yet remains underutilized. 

“These results suggest that simulation in accredited paramedic programs mirrors the proverbial three-legged stool,” the authors concluded. “To ensure simulation is used effectively, programs must have the appropriate equipment, faculty training and resources. If any of these elements is missing, the stool topples, and programs are less likely to use simulation.”

The NAEMSE Vision

The National Association of EMS Educators—some leading lights of which were among the SUPER authors—followed things up last November with a vision paper on simulation in EMS education. It examined those barriers to implementing it and looked at attributes of successful simulation programs. Finally it offered some recommendations for improving the use of simulation in EMS education.² 

First some barriers. The NAEMSE singled out: 

• Faculty training—Less than half of SUPER respondents reported their faculty training was appropriate, the vision paper noted. Users of advanced manikins were primarily trained by manufacturers’ representatives, but nearly one-fifth said their faculty had no training specific to the manikins they owned.
• Psychological fidelity—How realistic is the training experience? Beyond individual tools, developing skills and integrating knowledge in complex scenario-based simulations requires a “whole experience” fidelity that encompasses the environment and mind-set of the practitioner. Most paramedic program simulation happens in a skills lab, classroom or simulation lab, not a realistic, dynamic field setting. 
• Insufficient personnel resources—More than half of respondents to the SUPER survey said they had no staffing resources to support simulation beyond regular faculty hours, and 19% blamed inadequate personnel for their equipment’s disuse. 
• Shared resources—Having, or having access to, simulation equipment clearly doesn’t mean it’ll be used, and that’s especially true of shared resources: “Programs that reported ‘having access to’ a given simulation resource use that resource less frequently than programs that reported ‘having’ the same resource.”
• Inadequate funding—The money dedicated to simulation education often just isn’t enough. Fewer than half of those answering the SUPER survey said their simulation budget funding was adequate. 

The portrait that emerges here is that across much of EMS, simulation is handled in a cursory kind of way: Give the program head a tool or two and pay no more mind—they’ll work out the rest.

“It speaks to the problem we have with our general approach to simulation in EMS education,” says Elliot Carhart, EdD, NRP, EMS performance and research coordinator for Florida’s Pinellas County EMS & Fire Administration and a coauthor of the SUPER study. “That is, we take an educator, and we give them complex equipment and expect them to develop learning objectives, program a simulation and operate the manikin, all while observing the students and facilitating a debriefing, and all while achieving the desired educational outcome. When we look at the use of simulation in other healthcare disciplines, we see a much greater depth of resources.” 

To instructors in the field, that rings authentic. 

“My experience is that folks will buy, for example, something like a high-fidelity simulator through a grant,” says says Chris Boyer, MPA, NRP, FP-C, simulation coordinator at Delaware Technical Community College in Dover. “But when they buy the simulator, they never actually incorporate a cost for somebody to come in and train people how to use it. So they’re trying to learn how to use it on the fly, on top of trying to write the scenarios and run the simulations, and they never get comfortable with it.”

“Even when schools find grant funding or some other way to get this expensive stuff, very few of those grants provide any funding to pay or train the staff or pay for extra personnel,” says Gary Heigel, chair of the Emergency Services Department at Oregon’s Rogue Community College. “To use that kind of technology, you really have to have multiple instructors involved—one who’s running the scenario overall, another who’s just running the manikin, and preferably a third who can be sort of a participant observer. It gets very difficult on most of our budgets to figure out ways to do that.” 

Still, many paramedic education programs—both well-funded ones with whizbang toys and modestly appointed ones using more basic tools—make effective use of simulation in EMS education. What are the finer points of that? 

Secrets of Success

The first thing to know is that you don’t have to spend a lot of money to get some benefit from simulation. Even simpler tools and approaches can have value.

“When people think about doing simulation, they often think of these expensive high-fidelity simulators,” says Boyer. “Simulation is more of a concept than an actual purchase. You can use student role players with some of these fairly low-cost devices that are out now. It doesn’t have to be superexpensive. The focus should be less on the equipment and more on the process.”

“As much as we want to focus on the technologies, we can’t lose the importance of the simplified approach—the simple tabletop-type stuff as well,” says Carhart. “That is also effective, especially when we’re dealing with these discrepancies between the need for training and equipment sitting idle and resources being wasted. Sometimes simple is good too.”

The key is to make sure simulation is used and learning scenarios built around clear, succinct learning objectives. 

“We make sure every scenario we do has a specific objective, whether that’s focused on history-gathering or making a differential diagnosis and choosing a treatment path,” says Heigel. “Every scenario should have a really specific objective so the instructor knows going in exactly what it’s intended to teach and what the students are supposed to learn.”

Summative scenarios—concluding ones held at the end of a class—may be long and complex. Formative scenarios—those held during learning—need not be so lengthy, if a discrete objective can be accomplished more briefly.

With whatever equipment or tools you have, make the most of them. At Delaware Tech, manikins aren’t limited to the lab—Boyer brings them to the classroom to use as teaching aids. 

“So if I’m talking about pharmacology, for example—how medications impact the sympathetic nervous system—instead of speaking hypothetically, I can project the actual computer control for the simulator, with all its functions, and say, ‘OK, we’re going to give epinephrine. Let’s talk about what epinephrine does to the sympathetic nervous system.’ And then we can give it to the simulator and all watch together what happens as it starts to take effect,” he says. “By embracing that technology, you’re bridging that gap between knowledge acquisition and knowledge application.” 

Running simulations and scenarios smoothly requires familiarity with the equipment and practice beforehand. Master your technology—nothing dispels fidelity and flow like technological delays and interruptions. 

Fidelity, or the believability of the experience, is important on a number of levels. It applies to individual items of equipment, the environment in which training occurs and even the psychological realm: Good simulations help suspend disbelief and create a convincing mental “moment” that helps providers learn in what they perceive as credible, convincing situations. 

“We’ve all been doing simulation for as long as we’ve been doing EMS training,” notes Heigel. “The question is just how much technology you include and how much realism you try to apply to it. Simulation as a technology is great, if it contributes to the flow of the scenario—if you have well-trained people who know how to use the equipment and do it regularly. If you don’t have all those things, the technology can get in the way. In that case I’d rather have a non-technical manikin lying on the floor and just describe to the student what they’re finding. That may not be as effective, but it’s probably more effective than a scenario where the instructor has to stop the student while they try to get the manikin caught up to what was supposed to happen next.” 

Fidelity is holistic. It can be as simple as using a female manikin to represent a female patient, but can also span aspects of the training experience like backgrounds, sounds, smells, even the motion of an ambulance. 

There’s no doubt high-tech manikins that have vital signs and react to treatments can improve the fidelity of the learning experience. At this year’s EMS World Expo (Oct. 3–7 in New Orleans), Boyer will describe how simulation can be a disruptive innovation. 

“Some folks might object to that description, because disruptive innovations are typically lower cost, and all the high-fidelity simulators, for example, are expensive to purchase,” he says. “The efficiency of having students be able to train on a device that actually responds like a patient would is where the disruptive innovation part comes from. Because you’re basically utilizing the limited time you have in the lab much more effectively. You can get your students to understand and grasp concepts more quickly, and move them into more complicated tasks by actually demonstrating to them, especially your more hands-on learners: I give Drug A, Drug A takes two minutes to work. Well, instead of doing the magic ‘shake the fists’ type thing we do in labs sometimes—‘OK, two minutes has passed’—they actually have to watch this medication take effect and watch some of the side effects.”

Some other good ideas for simulation education: 

• When possible, seek trainers with EMS experience. Our environments and practices are unique. There’s benefit to intraprofessional training opportunities too. 
• Stay abreast of current literature and involved in user groups, professional organizations and the like, where you can exchange insights and lessons learned with colleagues. 
• Students may appreciate the chance to run through scenarios more than once. “We run them through a scenario where they may miss key things, stumble on something or not be particularly effective,” says Heigel. “We stop, do a debrief, talk about what was done well or not, and then we reset and let them redo the same scenario, knowing exactly what they’re walking into and how they’re supposed to manage it. It really helps reinforce positive behavior.”
• Post-training debriefing is often overlooked. “It’s that debriefing process, that reflective process, where the learning actually takes place,” says Carhart. “Sometimes we get so caught up in everything we’re juggling, we get to the end of the simulation and have more of a hasty hot wash than a proper debriefing. That is to our detriment.” 

‘Like Night and Day’

“Evidence seems to indicate,” the NAEMSE vision paper concludes, “that high-fidelity simulations facilitate learning when used under the right conditions.”² It offers, from authors led by the University of Miami’s Barry Issenberg, MD, 10 habits of effective simulation programs (feedback; repetitive practice; integration into the standard educational curriculum; range of difficulty level; adaptability to multiple learning strategies; capture of a wide variety of clinical conditions; controlled environment to make, detect and correct errors without adverse consequences; individualized, active learning; defined outcomes and tangible measures; and simulator validity).³ 

“What we’ve seen is a remarkable improvement in our students’ level of preparedness that shows up when they actually get to the field internship,” says Heigel. “For many years we did EMT classes in a classroom, and we’d just sort of talk about things: ‘OK, here’s your patient, you pick them up and put them on the table. Now pretend you’re in the back of an ambulance, and imagine you’re doing this.’ And we’d put students through a variety of steps, basically asking them to visualize what it’s going to be like when they get out in the real world, and assuming they’ll translate that pretty well. 

“Well, our students have been getting through those processes for years, and they’ve been successful. But when we really started incorporating the realism, the feedback we’ve gotten from our clinical sites and our internship sites is like night and day. The students actually understand what it’s like to work in the back of a moving ambulance, because we run scenarios where they have to package the patient in unpredictable situations and get them on a gurney and into the ambulance and do interventions while it’s moving. We used to just talk about those things. So I think each component of that better prepares our students for stepping into that real environment.” 

How the National Registry Is Transitioning to Scenario-Based Examinations

There are a few reasons behind the NREMT’s move toward scenario-based verification of psychomotor competency for paramedic students. One is to assess them in a way that more simulates actual practice. Another is improve students’ ability to integrate individual skills into their overall scene and patient management.

Certification standards starting this year require students to have a portfolio of vital skills and show progression from performing simple psychomotor skills and scenarios to integration of skills and assessments in complex simulated scenarios. Next year the NREMT will launch the first phase of its new scenario psychomotor exam. “Scenario examinations,” the organization says, “allow the NREMT to incorporate essential attributes of Team Leadership along with scene and patient management, thus better reflecting actual out-of-hospital care as opposed to continuing to test 12 isolated skills.”

To prepare educators to construct and deliver these new scenarios, NREMT and the National Association of EMS Educators (NAEMSE) are partnering to put on a series of regional scenario development workshops. These are free (except for ones at EMS World Expo and the NAEMSE Educator Symposium) two-day classes designed to help program leaders develop offerings of appropriate type, variety and difficulty. They cover areas like scenario writing, calibration and evaluation. Scenarios created in the workshops will be available for use afterward.

“While the Registry has not really ever prescribed or done education—we’re about verifying competency—in this case we decided educators needed some help,” says Heather Davis, EdD, NRP, a member of the NREMT’s executive committee and the planning committee for the workshops. “So the Registry partnered with NAEMSE to bring this opportunity to educators and program directors to help them understand the process, what they need to be doing in their skill labs, formative vs. summative scenarios and how scenarios are different than full-blown simulation. We’ll use folks who have already done this successfully in their programs to help answer questions and serve as mentors.”

Apart from the two at the big shows, there will be 20 regional workshops held across the country (see https://naemse.org/nremt) through December. Feedback is being actively courted and employed to improve the workshops as they go. Resources to help have also been assembled on the NREMT’s website, including sample video scenarios and things like sample rotation schedules, student job descriptions and an example of a portfolio collection tool.

“So often, particularly if you don’t have a lot of experience writing scenarios or aren’t in the habit of including your faculty in the calibration process, you can think you’ve written a great scenario, and then it turns out to be a mess when the students actually perform it,” says Davis, program director for the UCLA paramedic education program. “Then you don’t really get the feedback you’d hoped to gather. So it’s really valuable feedback for groups of instructors, and the hope is that they’ll leave these workshops and go model this behavior back in their own programs.”

References

1. McKenna KD, Carhart E, Bercher D, Spain A, Todaro J, Freel J. Simulation Use in Paramedic Education Research (SUPER): A Descriptive Study. Prehosp Emerg Care, 2015 Jul–Sep; 19(3): 432–40. 

2. National Association of EMS Educators. Simulation in EMS Education: Charting the Future, https://c.ymcdn.com/sites/naemse.site-ym.com/resource/resmgr/Docs/SimPressRelease15.pdf. 

3. Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach, 2005 Jan; 27(1): 10–28.