PCRF Research Alert: Measuring Skills Degradation Over Time

Each month the Prehospital Care Research Forum combs the literature to identify recent studies relevant to EMS education practices. 

Fisher J, Viscusi R, Ratesic A, et al. Clinical skills temporal degradation assessment in undergraduate medical education. J Adv Med Educ Prof, 2018 Jan; 6(1): 1–5. 

As EMS educators it seems we are forever in pursuit of that magic number that defines the moment of student competency in a psychomotor skill. But once a student achieves competency in a skill, we know little about the effect of time on its degradation. At what point do students drop below competency in a skill? How long after experiences or practice sessions might we see a decline in performance? Knowing this could help us establish ideal time increments between targeted skills sessions for particular skills. This would hold particular significance for more complex skills that don’t easily form stable motor memories or those that might be associated with rapid declines in competency. 

Research from the medical education literature shows that sustained deliberate practice improves skills proficiency; however, an optimal time frame for interval refreshers has not been established. Researchers from University of Arizona evaluated the temporal degradation of complex clinical skills in third-year medical students. The authors conducted a single-center, cross-sectional study at four separate time intervals in a cadaver laboratory after students underwent an education program on three procedures: femoral line placements, endotracheal intubation, and pigtail thoracostomy. These were chosen because they are complex skills that have a high impact on patient outcomes. 

The initial sessions were held just prior to the third-year clerkship rotations with students who had no previous procedural experience. Students underwent two hours of formalized didactic and hands-on instruction in small groups (2–3 students per faculty), which included practice time and immediate feedback during skills development. Additional time was also provided for supervised practice. Following the initial training all participants were evaluated for competency using validated checklists for each skill. Eight students were tested at one week to test retention and establish baseline. Then three groups of students were tested at 6 weeks, 12 weeks, and 18 weeks for competency in each skill using the same checklists. Authors hypothesized that scores would not significantly decline over the 18-week period. 

A total of 45 third-year medical students participated between June and November 2015. No significant difference was found in score distribution between baseline and 6 weeks; however, a significant decline was noted after 6 weeks for all three skills. The authors included a graphical representation of the data, which shows the dramatic decline in scores for endotracheal intubation and pigtail thoracostomy after 6 weeks. Interestingly, performance for femoral line placement improved between baseline and 6 weeks but then declined after 6 weeks. Educational experiences during early clerkship may explain the initial improvement in this one skill. 

Although other studies have shown retention of basic skills as long as 9 months after initial education, these authors hypothesize that the more complex skills, like endotracheal intubation, may require more frequent practice. This study should be replicated in EMS programs using psychomotor skills checklists validated for prehospital professionals, particularly for high-risk skills such as endotracheal intubation, needle thoracostomy, and cricothyrotomy. Given the current initiative to establish minimum competencies in CAAHEP-accredited paramedic programs, there’s no better time to become EMS educator-researchers in our own programs as active participants in establishing evidence-based education practices.

Megan Corry, EdD, EMT-P, is the program director and full-time faculty for the City College of San Francisco paramedic program and on the board of advisors of the UCLA Prehospital Care Research Forum.