Using Data Aggregation to Determine Thematic Knowledge Transfer and Lasting Performance Change in Multiple Continuing Education Activities on IBD
BACKGROUND: Often, implications of the effect of CME is difficult to determine due to reliance on single data points of pre-post knowledge questions. Using aggregation methodology, we were able to thematically combine the assessments of 3 similar educational activities to determine the immediate and lasting effect of education on IBD practice.
METHODS: Surveys were designed based on learning objectives of 3 accredited CME activities on IBD. Outcomes measurements were taken directly before and after the activities, as well as 3-months after activity completion. This 3-month follow-up sample was matched at the same time to a control sample of clinicians who did not participate in the education.
RESULTS: Based on the themes of the pre-post questions, we were able to aggregate the scores of the three activities and determine overall changes in understanding key risk factors, establishing appropriate goals, choice of guideline-based therapy, and knowledge of clinical trials. Compared to pre-activity, learners had significant improvements in each of these areas, showing effectiveness of the education. Hodge’s g effect sizes were run on all pre-post data to better compare the effect of education across themes. The most robust effects were seen in recognizing risk factors (g = 0.75) and clinical trial results (g = 0.71) while more modest effects were seen in establishing goals of treatment (g = 0.40) and selecting treatment (g = 0.59). Some of this modest effect seen in establishing goals can be attributed to a high baseline level of knowledge (a pre-score of 70.8%). However, looking at the scores of learners compared to a control group of similar clinicians that did not attend the education, robust effects can be seen even 30 days after the activity. Assessment of risk had an effect of 0.67 while establishing goals (g = 0.81) and treatment choice (g = 0.85) had even higher effects.
CONCLUSION(S): Aggregating the outcomes of multiple educational activities allows us to understand effect by theme, providing opportunities to inform key stakeholders of the value of educational interventions. Further it allows educators opportunities to understand the continuing need of the community that has not attended these activities, and focus future interventions on improving areas where they are needed most.
