How to Use Data and Technology in Quality Improvement and Training

EMS agencies have been collecting data for some time, but those data have not generally been used to develop targeted improvement or training opportunities. As EMS stands poised to create and implement EMS Agenda 2050, now is the time to develop the core technology and improvement principles to quickly adopt evidence-based recommendations as they appear elsewhere in medicine. American Medical Response’s West Michigan practice has been doing just that with help from both its local regulatory body and area hospitals.

Basics of EMS Data

The transition of EMS documentation from paper PCRs to an electronic platform drastically improves the availability of data from each patient encounter. Not only is the data more complete, but it’s also available in a timelier fashion. In contrast to previous data-driven quality improvement opportunities, the changing face of EMS data allows more detailed analysis that in some systems may verge on real time. This breadth of data allows QI professionals and medical directors to ask far more complex questions about EMS treatments and their impact on patients in the prehospital environment.

In 1993 the National Highway Traffic Safety Administration (NHTSA) developed and released the first version of a uniform data set for EMS provider agencies, which consisted of 81 fields. The fields largely represented patient demographic and call information but included elements dedicated to vital signs and cardiac arrest findings. The 1996 EMS Agenda for the Future contained recommendations for a uniform, collaborative data set that described the entirety of EMS events for purposes of research, quality improvement and provider feedback.¹ In 2001 NHTSA joined with the Health Resources and Services Administration to develop the National EMS Information System, now commonly known as NEMSIS.²

As U.S. locations moved forward with mandating electronic patient care records (ePCRs), NEMSIS adoption spread through the country. NEMSIS is currently in its third version, which contains 574 elements and the ability to describe EMS calls from both operational and clinical perspectives.³ The current version of NEMSIS is also approved by the American National Standards Institute, and during that approval NEMSIS partnered with Health Level 7, the standards developer of hospital electronic medical record data sets.⁴ This partnership increases the interoperability of EMS and hospital data.

The promise of a standardized EMS data set linked to hospital information is the ability to quantify the performance of a system and use that information as the starting point for targeted improvement projects and educational opportunities that directly benefit communities. In an effort to provide this type of snapshot to EMS system participants, NHTSA funded the EMS Compass initiative. EMS Compass strives to produce a “consistent set of measures [to] allow EMS caregivers, managers and governments to monitor performance and improve performance…”⁵ To engage in a performance improvement project, an EMS system or provider must first know what the current performance of their system is; only then can they determine how to measure potential improvements. As hospital reimbursement has been increasingly tied to performance and improvement, it is reasonable to think EMS reimbursement—and the future of EMS overall—will be tied to similar measures.

Building Trends from Data

The ability to trend EMS data is largely linked to the quality of that data. As mentioned, the first standardized EMS data set included only a few clinical metrics. From that data it would be difficult to craft broad answers about clinical questions. For instance, if a medical director were curious about the treatment of patients with respiratory distress, it would be impossible to draw a universal conclusion from a national data set that did not include elements quantifying a patient’s respiratory distress and how that level of distress changed with treatment. While the EMS Compass initiative makes great strides toward a group of quality indicators for EMS, adoption of those metrics requires implementation of NEMSIS version 3. In systems that haven’t yet implemented the latest version of NEMSIS, what is a supervisor or quality improvement analyst to do?

One feature of EMS Compass will be the ability to compare performance between systems, but the other benefit—the ability to trend performance and determine if it is improving—is something EMS agencies can begin doing now.

When developing an improvement opportunity, it is important to focus the initiative to create the best possibility for success. Following the Institute for Healthcare Improvement’s Model for Improvement, anyone interested in creating these opportunities should ask three questions:⁶

• What are we trying to accomplish?
• How will we know when a change constitutes an improvement?
• What specific change will result in an improvement?

When trending data, that second question is the focus. Start with a goal and then determine which outcome measures are pertinent to it. Once you’ve identified the measures you’re interested in, identify in your own agency’s data set where those data will come from. Including multiple data sources (e.g., ePCR, CAD, hospital discharge diagnosis) improves the quality of data but drastically increases the amount of time spent matching, combining and analyzing records. If your project does not require multiple data sources, don’t simply collect extra data for the sake of having it.

After identifying outcome measures, establish a baseline for performance of your system. By doing so, you’ll be able to determine if a change you make improves care or outcomes. Measuring baseline performance can occur with lean or other process improvement disciplines. Then implement small-scale changes in your system and analyze the impact on your performance measures. If a change improves performance, gradually spread that change throughout your system. If not, head back to the drawing board.

QI and Training Impacts

The terms quality assurance and quality improvement have been part of EMS since early in the history of prehospital care. In an era of paper records, however, the difficulty of collecting meaningful data meant that most QA/QI initiatives were limited to tangible interventions that were easily measured. Paramedics would receive feedback on a semiregular basis on metrics such as IV success rates and first-pass intubation success. While patients needing medication or airway management might have been affected by the skills performance measured by these outcomes, it simply is not logical to draw a direct correlation between skill success rates and patient outcomes.

In contrast to the data obtained manually from paper PCRs, data from ePCRs includes direct measurements of patient status (level of conscious, vital signs, ECG interpretation, etc.) and treatment and can be readily exported for analysis. Combined with process improvement tools and techniques, these data can be analyzed quickly to decrease the turnaround time for QA/QI processes.

By streamlining the collection and analysis of data from ePCR sources, EMS agencies can more quickly react to findings of these processes and implement meaningful changes. One word of caution: Data analysis is only as good as the data being analyzed. EMS providers and agencies alike must be confident in the accuracy and quality of the data entered into the ePCR; otherwise incorrect conclusions may result from a set that is not representative of system performance or patient presentations.

Nearly every EMS provider has participated in training that did not feel relevant to his or her practice. With employee time and training budgets both limited, it makes sense to target education to areas of greatest need. The collection of patient treatment data and analysis of how EMS care affects outcomes can identify training opportunities that make the most efficient use of employee training time. The extra time spent gathering and analyzing data can pay dividends in the long run (see sidebar).

The Next Steps

As attentive readers will have noticed, one of the hallmarks of effective data collection, analysis and creation of useful, targeted interventions is the inclusion of hospital data, particularly patient outcome indicators. In EMS systems with a robust reporting scheme for EMS data, participants should next look to their hospital partners to identify opportunities to collect inpatient data as well. CARES is a good example of an established program with excellent hospital participation that may ease hospital and EMS leadership into that transition.

In the longer term, however, direct integration of EMS and hospital data is the goal. By marrying those two data sets, hospital and EMS personnel alike can begin to answer questions about what EMS can do for patients that actually impacts outcomes. While there are some technological hurdles to this level of integration, those hurdles continue to fall. To take advantage of improving technology and advancing data analysis techniques, EMS providers, agencies and regulators will need to establish solid working relationships with their hospital partners. AMR in West Michigan is lucky to have had those relationships for some time, and we stand poised to move forward aggressively with improving the care our patients receive in the data-driven future of EMS.

Michigan Examples of Data-Driven Improvement Opportunities

AMR in West Michigan provides 9-1-1 and interfacility transport service to three counties around the Grand Rapids area with a volume of approximately 50,000 calls per year. The structure of the EMS system develops hospital partnerships by necessity: Operating the local medical control authority is tasked to area hospitals rather than government entities. This relationship provides for cooperative opportunities between EMS and hospital leadership and QI personnel. Additionally, sharing EMS and hospital data for joint improvement opportunities has become the norm. By using the trending techniques discussed here, Kent County EMS has instituted a variety of data-driven improvement projects directed at impacting patient outcomes.

STEMI—STEMI patients are among those who may benefit the most from timely and appropriate EMS intervention. Recognition of ECG changes by EMS providers, transport to an appropriately equipped hospital and prenotification of the hospital are all linked to shorter door-to-balloon times and smaller infract sizes—effective surrogates of improved outcome.⁷ In Kent County the three ALS transport agencies and three health systems have partnered to analyze and trend metrics on each STEMI-activated patient transported by EMS as well as every patient taken to the cardiac catheterization lab who arrived by EMS. In this way, false positive and false negative cases can be included in the analysis.

The data collection and reconciliation process is lengthy, but the analysis has produced several actionable findings that are influencing changes in the way EMS interacts with cardiac patients and receiving facilities. In 2015, the most recent year for which complete data exist, AMR and the other ALS provider agencies in Kent County transported 186 suspected STEMI cases. Of those, 123 were included in the data analysis. In 45% of the cases, the cath lab was activated before the patient arrived at the hospital, with a true positive rate of 84%. In 33% of cases, the cath lab was activated after patient arrival, with a true positive rate of 95%. In the remaining 22% of cases, the cath lab was not activated. The data analysis includes hospital performance measures (door-to-balloon time among others) as well as evaluation of the specific coronary artery that caused the STEMI.⁸

When looking for improvement opportunities, researchers found that patients for whom the cath lab was activated prior to hospital arrival had a mean door-to-balloon time 10 minutes shorter than those for whom the cath lab was activated after arrival (49 minutes vs. 59 minutes). This finding prompted further review of some factors that influence cath lab activation. It appears that early 12-lead acquisition and bedside radio reporting are associated with an increased likelihood of prearrival cath lab activation. To summarize: A detailed analysis of EMS and hospital STEMI data found that performing a 12-lead as soon as possible and calling the hospital right away (both low- to no-cost interventions) may be able to reduce door-to-balloon times by as much as 10 minutes.

Stroke—Similar to STEMI, stroke patients represent a population whose outcomes largely depend on recognition of symptoms and the underlying disease and transport to an appropriate facility. Adam Oostema, an emergency physician in Kent County, has established a local stroke registry, similar in functionality to the Cardiac Arrest Registry to Enhance Survival (CARES). Like CARES, Oostema’s registry includes both EMS and hospital data and seeks to demonstrate both the aspects of EMS care associated with better outcomes and opportunities for improvement.

In his most recent analysis of Kent County stroke patients, Oostema found documentation of a complete Cincinnati Prehospital Stroke Scale (CPSS) was associated with increased sensitivity of EMS screening for stroke. Additionally, for strokes not recognized by EMS, symptoms not included in the CPSS, such as vertigo and ataxia, were commonly present.⁹ This analysis and the continued submission of stroke data by EMS and hospitals has resulted in several educational undertakings by Kent County EMS agencies to encourage the documentation of a CPSS on every suspected stroke patient. Additionally, educational interventions are being considered to potentially add screening for common symptoms associated with missed stroke but not addressed in the CPSS.

Pediatric medication administration—Pediatric patients represent a relatively small portion of the volume for most EMS agencies, and critical pediatrics are even less common. That said, critical pediatric patients require accuracy when it comes to medication administration, and proper dosing requires an accurate weight. Expecting EMS providers to accurately estimate patient weight in pounds, convert to kilograms, remember the proper weight-based dose for a medication, accurately calculate the dose, calculate the proper volume of drug based on the dose and concentration, correctly draw up the medication and finally administer it is a recipe for disaster.

Michigan EMS Bureau medical director William Fales, MD, previously identified a rate of medication errors by EMS approaching 35%, compared to approximately 18% for pediatric patients in the hospital.¹⁰ While it is not correct to assume the medication error harms each of these patients, the risk is one that can be largely mitigated. After a series of follow-up analyses of pediatric dosing data (including a national survey of paramedics),¹¹ the Michigan EMS Bureau released a drug dosing safety system called MI-MEDIC. The system, packaged as a card set provided for every licensed ALS vehicle, is set up to mimic the colors of the Broselow tape but precalculates both drug dose and volume based on state standard protocols and drug concentrations. This tool combines the findings of a decade of EMS data analysis in Michigan with a low-tech intervention to improve patient safety.

References

1. Mears G, Ornato JP, Dawson DE. Emergency medical services information systems and a future EMS national database. Prehosp Emerg Care, 2002 Jan–Mar; 6(1): 123–30.
2. NEMSIS Technical Assistance Center. History of NEMSIS, https://nemsis.org/theProject/historyofNemsis.html.
3. NEMSIS Technical Assistance Center. Version 3—Frequently Asked Questions, https://nemsis.org/v3/faq.html.
4. NEMSIS Technical Assistance Center. NEMSIS Version 3—Overview, https://nemsis.org/v3/overview.html.
5. EMS Compass. About Performance Measures, https://emscompass.org/about-performance-measures/.
6. Institute for Healthcare Improvement. How to Improve, www.ihi.org/resources/Pages/HowtoImprove/default.aspx.
7. Kobayashi A, et al. STEMI notification by EMS predicts shorter door-to-balloon time and smaller infarct size. Am J Emerg Med, 2016; 34(8): 1,610–3.
8. Obiden D, Chassee T, Waddell B, et al. Cardiac Catheterization Activation Rates for ST-Segment Elevation Myocardial Infarction: Results from the Kent County STEMI Project. Poster session presented at the Annual Michigan Medical Control Seminar, Acme, MI; www.wmrmcc.org/MCA/Kent/PI-DASHBOARD.
9. Oostema JA, Konen J, Chassee T, Nasiri M, Reeves MJ. Clinical predictors of accurate prehospital stroke recognition. Stroke, 2015 Jun; 46(6): 1,513–7.
10. Hoyle JD, Davis AT, Putman KK, Trytko JA, Fales WD. Medication dosing errors in pediatric patients treated by emergency medical services. Prehosp Emerg Care, 2012 Jan–Mar; 16(1): 59–66.
11. Hoyle JD Jr., Crowe RP, Bentley MA, Beltran G, Fales W. Pediatric Prehospital Medication Dosing Errors: A National Survey of Paramedics. Prehosp Emerg Care, 2017 Mar–Apr; 21(2): 185–91.

An EMS practitioner for nearly 15 years, Patrick Lickiss, BS, NREMT-P, is currently located in Grand Rapids, MI. He is interested in education and research and hopes to further the expansion of evidence-based practice in EMS.