Outcomes Panel: Plenty of Signs of Hope for More Effective Evaluation

A roundtable discussion on outcome evaluation for substance use services at the National Conference on Addiction Disorders (NCAD) East identified several industry trends that could bode well for more effective and standardized measurement.

One that particularly resonated with panelist Thomas Kimball, PhD, clinical director of MAP Health Management, is the increasing prominence of the peer recovery support specialist in treatment organizations. He characterized this trend as “an amazing professional lane emerging in our field,” adding that peers often fare best in gathering relevant patient data. “Establishing trust and gathering accurate information is so difficult,” Kimball said.

Kimball and his fellow panelists, Evince Clinical Assessments president Norman Hoffmann, PhD, and Vista Research Group CEO Joanna Conti, suggested as a group that the addiction field appears to be on the cusp of overcoming a history marred by inconsistent measurement of treatment effects.

Hoffmann urged the audience to seek precision in the terms used to describe success. Standards for “remission” are clearly spelled out in the DSM-5, he said, but if you ask five treatment leaders to define the term “recovery,” you might get seven different answers.

Conti founded her consulting firm after a five-year odyssey of trying to identify viable treatment options for a daughter who battled alcoholism. At that time, the best answer she could receive from center directors on their facilities' success rates was “Trust us,” she said.

“I was forced to use sheer luck to decide where to send her for treatment,” Conti said. (Her daughter has been in recovery for six years.)

Evolving models

Panelists urged their colleagues not to think of standardized measurement sets as static documents.

“I don't want my tools mandated,” Hoffmann said, because that means if something better comes along tomorrow, the field remains locked into a less effective instrument.

“We have good models in [general] medicine,” Kimball said. Recalling his son's cancer diagnosis at a young age, Kimball said his family received the option of being included in a test model of care that would eventually become the standard of care.

In cancer care, he said, professionals essentially put ego and territory aside in order to save lives. This is what is needed for addiction treatment, he said. “We have to have a great cooperative movement,” Kimball said.

In addition, he said, outcome evaluation should feature a mixed design that reflects the multiple pathways to recovery, with both quantitative and qualitative components. More than just a set of data on client substance use, recovery “is an experience. It's a story,” he said.

And the metrics will change over time, even at the individual level. “At 30 days, [recovery] looks very different than it does at one year,” Kimball said.

When she started moving from the role of family advocate toward consulting in 2015, Conti could get only a small handful of centers to report reliable results on how their patients were faring post-treatment. Now her organization is publishing data on thousands of patients' progress both during and after a treatment stay.

She foresees the field soon having enough information to be able to predict what modality of treatment would be most effective for a patient based on his/her profile. Hoffmann pointed out, however, that predictive analysis tends to work better for patient groups than for individuals.

Hoffmann also emphasized that treatment programs' credibility gets called into question when reporting their own results. He compared the situation to that of company shareholders who put more trust in an outsider's audit than in company executives' own pronouncements.