USING TECHNOLOGY TO CONNECT THE DOTS

In 1993, William A. Anthony, PhD, director of the Center for Psychiatric Rehabilitation at Boston University, declared that the 1990s should be called the “Decade of Recovery.”¹ While many definitions of recovery were offered since then,² the Center's work has been guided by a parsimonious definition of a person's recovery: “the development of new meaning and purpose in one's life as one grows beyond the catastrophe of a mental illness.”¹

No matter what definition of recovery one prefers, the vision of recovery has entered the mental health field as a genuine hope for the millions of Americans living with mental illnesses. Fueled by the consumer self-help movement, more effective and less harmful medications, longitudinal research showing evidence for recovery outcomes, and new service delivery models, the recovery vision has manifested itself in people returning to work, living independently, and attending school and training programs.

Simultaneously, people are asking, “If recovery is real, can it be measured in real-world outcomes such as earnings, degrees, and reduction of benefits, as well as more intangible variables such as self-esteem, empowerment, and hope?” Furthermore, even if we agree on recovery outcome measures, how can we measure what factors are determining these outcomes? Are some programs better at helping people reach these outcomes and, if so, what intervention processes are most effective?

The Value of Measurement

In our quest for evidence-based mental health interventions, it is not enough to measure program structures and outcomes. We must know how the intervention's clinical process helps to bring about recovery outcomes. Monitoring and improving a program's outcomes are most relevant if we can link the clinical process to the outcomes achieved.

To deploy true recovery-oriented mental health services, providers need to examine the effects of their services and dynamically adjust those services to optimize recovery. Unfortunately, most programs have limited resources and are unable to perform these types of evaluation. Many mental healthcare programs, therefore, have not established effective, reliable measurements of their performance.

A Technologic Solution

Boston University, via the Center, has created ROMIS, a Recovery-Oriented Management Information System designed to assess intake, process, and outcome measurements in mental health services. Conceived during the 1990s, ROMIS was the result of two years of discussions among senior staff to describe and define various service processes (e.g., rehabilitation, case management, treatment) as accurately and completely as possible.

Karen Danley, a brilliant and innovative force in vocational rehabilitation who worked at the Center for 17 years until her death in 1997, said of those discussions, “We knew that if we could get down on paper a common, logical, and objective description of the service processes, we could create a management information system that would hold the field's feet to the fire in regards to facilitating recovery outcomes.”

Danley's words were prophetic, as the field in the past ten years has moved toward evidence-based practice and has begun to recognize the need to be able to describe and measure the processes producing recovery outcomes.³

Funded by consecutive grants from the Tower Foundation and the Fidelity Foundation, Center staff first developed, field-tested, and refined a paper-and-pencil version of ROMIS designed to measure recovery processes and outcomes. Next, we constructed a Microsoft Access version and repeated the field tests. The rapid shift from an office-based workforce to one that is much more mobile necessitated ROMIS's migration to a Web-based design (figure).

ROMIS is accessible by any Web browser and is being tested against Internet Explorer 6.0 for Windows, Firefox 1.5, and Safari 1.3. Our goal has been to make ROMIS quickly accessible, easily usable, and able to precisely track required intake, process, and outcome information. With this goal in mind, ROMIS is intuitively designed as a series of screens and drop-down menus that contain multiple variables regarding intake and process activity components. ROMIS can supplement information systems that already track demographic, administrative, and financial data, so that clinical processes and recovery outcomes can be incorporated within existing management systems. ROMIS will have a database dictionary that will tell how the data are stored so that anyone who can access the database should be able to retrieve data from ROMIS to use with any other application. The official launch of the new Web-based ROMIS (expected this fall) will be a great leap forward at the Center in terms of both flexibility and efficiency in our data collection.

ROMIS's Modules

ROMIS includes three modules.

Intake Module. The Intake Module includes a wide range of intake requirements, including role status, health status, residential status, financial status, clinical status, and demographics. Some of these items were chosen because they may change as a result of participation in recovery-oriented programs; these same items appear in the Outcomes Module.

Process Module. This is ROMIS's heart and soul. It is predicated on the assumption that the process between client and practitioner, regardless of the particular service process in which it is activated, consists of diagnosis, planning, and intervention activities that can be tracked. The essence of ROMIS is that by collecting data on the components of the interaction between the client and practitioner, measured by the adherence to a uniformly defined diagnostic, planning, and intervention process, information can be gained about the relationship between that process and recovery outcomes.

The types of service processes currently built into ROMIS include psychiatric rehabilitation, psychiatric treatment, health and wellness, care/case management, and family education and support. Other services easily adaptable to ROMIS include medication management, crisis intervention, and individual therapy. The table illustrates examples of the diagnostic process activities for psychiatric rehabilitation services and care/case management services that ROMIS tracks. In addition, the Process Module can monitor diagnostic, planning, and service activities regardless if they occur in individual sessions or groups.

TABLE. Examples of the diagnostic process activities that ROMIS tracks

Care/Case Management

• • Clarifying client problems

  • Setting service goals

  • Choosing service providers

Psychiatric Rehabilitation

• • Establishing an initial relationship
  • Readiness assessment/choosing a direction
  • Setting an ORG/choosing an ORG (overall rehabilitation goal)
  • Functional assessment-assessing critical skills for the goal
  • Resource assessment-assessing critical supports for the goal

Outcomes Module. The Outcomes Module captures changes in status, functional role recovery, and personal benefits related to recovery. This module includes some of the Center's most popular instruments, such as the Empowerment Scale ⁴ and the Status Update, ⁵ as well as numerous measures addressing symptoms, health, functioning, and personal benefits such as self-concept, quality of life, and satisfaction with services. The Outcomes Module allows for other widely used instruments in mental health to be integrated into the module if desired.

Putting ROMIS to Work

Here's an example of how ROMIS works, using all three modules. Let's say “Michael” comes to the program, and we collect information on his demographics for ROMIS's Intake Module. We collect information such as level of education, work history, health history, medication history, etc. Michael tells his practitioner that he would like to finish an undergraduate thesis to complete his BA, which he has not worked on for ten years. We help him choose an overall rehabilitation goal in the educational environment of writing his thesis by October 15, 2006.

Figure. An example of the Web-based version of ROMIS.
The practitioner working with Michael engages him in a rehab relationship, teaching Michael skills, providing him support, linking him to existing supports, etc. All the time, Center staff use ROMIS to document exactly what the practitioner is doing in their time together, differentiating between all of the aspects of the rehabilitation process. Staff are trying to answer the question, “What happened to Michael as a result of our work?” It allows us to understand how our services contributed to the outcome. Recovery outcomes that might be measured include changes in educational status, self-concept, empowerment, earnings, etc.

ROMIS has been used to conduct a program evaluation of our Recovery Center. The Recovery Center is an innovative model of service delivery based on a biopsychosocial conceptualization of psychiatric disability. Service delivery involves a psychoeducational intervention and semester-long courses in a variety of health, wellness, computer, and other types of classes.

ROMIS was used to examine the effectiveness of the Recovery Center along multiple objective and subjective dimensions of recovery. We tracked 178 individuals (97 who received the intervention and 81 who served as a comparison group) for one to two years after enrollment in the study. Using ROMIS, we tracked objective dimensions of recovery, including mental health service utilization, hospitalizations, diagnosis, and instrumental role functioning in residential and vocational areas. We also tracked subjective dimensions of recovery, including self-reported measures of symptoms, physical well-being, and self-esteem. Individuals were assessed at multiple time points with ROMIS, including at baseline and at follow-up at six-month intervals.

These data were transported from ROMIS into a statistical package for analysis, enabling us to examine change over time for the 97 individuals who received the Recovery Center intervention and the 81 comparison individuals who did not receive the intervention. In addition to tracking subjective and objective recovery outcomes, we tracked the process information that will allow us to link the specific interventions to specific outcomes. With this process information, we are aiming to determine if individuals who participated more intensively in the Recovery Center experienced better outcomes or if individuals who chose certain course offerings in the Recovery Center were more likely to improve along certain dimensions.

Conclusion

John Ahman, chief operating officer at WestBridge Community Services of Cambridge, Massachusetts, and Manchester, New Hampshire (early ROMIS adopters), says of ROMIS, “ROMIS stands out because it focuses on the persons served and tracks their recovery process. It looks at the clinical relationship and breaks it down into three simple stages: diagnosis, planning, and intervening. A lot of thought went into what data should be captured.” To learn how your organization can use ROMIS, contact the author at kohnman@bu.edu.

Larry Kohn, MS, is Director of Development at the Center for Psychiatric Rehabilitation at Boston University.

References

1. Anthony WA. Recovery from mental illness: The guiding vision of the mental health service system in the 1990s. Psychosocial Rehab J 1993; 16:11-23.
2. U.S. Substance Abuse and Mental Health Services Administration. Consensus statement defines mental health recovery. SAMHSA News 2006;14(2). Available at: https://www.samhsa.gov/SAMHSA_News/VolumeXIV_2/article4.htm.
3. Manderscheid RW, Henderson MJ. From many into one: An integrated information agenda for mental health. Behavioral Healthcare Tomorrow 2004; 13 (1): 38-41.
4. Rogers ES, Chamberlin J, Ellison ML, Crean T. A consumer-constructed scale to measure empowerment among users of mental health services. Psychiatr Serv 1997; 48:1042-7.
5. Center for Psychiatric Rehabilitation. Status update of clinical and role functioning. Boston:Center for Psychiatric Rehabilitation, Boston University; 1999.