Ana-Maria Orbai, MD, on Agreement Between HAQ-DI and PROMIS-PF
Dr Orbai, from Johns Hopkins University, discusses her examination of how well results from Health Assessment Questionnaire Disability Index and the Patient Reported Outcomes Measurement Information System for physical function agree when used among patients with psoriatic arthritis.
Ana-Maria Orbai, MD, is an assistant professor in the rheumatology division and director of the Psoriatic Arthritis Program at Johns Hopkins University in Baltimore, Maryland.
TRANSCRIPT:
Welcome to this podcast from the Rheumatology and Arthritis Learning Network. I'm your moderator, Rebecca Mashaw. I'm delighted to have with us today Dr. Ana‑Maria Orbai, who's an assistant professor in the rheumatology division and director of the Psoriatic Arthritis Program at Johns Hopkins University in Baltimore, Maryland.
She's going to talk to us today about some recent research that she conducted with colleagues on comparing results between the Health Assessment Questionnaire‑Disability Index and the PROMIS‑PF, or the Patient‑Reported Outcomes Measurement Information System for physical function. Thank you for joining us to talk ab out this today, Dr. Orbai.
Dr. Orbai: Thank you, Rebecca. It's a pleasure to be on your podcast again and to share our research. Let me tell you in a few words what we aim to do. This project was a research project led by Dr. Erin Chew, who was a rheumatology internal medicine resident at that time at Johns Hopkins. Now, he's a rheumatology fellow at Vanderbilt.
We have been collecting, for a long time, what we call legacy measures in psoriatic arthritis. The Health Assessment Questionnaire‑Disability Index is one of those measures that are meant to evaluate the patient's disability status over time in psoriatic arthritis.
The measure has been collected in every clinical trial of psoriatic arthritis as part of the FDA requirement for the American College of Rheumatology response indices, which are primary outcomes in these efficacy trials. As we can see from the name of this questionnaire, it's very much focused on disability.
However, since then, the concepts somewhat evolved. We all know that health is not just absence of disability. Health is achieving one's full potential on a number of dimensions and outcomes, from health-related quality of life, health symptoms, and so on.
In the psoriatic arthritis cohort, we set up a study where we administered the Health Assessment Questionnaire index as has been done previously, as well as the newer PROMIS measures. The Health Assessment Questionnaire‑Disability Index measures disability, whereas the PROMIS‑Physical Function measure physical function.
The direction of this is slightly different. The higher the score on the HAQ‑DI, the more the disability. Whereas the higher the score on the PROMIS measures, the higher the physical function capacity.
We administered two PROMIS‑Physical Function measures. One is a short form of only four items. The other one is a physical function computer adaptive test, which can range from anywhere between 4 to 12 items.
First of all, the concept of physical function, as we discussed, we focus on the positive, how much people can achieve on physical function. PROMIS is population normed, meaning it's easy to understand what the scores mean. A score of 50 on PROMIS measures is the average score of the US population. A standard deviation is a difference of 10.
We know that if somebody has a score of 60, their physical function is one standard deviation better than the US population mean. Then if it's 40, it's one standard deviation worse. We immediately can place people on that bell curve for the general population.
The third reason is that we use Epic, which is an electronic medical system at many institutions. This is now standard for documentation and health care. PROMIS measures are programmed in the electronic medical record. Meaning, they are scored instantly as the patients complete a questionnaire.
We could implement our study tools in the regular clinical care flow. We had the Health Assessment Questionnaire. Then, we said, "Let's compare the calculation of minimal disease activity, which is the treatment‑targeted psoriatic arthritis, calculated traditionally using the Health Assessment Questionnaire, versus calculating using PROMIS instead of the HAQ‑DI."
We were able to do this because, before us, Dr. Schalet, Dr. Cella, and their team demonstrated a walkway between the Health Assessment Questionnaire‑Disability Index and PROMIS scores.
As we looked at this walkway, a score of 0.5 on the HAQ‑DI, which is considered the maximum score where somebody doesn't have disability issues in psoriatic arthritis. It's the MDA threshold. If there's more than 0.5, then people don't meet the MDA criterion on that on physical function.
A 0.5 was equivalent to 41.3 in the general population. We decided to use the same cutoff. We asked whether in psoriatic arthritis, a physical function of 41.3 or better, would correctly classify people in MDA. We did classify it as well as using the Health Assessment Questionnaire‑Disability Index.
Basically, in the same set of 100 people with up to 5 longitudinal clinical observations each, we dichotomized, are they in MDA using the HAQ? The only thing that changed was how we defined the physical function criterion. MDA using the HAQ‑DI or MDA using one of these PROMIS scores.
We found excellent agreement between these classifications. Whether we used the HAQ‑DI or PROMIS, we were able to capture those people as in MDA status. It didn't really make a difference whether we used one or the other.
In our paper, we have this graph with the overlapping circles which very nicely demonstrates this concept. After we demonstrated this at every visit, we also wanted to see whether people who changed state...These are people followed longitudinally.
We wanted to make sure that if somebody changes from MDA and not MDA, or the other way around at their follow‑up clinic visits that the calculation wouldn't show that they would be indifferent categories if we use HAQ‑DI or PROMIS. We wanted to make sure there's longitudinal agreement as well when people transition.
The agreement between those was also very high. We measured this using the kappa statistic. One other sensitivity analysis we performed was to look in people with different levels of physical function. People who had more disability versus people who were more functional. There was no difference. The same between pain categories, men, women.
This correspondence held true. Our conclusion was that we can use PROMIS‑Physical Function to replace the HAQ‑DI in the calculation. Whoever is collecting PROMIS through their medical record doesn't need to also collect HAQ‑DI to calculate MDA. We're helping people who are just launching their cohorts.
If some clinics collect HAQ‑DI still and others PROMIS‑PF, we know that there's good agreement between these. We can make those data sets talk to each other having this walkway. Hopefully, as clinical trials pick up on PROMIS measures, we'll also be able to calculate MDA using that physical function measure, which is much more relevant in this day and age, we think.
RALN: How do these tools compare, in terms of the amount of time and effort that's required by the patient or the clinician to administer?
Dr. Orbai: That's an excellent point. If I had to administer the HAQ‑DI now in the clinic, the HAQ‑DI is 20 questions. Each has several response options. We would have to calculate this on paper because I don't have a calculator. That would involve some waiting and adding. It would take quite some time.
Also, what used to happen is patients completed it on paper. Then, the score was really available just for research and once we were ready to analyze the data. Whereas with PROMIS, it's implemented in the electronic medical record. We have the score instantly as soon as the patient finishes their questionnaire on the tablet.
RALN: Is the PROMIS instrument faster and easier to complete as well?
Dr. Orbai: It is. For the short form, they complete 4 questions. For the computer adaptive test, patients complete up to 12 questions, which obviously, is shorter than the 20 HAQ‑DI questions.
Rebecca: In your paper, you mentioned that PROMIS provides an additional anchor for the treat‑to‑target state. Now, considering how important a treat‑to‑target approach has become in all kinds of rheumatic diseases, this could be a critical factor. Can you explain what you mean by this additional anchor?
Dr. Orbai: What we meant here is more the transition towards we're just trying to improve disability, but we are trying to make sure that people achieve their best possible physical function score. If someone used to practice performance sports, we want to make sure that...A great achievement would be to get people back to what their real normal is.
Everybody's normal is different because we are on this curve. In a way, this would be raising the bar of how we can help people. If you just use a disability measure, your ability to show improvement stops. When somebody has a zero disability score, that questionnaire is not going to show you if you're actually doing more than avoiding disability.
PROMIS has no limit in a way because it measures ability. It's very interesting, especially the computer adaptive test, how people get harder questions if they're high performers on physical function. Then, the score zooms in on each individual's level.
RALN: As you mentioned earlier, the HAQ is designed to measure disability, whereas PROMIS has a more positive approach in looking at how are people improving? How physical function is getting better and being able to measure that more accurately in the clinic?
Dr. Orbai: Exactly. It measures ability, which has no ceiling.
RALN: What does this mean for the practicing clinician who is looking for the most efficient ways and effective ways to assess disease activity, to assess improvements, or conversely, to see if a patient is beginning to show signs of more disability? How does this information help that clinician make decisions about the best way to go about making these measurements?
Dr. Orbai: It's very useful. First of all, we have these scores. Then, we have the color codes for how far a patient is from the general population mean. It's very easy to discuss around these scores. The second advantage is the measures come in a package.
At the same time as administering the physical function questionnaire, we administer the fatigue, pain interference, sleep. We can see all of these domains at the clinic visit. We can discuss with the patient. Oftentimes, things are not perfect because of different reasons.
It's not always going to be psoriatic arthritis. Maybe we discover additional issues. This is very helpful in that direction. Also, tracking scores. Sometimes we tend to forget. If the last week before the visit was really bad, we may not realize, overall, we came such a long way. This is not so bad, or the other way around. It's very, very useful.
RALN: It helps with perspective?
Dr. Orbai: Yes.
RALN: Very interesting. Where do you go from here with your research? Are you going to be looking at this in more depth? Do you think PROMIS is showing that it's going to become a dominant way of measuring psoriatic arthritis activity and ability and disability in patients?
Dr. Orbai: I think so. The good news is that PROMIS is now being implemented internationally as well. Questionnaires are being translated, implemented in clinics. Its validity is becoming more universal. Our data and our research questions will be more meaningful because they'll be applicable to many more people.
It's certainly our standard of care at Johns Hopkins, and in the psoriatic arthritis cohort, as the measure of quality of life. We are looking at patients' profiles and health‑related quality of life scores, whether they are in a treat‑to‑target state or not. We're looking at multiple dimensions, not just as physical function.
These questionnaires very nicely distinguish people who are a target and who are not. Potentially, the physician can be warned, as soon as the patient completes the questionnaire that this is someone who needs to come to the office sooner.
The more we work with these measures, we'll be able to understand how to use them to intervene as soon as possible if someone is not doing well.
RALN: Thank you so much for talking with us today. This is very interesting. We'll look forward to what a future research shows.
Dr. Orbai: My pleasure. Thank you for inviting me.
