Susan Bartlett, PhD, on Minimal and Meaningful Change in PROs for RA

In this podcast, Dr Barlett of the McGill University Centre for Outcomes Research and Evaluation, discusses her research in identifying minimal and meaningful change in patient-reported outcomes among patients with rheumatoid arthritis.

Susan Bartlett, PhD, is professor of medicine at McGill University and a senior scientist at the Centre for Outcomes Research and Evaluation.

TRANSCRIPT:

Welcome to this podcast from the Rheumatology & Arthritis Learning Network. I'm your moderator, Rebecca Mashaw. Today, I'm pleased to have Dr. Susan Bartlett from the McGill Centre for Outcomes Research & Evaluation here to talk about her research in identifying minimal and meaningful change in patient reported outcomes among patients with rheumatoid arthritis. Thanks for your time today, Dr. Bartlett.

Dr Bartlett:  Delighted to be here, Rebecca.

RALN: In a recent article about your research, you stated that the objective of the study was to evaluate the responsiveness of patient-reported outcomes, measurements, information system, or PROMIS short forms, and to identify the meaningful and minimal score changes. What led you to choose this subject?

Dr Bartlett: I've always been interested in patient-reported outcomes, or PROs as we tend to refer to them, because they really help us to understand how patients feel and function. And they let us know if a person is responding to treatment as well as we had hoped. So what a good PRO does is it puts a number on what the patient is telling us. In the same way that a thermometer would give us an indication of a person's temperature, for example, or a weight scale tells us somebody's mass, a PRO puts that number on various symptoms that they might experience related to their rheumatoid arthritis or the ways in which they're able to function in their everyday life. The measurement is associated with some error though. And so when you're looking at a change in a patient reported outcome measure, you need to understand how much of that change might be due to just error, the error that's inherent in the tool. That's minimal change.

And then there's the idea of how much change is actually meaningful and meaningful to whom. So this notion of responsiveness or meaningful change has been around for a while, but it was often defined rather arbitrarily either using statistical methods like a half of standard deviation or other complex statistical formulas. There's growing emphasis now though, on looking at outcomes that matter to patients and understanding the amount of change that is relevant to a patient. And so that's what we did in our study, is we compared meaningful change from the perspective of patients, rheumatologists, and then using the traditional approach to evaluating change.

RALN:  you give us a brief overview of your study?

Dr Bartlett: In our study, we had several hundred patients from a number of different settings. Our group of patients was actually quite diverse and we did that specifically because we wanted to represent a broad group of people with rheumatoid arthritis. So we had people who were younger and older, people who were relatively new in their journey with RA, they'd only had it for a year or two. And we had some people who had had it for more than 50 years. Some people had minimal disability and others, particularly people who had had it before we had really good drugs that could help change disease trajectories, some of those people had considerable disability. So we had a good, robust sample of several hundred people. And then we were following them over time as they were receiving their regular clinical care.

Now, one of the things we asked them was compared to your last visit, how would you rate your RA? Is it a little better, a little worse, the same, a lot better or a lot worse? And then we looked at the change in their PRO numbers to see what that level of change was correlated with. Let me say a little bit about the measures that we use too, which are the NIH PROMIS measures. Starting in about 2004, NIH has invested a considerable amount of money. And by that, I mean about $800 million into improving the outcome measures that we use in clinical trials and in patient care. And so our study was focusing specifically on these NIH PROMIS measures. We were the first to do so in rheumatoid arthritis. And because these measures are now widely used around the world, we felt it was really important to identify minimal and meaningful change. So clinicians would be able to use these measures more effectively.

RALN: What were your top line findings?

Dr Bartlett: We found a number of things, several of which were interesting. One was there was a difference in the direction of change. So when patients were getting worse, the amount of change they were reporting was smaller. And what that suggested to us was that patients were more attuned to feeling worse. They were paying more attention to when they were feeling worse and it took less change for them to recognize and kind of define it, that I'm a little bit better or a lot worse. Those changes were in the order of anywhere from 2 to 4 points on average. Physicians on the other hand required even smaller amounts of change to judge the patient as worse. But those differences were relatively small. So I think on average, what we were seeing was that for worsening somewhere between 2 to 4 points on average. For improvement, it was a different story.

When patients reported they were feeling a lot better, their scores were changing anywhere from 6 to 12 points. What that suggested to us is that patients need to see a larger amount of change before they're really convinced that things are going in the right direction. For the rheumatologists, the amount of change for improvement was about the same, anywhere from 6 to 12 points. So in brief, what we're finding is that when patients do come in and they say I'm a little bit better, I'm a lot better, there's a real difference numerically between that. A little bit better, somewhere around 1 to 3 points, a lot better, it could be anywhere from 6 to 12 points on average.

RALN:  Was that surprising?

Dr Bartlett: We are not the first to report that there's this asymmetry with change, that it's easier to notice worsening, or perhaps maybe more accurately we're hardwired to pay more attention to when symptoms are getting worse than when they improve. But the magnitude of change, I think did surprise us. One other thing that we noticed that I guess wasn't that surprising in the long run was that patients were clearly able to differentiate when they were feeling a little better from a lot better and a little worse from a lot worse, but we didn't see that with the rheumatologists.

What we saw in fact was a lot of overlap between when rheumatologists reported that the patient was the same versus a little worse or a little better whereas the numbers were quite distinct when patients were making those assessments. What does that say to us? I think it makes sense. Patients are much more attuned to and better able to detect when their disease is in fact, a little bit better or a little bit worse. And as clinicians, of course, we're relying on the patient report. And so there's less clarity when that is happening.

RALN: So the patients are reporting their symptomatic state with rheumatoid arthritis and the rheumatologists are relying on that, but are there other markers as well that rheumatologists should be looking at that would help them correlate these changes?

Dr Bartlett: Well, that's an excellent point, Rebecca. So we're focusing on the patient report or the patient experience, but rheumatologists also are looking at other things. They're looking at their objective findings. So they're looking at the joints specifically in the joint exam that they conduct to see if the joints look different. And the rheumatologists are also relying on their memory, was the patient the last time I saw them. I think it does make sense that patients are much better able to remember how they felt at the last visit than the rheumatologist necessarily remembers because a rheumatologist is seeing many, many patients over the course of the week.

But I think what reassured us from this research is that those changes in the values for the scores we were seeing were very robust and very consistent across all groups. And so that I think is an important message for clinicians, that patients really are able to assess what's happening and to report reliably. And we can trust those reports on symptoms and frankly, we have to be able to trust those reports. It does suggest to us the measures are working just as we hope that they would, that they're reliable, that they're valid, and that they're responsive to change. So when a patient changes in state, those changes will be reflected on the measures of the score.

RALN: So that more or less answers my next question, which was how might your results affect clinical rheumatology practice. And I guess one of the key ways is that clinicians can now trust that what they're hearing from their patients is, as you say, reliable.

Dr Bartlett: It's reliable and it's valid. And it's very important information. Think of the situation where somebody is being started on a new treatment. And one of the questions, both the patient and the clinician has right away is, is this making a difference? Is this new treatment working the way in which we hope it will work? And we often will see patients fairly soon after a change is made just to double check that things are going in the right direction.

I think what our study shows rather conclusively, is that when a patient comes in and reports, I feel like I am a little better, or I feel like I am a little worse that, that's really important information. And that's information that actually may be much more sensitive than the rheumatologist might pick up through looking at the lab work or looking at the joints in particular. When patients tell us things are moving in the right direction, I think we can trust that that change is occurring. And of course, wait to see as time goes on, to make sure that this persists, but it is good reassurance that our measures are indeed helpful and are giving us additional information, giving clinicians additional information beyond what the lab values and what the physical exam is telling them.

RALN: What’s next on your agenda for research in this area?

Dr Bartlett: We've been doing a lot of work with the PROMIS measures to try and understand what are the measures that are most important to patients. And I think those have been pretty well identified. Fortunately, they're also the measures that are part of a PROMIS profile. A PROMIS profile is 29 questions. And those 29 questions can be completed in as little as 3 minutes in a waiting room. With the PROMIS profile, you're getting information on how patients are feeling physically, emotionally, and their social health. Are they able to do the things that they want to do and not just the ADLs, but are they able to participate in the aspects of life that bring meaning to life? Can they do the things they want to be able to do? Can they garden? Can they go to sporting events? Do they have enough energy at the end of the week, for instance, to have friends and family over for dinner?

So what we're finding is that by having patients complete these measures, either at home or even in the waiting room, we can get a good snapshot of how they're functioning overall. Three to 4 minutes of the patient completing the PROs can give the equivalent information to probably much longer of a discussion between patients and the clinicians. And why this is so helpful is that we found in our clinics at Hopkins that 80% of the time what the PROs was telling us was that the patient was doing really well overall. Their scores were exactly what we would expect them to be. And that was good reassurance for the clinician who asked the patient, how they were doing, and the patient said, I think I'm doing okay. Now we've got some additional evidence that supports that. That allows a clinician to have perhaps a briefer visit with that patient and save the extra couple of minutes for that 1 in 5 who's coming in whose not doing all right, whose condition really is changing. And for whom more time is needed to tease out what's going on and whether a change in treatment might be indicated.

RALN: And do the patient answers in those cases to the PRO questionnaire help direct that conversation between the doctor and the patient? Does that help the doctor focus on the issues that really require more discussion and more in-depth examination?

Dr Bartlett: So we've actually done specific work on that. That's published in another paper, but we look specifically at that. We asked both patients and clinicians, did they find it was helpful? Did it focus the conversation on things that were meaningful? And we heard from patients, it absolutely did. In fact, they really liked the fact that it got them to think about a lot of things. How were they sleeping? How did they feel overall? What was their energy level? And so in thinking about that before they met with their doctor, they were primed in some ways to talk about the things that were important to them. For the rheumatologists, we heard much the same thing. First it was a really good snapshot of how the person was doing. And so as they began the conversation with the patient, looking at the dashboard, they were able to understand what to focus in on if something wasn't where it should be.

For instance, if you saw that sleep was problematic in the last few weeks, but everything else looked fine, that was a place to maybe begin the conversation about what's going on in the person's life. And for example, this time of year, if you're an accountant you're involved in a lot of tax work. And so maybe you're getting less sleep because of extra demands at work. Is that going to impact how you feel, how your RA, how well controlled your RA is? Probably not. But on the other hand, if a clinician noticed that sleep was impaired, fatigue was much higher, patients were reporting more pain, and more problems with physical function, then that of course would alert them to the fact that disease control might be changing. And this might be a person for whom, a few more minutes talking about what's going on and what seems to be happening may be really important.

RALN: Well, I hope that we can get together again and discuss this other study that you just mentioned and go into more detail with that. But for today, I really appreciate your time. And I look forward to talking with you again.