ADVERTISEMENT
Quality Measurement in Oncology: Is the Race Worth Running?
There is good news and there is bad news on efforts to measure quality of care in cancer.
The good news: numerous innovations in treatment are improving the health outcomes across a variety of cancers. Immuno-therapies, for instance, have improved survival outcomes for patients across a variety of tumors. CAR-T therapy uses with genetically engineered immune cells to fight cancer. These innovations are not “me too” incremental advances. One study showed that CAR-T therapies delivered large health gains and “broke the trend” of modest health improvements for many previous treatments for hematological cancers. Cancer mortality rates are falling in large part due to these new, innovative treatments.
The bad news: measuring quality of care in cancer has now become a moving target. A recent commentary by Valuck, Blaisdell, and Schmidt (“Catch Me If You Can: Aligning Quality Measurement With Oncology Innovation”) highlights a number of the challenges in measuring quality in a disease area with so much innovation. The challenges mentioned included: the long time-lag to develop new guidelines and then incorporate these into validated quality measures, the potential narrow populations to which guidelines would be applied in a world of precision medicine, the need to use real-world data to better extrapolate clinical trials into the real world, and better incorporating patient preferences into decision-making. All of these challenges are real.
In this commentary, however, I will ask a more fundamental question: is the quality measurement race worth running? In other words, is it worth continually updating quality measures in such a disease area where innovation moves so fast? Before answering this question, measure developers should consider at least three key factors.
First, measure developers need to consider the degree to which physicians have access to information not available to administrators. If all patients were all the same and measure developers had exactly the same information as physicians about individual patients, then creating quality measures is sensible. In the real world, however, patients may have multiple comorbidities, which may make simplified quality measures irrelevant or even inappropriate. Additionally, patients have different preferences. Some patients prefer more efficacious treatments even if they are at higher risk of adverse events; other patients are willing to sacrifice efficacy for a better quality of life. These preferences are difficult for measure developers to capture. Valuck, Blaisdell, and Schmidt rightly argue that patient-reported outcomes that measure quality of life may be difficult to implement in practice because of inaccurate risk adjustment, and instead advocate for structural measures capturing providers ability to collect data on patient preferences. Simply capturing structural measures, however, is an inadequate solution to how physicians in the real world discuss treatment options with patients. Economist Charles Manski argues more broadly that physicians will always have more information on individual patients than administrators or measure developers. For instance, physicians may know more about a specific patient’s clinical situation, preferences, likelihood of being adherent to recommended treatment and other factors. This information asymmetry limits the utility of top-down quality measures.
Takeaway #1: Measure development is more useful when patients are more homogeneous and measure developers have similar levels of information as physicians. Measures development is less useful when physicians have significant patient-specific information about a patient’s clinical state, preferences, or other dimensions that are not available to measure developers.
The second consideration is the cost of measure development. Developing measures is not a costless exercise. The extensive research needed to develop these measures, however, represents just a small share of the cost of actually implementing a quality measure regime. According to the Medicare Payment Advisor Commission (MedPAC), having a large number of measures creates “unneeded complexity” for providers. One study found that the cost physicians to comply with quality reporting measures was $15.4 billion. Another study found that it would take physicians 7.4 hours per working day to comply with preventive care recommendations from US Preventive Services Task Force (USPSTF); in other words the USPSTF guidelines barely leave physicians any time in the day to address the issues that caused the patient to come to the doctor’s office in the first place.
While Valuck, Blaisdell, and Schmidt make the case that broad, cross-tumor quality measures are needed to simplify data collection, solving one problem creates others. Simple cross-cutting measures do not take into account that cancer patients have different tumor types, tumors at different stages, are being treated in different lines of therapy, have different comorbidities, have different levels of adherence and differ on many other dimensions. Risk adjustment is generally insufficient to overcome these barriers. Thus, balancing between simpler but cruder cross-cutting measures against a multitude of detailed measures that add complexity is difficult. Regardless, measure developers should be taking into account the cost of measure development and implementation.
Takeaway #2: Measure developers should examine not only the potential clinical benefits of adopting best practices, but also the cost for physicians and hospitals to comply with such measures.
Even if central planners had perfect information and quality metrics were costless to collect, one must still worry about unintended consequences. Health care quality is a multi-dimensional outcome and evaluating providers based on measures that only capture a subset of quality is inevitable. Nobel prize winner Bengt Holmstrom and his co-author Paul Milgrom outline this problem in their 1991 paper. They argue that in many industries, individuals are paid a salary even when outcomes are observable because employers do not want their employees to provide low effort on unmeasured dimensions of quality. Consider an example in the health care area. A 2012 JAMA article by Zgierska, Miller, and Rabago showed that evaluating physicians based on patient satisfaction increased prescriptions for opioids and other addictive medications. The reason is that physicians who were stricter with opioid prescribing were more likely to receive lower patient satisfaction scores; thus, patient satisfaction metrics may have worsened the opioid crisis.
Further, using quality metrics to incentivizing physicians to adopt identical treatment patterns may also have unintended consequences. Consider the case where there are two new drugs: Drug A and Drug B. Assume that Drug A is superior to Drug B in clinical trials but in the real-world Drug B is superior to Drug A for a subset of patients not captured in the trial. In this case, only by allowing variability in practice patterns could researchers identify this disconnect between the clinical trial and real-world results. If all physicians were coerced to prescribe Drug A based on guidelines or quality measures, there would be no variability from which researchers could draw inference.
Takeaway #3: Measure developers should consider not only what quality measures incentivize physicians to do, but the unintended consequences of what dimensions of quality physicians will be incentivized to ignore.
In short, the Valuck, Blaisdell, and Schmidt commentary does an excellent job of identifying some of the upcoming issues in cancer measure development due to the increasing pace of innovation. In some cases, dedicating more resources to improve quality measurement is the right way to go. In other cases—and even perhaps in most cases—abandoning quality measurement and trusting doctors judgement may be preferred.
