Skip to main content
Chief Medical Editor Message

Assessing Performance

March 2017

I recently flew into Ronald Reagan Washington National Airport to attend a professional event, arriving early so that I would have time to visit my family there, too. Because my scheduled arrival was so early, unlike many of my fellow passengers, I was not concerned that we were landing an hour behind schedule. Returning home the next day there was another flight delay, but fortunately, I was not rushing back for anything important.

The airline I flew, American Airlines, did not have the best ontime record. This past summer, based on data from the US Bureau of Transportation Statistics, American had briefly dropped to the bottom among US airlines with fewer than 75% of flights on-time.1

Being on time should not be that hard, should it? Hawaiian Airlines apparently has it figured out. Hawaiian Airlines routinely has over 90% of their flights land on time; in 2016, they canceled only 9 of 6347 flights. Also in 2016, Hawaiian Airlines remained the nation’s top carrier for punctuality for the 13th consecutive year.2

But here is the rub. Hawaiian Airlines does not have a lot of flights in and out of busy Northeast US airports that are plagued by bad weather. The month that American Airlines had a poor on-time record, its Dallas/Fort Worth hub was beset by thunderstorms. An airline could be doing everything right and still score poorly because of factors out of its control, like the weather in the cities it serves.

The idea of tracking performance and rewarding better outcomes is coming to medicine. The theory is very appealing: we can incentivize people to provide better care by measuring performance and rewarding better outcomes. In practice, this is more difficult than it looks at first.

Doctors providing equally good service might get very different ratings—for objective treatment outcomes and for subjective patient satisfaction—simply because of differences in their patient populations. Controlling for differences in the patients is not straightforward or easy.

If airlines were paid primarily for their on-time record, airlines might schedule more flights to Hawaii and fewer to New York, Philadelphia, and Washington, DC. If performance were a major determinant of how much health care professionals are paid, there would be a strong incentive to only care for healthy patients, easy to treat patients, and patients who adhere to their treatment, not to the sickest patients most in need of care.

The goal of rating physicians and paying more for better outcomes is a laudable goal, but an effective system has many hurdles to address. Just as with airlines, simple measures of performance will not tell the whole story and could be very misleading. 

Steven R. Feldman, MD, PhD
Chief Medical Editor

Dr Feldman is with the Center for Dermatology Research and the Departments of Dermatology, Pathology, and Public Health Sciences at Wake Forest University School of Medicine in Winston-Salem, NC.

References
1. Rivers M. Does American Airlines really have worst on-time performance rate in the U.S.? August 20, 2016. Forbes. https://www.forbes.com/sites/martinrivers/2016/08/20/does-american-airlines-really-have-the-worst-on-time-performance-rate-in-the-u-s/. Accessed March 1, 2017.
2. Hawaiian Airlines’ passion for punctuality [news release]. Honolulu, HI. Hawaiian Airlines. February 14, 2017. https://newsroom.hawaiianairlines.com/releases/hawaiian-airlines-passion-for-punctuality. March 1, 2017.

I recently flew into Ronald Reagan Washington National Airport to attend a professional event, arriving early so that I would have time to visit my family there, too. Because my scheduled arrival was so early, unlike many of my fellow passengers, I was not concerned that we were landing an hour behind schedule. Returning home the next day there was another flight delay, but fortunately, I was not rushing back for anything important.

The airline I flew, American Airlines, did not have the best ontime record. This past summer, based on data from the US Bureau of Transportation Statistics, American had briefly dropped to the bottom among US airlines with fewer than 75% of flights on-time.1

Being on time should not be that hard, should it? Hawaiian Airlines apparently has it figured out. Hawaiian Airlines routinely has over 90% of their flights land on time; in 2016, they canceled only 9 of 6347 flights. Also in 2016, Hawaiian Airlines remained the nation’s top carrier for punctuality for the 13th consecutive year.2

But here is the rub. Hawaiian Airlines does not have a lot of flights in and out of busy Northeast US airports that are plagued by bad weather. The month that American Airlines had a poor on-time record, its Dallas/Fort Worth hub was beset by thunderstorms. An airline could be doing everything right and still score poorly because of factors out of its control, like the weather in the cities it serves.

The idea of tracking performance and rewarding better outcomes is coming to medicine. The theory is very appealing: we can incentivize people to provide better care by measuring performance and rewarding better outcomes. In practice, this is more difficult than it looks at first.

Doctors providing equally good service might get very different ratings—for objective treatment outcomes and for subjective patient satisfaction—simply because of differences in their patient populations. Controlling for differences in the patients is not straightforward or easy.

If airlines were paid primarily for their on-time record, airlines might schedule more flights to Hawaii and fewer to New York, Philadelphia, and Washington, DC. If performance were a major determinant of how much health care professionals are paid, there would be a strong incentive to only care for healthy patients, easy to treat patients, and patients who adhere to their treatment, not to the sickest patients most in need of care.

The goal of rating physicians and paying more for better outcomes is a laudable goal, but an effective system has many hurdles to address. Just as with airlines, simple measures of performance will not tell the whole story and could be very misleading. 

Steven R. Feldman, MD, PhD
Chief Medical Editor

Dr Feldman is with the Center for Dermatology Research and the Departments of Dermatology, Pathology, and Public Health Sciences at Wake Forest University School of Medicine in Winston-Salem, NC.

References
1. Rivers M. Does American Airlines really have worst on-time performance rate in the U.S.? August 20, 2016. Forbes. https://www.forbes.com/sites/martinrivers/2016/08/20/does-american-airlines-really-have-the-worst-on-time-performance-rate-in-the-u-s/. Accessed March 1, 2017.
2. Hawaiian Airlines’ passion for punctuality [news release]. Honolulu, HI. Hawaiian Airlines. February 14, 2017. https://newsroom.hawaiianairlines.com/releases/hawaiian-airlines-passion-for-punctuality. March 1, 2017.

I recently flew into Ronald Reagan Washington National Airport to attend a professional event, arriving early so that I would have time to visit my family there, too. Because my scheduled arrival was so early, unlike many of my fellow passengers, I was not concerned that we were landing an hour behind schedule. Returning home the next day there was another flight delay, but fortunately, I was not rushing back for anything important.

The airline I flew, American Airlines, did not have the best ontime record. This past summer, based on data from the US Bureau of Transportation Statistics, American had briefly dropped to the bottom among US airlines with fewer than 75% of flights on-time.1

Being on time should not be that hard, should it? Hawaiian Airlines apparently has it figured out. Hawaiian Airlines routinely has over 90% of their flights land on time; in 2016, they canceled only 9 of 6347 flights. Also in 2016, Hawaiian Airlines remained the nation’s top carrier for punctuality for the 13th consecutive year.2

But here is the rub. Hawaiian Airlines does not have a lot of flights in and out of busy Northeast US airports that are plagued by bad weather. The month that American Airlines had a poor on-time record, its Dallas/Fort Worth hub was beset by thunderstorms. An airline could be doing everything right and still score poorly because of factors out of its control, like the weather in the cities it serves.

The idea of tracking performance and rewarding better outcomes is coming to medicine. The theory is very appealing: we can incentivize people to provide better care by measuring performance and rewarding better outcomes. In practice, this is more difficult than it looks at first.

Doctors providing equally good service might get very different ratings—for objective treatment outcomes and for subjective patient satisfaction—simply because of differences in their patient populations. Controlling for differences in the patients is not straightforward or easy.

If airlines were paid primarily for their on-time record, airlines might schedule more flights to Hawaii and fewer to New York, Philadelphia, and Washington, DC. If performance were a major determinant of how much health care professionals are paid, there would be a strong incentive to only care for healthy patients, easy to treat patients, and patients who adhere to their treatment, not to the sickest patients most in need of care.

The goal of rating physicians and paying more for better outcomes is a laudable goal, but an effective system has many hurdles to address. Just as with airlines, simple measures of performance will not tell the whole story and could be very misleading. 

Steven R. Feldman, MD, PhD
Chief Medical Editor

Dr Feldman is with the Center for Dermatology Research and the Departments of Dermatology, Pathology, and Public Health Sciences at Wake Forest University School of Medicine in Winston-Salem, NC.

References
1. Rivers M. Does American Airlines really have worst on-time performance rate in the U.S.? August 20, 2016. Forbes. https://www.forbes.com/sites/martinrivers/2016/08/20/does-american-airlines-really-have-the-worst-on-time-performance-rate-in-the-u-s/. Accessed March 1, 2017.
2. Hawaiian Airlines’ passion for punctuality [news release]. Honolulu, HI. Hawaiian Airlines. February 14, 2017. https://newsroom.hawaiianairlines.com/releases/hawaiian-airlines-passion-for-punctuality. March 1, 2017.