Association Between Nursing Home Compare Five-Star Ratings and Clinical Outcomes After Hip Fracture

The Centers for Medicare & Medicaid Services (CMS) launched the Five-Star Quality Rating System (FSQRS) on the Nursing Home Compare (NHC) website in December 2008.¹  Every skilled nursing facility (SNF) that participates in Medicare and Medicaid receives a rating in the following domains: (1) Overall Quality (Overall); (2) Health Inspection (HI); (3) Staffing; (4) Registered Nurse (RN) Staffing; and (5) Quality Measures (QM). The primary goal of the ratings is to serve as an easily accessible, user-friendly tool for residents and families to compare nursing home quality.²

The FSQRS has received criticism, however, because limited evidence supports the reliability and applicability of ratings. Current data suggests a lack of sensitivity and calls into question the validity of the rating system.^3-12 In one study, accredited SNFs had higher HI, Staffing, and QM ratings, but paradoxically lower Overall ratings, compared with nonaccredited SNFs.¹² Despite concern, CMS continues to publish FSQRS ratings for public use.

The association between the FSQRS and clinical outcomes is largely unknown. Existing data are inconsistent and contradictory. Neuman et al reported Staffing and HI ratings were not significantly associated with readmissions or death after acute care hospitalization.⁷ Similarly, NHC ratings did not predict 90-day readmissions or complications in patients after total joint arthroplasty.⁹ In contrast, a retrospective analysis of patients who received total joint arthroplasty reported an inverse relationship between Overall ratings and readmissions.⁴ Studies of patients with heart failure, meanwhile, support an association between higher ratings and improved outcomes. One such study found patients at SNFs with 1-star Overall ratings had a higher risk of mortality and 90-day readmissions compared with patients at SNFs with 5-star Overall ratings.¹⁰ As the use of quality measures continues to grow in health care, assessing the relationship between the FSQRS and clinical outcomes becomes increasingly important.

In the United States, hip fracture is a major public health concern in older adults. More than 90% of hip fractures occur in patients aged 50 years or older.^13,14 Braithwaite et al estimated the lifetime attributable cost of hip fracture is $81,300 per patient.¹⁵ Hip fractures are associated with 1-year mortality rates that range from 26% to 30% and 30-day readmission rates of 9% to 18%.^16-22  Predictors of readmission include age, male sex, electrolyte abnormalities, time to surgery, length of stay, kidney disease, cardiovascular disease, dementia, malignancy, Cumulative Illness Rating score, and Charlson Comorbidity Index (CCI).^20,22,23 Increased risk of mortality is associated with lab abnormalities, race, dementia, fracture type, pre-fracture function, and readmission after hip fracture.^20,24,25 After hip fracture, the majority of patients are discharged to short- or long-term care facilities. Just 9% of patients are discharged home, after a hip fracture, while 53% to 69% are discharged to SNFs.^16,17,19,22

This study explores the association between FSQRS ratings and 1-year readmissions, emergency department (ED) admissions, and mortality in older adults after hip fracture.

This retrospective analysis compares outcomes among patients with hip fracture treated at a level 1 trauma center and discharged to SNFs rated with three stars or less with outcomes of patients discharged to SNFs rated with four or five stars in Rhode Island.

Study Design and Patient Selection

A retrospective chart review was conducted of patients aged 65 years or older treated for hip fracture at a single-center hospital from January 7, 2012, to January 8, 2014, who were discharged to SNF care. The study was approved by the hospital institutional review board.

There were 325 patients admitted with hip fracture. Of these patients, five died during hospitalization (n=320), and one was excluded due to numerous readmissions (n=319). Other patients were excluded based on discharge location: six to hospice (n=313), 16 to home (n=297), 36 to inpatient rehabilitation (n=261), 15 to facilities outside of Rhode Island (n=246), and one who remained hospitalized (n=245).

NHC Five-Star Quality Rating System

The HI rating is a weighted score that factors 3 years of documented deficiencies and compliance findings from yearly unannounced health inspections and the SNF’s relative performance within each state. The Staffing rating is based on self-reported RN and total nursing hours (ie, RN, licensed practical nurse, and nurse aid hours). RN and total nursing hours are equally weighted and case-mix adjusted for resident needs. The QM rating reflects SNF performance on 13 Minimum Data Set quality measures for long- and short-stay residents.

The Overall rating is a composite of HI, Staffing, and QM ratings that is calculated as follows: CMS starts with the HI rating and adds one if the Staffing rating is four or five stars and greater than the HI rating. If the Staffing rating is one star, CMS subtracts one. If the QM rating is five stars, CMS adds one to the Overall rating; CMS subtracts one if the QM rating is one star. If the HI rating is one star, the Overall rating cannot be greater than two stars. The Overall rating cannot be less than one star or more than five stars.²

Data Collection

Data was collected from the American Trauma Society registry and LifeLinks electronic records. Discharge location and date were obtained from LifeChart using Epic.²⁶ Baseline demographics recorded included age, body mass index (BMI), sex, race, and number of CCI comorbidities.²⁷ Antidepressants, anticholinergics, antipsychotics, and sedatives were recorded as at-risk medications.²⁸ Abnormal lab values (eg, sodium, potassium, creatinine, hematocrit) on admission, hospital length of stay, and complications (eg, pneumonia, urinary tract infection, heart failure) were recorded for each patient.²⁹

NHC star data were collected from the NHC website. The NHC Five-Star Ratings 2009-2013 and NHC data archive publish ratings monthly.^30-31 The SNF rating in each category was recorded for the month of patient discharge.

There is no standard for comparing outcomes by star ratings, with past studies stratifying SNFs in multiple ways.^4,7,9,10 The NHC website defines an Overall star rating of three as average.¹ In this study population, 50% of patients were discharged to SNFs with 4- to 5-star Overall ratings. SNFs were stratified into 1- to 3-star facilities (below average and average) or 4- to 5-star facilities (above average) for each rating category.

Primary outcomes included mortality, ED visits, and hospital readmission within 1 year for patients discharged to SNFs with 1- to 3-star and 4- to 5-star Overall ratings. Mortality was defined as patients who died at home, SNF, or hospice within 1 year of discharge. The Social Security Death Index was used to verify living status of patients who were not confirmed to be alive at 1 year. Living status, cause, and location of death were recorded.³² The status of patients not found on the death index was obtained through the vital records department at the Rhode Island Department of Health. ED visits after initial discharge were defined as any ED visit that did not result in admission. The first three ED visits were accessed through LifeLinks records. Readmissions were defined as any admission to Lifespan institutions, and the first three dates until 1 year after initial discharge were recorded.

Secondary outcomes included analyzing the association of mortality, ED visit, and readmission rates with discharges to SNFs with 1- to 3-star and 4- to 5-star ratings in the HI, QM, Staffing, and RN Staffing categories.

Statistical Analysis

All analyses were conducted using SAS software, version 9.3. Chi-square, Fisher exact, and Student’s t tests were used for bivariate analysis to report demographics, comorbidities, and star ratings. Covariates that were significantly associated (P=<.05) with the Staffing rating were entered into a logistic regression model.

Power analysis showed this study had more than 99% power to detect differences at 0.05 level in at least one ED visit when discharged to SNF with a 4- to 5-star rating compared with a 1- to 3-star rating.

Demographic Data

The final data set consisted of 245 patients: 122 were discharged to SNFs with 1- to 3-star Overall ratings and 123 were discharged to SNFs with 4- to 5-star Overall ratings. The groups were relatively homogenous with similar baseline demographic characteristics (Table 1). The mean age was 84.4 years (P= .7608), and men made up 24.6% of patients discharged to 1- to 3-star SNFs and 25.2% of those discharged to 4- to 5-star SNFs (P= .9116). There was no significant difference in race or BMI between patients discharged to SNFs with 1- to 3-star and 4- to 5-star Overall ratings (P= .3686, .1814, respectively).

The CCI was 1.9 for both groups (P=.9661). There were no significant differences between groups for labs collected and use of at-risk medications at admission (Table 1). Average hospital length of stay was 5.7 days for each group (P=.9686). Complications of pneumonia (1-3 star: 3.3%, 4-5 star: 1.6%; P=.4463), urinary tract infection (1-3 star: 9.0%, 4-5 star: 6.5%; P=.4623), and heart failure (1-3 star: 4.9%, 4-5 star: 3.3%; P= 0.5394) were similar between groups.

Primary Outcome

Patients discharged to SNFs with above-average Overall ratings had similar 1-year outcomes as patients discharged to SNFs with lower Overall ratings. Multivariate analysis showed no difference in mortality (P=.3581), readmission (P=.1412), or ED visits (P=.4933; Figure 1).

Secondary Outcomes

A higher Staffing rating was associated with a higher rate of ED visits. Patients discharged to SNFs with 4- to 5-star Staffing ratings were 70% more likely to have at least one ED visit compared with those discharged to SNFs with 1- to 3-star Staffing ratings (P=.0384). However, the rate of readmission at 1 year was not significantly different between groups; 34.2% of patients discharged to SNFs with 1- to 3-star Staffing ratings had readmissions compared with 46.3% at SNFs with  4- to 5-star Staffing ratings (P=.1528). Mortality rates among patients discharged to SNFs with 1- to 3-star Staffing ratings and 4- to 5-star Staffing ratings were also similar (P=.7086; Figure 2).

Patients discharged to SNFs with higher QM ratings had similar mortality rates, ED visits, and readmission rates than those discharged to SNFs with lower QM ratings (Figure 2). The mortality rate at SNFs with 4- to 5-star QM ratings was 16.7% compared with 12.4% at SNFs with 1- to 3-star QM ratings (P=.3652). The proportion of patients with at least one ED visit was 19.9% at SNFs with 4- to 5-star QM ratings compared with 16.9% at SNFs with lower QM ratings (P=.5607). Readmission rates among patients discharged to SNFs with 4- to 5-star QM ratings were 46.8% compared with 40.5% among those discharged to SNFs with 1- to 3-star QM ratings (P=.3364).

Mortality rate, ED visits, and readmission rates were also similar in patients discharged to SNFs with 1- to 3-star and 4- to 5-star HI ratings (P=.2410, .2920, and .8711, respectively) and RN Staffing ratings (P=.5963, .6165, and .9501, respectively; Figure 2)

Readmission rates for Overall and Staffing ratings were plotted on a diamond graph adapted from Li et al (Figure 3).³³ Higher readmission rates were skewed in favor of higher Staffing ratings.

The CMS FSQRS remains controversial because ratings do not consistently correlate with clinical outcomes or patient satisfaction.^7,10,34 Despite a lack of supporting evidence since implementation, the government continues to market the ratings as a quality assessment tool. Furthermore, some SNFs use FSQRS ratings to guide quality improvement efforts.^35,36

This study contributes to the limited data on association of SNF ratings with patient outcomes. Similar to previous studies, we observed no significant difference in outcomes between patients discharged to SNFs with higher Overall ratings (4-5 stars) compared with those discharged to SNFs with lower ratings.^7,9,34 However, we did observe a trend in which patients with hip fracture discharged to above-average (4-5 star Overall rating) SNFs had lower 1-year mortality, ED visit, and readmission rates. This trend is consistent with retrospective studies that reported higher Overall ratings were associated with lower mortality and readmission rates.^9,10 Although not statistically significant, the primary outcome of this study may provide insight into the relationship between FSQRS ratings and outcomes in patients with hip fracture. Currently, there are no data to guide health care professionals or patients considering SNF placement after hospitalization for hip fracture. Future studies with larger study populations and across multiple states are needed to further explore this trend and possibly define the relationship between FSQRS ratings and outcomes after hip fracture.

In the present study, a higher Staffing rating predicted an increase in ED visits. Patients discharged to above-average SNFs with 4- to 5-star Staffing ratings were 70% more likely to experience an ED visit compared with patients discharged to 1- to 3-star SNFs. Staffing ratings, based on total nursing hours, are dependent on expected patient need. SNFs with sicker patients require more staffing hours and consequently receive higher Staffing ratings. The importance of staffing hours in providing quality of care is widely accepted, with experts recommending minimum staffing hours at SNFs.³⁷ All SNFs are required to meet staffing hour requirements to receive Medicare reimbursement. The findings in this study suggest increased staffing hours do not reduce the rate of ED visits. This is supported by recent data suggesting there is no consistent relationship between quality of care and nurse staffing in SNFs.³⁸ When choosing an SNF, patients and providers should be aware that higher Staffing ratings are not consistently correlated with improved outcomes after hip fracture.

This study also observed a trend between higher Staffing ratings and higher readmissions. When readmission rates by Staffing vs Overall star ratings were plotted on a diamond graph, Staffing ratings were the more sensitive indicator of readmission rate (Figure 3). This observation that the Staffing rating, which is based solely on total staffing hours, may be more predictive of readmission rates than the composite Overall rating calls into question the validity and sensitivity of the FSQRS. Readmission rates are often used as markers of quality of care.^39,40 CMS recently implemented the SNF Value-Based Purchasing Program, an initiative in which SNFs receive incentive payments based on performance on hospital readmissions.⁴¹ As readmission rates become increasingly important in health care reimbursement models, the need for a validated and reliable system to predict readmissions is clear.

This study observed no significant differences between mortality, ED visits, and readmission rates in patients discharged to SNFs with 1- to 3-star and 4- to 5-star ratings in HI, QM, and RN staffing. When comparing trends in outcomes among rating categories, the data was inconsistent and contradictory. For example, patients discharged to SNFs with 4- to 5-star Overall ratings had better outcomes than patients discharged to SNFs with lower Overall ratings. In contrast, patients discharged to SNFs with 4- to 5-star QM
ratings had worse outcomes than those discharged to those with 1- to 3-star QM ratings. Given the composite nature of the Overall rating, one would assume outcomes at SNFs with higher Overall and QM ratings would correlate. The lack of statistical significance between lower ratings and unfavorable outcomes, coupled with the variability among domain-specific ratings, further casts doubt on the validity of the FSQRS.

This retrospective chart review has limitations. In July 2016, CMS added five new short-stay measures and implemented methodological changes to the QM domain. The impact of these minor adjustments remains unknown. Due to the state-specific nature of the ratings, this research only applies directly to Rhode Island. However, the rating system is universal, and this study could be replicated in other states. Finally, if a patient was treated at a non-Lifespan affiliate or an out-of-state hospital, the readmission or ED visit would not be included in the data set. However, this scenario is unlikely because Lifespan serves as the major health care provider for Rhode Island, with approximately 697,377 outpatient visits and 250,613 ED visits in 2016.⁴²

This study observed no significant association between CMS Overall ratings and 1-year outcomes in patients discharged to SNFs after hip fracture. Above-average Staffing ratings were associated with an increase in ED visits. There was lack of consistency in outcomes when comparing Overall, Staffing, HI, and QM ratings. In evaluating the relationship between the FSQRS and outcomes in older adults after hip fracture, this study builds upon the existing literature calling into question the validity of the FSQRS. Health care professionals, patients, and their families should approach these ratings with prudence until future studies with larger cohorts across many states can better characterize the relationship between the NHC FSQRS and clinical outcomes.

Authors: Patricia Giglio, MD¹ • Nadia Mujahid, MD^1,2 • Joao Filipe G Monteiro, PhD² • Lynn McNicoll, MD^1,2

Affiliations:
¹Warren Alpert Medical School of Brown University, Providence, RI
²Department of Medicine, Rhode Island Hospital, Providence, RI

Disclosures:
This work was supported by the Brown Univer- sity Summer Assistantship. It had no influence in the design and conduct of the study; col- lection, management, analysis, and interpreta- tion of the data; or the preparation, review, or approval of the manuscript. Dr Giglio has no disclosures.
Dr Mujahid reports serving as a paid consultant for AGS Co-Care Consultant, CVS, and Neighborhood Healthcare. Dr McNicoll has served as a consultant for AGS CoCare: Ortho Project. Dr Monteiro has served as a paid consultant for Brown Medicine.

Acknowledgments:
The authors would like to thank Drs Chris- topher Born and Roman Hayda from the Department of Orthopedics, Rhode Island Hospital, for their collaboration with us, as well as Ms Anne George and Ms Naomi Hagan from the Trauma Registry, who helped us with the data collection. We greatly ap- preciate the time and effort the GFP Steering committee members and the providers and staff on the GFP unit have put into this ongo- ing collaboration.

Address correspondence to:
Patricia Giglio
Warren Alpert Medical School of Brown University
222 Richmond Street
Providence, RI 02903

Phone: 978-884-2087
Email: Patricia_giglio@brown.edu