Validation of STS/ACC TVT-TAVR Score in Veterans Undergoing Transcatheter Aortic Valve Replacement
Abstract: Background. The Society of Thoracic Surgeons (STS)/American College of Cardiology (ACC) transcatheter valve therapy (TVT) transcatheter aortic valve replacement (TAVR) score was developed to predict in-hospital mortality in patients undergoing commercial TAVR in the United States. Veterans Affairs (VA) hospitals are not included in the TVT registry. Methods. The STS/ACC TVT-TAVR score was estimated in 195 veterans undergoing TAVR from 2015-2017. Discrimination was estimated by calculating the area under the receiver operating characteristics curve (AUC) for two outcomes of interest: in-hospital and 30-day mortality. The cohort was then divided into quartiles of TAVR and STS predicted risk of mortality (PROM) scores and long-term mortality was assessed with Kaplan-Meier curves. Results. The mean age of the cohort was 77 ± 8 years and the population was 99% males. The median TAVR and STS-PROM risk scores were 3.1 (interquartile range [IQR], 2.1-5.1) and 4.5 (IQR, 2.6-7.4), respectively. Observed in-hospital and 30-day mortality rates were 2.6% and 4.6%, respectively. The AUCs for the TAVR risk score were 0.68 and 0.64 for in-hospital and 30-day mortality, respectively. During a mean follow-up period of 1.9 years, a total of 37 patients (20%) died. Long-term survival was similar in different quartiles of STS-PROM scores (P=.52). In contrast, patients in the highest quartile of TAVR risk score (8.4; IQR, 5.8-9.9) had significantly worse survival (P<.01). This group included 20 out of the 37 deaths (>50%). Conclusions. Developed and validated in commercial United States cases, the TAVR risk score has a similar performance in the veteran population for predicting short-term outcomes. In addition, the TAVR score predicts long-term mortality. Our results have implications for optimal patient selection.
J INVASIVE CARDIOL 2018;30(12):447-451. Epub 2018 September 15.
Key words: prognosis, risk assessment, transcatheter aortic valve replacement
Having applicable, valid, and reliable procedure-specific risk-prediction tools is paramount to providing high-quality care and obtaining informed consent. As transcatheter aortic valve replacement (TAVR) rapidly becomes the standard of care for patients with severe symptomatic aortic stenosis, developing appropriate procedure-specific risk-prediction tools is clinically necessary. The Society of Thoracic Surgeons (STS) predicted risk of mortality (PROM) score is used in clinical practice to estimate surgical risk prior to surgical aortic valve replacement (SAVR). However, the STS-PROM score has poor correlation with in-hospital and 30-day mortality in patients undergoing TAVR and significantly overestimates mortality risk.1
To overcome these limitations, a TAVR-specific risk score was created and validated as a predictor of in-hospital mortality among patients undergoing TAVR.2 The risk model was developed from the Society of Thoracic Surgeons/American College of Cardiology Transcatheter Valve Therapy (STS/ACC TVT) registry. However, hospitals within the Veterans Affairs (VA) system were not included in the STS/ACC TVT registry, which limits the validity of the TAVR risk score in this unique population. We sought to validate this score by evaluating in-hospital and 30-day mortality in veterans undergoing TAVR within the VA system. In addition, we assessed the discrimination of both STS-PROM and TAVR risk scores to predict long-term mortality after TAVR.
Methods
Setting. The Minneapolis VA Healthcare System (MVAHCS) is a tertiary, 250-bed hospital within the VA Midwest Heath Care Network (Veterans Integrated Service Network VISN 23). The network serves more than 440,000 enrolled Veterans residing in the states of Iowa, Minnesota, Nebraska, North Dakota, South Dakota, and portions of Illinois, Kansas, Missouri, and Wyoming. The MVAHCS is the only approved TAVR program in a nine-state area and has an academic affiliation with the University of Minnesota Medical Center (UMMC). The TAVR program at the Minneapolis VAMC was established in April of 2015 and has performed over 225 procedures to date.3,4
Patients, study variables, and outcomes. We enrolled 195 consecutive patients treated with TAVR at MVAHCS from April 2015 to December 2017. We excluded those patients who underwent TAVR in a non-aortic valve position (ie, mitral valve-in-valve, pulmonary position), but we included those who underwent TAVR for off-label indications (bicuspid, aortic insufficiency) and/or aortic valve-in-valve procedures in the analysis. All patients undergoing TAVR procedures at the MVAHCS are prospectively entered in a database modeled after the STS/ACC TVT registry, version 2.0.5 Variables captured include demographics, baseline comorbidities, laboratory parameters, procedural variables, frailty assessment, quality of life assessed by the Kansas City Cardiomyopathy Questionnaire (KCCQ), in-hospital complications, disposition at discharge, 30-day follow-up, and long-term follow-up. A list of variables collected is provided in Supplementary Appendix A1. Unlike the TVT registry, 30-day and long-term mortality rates are available for all patients in a common electronic medical record system (computerized patient records system [CPRS]). A dedicated research coordinator updates vital statuses biannually by reviewing electronic medical records.
Supplementary Appendix A1
STS/ACC TVT-TAVR risk scores and STS-PROM surgical risk scores were calculated for all patients using available online tools. For patients needing percutaneous revascularization prior to TAVR, the STS-PROM scores were calculated under the assumption they would need coronary artery bypass graft (CABG) surgery in addition to aortic valve replacement. All TAVR risk scores were calculated using the ACC website [https://tools.acc.org/tavrrisk/#!/content/evaluate/]. STS-PROM scores were calculated using the Online STS Adult Cardiac Surgery Risk Calculator utilizing the STS Adult Cardiac Surgery Database, version 2.8 [https://riskcalc.sts.org/stswebriskcalc/#/calculate]. No variables were missing for any patient for either risk score. Components of the TAVR risk score and their respective odds ratios are listed in Table 1.
The primary outcome measure was in-hospital and 30-day mortality. As a secondary objective, we assessed the ability of these scores to predict long-term mortality by comparing different quartiles of TAVR and STS-PROM risk scores.
Statistical analysis. Continuous variables are presented as mean ± standard deviation, or as median with interquartile range (IQR) when appropriate. Categorical variables are reported as frequencies and percentages. Discrimination was estimated by calculating the area under the receiver operating characteristics curve (AUC) for the two primary outcomes of interest (in-hospital and 30-day mortality). An AUC of 0.5 indicated no discriminatory ability, whereas an AUC of 1 indicated perfect discrimination.6 The cohort was divided into quartiles of TAVR and STS-PROM risk scores. Comparison of long-term mortality was subsequently evaluated by Kaplan-Meier curves using the log-rank t-test. Medcalc version 17.2 was used for analysis. This study was approved by the Institutional Review Board of the Minneapolis VA Medical Center. Individual consent requirement was waived.
Results
A total of 195 patients underwent TAVR at MVAHCS during the study period. All patients were included in the analysis. The mean age of the cohort was 77 ± 8 years, and 99% of patients were male. The median TAVR and STS-PROM risk scores were 3.1 (IQR, 2.1-5.1) and 4.5 (IQR, 2.6-7.4), respectively. A previous coronary artery bypass graft (CABG) surgery was present in 31% of patients and 48% had chronic obstructive pulmonary disease. Median baseline aortic valve area was 0.7 cm2 (IQR, 0.6-0.9), and median aortic valve mean gradient was 37 mm Hg (IQR, 28-46). The mean aortic valve peak velocity was 4.2 ± 1m/s and baseline ejection fraction (EF) was 49%.16 Quality of life at baseline was poor (baseline median KCCQ score, 38; IQR, 30-50), with 70% of patients having New York Heart Association (NYHA) functional class III-IV symptoms and a median N-terminal pro-brain natriuretic peptide (NT-proBNP) of 1440 ng/L (IQR, 569-3948).
Transfemoral access and balloon-expandable valves were used in 88% and 80% of patients, respectively, while 53% of cases were done under conscious sedation.
Baseline characteristics are presented in Table 2. Values defining TAVR and STS-PROM risk score quartiles are listed in Table 3.
The observed in-hospital and 30-day mortality rates after TAVR were 2.6% and 4.6%, respectively. The AUCs for the TAVR risk score were 0.68 and 0.64 for in-hospital and 30-day mortality, respectively (Figure 1). The TAVR risk score (AUC, 0.68; 95% confidence interval [CI], 0.61-0.74) was superior to the STS-PROM score (AUC, 0.52; 95% CI, 0.45-0.59) for predicting in-hospital mortality (Figure 2).
The observed/expected (O/E) in-hospital mortality ratios based on the STS-PROM and TAVR risk scores were 0.57 and 0.83, respectively.
During a mean follow-up period of 1.9 years, a total of 37 patients (19%) died. Long-term survival post TAVR did not change according to quartiles of STS-PROM scores (Figure 3A; P=.52). In contrast, patients in the highest quartile of TAVR risk score (8.4; IQR, 5.8-9.9) had significantly worse survival (Figure 3B; P<.001). This group comprised >50% (20 out of 37) deaths.
Discussion
We validated the STS/ACC TVT-TAVR risk score in a cohort of male veterans undergoing TAVR within the VA system and found that when compared to United States commercial cases, the score had similar discrimination to predict in-hospital mortality. We also found that both STS-PROM and TAVR risk scores overestimated in-hospital mortality after TAVR. The degree of overestimation is significantly higher for the STS-PROM score, with an O/E ratio of 0.57 vs 0.83 for the TAVR risk score. Finally, we found that the TAVR risk score predicts long-term mortality whereas the STS-PROM score does not. Our findings confirm and expand the clinical applications of the TAVR risk score and highlight the importance of TAVR-specific risk assessment tools to optimize patient selection.
The STS/ACC TVT-TAVR risk score was developed to fill a gap in procedure-specific risk-assessment tools after recognizing that risk models perform best when developed from a population undergoing the 
procedure that is the focus of the model.7 The STS-PROM score, widely used as entry criteria in all pivotal TAVR trials, has been designed for predicting in-hospital mortality for SAVR.8-11 Previous efforts to validate the STS-PROM score in patients undergoing TAVR have yielded conflicting results.1-12 A previous analysis of the PARTNER I (Placement of Aortic Transcatheter Valve) trial showed that the STS-PROM score overestimated (O/E ratio = 0.57) and was a poor discriminator of in-hospital mortality.1 Interestingly, the degree of overestimation of in-hospital mortality after TAVR with the STS-PROM score in the PARTNER I trial was identical to the one reported in our dataset.
Approximately 30% of patients in the TVT registry have missing follow-up data at 30 days because of inability to link registry data to patient-specific Centers for Medicare and Medicaid Services (CMS) administrative claims.13 Therefore, the TAVR risk score focused on in-hospital mortality instead.2 A significant number of patients survive the procedure, but succumb within 30 days.14 Our complete dataset allowed us to determine vital statuses in all patients and validate the score for this important outcome. We showed that the TAVR score had similar discrimination for predicting mortality at 30 days post procedure (AUC, 0.64 vs 0.68).
The finding of increased long-term mortality among patients in the highest quartile of the TAVR risk score is novel and could have implications for patient selection. Notwithstanding this observation, improvements in quality of life and heart failure symptoms may be as important as longevity in the elderly population.13,15 Developing risk models that can predict combined measures of survival and quality of life may help clinicians identify patients unlikely to benefit from valve replacement.
Study limitations. Our study has important limitations. First, the study cohort in which the score was validated is small and derived from one VA hospital in the Upper Midwest region of the United States. Currently, there are eight VA structural heart programs performing TAVR procedures; additional studies are ongoing to validate the score in the broader VA population. Second, the cohort is predominantly male. Finally, we did not include quality of life measures in our assessment. Survival alone may not be an adequate measure of optimal patient selection in the elderly.
Conclusion
The TAVR risk score has a similar performance in the veteran population for predicting short-term outcomes. In addition, the TAVR score predicts 30-day and long-term mortality. More than 50% of deaths were clustered in the highest quartile of the TAVR score.
References
1. Beohar N, Whisenant B, Kirtane AJ, et al. The relative performance characteristics of the logistic European System for Cardiac Operative Risk Evaluation score and the Society of Thoracic Surgeons score in the placement of aortic transcatheter valves trial. J Thorac Cardiovasc Surg. 2014;148:2830e1-2337e1.
2. Edwards FH, Cohen DJ, O’Brien SM, et al. Development and validation of a risk prediction model for in-hospital mortality after transcatheter aortic valve replacement. JAMA Cardiol. 2016;1:46–52.
3. Gurevich S, Oestreich B, Kelly RF, et al. Outcomes of transcatheter aortic valve replacement using a minimalist approach. Cardiovasc Revasc Med. 2018;19:192-195. Epub 2017 Aug 12.
4. Gurevich S, John R, Kelly RF, et al. Avoiding the learning curve for transcatheter aortic valve replacement. Cardiol Res Pract. 2017;2017:7524925. Epub 2017 Jan 26.
5. Holmes DR Jr, Nishimura RA, Grover FL, et al; for the Registry SAT. Annual outcomes with transcatheter valve therapy: from the STS/ACC TVT Registry. Ann Thorac Surg. 2016;101:789-800.
6. Zou KH, O’Malley AJ, Mauri L. Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models. Circulation. 2007;115:654-657.
7. Dewey TM, Brown D, Ryan WH, Herbert MA, Prince SL, Mack MJ. Reliability of risk algorithms in predicting early and late operative outcomes in high-risk patients undergoing aortic valve replacement. J Thorac Cardiovasc Surg. 2008;135:180-187.
8. Mack MJ. Risk scores for predicting outcomes in valvular heart disease: how useful? Curr Cardiol Rep. 2011;13:107-112.
9. Thourani VH, Suri RM, Gunter RL, et al. Contemporary real-world outcomes of surgical aortic valve replacement in 141,905 low-risk, intermediate-risk, and high-risk patients. Ann Thorac Surg. 2015;99:55-61.
10. Leon MB, Smith CR, Mack MJ, et al; for the PARTNER 2 Investigators. Transcatheter or surgical aortic-valve replacement in intermediate-risk patients. N Engl J Med. 2016;374:1609-1620. Epub 2016 Apr 2.
11. Thourani VH, Kodali S, Makkar RR, et al. Transcatheter aortic valve replacement versus surgical valve replacement in intermediate-risk patients: a propensity score analysis. Lancet. 2016;387:2218-2225.
12. Balan P, Zhao Y, Johnson S, et al. The Society of Thoracic Surgery risk score as a predictor of 30-day mortality in transcatheter vs surgical aortic valve replacement: a single-center experience and its implications for the development of a TAVR risk-prediction model. J Invasive Cardiol. 2017;29:109-114.
13. Arnold SV, Spertus JA, Vemulapalli S, et al. Quality-of-life outcomes after transcatheter aortic valve replacement in an unselected population: a report from the STS/ACC Transcatheter Valve Therapy Registry. JAMA Cardiol. 2017;2:409-416.
14. Arnold SV, Reynolds MR, Lei Y, et al. Predictors of poor outcomes after transcatheter aortic valve replacement: results from the PARTNER (Placement of Aortic Transcatheter Valve) trial. Circulation. 2014;129:2682-2690.
15. Gurevich S, Reiff C, Bertog S, et al. Transcatheter aortic valve replacement improves health status in elderly veterans. J Invasive Cardiol. 2018;30:207-211.
From the 1Department of Medicine, Division of Cardiology, University of Minnesota Medical Center, Minneapolis, Minnesota; 2Department of Medicine, Division of Cardiology, Minneapolis VA Healthcare System, Minneapolis, Minnesota; 3Minneapolis Heart Institute, Minneapolis, Minnesota; and 4Department of Surgery, Division of Cardiothoracic Surgery, University of Minnesota, Minneapolis, Minnesota.
Disclosure: The authors have completed and returned the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Garcia reports grant support from Edwards Lifesciences; personal fees from Edwards Lifesciences, Medtronic, and Abbott. The remaining authors report no conflicts of interest regarding the content herein.
Manuscript submitted June 15, 2018, final version accepted July 3, 2018.
Address for correspondence: Santiago Garcia, MD, Minneapolis VA Medical Center, Associate Professor of Medicine, University of Minnesota, One Veterans Drive, 111-C, Minneapolis, MN 55417. Email: garci205@umn.edu
