Variability in Freedom From Loss of Primary Patency Results in Trials Assessing Stent Implantation in the Superficial Femoral Artery

Abstract: Background. Primary patency (PP) in trials assessing superficial femoral artery (SFA) stenting is defined as a combination of vessel patency assessed by duplex ultrasound (DUS) at the 12-month follow-up exam and freedom from revascularization of the index vessel through 12 months of follow-up. Loss of PP is thus more likely to be identified during the mandated DUS assessment. Moreover, DUS is performed within a prespecified allowed window of time for the visit that exceeds 12 months (typically by 30 days). Therefore, the time frame for detecting patency with DUS exceeds the time frame in which revascularization is captured. Survival analyses are often applied to present estimates of freedom from loss of PP, but there are no clear guidelines as to the correct method for presenting these analyses in reports from clinical trials. We aimed to analyze the implications of applying different methods in assessing freedom from loss of PP in studies assessing stenting for diseased SFA. Methods. Data were simulated based on existing available results from SFA bare-metal nitinol stent trials published between 2009 and 2013 and summarized in a previous analysis (STROLL, SUPERB, RESILIENT, DURABILITY I, DURABILTY II, COMPLETE SFA). Six different approaches to Kaplan Meier (KM) analyses were applied based on entry criteria into and time frame of the KM model. Results. Six KM estimates of PP were generated for each of the 10,000 simulated datasets. The average exact PP rate was 70.6%, while the average estimated KM rates using the six different methods ranged between 68.0% and 81.9%. Conclusion. KM estimates of PP vary substantially according to the methods employed. These may lead to misrepresentation of results from clinical trials. The development of a unified approach is advocated.  

J INVASIVE CARDIOL 2014;26(11):614-617

Key words: peripheral arterial disease, peripheral interventions, primary patency, Kaplan-Meier analysis

_______________________________________

Peripheral arterial disease (PAD) is common, affecting almost 8 million Americans, including 12%-20% of individuals over age 65 years.^1,2 The manifestations of lower-extremity PAD inflict a large personal, medical, social, and economic burden on patients each year, with symptoms ranging from mild claudication to limb-threatening ischemia.³ PAD is associated with an increased risk of cardiovascular morbidity and mortality and with impaired function and quality of life.^4-7 In some patients, revascularization of the culprit vessel may be pertinent. According to the revised 2007 TransAtlantic InterSociety Consensus Classification (TASC) II recommendations, endovascular treatment should be utilized to treat type C lesions.⁵

Primary stent implantation for diseased SFAs has been extensively studied in recent years, with favorable results either when compared to percutaneous transluminal angioplasty (PTA) with provisional stenting, or when compared to a predefined performance goal (PG).^8-15 These PGs are most often based on the VIVA Physicians, Inc (VPI) analysis published in 2007,¹⁶ in which safety and efficacy benchmark rates were set based on historical studies of patients treated with PTA. The primary efficacy endpoint proposed was vessel patency at 12 months, defined as freedom from more than 50% restenosis based on duplex ultrasound (DUS) peak systolic velocity ratio. Current SFA studies commonly utilize an efficacy outcome of primary patency (PP), which includes freedom from restenosis per DUS assessment at an allowed window around 12 months after the index procedure, combined with freedom from reintervention of the target vessel or lesion through 12 months after the index procedure. PP refers to the exact rate of subjects who have not lost their vessel patency, out of all evaluable subjects (ie, with sufficient follow-up and a valid 12-month DUS). The conventional approach to reporting these results has been to present both crude rates and Kaplan-Meier (KM) estimates for freedom from loss of PP, the product of a survival analysis. In a recent meta-analysis, we observed that these rates significantly differ.¹⁷ We hypothesize that these significant differences are primarily due to diversity in utilization of survival analysis techniques for assessing primary patency, an outcome largely based on results from a diagnostic tool applied at a prespecified time point. As there is currently no convention for the methodology by which survival analyses should be executed in these settings, we set to analyze the implications of applying different methods in assessing freedom from loss of PP in studies assessing stenting for SFA disease.

Methods

We calculated the KM estimate for freedom from loss of PP by six different methods employed on the data. We studied the implications of two variables thought to primarily affect the outcome of the KM model: subject entry into the model and subject censorship rules (ie, when a subject is excluded from the analysis before an event occurs). For subject entry, it is not uncommon that 12-month DUS is either not performed or performed outside the allowed window. Subjects without a valid 12-month DUS assessment who did not experience revascularization during follow-up are considered non-evaluable for PP assessment, and can be either excluded from the model at entry or censored at the last day of follow-up. As for censorship, a discrepancy in follow-up time for clinical and diagnostic outcomes is inherent to the assessment of PP: exactly 12 months for reintervention, and 12 months + 30 days for DUS evaluation of patency, reflecting an allowed window for patients to return for assessment. Censorship can thus occur in several ways. We executed six different approaches to the survival analysis incorporating variations of the above variables (Table 1). 

We carried out a simulation study to examine the implications of the above methods on the PP estimates at 12 months. We generated 10,000 datasets with 512 subjects in each group. For each subject, we generated a time to target lesion revascularization (TLR) and a time to censoring using exponential distributions with rates informed by existing data available from superficial femoral artery (SFA) stenting trials. If the time to TLR was longer than the time to censoring, then subject data were censored at the censoring time. Data were also censored at 360 days for TLR. For subjects who did not have an event and were not censored before the 330-day mark in the study, we generated a time of DUS uniformly distributed between 330 and 390 days. A subgroup of these patients was randomly assigned to have an event using a binomial distribution. The rate of DUS assessment and the rate of restenosis detected by DUS were informed from the published data. Subject status for the patients who did not undergo DUS was determined based on the time to TLR and censoring, while the status of the patients who underwent DUS was determined based on their DUS status and TLR (the latter only through 360 days). 

For each dataset, we initially calculated the exact rate of PP as the number of subjects with a patent SFA by the 12 months ± 30 days DUS, who were free from revascularization at 360 days, divided by the number of evaluable subjects, defined as subjects with TLR through 360 days, or with sufficient follow-up (at least 330 days post index procedure) and DUS performed within the allowed window of 12 months of ± 30 days. We then generated the KM estimates of the PP rate using each of the six methods. The estimates at 12 months along with ranges for these estimates are reported. 

Results

We generated 10,000 datasets, each with 512 subjects. For each of these sets, we calculated exact rates of PP and six KM estimates of freedom from loss of PP. The average exact rate of PP was 70.6%, with 95% of the exact rates falling between 66.2% and 74.9%. The average KM estimates for the six methodologies ranged between 68.0% and 81.9% (Figure 1). 

Figure 2 plots the freedom from loss of PP curves through 360 or 390 days for the six different methodologies employed. As expected and in accordance with the published literature, loss of patency is homogenously distributed between 0 and 330 days, driven primarily by revascularizations, accelerating beyond 330 days for the remaining follow-up duration primarily due to loss of patency as documented in premandated DUS evaluation.

As shown, when all of the subjects (evaluable and non-evaluable) were entered into the model, higher KM estimates were generated compared to those obtained when only evaluable subjects were entered into the model. There was no substantial difference between extending the duration of clinical follow-up to 390 days versus including all post 360-day DUSs as if they were performed at 360 days. However, censoring subjects with DUS performed at 361-390 days at 360 days (ie, excluding them from the analysis regardless of the DUS results) provided considerably higher KM estimates, with the 95% range not overlapping with those in which these subjects were included (method 1 vs methods 2 and 3; method 4 vs methods 5 and 6). 

Discussion

Survival analyses have the advantages of studying events over time, providing input on both timing and occurrence of events, and allowing the contribution of data from patients who are lost to follow-up.¹⁸ The KM method¹⁹ is the most widely used for analyzing survival from events over time. KM estimates of survival are highly dependent on the assumption that patients’ survival prospects stay the same throughout the study.²⁰ The requirements for the analysis of survival data also include uninformative censoring and completeness of follow-up.²¹

Primary patency is a unique endpoint composed of a clinical event (revascularization) combined with a component derived from a diagnostic evaluation (12-month patency per DUS). In this analysis, we used simulated datasets to analyze the effect of different approaches to evaluate freedom from loss of PP with the KM method. We have shown that inclusion of all evaluable and non-evaluable subjects into the model at time 0 and censorship of DUS performed beyond 12 months are contributors to higher KM estimates of the 12-month PP rate. 

In our meta-analysis of studies assessing primary implantation of nitinol stents in diseased SFA,²² we report a meta-analytic 12-month PP rate of 71.6% (95% confidence interval [CI], 66.4%-76.7%), and a meta-analytic KM rate of freedom from loss of PP of 81.0% (95% CI, 76.0%-85.4%). These were calculated from results of studies assessing primary nitinol stent implantation in the SFA in which PP was defined as a composite of freedom from revascularization or loss of patency per DUS, and include results from the DURABILITY I,¹⁵ DURABILITY II,²³ COMPLETE SE SFA, RESILIENT,¹⁴ SUPERB, and STROLL trials. When presented side by side, the differences between KM estimates and exact rates of 12-month PP range between 7.31% (SUPERB) and 13% (STROLL). Recently, 360-day KM estimates of PP from the COMPLETE SE study on SFAs became publicly available and are in the order of 18% higher than the exact rates previously reported for this study.²⁴ 

These apparent and substantial differences in outcomes from two methods by which PP is evaluated pose challenges in interpretations to clinicians, regulators, patients, and other stakeholders. DUS-verified restenosis consistent with loss of patency is not distributed equally throughout the follow-up period, and its identification heavily relies on adherence to visit schedules. This leaves several degrees of freedom by which the KM models can be constructed. The exact mechanisms by which KM estimates are populated are not typically detailed with the presentation of results, which even further complicates their interpretation. Thus, educated readers are left with uncertainty as of the meaningfulness of the presented rates. 

Study limitations. Our study is limited by its reliance on simulated data. While making every effort to simulate data that incorporate revascularizations, DUS assessments, loss to follow-up, and percentage of missing DUS assessments, and feeding the model with parameters based on true study results, we acknowledge that it is weakened by the lack of availability of real patient-level data. To mitigate this limitation, we generated 10,000 simulation sets closely mimicking the event distribution and event rates from the published literature.

Conclusion

We believe that survival analysis is deficient in its availability to portray a true representation of event rates over time in outcomes that are largely based on diagnostic procedures, such as in the case of PP. We have shown that different assumptions introduce large variability in results. Thus, manipulation of these assumptions can in theory create false representation of efficacy. Survival analysis techniques should ideally be reserved for the evaluation of clinically driven time-to-event analysis of outcomes (such as rates of revascularization, death, amputation, etc). If used to assess freedom from loss of PP, we advocate for a clear and unified directive regarding the best approach by which these techniques should be used. A conservative approach would be to analyze PP at 390 days. 

References

1. Roger VL, Go AS, Lloyd-Jones DM, et al. Heart disease and stroke statistics--2012 update: a report from the American Heart Association. Circulation. 2012;125(1):e2-e220.
2. Ostchega Y, Paulose-Ram R, Dillon CF, Gu Q, Hughes JP. Prevalence of peripheral arterial disease and risk factors in persons aged 60 and older: data from the National Health and Nutrition Examination Survey 1999-2004. J Am Geriatr Soc. 2007;55(4):583-589.
3. Hirsch AT, Haskal ZJ, Hertzer NR, et al. ACC/AHA 2005 practice guidelines for the management of patients with peripheral arterial disease (lower extremity, renal, mesenteric, and abdominal aortic): a collaborative report from the American Association for Vascular Surgery/Society for Vascular Surgery, Society for Cardiovascular Angiography and Interventions, Society for Vascular Medicine and Biology, Society of Interventional Radiology, and the ACC/AHA Task Force on Practice Guidelines (Writing Committee to Develop Guidelines for the Management of Patients With Peripheral Arterial Disease): endorsed by the American Association of Cardiovascular and Pulmonary Rehabilitation; National Heart, Lung, and Blood Institute; Society for Vascular Nursing; TransAtlantic Inter-Society Consensus; and Vascular Disease Foundation. Circulation. 2006;113(11):e463-e654.
4. Murabito JM, D’Agostino RB, Silbershatz H, Wilson WF. Intermittent claudication. A risk profile from the Framingham Heart Study. Circulation. 1997;96(1):44-49.
5. Norgren L, Hiatt WR, Dormandy JA, Nehler MR, Harris KA, Fowkes FG. Inter-Society Consensus for the Management of Peripheral Arterial Disease (TASC II). J Vasc Surg. 2007;45(Suppl S):S5-S67.
6. Murphy TP, Ariaratnam NS, Carney WI Jr, et al. Aortoiliac insufficiency: long-term experience with stent placement for treatment. Radiology. 2004;231(1):243-249.
7. Murabito JM, Evans JC, D’Agostino RB Sr, Wilson PW, Kannel WB. Temporal trends in the incidence of intermittent claudication from 1950 to 1999. Am J Epidemiol. 2005;162(5):430-437.
8. Duda SH, Bosiers M, Lammer J, et al. Sirolimus-eluting versus bare nitinol stent for obstructive superficial femoral artery disease: the SIROCCO II trial. J Vasc Interv Radiol. 2005;16(3):331-338.
9. Krankenberg H, Schluter M, Steinkamp HJ, et al. Nitinol stent implantation versus percutaneous transluminal angioplasty in superficial femoral artery lesions up to 10 cm in length: the femoral artery stenting trial (FAST). Circulation. 2007;116(3):285-392.
10. Zeller T, Tiefenbacher C, Steinkamp HJ, et al. Nitinol stent implantation in TASC A and B superficial femoral artery lesions: the Femoral Artery Conformexx Trial (FACT). J Endovasc Ther. 2008;15(4):390-398.
11. Dick P, Wallner H, Sabeti S, et al. Balloon angioplasty versus stenting with nitinol stents in intermediate length superficial femoral artery lesions. Catheter Cardiovasc Interv. 2009;74(7):1090-1095.
12. Schillinger M, Sabeti S, Loewe C, et al. Balloon angioplasty versus implantation of nitinol stents in the superficial femoral artery. N Engl J Med. 2006;354(18):1879-1888.
13. Dake MD, Ansel GM, Jaff MR, et al. Paclitaxel-eluting stents show superiority to balloon angioplasty and bare metal stents in femoropopliteal disease: twelve-month Zilver PTX randomized study results. Circ Cardiovasc Interv. 2011;4(5):495-504.
14. Laird JR, Katzen BT, Scheinert D, et al. Nitinol stent implantation versus balloon angioplasty for lesions in the superficial femoral artery and proximal popliteal artery: twelve-month results from the RESILIENT randomized trial. Circ Cardiovasc Interv. 2010;3(3):267-276.
15. Bosiers M, Torsello G, Gissler HM, et al. Nitinol stent implantation in long superficial femoral artery lesions: 12-month results of the DURABILITY I study. J Endovasc Ther. 2009;16(3):261-269.
16. Rocha-Singh KJ, Jaff MR, Crabtree TR, Bloch DA, Ansel G. Performance goals and endpoint assessments for clinical trials of femoropopliteal bare nitinol stents in patients with symptomatic peripheral arterial disease. Catheter Cardiovasc Interv. 2007;69(6):910-919.
17. Vardi M, Novack V, Pencina MJ, et al. Safety and efficacy metrics for primary nitinol stenting in femoropopliteal occlusive disease: a meta-analysis and critical examination of current methodologies. Catheter Cardiovasc Interv. 2014;83(6):975-83.
18. Kim J. Survival analysis. Pediatr Rev. 2012;33(4):172-174.
19. Kaplan EL, Meier P. Nonparametric estimation from incomplete observations. J Am Stat Assoc. 1958;53:457-481.
20. Altman DG, Bland JM. Time to event (survival) data. BMJ. 1998;317(7156):468-469.
21. Clark TG, Bradburn MJ, Love SB, Altman DG. Survival analysis part I: basic concepts and first analyses. Br J Cancer. 2003;89(2):232-238.
22. Vardi M, Novack V, Pencina MJ, et al. Safety and efficacy metrics for primary nitinol stenting in femoropopliteal occlusive disease: A meta-analysis and critical examination of current methodologies. Catheter Cardiovasc Interv. 2014;83(6):975-983. Epub 2014 Jan 31.
23. Matsumura JS, Yamanouchi D, Goldstein JA, et al. The United States study for evaluating endovascular treatments of lesions in the superficial femoral artery and proximal popliteal by using the Protege Everflex Nitinol Stent System II (DURABILITY II). J Vasc Surg. 2013;58(1):73-83.e1. Epub 2013 May 2.
24. Complete SE Vascular Stent System Summary of Safety and Effectiveness Data (SSED). Volume 2014.

___________________________________________________

From ¹Harvard Clinical Research Institute, Boston, Massachusetts; ²Department of Global Health, Harvard School of Public Health, Boston, Massachusetts; and ³Department of Biostatistics, Boston University, Boston, Massachusetts.

Disclosure: The authors have completed and returned the ICMJE Form for Disclosure of Potential Conflicts of Interest. The authors report no conflicts of interest regarding the content herein.

Manuscript submitted January 28, 2014, provisional acceptance given May 15, 2014, final version accepted May 22, 2014.

Address for correspondence: Dr Moshe Vardi, Harvard Clinical Research Institute, 930 Commonwealth Ave, Boston, MA 02215. Email: Vardi.moshe@gmail.com or Moshe.vardi@hcri.harvard.edu