Minimizing Bias in a Diabetic Foot Ulcer Clinical Evaluation: Analysis of the HIFLO Trial

Clinical research is important for many reasons, including determining whether a medical product is safe and effective, generating patient outcomes data that may help providers select appropriate therapies for their patients, and furthering understanding of a disease state, as well as for regulatory approval, reimbursement, and health economics purposes.

Study sponsors are responsible for ensuring that clinical research results in high-quality scientific information while protecting the rights and welfare of study subjects, minimizing bias wherever possible, and providing clinically significant and reproducible results. In addition to standardizing factors such as randomization, blinding, wound assessment, choice of study population, and consistency in SOC, the FDA¹ and the European Wound Management Association² have identified reducing study bias as a critical factor in improving the quality of evidence in wound care clinical research.

Well-designed RCTs are considered to be the most appropriate method of evaluating the efficacy of an intervention.³ The present report summarizes the steps taken to reduce bias and achieve the highest evidence possible (Level 1) in the HIFLO Trial, an RCT (ISRCTN #24783859, conducted in accordance with WCG IRB [formerly Western Institutional Review Board] study #1175398 protocol #20171089, and South Shore Health reference #17-013) in which use of PMVT (mVASC; MicroVascular Tissues, Inc) increased complete healing of Wagner grade 1 and 2 DFUs at 12 weeks.⁴ To demonstrate that the methods used during the HIFLO Trial to reduce bias were effective, the authors of the present study analyzed the predefined, peer-reviewed criteria adopted in the HIFLO Trial for its primary study endpoint of “healed” ulcers at 12 weeks in this complex patient population.

Bias in Clinical Trials

The 5 main sources of bias reported in clinical research literature are selection, performance, detection, attrition, and reporting.^5-7 A literature review of RCTs on treatment of chronic wounds noted potential sources of bias in the study designs. Some of these study design features included not using the FDA definition of wound closure¹ (ie, 100% epithelialization with no wound drainage), allowing investigators to select the SOC for study patients, and using per-protocol data analysis. With these limitations in mind, Table 1 defines each type of bias and summarizes the methods used and actions taken to minimize each of them in the HIFLO Trial.

Primary Endpoint Detection Bias
Minimization

The position of the FDA is that complete wound closure of a chronic, nonhealing wound is “one of the most objective and clinically meaningful wound healing endpoints.”¹ However, in reviewing clinical trials and real-world literature between March 2010 and March 2018, Gould and Li⁸ found that “the concept of wound healing and the method of assessment are very malleable, resulting in research findings that cannot be easily compared. Even more worrisome is that the lack of a universal definition of healing and standard assessment method in clinical practice is likely resulting in inflated healing rates.” Using that 2019 article as a guide, physician experts (P.G., L.G.) assisted the study sponsor (MicroVascular Tissues, Inc.) in developing 4 criteria (ie, the 4-part test) for use in determining whether a wound was healed in the HIFLO Trial. The 4-part test required 100% epithelialization, normal coloration and appearance of skin tissue around the former wound site, absence of exudate, and no infection (Figure 1).

The 4-part test was independently conducted by study sites and a 3-member, blinded panel of plastic surgeon adjudicators who are experts in wound healing to determine whether a study wound was healed. All site investigators and study adjudicators were trained on the HIFLO Trial test criteria and adjudication process workflow. For a HIFLO Trial subject to be deemed healed, all 4 criteria had to be met for each subject by 2 of the 3 adjudicators.

Figure 2 illustrates the workflow used in the HIFLO Trial to determine whether a wound was healed. Upon determination of a healed wound by both the site investigator and a physician or podiatrist at the site not associated with the trial, a high-resolution, close-up photograph of the wound was independently provided to each adjudicator by the study CRO. Adjudicators were blinded to the treatment allocation and assessed the wound photograph based on each of the 4 criteria. Adjudicators were not allowed to discuss adjudications with each other or the CRO. Adjudicators returned individually completed responses to the CRO, which compiled the results. The results of the adjudication—healed or not healed—were provided to the site investigator by the CRO so the investigator could determine next steps for the subject—if not healed, another study treatment visit the following week, or, if healed, a wound healing confirmation visit 2 weeks later.

The HIFLO Trial is the first RCT to use the peer-reviewed, 4-part definition for wound healing.⁸Table 2 shows examples of adjudicated healed and nonhealed wounds from the trial. Upon completion of the trial, the study sponsor analyzed the level of agreement among the adjudicators, using the 4-part criteria for healing. There were 8 “failed” site adjudications during the HIFLO Trial. As shown in Figure 3, for the primary endpoint of wound closure within 12 weeks, the level of agreement among the adjudicators was greater than or equal to 90% for each of the 4-part criteria. The highest disagreement rate between investigators and adjudicators was 9.6%, for criterion 1 (epithelialization).

High-level agreement on the 4-part closure criteria, within a group of blinded plastic surgeons who are experts in wound care, provided assurance that HIFLO Trial wounds were healed using the most rigorous visual assessment criteria to date. Additionally, these data serve as validation of the recommendations of Gould and Li⁸ in a large, prospective RCT.

The nearly 10% disagreement rate between investigators and adjudicators regarding epithelialization, however, indicates that epithelialization is the most difficult aspect in judging DFU healing. As a result, initial epithelialization without re-creation of the moisture barrier function may be unstable and supports the reasoning behind the FDA recommendation (used in the HIFLO Trial) for a confirmatory study visit 2 weeks after initial determination of closure to ensure maintenance of skin reepithelialization without drainage or dressing requirements.

Seven of the 8 failed adjudications in the HIFLO Trial resulted in study subjects receiving another week of treatment. These 7 were subsequently adjudicated as healed, while the eighth subject had reached the end of the study. If the additional layer of adjudication had not been used in this trial, the study results may have been different.

Pre-study investigator training, consistent SOC, data monitoring, independent statistical and ITT-only analysis⁹ using last observation carried forward¹⁰ methodology, along with the other implemented strategies detailed in Table 1, further ensured rigor and comparability across HIFLO Trial sites. Using these bias-reducing methodologies and the rigorous predefined closure criteria, blinded adjudication established that 74% (37 of 50) of DFUs treated with PMVT closed, compared with 38% (19 of 50) of wounds managed with SOC (P = .0003).⁴ This highly significant outcome, which on statistical analysis (detailed in Gould et al⁴) determined that DFUs were 9 times more likely to heal when treated with PMVT compared with SOC, is representative of the 96% statistical power of that high-quality study.⁴ This degree of statistical power was achievable only by minimizing sources of potential study bias.

While the methods used in conducting the HIFLO Trial were aimed at substantially reducing bias, there is room for improvement. One limitation of the trial is that although the high-resolution photographs provided the remote adjudicators with a clear image of the wounds, in-person assessment and adjudication of the state of healing is ideal. Regardless of the rigor of a definition of the term healed, healing is not always easily assessed using a photograph. The need for timely, blinded, and independent multiple adjudications in a multicenter trial, however, makes in-person adjudication extremely challenging from a logistical perspective.

Designing, executing, and reporting quality clinical research involves a commitment to reducing bias throughout the study. Performing a prospective, single-blind RCT reduces some study selection and detection bias, but it does not completely address other sources of bias. The HIFLO Trial was designed to minimize bias across the 5 main sources of bias. In addition, using the most critical visual assessment criteria to date, high-level agreement within an adjudication panel of blinded plastic surgeons who are experts in wound care assured that DFUs in the HIFLO Trial were truly healed. The findings reported herein concerning these measures and criteria may prove beneficial for others seeking to minimize bias in wound studies.