The Reliability of a Novel Mobile 3-dimensional Wound Measurement Device

Background. Objective assessment of wound dimensions is essential for tracking progression and determining treatment effectiveness. A reliability study was designed to establish intrarater and interrater reliability of a novel mobile 3-dimensional wound measurement (3DWM) device. Methods. Forty-five wounds were assessed by 2 raters using a 3DWM device to obtain length, width, area, depth, and volume measurements. Wounds were also measured manually, using a disposable ruler and digital planimetry. The intraclass correlation coefficient (ICC) was used to establish intrarater and interrater reliability. Results. High levels of intrarater and interrater agreement were observed for area, length, and width; ICC = 0.998, 0.977, 0.955 and 0.999, 0.997, 0.995, respectively. Moderate levels of intrarater (ICC = 0.888) and interrater (ICC = 0.696) agreement were observed for volume. Lastly, depth yielded an intrarater ICC of 0.360 and an interrater ICC of 0.649. Measures from the 3DWM device were highly correlated with those obtained from scaled photography for length, width, and area (ρ = 0.997, 0.988, 0.997, P < 0.001). The 3DWM device yielded correlations of ρ = 0.990, 0.987, 0.996 with P < 0.001 for length, width, and area when compared to manual measurements. Conclusion. The 3DWM device was found to be highly reliable for measuring wound areas for a range of wound sizes and types as compared to manual measurement and digital planimetry. The depth and therefore volume measurement using the 3DWM device was found to have a lower ICC, but volume ICC alone was moderate. Overall, this device offers a mobile option for objective wound measurement in the clinical setting. 

Chronic wounds affect approximately 1% to 2% of the population in developed nations. Chronic wounds are estimated to cost $25 billion per year in the United States alone, with the majority spent on wound care products.^1-3 Diabetic foot wounds represent a large portion of this cost, with 15% to 25% of patients with diabetes developing chronic ulcers during their lifetime.^3-5 These wounds can be difficult to manage and heal slowly; treatment time averages 2 months.³ A retrospective study of men aged 40-64 years found the cost of care for patients with diabetic foot ulcers was 5.4 times that of a control group.⁶ In addition, the rate of recurrence is high, at 50% to 70% over the next 5 years.³ The incidence of chronic wounds is only expected to increase as diabetes and obesity become more prevalent.

A key component of wound care is the collection of objective data on wound size and composition. Clinicians must be able to track wound progression over time to determine treatment effectiveness and the necessity for alternative interventions.^7-9 A randomized, controlled trial (RCT) conducted by Sheehan et al¹⁰ found the percent change in ulcer size after 4 weeks of treatment is a robust predictor of wound healing at 12 weeks. Further research¹¹ has shown the percent change in ulcer size as early as 1 week following the start of treatment can predict the likelihood of healing. By closely monitoring the healing process, clinicians can identify ineffective treatments early and make timely adjustments, avoiding prolonged healing and costly complications. 

There is a need for a 3-dimensional (3D) wound assessment solution that is accurate and reliable; it also has to integrate seamlessly with the clinical workflow through intuitive interface and portability. Such requirements are not met by any wound assessment device currently on the market. Here, the authors present a novel assessment solution that satisfies these design requirements. By combining the iPad (Apple, Cupertino, CA) with infrared-based structure sensing and computer vision for wound border delineation, the authors present an intuitive, portable 3D measurement device to improve wound assessment.  This study was designed to establish the intrarater and interrater reliability of this 3D wound measurement (3DWM) device camera (inSight, eKare Inc, Fairfax, VA) and to determine if this novel 3D structure sensor provides valid wound measurements when compared with empirical values obtained by digital planimetry.

The primary endpoint of this study was to establish the intrarater and interrater reliability of the 3DWM device. Intrarater reliability (also referred to as test-retest reliability) is the reliability of measurements conducted at different times by the same rater. This was assessed by having the same rater assess a wound at 2 different times within a 15-minute interval, so the wound would remain morphologically unchanged. Interrater reliability is the reliability of independent raters’ measurements. It was determined by conducting an assessment of the same wound by 2 independent raters. 

The secondary endpoints included comparing measurements obtained from (a) the 3DWM device to those obtained from the digital planimetry and (b) the 3DWM device to those obtained from the manual metric measurements. 

The intraclass correlation coefficient (ICC) was used to establish both test-retest (ICC 1, 1) and interrater (ICC 2, 1) reliability.^12-14 Henceforth, ICC 1, 1 will be referred to as the intrarater ICC, and ICC 2, 1 will be referred to as the interrater ICC.  All analyses were conducted in R.¹⁵ The 3DWM device measurements were compared to digital and manual measurements using the Wilcoxon signed rank test. 

The researchers wished to have adequate power to detect a test-retest correlation of at least 0.55. Using the Bonferroni correction to account for multiple comparisons, so as to maintain a family-wise type I error of 0.05, 28 wounds were needed to provide 80% power. After accounting for attrition at roughly 10%, the researchers aimed to collect data on 31 wounds but  oversampled and collected measurements on 45 wounds. 

Individuals were deemed eligible if they were 18 years or older, were willing to participate and give consent, did not have external fixation devices, did not have circumferential wounds, and had wounds in easily visualized anatomic areas (eg, not within skin folds).

Two raters were trained to use the 3DWM device and were given 10 wounds to measure prior to beginning the trial to familiarize themselves with the technology. During each patient encounter, the index wound was measured a total of 5 times: (1) 3DWM device by rater 1; (2) length, width, and depth using a ruler and a cotton swab for maximum depth by rater 1; (3) 3DWM device by rater 2; (4) after placing a reference marker with a diameter of 1.90 cm, a scaled high-resolution 2D photo with the iPad’s native 5-megapixel camera by rater 1;  and (5) 3DWM device by rater 1. Raters conducted their assessments independently. Additionally, raters were instructed to refrain from disclosing their assessments to each other.  All digital images were taken at an angle perpendicular to the index wound, at a 40 cm–50 cm distance, within a 15-minute interval so the wound was morphologically unchanged. 

The 3DWM device was used to obtain length, width, area, and volume measurements for each wound. The 3DWM device utilized was an iPad fitted with a 3D structure sensor, similar to a portable Kinect for Xbox One (Microsoft, Redmond, WA)¹⁶ with integrated software that performs wound border segmentation using the interactive Graph Cuts algorithm implemented by Boykov et al.¹⁷ An outline of the wound border, semiautomatically defined by the user, was used to define the region of interest on the depth-map provided by the structure sensor. The largest rectangle encapsulating the wound boundary is used by the device to generate length and width values. The area of the wound is calculated as the surface area of the reference plane enclosed within the wound boundary. The region of interest is then transformed into a 3D space, which is a depth-map of a 2D image with depth values at pixel positions, leading to a volume value. 

For wounds manually measured with a disposable ruler, the area was calculated as length multiplied by width. The length and width of each wound was defined as the longest perpendicular distance of the wound. The maximum depth of the wound was measured using a cotton swab, which was then held next to a metric ruler. 

Additionally, wounds were measured using digital planimetry, which used scaled photography with a reference marker.  A 2D image was captured by the iPad’s native 5-megapixel camera and later uploaded to a computer to calculate the 2D dimensions using ImageJ scientific image analysis software (National Institutes of Health, Bethesda, MD). Measurements were calibrated with a green dot sticker as a reference that measured 1.9 cm in diameter and was placed in the same plane as the wound. Rater 1 then traced the wound margins via mouse to define the wound perimeter using the scientific image analysis software. The software then computed the area of the wound. Rater 1 also used the software to calculate the length and width of each wound, defined as the longest perpendicular distances in the 2D plane of the wound.

The study was conducted at the Center for Wound Healing and Hyperbaric Medicine at MedStar Georgetown University Hospital (Washington, DC). Patient accrual occurred in September 2015.  The study was granted institutional review board approval from Georgetown University (#2015-0250). 

Forty-five wounds were included in this study from a total of 31 patients (Figure 1). Four patients declined to participate; excluded patients were 1 patient with an external fixation device, 5 patients with circumferential wounds, and 3 patients whose wounds were in skin folds. 

An intrarater ICC of 0.998 was obtained for area, indicating a high level of reliability between a rater’s measurements at 2 time points (Table 1). Similarly, an interrater ICC of 0.999 was obtained, indicating strong agreement between the 2 raters (Table 2). Similar trends were noted for length and width. Intrarater ICCs of 0.997 and 0.995 were obtained for length and width, respectively. The interrater ICCs for length (0.997) and width (0.995) also demonstrated high levels of agreement between the 2 raters.  As far as depth, an intrarater ICC of 0.360 was obtained, demonstrating a low level of reliability for a rater’s measurements at 2 time points. In the same vein, an interrater ICC of 0.649 was obtained, indicating moderate agreement between the raters. Volume had moderate levels of intrarater (ICC = 0.888) and interrater (ICC = 0.696) agreement. 

To compare measurements from the 3DWM device with those from digital planimetry, the 3 measurements from the 3DWM device were averaged (unweighted) and compared to those from the digital planimetry measurements (Table 3). Statistically significant correlations of 0.997, 0.988, and 0.997 were obtained for length, width, and area, respectively. Similarly, statistically significant correlations of 0.990, 0.987, and 0.996 were obtained for length, width, and area when comparing the 3DWM device measurements and the manual measurements (Table 4). 

Wilcoxon signed rank tests were performed to test the null hypotheses of no difference in medians for area, length, and width for the 3DWM device and the digital planimetry (Table 5). P values of 0.911, 0.001, and 0.001 were obtained for area, length, and width, respectively. Therefore, the authors concluded that the medians for length and width differed significantly between the 3DWM device and digital planimetry. The 3DWM device tended to result in greater width measurements and smaller length measurements compared to digital planimetry. In fact, the 3DWM device recorded a greater width 71.11% of the time (32 of 45 measurements). On the other hand, the 3DWM device recorded a lesser length in 68.89% of the measurements (31 out of 45). Of note, the authors found medians for area did not differ significantly between the 3DWM device and digital planimetry measurements. 

Interest also lay in comparing the measurements from the 3DWM device to those from the manual measurements. As before, the 3 measurements from the 3DWM device were averaged (unweighted) and then compared to those from the manual measurements. Wilcoxon signed rank tests yielded P values of 2.797 x 10^-8, 0.4728, 0.1192 for area, length, and width, respectively. Hence, the authors concluded that median length and width measurements did not significantly differ between the 3DWM device and the manual measurements. On the other hand, it was also found that the medians for area differed significantly between the 3DWM device and manual metric measurements. 

Despite the enormous clinical burden posed by chronic wounds, wound management technologies are woefully underdeveloped. Current methods for wound measurement are inaccurate, invasive, and inconsistent, with many clinicians relying on crude visual assessment. One of the most widely used methods of wound measurement is the ruler-based method. This involves using a ruler to measure wounds and multiplying length and width to determine area.  Although this method is easy and inexpensive, it is inaccurate, particularly for irregularly shaped wounds.^7,9,18 Further, measurements vary widely from clinician to clinician. Lastly, measurements, particularly depth, require manipulation of the wound, which may be uncomfortable for the patient and may make the wound vulnerable to contamination. 

Digital planimetry has been found to give a better estimate of wound size. Digital planimetry involves creating a 2D image of the wound through either scaled imaging or direct tracing.  The area is then calculated by a computer after scanning the 2D image. While more accurate than ruler-based methods, digital planimetry is time consuming.⁷ Manually tracing the wound can lead to patient discomfort, while issues of magnification can lead to errors with scaled imaging.

More advanced vision-based technologies include stereophotogrammetry and structured light. Stereophotogrammetry (SPG) involves taking pictures of the wound from slightly different angles and using a computer program to create a 3D model of the wound. Blustrode et al¹⁴ found the accuracy of SPG was > 99% with a precision of < 2%, which was significantly superior to direct tracing. While SPG provides highly accurate measurements, the equipment required is expensive, bulky, and time consuming to use, making it a poor choice for most clinicians.¹⁹  Structured light uses strips of light (eg, laser, infrared) for scanning a wound to create 3D mappings of wound topography. This depth map can then be processed to calculate wound dimensions such as area, depth, volume, and perimeter. The method is noninvasive and fairly accurate, although costs remain a concern. In experienced hands, wound measurements can be obtained in just under 3 minutes.²⁰ While both of these methods can provide detailed topographical mapping of a wound, both require transfer of the wound image to a computer for computation of the desired measurements. This added step severely disrupts the clinical workflow and compromises the utility of these devices. 

The 3DWM device was found to be highly reliable for measuring wound area for a range of wound sizes and types. The intrarater and interrater ICC values were both high for wound length and width.  Therefore, length and width are consistently reproducible independent of the rater or time of collection using this device. As anticipated, the area values, which are related to length and width, also showed high ICC values of 0.998 and 0.999 for intrarater and interrater reliability. It is important to note that instead of being calculated as a direct function of length and width, area values were computed using wound image segmentation. 

Maximum depth was found to have a lower ICC, more so for intrarater than interrater agreement. Accurate singular maximum depth measurements are difficult to achieve due to poor lighting and difficulty capturing images perpendicular to the shallowest portion of the wound. Additionally, depth measurement of shallow wounds can result in a disproportionally large standard error due to the empirically small depth value. It is also challenging to accurately capture depth for tunneling wounds. The low intrarater agreement for maximum depth is hypothesized to be attributable to the first rater’s lack of consistency in reproducing the camera angle during the second measurement and the shallow nature of the wounds in this study. Since the 2 raters were independent, reproducibility of the camera angle, or lack thereof, was not of concern while assessing interrater agreement. As such, the interrater agreement for maximum depth was much higher. 

Finally, volume was found to have a moderate intrarater ICC of 0.888 and interrater ICC of 0.695. Considering the high values for area and moderate-to-poor values for depth, one would anticipate an ICC value between the 2. Readers should note the device does not calculate volume by multiplying deepest measured depth by area, but by using a depth-map defined by the wound perimeter. The depth-map is a 2D image with depth values at pixel positions that can be transformed into a 3D surface using the intrinsic camera parameters of the structure sensor. Therefore, concavity, often a feature of healing wounds, is appropriately represented in the volume value. This also explains why a lower intrarater ICC depth measurement did not result in a much lower intrarater ICC for volume.

Measurements recorded by manual measurement and the 3DWM device, as well as those recorded by digital planimetry and the 3DWM device, were almost perfectly positively correlated with each other. 

Median values of length (P = 0.4728) and width (P = 0.1192) were not found to be significantly different for 3DWM and manual measurements. This was because both the 3DWM device and the manual measurements appropriately measured distances influenced by body surface contour and topography. 

Conversely, median values of length (P = 0.001) and width (P = 0.001) between digital planimetry and 3DWM were found to be significantly different. The 3DWM device showed a tendency to overestimate width and underestimate length compared to digital planimetry, recording a greater width 71.11% of the time and a lesser length in 68.89% of the measurements. This resultant discrepancy is due to the fact that 2D images used in digital planimetry ignore wound topography and body contour, thereby inaccurately capturing length and width. On the other hand, since the 3DWM device takes wound topography and body contour into consideration, its measurements were consistent with manual metric measurements, as previously shown. Additionally, the authors speculate the 2D images captured for scaled scientific image software analysis were taken along the length plane, maximizing length and limiting width due to body contouring. 

Area calculated by manual tracing on a 2D scaled image using the software was not found to differ significantly from 3DWM device values. Although there are drawbacks to measuring length and width alone using digital planimetry, area can be measured accurately via this method. Unfortunately, the process of recording the image and manually tracing the wound margins is time consuming and unrealistic in a clinical setting. As such, the 3DWM device appears to be a more efficient alternative. 

Wound area was overestimated by manual metric measurement in 41 of 45 wounds (91.11%). Manual median wound area values were significantly different from 3DWM device values. This was anticipated as several studies^9,18,21 have shown that ruler-based assessments overestimate wound area by up to 44%. The conventional calculation of area, defined as length multiplied by width applied to a regularly shaped quadrilateral form, yields incorrect estimates for irregularly shaped wounds. The 3DWM device is able to measure area using wound image segmentation and therefore provides a more accurate assessment of wound area. 

As with all methods of wound measurement, definition of the wound border can be challenging. Often, subjective assessment by the clinician is used to demarcate wound dimensions. On the other hand, the structure sensor of the 3DWM device automatically defines the wound borders. This feature undoubtedly contributed to the discrepancy between manual measurements and the sensor’s measurements.  The user also has the option of manually adjusting the wound borders, which introduces further subjectivity into the measurements.

Readers should also note the 3DWM device performed comparably to both manual measurements and the digital planimetry at detecting changes in wound area (and consequently length and width). In instances where the same wound was measured at multiple time points, the 3DWM device, digital planimetry, and manual measurements comparably captured differences in area over time (Figure 1). 

This technology was found to have several limitations relating to nonoptimal orientation or lighting, low resolution, wound type, and lack of homeostasis. These limitations can be categorized as device-dependent and device-independent. Device-dependent limitations included differences in sensor orientation that introduced variability in measurements, the device’s inability to visualize wound borders in the presence of inadequate lighting, and the device’s failure to accurately measure small wounds (< 4 cm²) due to its wound delineation procedure. On the contrary, wound location and type were device-independent limitations. Wounds on the posterior leg were difficult to measure since the majority of patients recruited were sitting in a reclined chair. Wounds that tunneled, were circumferential, or required external fixation were not adequately visualized by the device and thus excluded from this study. 

Overall, this device has implications for improving clinical practice. As shown, the 3DWM device’s ability to reliability measure wound area, albeit with limitations regarding the reliability of maximum depth, serves to monitor wound morphology and guide clinical decision making. Percent area reduction after 4 weeks of treatment identifies venous, pressure, or diabetic chronic ulcers that are not on a path to healing.^10,22-25 Additionally, maximum depth of a chronic wound has been validated as a predictor of amputations in diabetic foot ulcers²⁶; thus, measurements captured by the 3DWM device could aid in determining risk factors for amputation. Overall, the 3DWM device is able to simply and reliably provide area measurements along with depth measurements that may be useful in clinical practice.

The 3DWM device is an accurate and reliable method of measuring wounds. Its performance is superior to manual measurement and comparable to more accurate methods such as digital planimetry.  At the same time, it provides a number of advantages over existing methods. It eliminates the calibration errors of digital planimetry because it does not require a reference. It provides a true 3D model of the wound rather than an estimate based on a 2D projection. Measurements can be obtained without having to manipulate the wound, decreasing patient discomfort and risk of contamination. Furthermore, the device is fast and easy to use, making it easy to integrate into clinical workflow. 

While there is value in measuring length and width with a ruler, it should be noted that manual metric measurements do not accurately calculate the wound area. Since area is most useful for clinical decision making, this is an important limitation of the manual method. While digital planimetry appropriately captures area, its failure to capture length and width as accurately is an inherent drawback.  This is likely due to the inherent lack of depth and contour perception in digital planimetry, which stands in stark contrast to both manual measurements and the 3DWM device. In light of this, the advantage of the 3DWM device is that it can accurately measure both length and width, while also providing a reliable and accurate measure of area. Although the device does not reliably capture depth, it captures volume with moderate reliability.

Considering the clinical implications of tracking changes in wound area as a marker for normal healing, this device offers great utility. In instances where the same wound was measured at multiple time points, the 3DWM device, digital planimetry, and manual measurement comparably captured differences in area over time. Overall, the 3DWM device provides reliable measurements that maximize user objectivity, correlate highly with digital planimetry, and exceed the quality of measurements obtained manually. 

Affiliations: Department of Plastic Surgery, MedStar Georgetown University Hospital, Washington, DC; MedStar Health Research Institute, Hyattsville, MD; and College of Medicine, Georgetown University Hospital, Washington, DC 

Correspondence:
Ersilia L. Anghel, BS, BA
Limb Salvage Research Fellow
Department of Plastic Surgery
Center for Wound Healing
Georgetown University Hospital
3800 Reservoir Road, 1-Bles
Washington, DC 20007
ersilia.a@gmail.com

Disclosure: The authors disclose no financial or other conflicts of interest.