Validity of Diagnosis of Superficial Infection of Laparotomy Wounds Using Digital Photography

Abstract: Background. The use of digital photography to assess and document the wound healing process has become increasingly popular. One of the most common complications of wound healing is infection, but the validity of digital photography for the diagnosis of wound infection is unknown. We intended to measure the degree of inter and intra-observer agreement on the diagnosis of superficial wound infection using digital photography. Methods. In a prospective, observational clinical study, abdominal wounds were photographed daily and signs of infection were documented in a standard manner. Four surgeons independently assessed photographs of 50 wounds opened for infection within hours after photography and 50 normally healed wounds (n = 50). Wound pain scores, morning temperature, and postoperative day were noted. Surgeons recorded the presence of infection and treatment for each wound. Paired kappa (κ) values were calculated and intra-observer agreement was measured after 4–6 weeks. Results. Mean specificity with regard to infection was 97% (94%–100%) and mean sensitivity was 42% (32%–48%). Paired κ-values with regard to wound infection were: 0.54, 0.67, 0.68, 0.63, 0.58, and 0.61. Agreement on treatment was present in 76 of 100 wounds (κ values: 0.15, 0.17, 0.20, 0.72, 0.63, 0.68). Kappa values for intra-observer agreement on infection were: 0.66, 0.43, 0.74, and 0.76 for surgeons A, B, C, and D, respectively. Conclusion. Inter- and intra-observer agreement on the diagnosis of superficial infection with digital photography are moderate, but specificity is high. Physical examination findings should also be reported.

Address correspondence to: Gabriëlle H. van Ramshorst, MD Department of Surgery, Erasmus University Medical Center Room Z-836 PO Box 2040, 3000 CA Rotterdam, The Netherlands Email: g.vanramshorst@erasmusmc.nl

     Recently, the use of digital photography has become increasingly popular and highlighted in literature for documentation and evaluation of wound healing progression in addition to its usefulness in telemedicine for diagnosis in dermatology and vascular surgery.^1–3 For chronic and burn wounds in particular, photography can be used to assess treatment results and support continuation or alteration of treatment strategy.⁴ One of the most common complications of surgery is wound infection, as approximately 10% of all abdominal wounds are affected.^5,6 Although the validity of digital photography has been reported for a number of indications, its validity of diagnosing infection in surgical wounds remains unclear.³      The diagnosis of infection of the acute wound has been based on symptoms such as rubor, dolor, calor, tumor, and functio laesa, which have been around since the time of Hippocrates. Physicians diagnose wound infection based on subjective and objective criteria and experience. The international gold standard for diagnosis of surgical wound infection is represented by the criteria for surgical site infection (SSI) as defined by the Centers for Disease Control and Prevention (CDC).⁷ According to the CDC, the surgeon’s judgment is very important in the diagnosis of superficial SSI. In several studies where wound photography was used to assess healed lacerations and incisions, moderate to good inter- and intra-observer agreement was found on wound appearance scales.^8–10 Few reports exist on agreement among surgeons with regard to wound assessment. In the literature, kappa (κ) values for inter-observer agreement on infection vary between 0.08 and 1.00.^1,11,12 Unfortunately, neither the absolute numbers of infected wounds nor the levels of intra-observer agreement were reported in these studies.^1,11,12 The goal of the present study was to measure the degree of inter- and intra-observer agreement on the diagnosis of superficial infection of laparotomy wounds using digital photography thereby assessing its validity.

Materials and Methods

     Between May 2007 and January 2009, 1000 patients were included in a prospective, observational clinical study on surgical wound healing. The Erasmus University Medical Center’s ethics committee approved the research protocol. After informed consent was obtained, the abdominal wound was photographed on a daily basis (including weekends and holidays) until discharge or until the 21st postoperative day using a Fujifilm (Tokyo, Japan) Finepix S5700 digital camera (7.1 megapixels, 10x optical zoom) with standardized multi-auto focus and macrosettings. Each day, two photographs (resolution 3072 x 2304 pixels) were taken according to a standardized protocol. The first photo was of the entire abdomen from sternum to the pubic bone at a distance of approximately 40 cm. The second photo was a close-up photograph of the wound at a distance of approximately 20 cm. Photographs were loaded onto a personal computer and saved in Joint Photographic Expert Group (JPEG) format, coded for each patient, postoperative day, and number of sequence. Signs of infection were documented using a standard procedure and relevant data on wound infection were retrieved prospectively from hospital and nursing charts. Four gastrointestinal surgeons (A, B, C, D) with clinical experience ranging between 10–30 years independently assessed 100 randomly ordered sets of abdominal wound photographs. Each set consisted of one overview photograph and one close-up photograph. Fifty of these sets consisted of photographs of wounds that had been opened within hours on suspicion or presence of infection and had met the criteria for superficial SSI of the CDC.      According to the CDC, the following criteria have to be met for diagnosis of a superficial wound infection (SSI).⁷ Infection occurs within 30 days after the operation and infection involves only skin or subcutaneous tissue of the incision and at least one of the following:      1. Purulent drainage, with or without laboratory confirmation from the superficial incision.      2. Organisms isolated from an aseptically obtained culture, fluid, or tissue from the superficial incision.      3. At least one of the following signs or symptoms of infection: pain or tenderness, localized swelling, redness, or heat, and superficial incision is deliberately opened by a surgeon, unless incision is culture-negative.      4. Diagnosis of superficial incisional SSI by the surgeon or attending physician.      Photographs of infected wounds were matched by postoperative day, type of incision, and skin color with 50 sets of photographs of wounds that had healed without complications, as verified by surveillance by means of outpatient clinic visits following discharge and review of hospital charts, discharge letters, and complication registration systems.      Surgeons were asked to read the CDC criteria for superficial SSI and to apply these criteria if possible before all sessions. Wound pain scores (visual analogue scale ranging from 0 = no pain to 100 = worst imaginable pain) of the current and previous day, morning temperature, and postoperative day were noted for each wound. All photographs were viewed on one laptop using standardized settings with the possibility to adjust the viewing screen (Toshiba A100 portable personal computer, 17-in screen). Surgeons were requested to record for all wounds whether or not superficial infections were present and whether the wounds should be treated conservatively (ie, remain closed) or be opened (either partially or fully). Four to 6 weeks after the initial sessions, all photographs were placed in a different, random order and were re-evaluated in order to measure intra-observer agreement among all surgeons. Repeat evaluations took place in the same room at approximately the same time of day as the initial evaluations.

Statistical Analysis

     Statistical analysis was performed by the first and fourth author (GHvR, WCJH) by calculating paired κ values with 95% confidence intervals (CI) calculated as 1.96 ± SE between all observers (A-B, A-C, A-D, B-C, B-D, C-D) for inter-observer agreement. For each observer intra-observer agreement was measured by calculating κ values (including 95% CI). In general, κ values of 0.80 or greater are considered to represent a good level of agreement.¹³ Sensitivity and specificity was calculated for each observer.

Results

     On average, abdominal wounds had been opened on the seventh postoperative day (range 3–15). Mean specificity with regard to wound infection was 97% (94%–100%) and mean sensitivity was 42% (32%–48%; Table 1). Paired κ values with regard to wound infection varied between 0.54 and 0.68 (Table 2). Agreement on treatment (conservative or opening of wound) was present in 76 of 100 wounds (κ values: 0.15, 0.17, 0.20, 0.72, 0.63, 0.68). The diagnosis of wound infection was unanimous in 12 of 50 cases; two examples of these cases are shown in Figures 1 and 2.      In some cases, surgeons preferred not to open wounds in presence of infection. In 13 patients, symptoms of infection were considered minimal by one or more surgeons, and in five cases spontaneous drainage of pus was present and further opening of the wound was therefore not considered compulsory. Surgeon A was the only surgeon to report low morning temperature as a reason for not opening infected wounds. None of the additional information given on morning temperature or wound pain scores was significantly associated with wound infection in this group of patients (all P > 0.05).      Kappa values for intra-observer agreement varied between 0.43–0.76 for wound infection and 0.52–0.87 for wound treatment (Table 3).

Discussion

     Wound assessment is normally based on a combination of both subjective and objective information, visual and physical information, and experience. This study demonstrated that the inter-observer agreement regarding laparotomy wound infection is moderate among surgeons when using digital photography. The inter-observer agreement on the treatment of wound infection is also moderate and shows high variability among different surgeons. Moreover, the intra-observer agreement on wound infection and treatment differs among surgeons. This implies that wounds are possibly assessed and treated differently depending on which individual is supervising the patient’s care. Infection rates, as collected in several national surveillance programs, might vary between hospitals partly as a result of differences in judgment among physicians.^14–18      Standard protocols for the assessment of acute wounds such as ASEPSIS and the Southampton Wound Assessment Scale are time consuming and have yet to be implemented widely.^14,19,20 Therefore, wounds remain subject to the individual surgeon’s or attending physician’s experience. Two wound examples were rated as infected by two surgeons and as uninfected by the remaining two surgeons (Figures 3, 4). Mild erythema can be seen across the upper two-thirds of the wound (Figure 3). The contrast with the skin-colored lower third of the wound suggests that the erythema is a symptom of abnormal wound healing resulting in the diagnosis of wound infection by two of the four surgeons. A subcostal wound is shown with minimal bloody discharge from the cranial aspect of the wound (Figure 4). The minor dehiscence in combination with the discharge on postoperative day 5 suggests that the wound healing in this part of the wound shows less progression than the rest of the wound; however, the criteria for wound infection are not met by these symptoms alone. The two surgeons who rated the wounds as infected (Figures 3 and 4) also proposed to open the wounds—a sharp contrast with the two other surgeons.      The predictive value of the criteria for wound infection used in the aforementioned protocols is unclear. The European Society for Wound Management reported the results of a Delphi approach in order to identify criteria for SSI in various types of wounds.²¹ In the Delphi approach of the acute wound, 8–10 panel members were asked to list relevant clinical indicators of infection. Panel members were offered the opportunity to review scores for the most important criteria in view of the position of the group as a whole (the “group score”). Cellulitis and pus/abscess were considered the most important factors, followed by delayed healing, erythema with or without induration, hemo- or seropurulent exudate, malodor, and wound breakdown/enlargement. Assumed early signs of infection included increase of local skin temperature, edema, serous exudates with erythema, swelling with increase in exudate volume, and unexpected pain/tenderness.²¹ The predictive value of these signs are yet unknown for acute wounds. Gardner et al²² found positive predictive values of 1.00 for increasing pain and wound breakdown in chronic wounds. Sensitivity of classic signs of infection in chronic wounds showed large variability among different items: heat and purulent exudate 0.18, increasing pain 0.36, erythema 0.55, and edema 0.64.²²      Moreover, from the few studies that exist on inter-observer agreement in wound assessment, it appears that κ values for many of the important variables in the Delphi approach were not high. Hollander et al¹¹ reported inter-observer concordances (κ values) of 0.51 for erythema, 0.39 for warmth, 0.38 for tenderness, and 1.00 for infection of 100 wounds registered in the emergency department by two independent physicians. Allami et al¹² reported inter-observer variations in the evaluation of 50 lower limb arthroplasty wounds between four observers. In this study, poor inter-observer agreement (κ values < 0.40) was reported for tenderness, localized swelling, redness, heat, and moderate agreement for pain (κ values 0.60–0.80), and good agreement (κ values 0.80–1.00) for clinical diagnosis of superficial SSI, purulent drainage, dehiscence, and fever. In a study by Wirthlin et al¹, agreement amongst surgeons in the “remote” assessment of digital photographs of 38 vascular surgery wounds, similar to the present study, proved lowest on the aspects cellulitis/infection and erythema (κ values of 0.08 and 0.28, respectively).¹ The mean κ value for inter-observer agreement on wound infection of 0.62 found in the present study may be fair considering the results of previous studies in which, presumably, fewer infected wounds were included.      In the present study, digital wound photographs were assessed with additional information available on wound pain—expressed as visual analogue scale scores—postoperative day, and morning temperature, which was thought to have been of additional value for the diagnosis of infection and to better simulate the clinical setting. The 2-dimensional aspect of digital photographs hampered assessment of swelling of the wound edges. Palpation of the wound was an aspect that was considered an omission from the regular physical examination of wounds by the surgeons participating in this study. Palpation can provide valuable information in view of expression of wound pain and pus production during pressure exertion and elicit increased capillary refill. Digital photography, even with the provided additional information, seems adequate for diagnosis of “normal wound healing” (ie, no infection) in wounds based on a high specificity of 97%, but at a mean sensitivity of 42% not be sensitive enough to diagnose infections in all wounds. We recommend proper assessment of the entire wound and wound surroundings for erythema, dehiscence, and (purulent) discharge by physicians who are familiar with wound infection criteria to avoid diagnosis mistakes.      Besides the discussion on the use and validity of digital photography in wound assessment, it would appear that criteria for wound infection are not objective enough to establish uniformity in the diagnosis of wound infection. It may also be necessary for doctors to be educated more about present criteria for wound infection. In addition, more research is needed to evaluate the predictive value of wound characteristics for wound infection such as wound temperature and production of exudate, to be incorporated in a standardized wound appraisal tool. Structural assessment of wounds, combined with onsite registration of SSI and plenary discussion will undoubtedly result in more uniformity amongst surgeons and higher reliability of reported infections and infection rates.

Conclusion

     Inter- and intra-observer agreement on the diagnosis of wound infection when using digital photography were both moderate, but specificity was very high. Findings of physical examination, palpation in particular, could present valuable information for electronic wound assessment. We recommend that these findings be documented in detail and presented in conjunction with digital wound photographs in the electronic assessment of infection in wounds. Furthermore, we believe more education is needed on wound assessment and criteria for wound infection, and that more data are needed on the predictive value of wound characteristics for infection.