Inter- and Intra-observer (Dis)agreement Among Physicians and Nurses as to the Choice of Dressings in Surgical Patients with Ope

The first step in the management of patients with open wounds is determined by the patient’s condition and the wound’s underlying etiology, which guide systemic treatment. The next step is local wound care (ie, wound bed preparation and appropriate dressing choice).^1,2 For this purpose, an overwhelming number of wound care products that help protect, cleanse, debride, and eventually heal the wound have flooded the market. Currently, almost 2 generations of dressings are in use. The first, classical generation is comprised of gauze-based dressings. The second and more modern generation of dressings (eg, foams, hydrocolloids, alginates, hydrofibers, gels, films)³ are based on the principle of occlusive wound healing.⁴
The choice of the best dressing for local wound management is influenced by the type and appearance of a wound (ie, color and amount of exudate), presence or absence of pain, required dressing change frequency, and dressing and personnel costs.⁵ Decision-making regarding local wound care is difficult due to the abundance of influencing factors (eg, age, wound size and etiology, nutritional status) and large number of available dressings.⁶ However, appropriate dressing choice not only is an essential part of wound bed preparation^1,7,8 but also has patient and economic consequences.⁶
Ideally, dressing choices should be based on scientific evidence. At present, however, dressings are still being chosen on the basis of local traditions, empirical experience, and limited evidence from few controlled trials and systematic reviews.^9–12 In the literature, many reviews have attempted but failed to facilitate this choice and merely stress the importance of matching the appropriate dressing with the appropriate wound type.^13–15
For this purpose, several classification systems have been introduced.^2,16
The large number of wound care materials,¹² the many types of wounds, and the limited evidence available on the effectiveness of various dressings are a threat to consistent choice of the best suitable dressing material among wound care professionals and, thus, optimal wound care. The authors investigated this possible inconsistency by assessing inter- and intra-observer agreement among surgical physicians and nurses and their agreement with the expert panel regarding choice of gauze-based and occlusive dressings in hospitalized surgical patients with open wounds.

Patients and Methods

Study setting. The study was performed in the Academic Medical Center of the University of Amsterdam (Department of Surgery), a tertiary referral center. All patients admitted to the surgery ward during the second half of 2004 with open wounds that were treated with gauze-based or occlusive dressings were asked to participate in the study. Digital photos were taken of the wounds of the 90 patients who agreed to take part in the study. The Academic Medical Center’s medical ethics committee approved the study.
Expert panel. An expert panel was comprised of 4 experts in wound care from the Academic Medical Center—a surgeon experienced in wound care; 2 tissue viability nurses from the outpatient surgery and dermatology departments; and 1 tissue viability nurse from the surgery ward. These experts were all members of Academic Medical Center’s Wounds and Materials Committee, which coordinated local wound care policy.
This panel was asked to decide which gauze-based dressing (ie, dry, wet, or paraffin gauze) and which occlusive dressing (ie, foam, hydrocolloid, hydrogel, alginate, and film) they would choose for local wound care (Figure 1).
From the 90 digital photos, the panel chose 18 of high quality to represent the broad spectrum of wound types and etiologies seen in clinical surgical practice (eg, postoperative and traumatic wounds, diabetic and pressure ulcers, and wounds caused by vascular insufficiency). Size, color (red, yellow, or black), and amount of exudate (wet, moist, or dry) were also considered. The panel was unanimous with their dressing choice in these 18 cases. Their judgment was regarded as the reference standard.
Observers. All 79 Department of Surgery physicians (surgeons, residents, surgical researchers, and interns total) and 63 nurses (registered and in training) of all surgical wards were asked to judge these photos in order to calculate the inter-observer agreement among nurses and physicians. The observers were guaranteed confidentiality and anonymity regarding the data obtained. The authors chose observers with a wide range of clinical experience to mimic daily clinical practice and to investigate whether experience would influence decision making. Observer answers were compared to the expert panel to estimate whether observer judgment was “correct.” Later, observers were invited to judge these photos a second time in order to calculate individual intra-observer variation. For this purpose, the same slide set was presented to the nurses on duty from all surgical wards 8 times in 5 weeks to reach a sufficient number of nurses (63) to rate the slides twice.
The photos were shown to the physicians twice with a 2-month interval. For both groups of observers, the order of the slides was altered at each presentation.
Data collection. During each viewing session, basic demographic data of the 2 observer groups were collected anonymously (age, sex, present educational level, and years of medical experience). Subsequently, each wound slide was shown on a large projection screen for 30 seconds while the observers chose their preferred gauze-based and occlusive wound dressing. In the case where more than 1 dressing should be applied (eg, an alginate covered by a foam dressing), the authors asked the observers to record only the first material (alginate) that would be applied. Any additional wound care interventions desired, such as additional antiseptics or wound debridement, were neglected. No discussion was allowed during the presentation. The gauze-based and occlusive dressing materials the observers had to choose from were commonly available in the authors’ hospital.
Data analysis. Inter- and intra-observer agreement was calculated using AGREE version 7 (Science Plus Group, Groningen, The Netherlands), a software program used to calculate weighted or group kappa (κ) values. The group κ parameter is a measure of the agreement between categorical assessments in 2 groups in excess of chance. Kappa values usually lie between 0 and 1. A κ value > 0.8 is considered “very good;” between 0.8 and 0.6 is regarded as “good;” between 0.6 and 0.4 “moderate;” and below 0.4 “poor.”¹⁷
To assess inter-observer agreement in the physician group and nurse group and their agreement with the expert panel, the pairwise κ value was calculated, which allows for computing an unweighted group κ for several fixed raters based on the mean observed and the expected mean agreement amount.¹⁸ To assess the effect of any missing values on the resulting κ values, missing data were imputed using the AGREE software “series mean value” option, which replaces missing values with the mean for the entire series. If more than 50% of the answers to a certain wound characteristic were missing or unclear, the data for that particular physician or nurse and dressing material were excluded from the analysis.
To assess the intra-observer agreement between scores of an individual nurse or physician, the standard, unweighted Cohen’s κ value was calculated.¹⁹ Differences in κ values between nurses and physicians were statistically analyzed using the Mann-Whitney U test because the values were likely to be unevenly distributed. Any correlation between observer characteristics and κ scores was expressed using the Spearman correlation coefficient ρ. Statistical analysis was performed using SPSS version 12 (SPSS Inc, Chicago, Ill).

Results

Observers. A total of 79 physicians from the Department of Surgery and 63 nurses from the surgery wards attended at least 1 slide presentation and completed a score form.
Personal characteristics of the observers are shown in Table 1. Male physicians were significantly (P < 0.001) older than female physicians. Consequently, male physician experience with local wound care in the Department of Surgery (median, 2 years) was slightly longer than female physicians (median, 1 year; P < 0.05). However, the total clinical experience for surgeons did not differ significantly between male and female physicians.
Not all score forms were entirely completed. Missing data were particularly seen regarding occlusive dressing choice (physicians 33%; nurses 5%) due to unfamiliarity with these dressings.
Inter-observer agreement. In both groups, the inter-observer agreement regarding occlusive and gauze-based dressings was low with κ scores between 0.07 and 0.23 (Table 2). Kappa values did not change substantially after the missing values were added. Although the nurses performed slightly better, this was not significantly different from the physicians’ performance.
Agreement with expert panel. Agreement regarding dressing choices was low (κ scores between 0.14 and 0.32), indicating both groups showed a considerable discordance with the reference standard (Table 3). All agreement results had acceptable 95% confidence intervals.
Intra-observer agreement. Thirteen physicians rated the photos twice. These 13 physicians were regarded as representative for the entire physician group because they were able to attend a second viewing session and did not know the results of the previous session. Ten physicians correctly completed their choice of gauze-based dressings, and 5 for occlusive dressings during the 2 sessions. Physician intra-observer agreement, expressed as medians, was “moderate” for gauze-based dressing choice (κ = 0.56; range: 0.05 to 1.0) and “poor” for occlusive dressings (κ = 0.19; range: -0.01 to 0.44).
Fourteen nurses judged the wound photos twice. These 14 nurses were regarded to be representative for the same reason previously mentioned for the physicians. Twelve nurses completely filled out their score forms for occlusive dressing choice. The nurses’ median intra-observer agreement was “moderate” for gauze-based dressing choice (κ = 0.49; range: 0.01 to 0.90) and “poor” for occlusive dressings (κ = 0.26; range: 0.10 to 0.47).
Although these intra-observer data are limited, the data suggest that the opinions of physicians and nurses regarding dressing choice vary from time to time.
Correlations between expertise and agreement. The correlation between the agreement in wound dressing choice and the amount of total clinical experience was examined, as derived from observer age. A small but significant positive correlation was found regarding the physicians’ ages and their agreement with the expert panel regarding occlusive dressing choice (ρ: 0.29; P = 0.033) and a reasonable correlation for gauze-based wound dressing choice (ρ: 0.63; P < 0.001). Kappa values remained low in occlusive dressing choice even for senior surgeons (Figures 2 and 3). No such correlation was found in nurses.
In general, a higher degree of observer experience did not increase agreement substantially.

Discussion

This study shows that the variation in occlusive and conventional gauze-based wound dressing choice is significant between physicians and nurses in surgical practice and diverges strongly from the referenced standard as generated by an expert panel. The level of experience did not substantially influence the results. The tertiary teaching hospital surgical department setting where a certain level of knowledge on this matter could be expected makes these secondary findings even more surprising.
The findings in this study regarding the poor agreement on appropriate dressing choice and the poor knowledge of occlusive dressings suggest much work is needed on the part of wound care researchers and instructors. Convincing evidence on the effectiveness of the various dressing types must be obtained first.
These findings should be incorporated into the standard or dedicated educational programs in order for physicians and nurses to significantly improve the quality of care for patients with (surgical) wounds.
The poor inter- and intra-observer agreement may be an overestimation of the true agreement for several reasons. First, the fact that not all score forms were filled out completely, especially regarding physician occlusive dressing choice, may have overestimated the agreement. However, no substantial differences in the sensitivity analysis were found by adding the missing data.
Second, the use of more alternatives when choosing a dressing (5 options for occlusive materials) may have further reduced the measured agreement, although agreement was already low when choosing between only 3 gauze-based dressing options. Third, κ values can be influenced by the prevalence of the various groups to be classified.²⁰ However, the 18 digital photos were a clinically representative sample in which the wound condition etiologies were evenly distributed.
A study by Bux and Malhi²¹ corroborates these findings. Bux and Malhi conducted an audit of wound dressing selection and use. They found an appropriate dressing selection was made for 48% of the wounds, while appropriate dressing selection and use were identified for 20% of the wounds. Additionally, the authors of the present study found that nurses performed just as well as the physicians in dressing choice. A significant positive correlation was found for the more experienced surgeons who performed better than younger physicians in the choice of a gauze-based dressing but still performed poorly when choosing an occlusive dressing.
Previous studies have shown that wound classification based on color is feasible and shows moderate to good observer agreement.^22–24 A common and simple wound classification scheme is the red-yellow-black scheme¹⁶ that yields 9 categories based on wound color and exudate. Physicians and nurses had moderate to good inter-observer agreement regarding wound classification based on this scheme.²⁵ Suitable wound dressing suggestions can be given for each wound category. Thus, such a scheme can be used as a guide to help improve optimum dressing choice, particularly when experience or knowledge regarding dressing materials is deficient. Use of this scheme should improve local wound care uniformity and quality, although presently there is a lack of evidence to support a direct relation between correct dressing choice and patient outcomes. Ideally, uniform dressing choice should be based on scientific data because only then is quality of care secured. Subsequently, successful implementation of such a classification scheme to guide dressing choice needs to be assessed. This assessment is needed for dressing-related education programs as well. Success should be defined not only as an increase in uniform dressing choice but also in terms of patient-related outcomes. Furthermore, it has been suggested that local wound care and wound healing may benefit by being integrated into an effective multidisciplinary care program.²⁶
The use of digital photos may have been a drawback of this study. These 2-dimensional images do not allow assessment of the depth, odor, and exact moistness of the wound, which would help in dressing material selection. Moreover, odor and other physical parameters cannot be captured in photos. For practical reasons, nurse and physician judgment of the wounds could not have been done simultaneously, but in a certain amount of time, in which the wound characteristics could have changed. Other studies also preferred the use of color images instead of in-vivo appraisal.^23,27
For practical reasons, only 1 dressing was regarded to be a correct choice. This stipulation may be disputable but was based on the expert panel consensus. Currently, high-level evidence on which dressing is most effective and should therefore be the optimum choice is limited. Additionally, it is not known to what extent a “wrong” dressing selection would have in terms of patient-relevant outcomes. Moreover, this study was conducted to measure the (dis)agreement among healthcare professionals rather than to assess the ability to choose the “correct” dressing.

Conclusion

Disagreement was found regarding the choice of wound dressings among nurses and physicians. The many materials available and varying local wound care opinions hamper creation of a uniform approach to treat open wounds. Better education for wound care professionals regarding dressing materials and wound classification and an evidence-based selection from the wide variety of available wound care products is advocated to optimize wound care.

Acknowledgment

The authors thank the expert nurses at Academic Medical Center at the University of Amsterdam, Wounds and Materials Committee, Nel J. Lageweg, Els Dam, and Wilna H. Maarleveld, for judging the digital photos used in this study. Their assistance made this study possible.