The SYSTEMIC Project: A Pilot Study to Develop a Registry of Patients With an Ostomy for Predictive Modeling of Outcomes

Ostomy surgery is a life-saving procedure that allows body waste to pass through a surgically created stoma in the abdomen and is used to treat several medical conditions including congenital deficiencies as well as rectal and bladder cancer.¹ In Italy, in 2015, more than 74 000 people used an ostomy device.² It is anticipated that the prevalence of ostomies will continue to increase due to the increasing prevalence of colorectal cancer, as reported in the AIOM, AIRTUM report in 2016.² The prevalence of postprocedural complications after stoma surgery ranges from 6% to 70%^3,4 and depends on various factors.⁵ Peristomal complications negatively affect morbidity, health care costs, and health-related quality of life.⁶

Preoperative stoma site marking by a wound ostomy and continence (WOC) nurse has been shown to reduce stoma and peristomal complications, as reported in a recent meta-analysis on preoperative site marking.⁷ WOC nurses have the educational preparation and skills to manage patients with an ostomy and are experts in procedures ranging from ostomy site selection to postprocedural follow-up management. In Italy, a 1-year postgraduate course is required to become a qualified WOC nurse. The training is based on lectures and traineeship in specialized centers. However, skilled postoperative care is not always guaranteed for patients with an ostomy. In many cases, short hospital stays and long-term follow-up planning based on only a postoperative assessment make patient supervision problematic, even for experienced professionals.⁸ In the United States and Canada, the ratio of WOC nurses to patients with an ostomy is 1 to 200.⁸ In Italy, few hospitals offer a structured follow-up program by skilled professionals for these patients. Moreover, current active programs are hospital-based and rarely involve rural areas, as reported by Pittman et al⁸ in a study evaluating web-based support for patients who had recently undergone ostomy surgery. Hence, in many cases, ostomy care is provided by nonspecialized health care workers or unskilled caregivers.⁹ This may affect the risk of the development of complications.¹⁰

Understanding the risk factors influencing ostomy complications can help prevent them, if the factors are modifiable and addressed, as also suggested in the Italian Guidelines for the Surgical Management of Enteral Stomas in Adults (level of evidence Grade 1C¹¹); thus, this is crucial for appropriate patient management. A risk profiling procedure may help when ostomy care will be provided by unskilled persons, and standardized risk profile tools can help to prevent ostomy complications. A standardized tool is not intended to replace the knowledge and expertise of WOC nurses. However, as suggested by Beitz et al,¹² such a tool could be useful to nonspecialized clinicians. In addition, a registry of patients with an ostomy would be useful as a surveillance instrument and as a continuous quality-improvement tool. Most importantly, it may facilitate future studies in ostomy management, as suggested by Hollander et al¹³ in their study on the development and validation of a wound registry. In Italy, a national ostomy registry is not available; the unique data source for patients with an ostomy is the database of the Federation of People with Incontinence and an Ostomy Association, which does not collect data systematically.¹⁴

The oStomY regiSTry prEdictive ModelIng outCome (SYSTEMIC) project aims to define a new concept of the ostomy registry that is tailored not only to an etiological investigation, but also to prospectively improve patient-oriented outcomes (eg, a reduction in ostomy complications). The present work aims to present the SYSTEMIC initiative by describing the 1) setup of the registry, 2) preliminary data collected since the start of the project, and 3) development and implementation of an algorithm that predicts the presence of ostomy complications based on the registry data.

Registry development and structure. The SYSTEMIC registry collects detailed data about patients (sociodemographic characteristics and clinical information) as well as baseline and follow-up characteristics of the ostomy and pouching system (eg, shape, use of additional aids). The registry database is accessed only through the Research Electronic Data Capture (REDCap) data management application.¹⁵ The project was developed under an umbrella cooperation agreement between the Health Professional Management Service of the University Hospital of Padova and the Unit of Biostatistics, Epidemiology and Public Health of the Department of Cardiac, Thoracic, Vascular Sciences, and Public Health at the University of Padova, which is also in charge of its maintenance. Registry development included the clinician and the research team and included the following steps: 1) creation of a new project in REDCap with the completion and submission of all required information regarding the project settings, 2) creation of data collection instruments after the definition of variables and their properties, 3) preview and test of data entry, 4) configuration of study members’ permissions and users’ rights according to their role in the project, and 5) pilot data collection. Access to the final dataset was authorized only to the Unit of Biostatistics, Epidemiology and Public Health staff through their usernames and passwords. The clinical data are encrypted and stored on a secure server. Files downloaded from REDCap for analysis are deidentified using the export page deidentifiers.

Inclusion criteria. In the registry, data were collected from adult patients with any type of gastrointestinal or urinary stoma. For preliminary data analysis, patients with ureter-ileum ostomy and jejunum ostomy were excluded due to the limited number of cases.

Study design and setting. The prospective observational registry was started in June 2018 at the University Hospital of Padova. Patients admitted for the first time to the ostomy ambulatory were consecutively included in the database. The Padova University Hospital ostomy ambulatory is 1 of 2 referral centers for ostomy and incontinence care in the Veneto region, Italy. Services include management, consultation, and protocol creations for the continuity of care after hospital discharge.

Study variables. All variables included in the registry were selected by consultation with the literature and discussion with WOC nurses in the ostomy ambulatory. The variables used for study analysis were as follows: 1) sociodemographic characteristics (sex, age, diagnosis, comorbidities [yes/no], and body mass index (BMI) [healthy, overweight, obese]) and social situation (living alone, with a spouse, with family, or in a community); 2) baseline characteristics of the ostomy (elective or emergency surgery, type of ostomy [colostomy/ileostomy], preoperative ostomy marking [yes/no], size and features of the ostomy location, ostomy duration [temporary or permanent], characteristics of the pouching system [one/two pieces, flat/convex/convex light, flange presence, manufacturer], ostomy complications [yes/no], presence or absence of preoperative radiation therapy and chemotherapy); and 3) follow-up evaluation (management of the ostomy [caregiver assistance, can the patient see the stoma?], ostomy complications, and changes in the pouching system used). A half-circular ostomy guide in millimeters was used to measure the diameter and height of the stoma, and the measure was taken from the base to the edge.

Data management and safety. Data entered in the registry are deidentified and collected in the context of routine clinical practice. During hospital admission, patients provide their consent for data use for scientific purposes. For this reason, it was not necessary to obtain ethical board approval for the current study. Electronic case report forms developed within the REDCap platform¹⁵ were used for data collection. The study design, along with the data collection procedure, was performed while adhering to clinical practice regulations. The treatment of personal information was carried out in agreement with the Italian legislative decrees 211/2003-196/2003 and the European Regulation 2016/679.

If peristomal complications were detected at the follow-up evaluation, they were treated directly in the ambulatory department or referred to the appropriate clinicians.

Data collection procedures. All data were collected during routine visits (Figure 1). Baseline data were collected at the first examination before hospital discharge but after the procedure; follow-up data were collected during the first follow-up visit after hospital discharge, which usually took place at least 30 days after surgery. Patients not requiring an ambulatory evaluation were contacted by telephone. They were asked for information on stoma characteristics and possible complications.

Patients reporting complications (without an evaluation performed by other health care professionals) were referred to the ambulatory department for further assessments. For patients evaluated by other health care providers, complications are reported when their pouching system is changed after the baseline visit.

Data entry and quality control. Data were entered directly into the database created for the study. REDCap allows restricted data formatting, ranges of numbers and dates, data validation, and warnings if the entered data violate specific limits. When outliers or discrepancies were encountered, health care personnel were consulted.

Bayesian machine learning framework. A machine learning (ML) algorithm within a Bayesian framework was used to develop an algorithm for predicting complications. The advantage of using an ML algorithm lies in the fact that it is not looking for inferring associations between the variables and outcomes as in traditional statistical models (eg, linear or logistic regression), but instead is looking for a prediction model. ML algorithms are more suitable to identify complex and nonlinear relationships in the data. Moreover, Bayesian inference was chosen because it can incorporate in the final inference the information provided by the data (objective prior) or by the expert’s opinion (subjective prior).^16,17 This information is considered “prior” because it exists before or regardless of the data.¹⁸ This approach allows explicit probability statements about the hypothesis to be made, which cannot be inferred through classic methods.¹⁹ Noticeably, Bayesian inference treats all sources of uncertainty in the modeling process, allowing maximum flexibility in the modeling procedure.²⁰ More specifically, the authors used the naïve Bayes classifier, which is a method useful for modeling probabilistic associations between categorical variables.²¹ This method has been used with good performance in studies with small sample sizes, as in the current study.²²

Use of the Bayesian ML technique is increasing in clinical studies, especially for predicting outcomes. For example, Fojo et al²³ developed a Bayesian algorithm in 2444 participants from 2 National Network of Depression Centers and the Johns Hopkins HIV Clinical Cohort to predict mental health symptoms (eg, depression, anxiety, and mania) and substance use (eg, alcohol, heroin, and cocaine). Another recent study²⁴ compared 3 Bayesian ML techniques, naïve Bayes, Bayesian Network, and Bayesian Additive Regression Trees, to predict extraintestinal manifestations of Crohn’s disease; in this case, the algorithm could be important for clinical decision-making. The naïve Bayes classifier also was used to explore the health care orientation process indicators in newly hired nurses and physicians.²² It was also applied to predict patient hospitalization and emergency department visits in home health care clinical notes²⁵ and to predict hospital admissions at the emergency department.²⁶

Algorithm development. The naïve Bayes algorithm²¹ was employed to predict the risk of developing complications. The naïve Bayes model predictors were ostomy type, ostomy duration (temporary or permanent), condition necessitating the ostomy surgery, BMI, surgery type, preoperative site marking, ostomy location, and the patient’s ability to see the ostomy site. Predictors were chosen based on those reported in the literature as risk factors for ostomy complications.^7,27,28

Two scenarios concerning the prior choices were implemented to illustrate how the prior information may affect and enhance the algorithm’s performance.

Data-driven prior model. The ostomy complication distribution frequency, provided by the SYSTEMIC registry data, was considered to define the prior probability distribution.

Objective prior model. The data provided by studies conducted in similar research settings were considered to derive a prior distribution for the frequency of complications among patients with an ostomy. A frequency of ostomy complications of 50% was seen in similar studies in the literature.^3-5 Bayesian ML models were adopted to obtain the posterior probability distributions from the data.²⁹

The naïve Bayesian predictive performances were evaluated using the following measures: accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value, and the area under the ROC curve (AUC).

SYSTEMIC registry expected sample size. To determine a reasonable minimal sample size for this pilot study, the authors assumed an incidence rate of complications of 40%.²⁷ A sample size estimation procedure for the proportion of ostomy complications was performed considering a precision approach on the confidence interval estimate. A pilot study on 52 patients will ensure a precision (error) of the final estimate of 13.5% for a 95% confidence interval. Considering a sample size of 200 patients (approximately 6 months of expected recruitment), the confidence interval error will decrease by 7%.

Statistical analysis. Continuous variables were reported as quartiles I, II (median), and III and percentages for categorical variables. The Wilcoxon test was performed for continuous variables; the likelihood ratio chi-square test from the proportional odds model was performed for categorical ordered variables, and the Pearson chi-square test was performed for categorical nonordered variables.³⁰ For the analysis, diagnosis was grouped into 4 main categories (diverticula and bowel perforation, inflammatory bowel disease, neoplasia, and other). BMI was calculated by dividing weight (in kilograms) by height (in square meters). Statistical analysis was performed using the BlueSky Statistics System and R statistical software.

The current analysis included preliminary data gathered in the SYSTEMIC registry on 52 patients admitted to the ambulatory ostomy department from June to September 2018.

Clinical and demographic data at baseline. Among the 52 patients ambulatory department patients, 28 (53.3%) had a colostomy and 24 (46.7%) had an ileostomy. Population characteristics and medical history are summarized in Table 1. Cancer was the leading cause of ostomy surgery in those with (16 patients, 52%) and without (7 patients, 33%) complications (Table 2, Part 1 and Part 2). None of the patient demographic or clinical characteristics were significantly different between persons with a colostomy or ileostomy. Among the patients with an ostomy, 70% underwent elective surgery and 30% underwent emergency surgery. Among the total 52 patients, 33 had the preoperative site marked by a WOC nurse (Table 2, Part 1 and Part 2).

Ostomy and pouching system characteristics. Twenty-three (23) stomas (44%) had a round shape, and 29 (54%) had an irregular peristomal area (Table 2). Thirty-one (31) patients reported complications, and no differences were detected in the porportion of patients with ileostomy and colostomy (Table 1).

Follow-up data. Only 12 patients had access to outpatient follow-up 30 days after surgery; the remaining patients were contacted by telephone. Complications were reported in 31 patients (59.6%). In terms of postoperative complications, no significant differences were observed between patients with different BMIs (P = .16), those who underwent elective surgery compared with those who underwent emergency surgery (P = .66), and those who did or did not have the site marked preoperatively (P = .44). Radiation therapy before surgery and pouching system convexity showed significant differences between patients with or without complications (P = .012 and P < .001, respectively). Patients using convex pouching systems experienced fewer complications (1 patient [3%]) (P < .001). Forty-five (45) patients (87%) could see the stoma, and 30 of these developed complications (P = .009) (Table 2).

Predictive modeling. Table 3 presents the results of the naïve Bayes algorithm: the data-driven prior model (DDPM) includes a 0.596 prior complication probability in the final naïve Bayes inference. This probability value is derived from the registry data in Table 2. The objective prior model (OPM), consistent with the literature, considers a prior probability of peristomal complications equal to 0.5.^3-5 The probability distribution of ostomy complications is coherent in the 2 models according to the different covariates; that is, the higher the number is, the higher the probability of developing complications. The probability of developing complications in the DDPM was greater for persons with neoplasia or other diseases (0.3314 for both levels) than for patients with diverticula and bowel perforation (0.1453) or inflammatory bowel disease (0.1918) (Table 3).

Predictive accuracy and other performance measures of both models are reported in Table 4. Similar performance measures were observed for both models. The accuracy was 69.4% (95% confidence interval: 0.546–0.817) for both DDPM and OPM models. Sensitivity was 52.4% and 33.3% for DDPM and OPM, respectively. OPM showed a greater specificity (96.4%) than DDPM (82.1%). An increased PPV (87%) was observed for the OPM versus the DDPM (68.7%). The 2 models were similar in terms of the capacity to distinguish between classes (ie, the same ability to discriminate between subjects with and without complications) (AUC, 71.51%) (Table 4).

One of the most critical care issues for patients with an ostomy is preventing peristomal complications. The study by the Federation of People with Incontinence and an Ostomy Association revealed that patients are concerned about quality of life and about identification of the most suitable ostomy pouching system (99%).¹⁴ Correct ostomy pouching system identification and subsequently reduced complication rates could improve the quality of life of patients, avoiding embarrassment due to leakage, loss of working days, and altered social life.^28,31 Considering the small size of the sample and short follow-up time, the incidence of ostomy-related complications in the current study (31 patients, 59%), was high compared with the study by Carlsson et al³² (207 patients, 35%). This difference could be related to the fact that in the current study, the authors evaluated complications at a 30-day follow-up, and the results of Carlsson et al³² were based on a 1-year follow-up.

Conversely, the incidence of ostomy-related complications in the current study was consistent with the results of the studies by Herlufsen et al³³ (202 patients, 45%) and Lyon et al³⁴ (235 patients, 73%). The varying results among studies could be related to the person who reported the complications. In the studies by Herlufsen et al³³ and Lyon et al,³⁴ complications were reported by the patients. In the study by Carlsson et al,³² complications were evaluated using the definitions established in the study by Colwell and Beitz.³ In the current study, the presence or absence of complications was evaluated by a WOC nurse. Other studies also reported a lower level of complications. Ratliff³⁵ reported a 47% complication rate as determined by WOC nurses; however, that study had a 2-month follow-up. Additionally, Ayik et al³⁶ reported a lower rate of complications (36.2%); however, those complications were evaluated 30 days after the procedure. Prompt identification of peristomal complications, especially with a decision-support tool, can reduce the number of visits and purchases of equipment with a consequent decrease in related costs.¹⁰

The developed algorithms show a similar ability to correctly classify patients according to the presence of peristomal complications (accuracy of nearly 70% in both models) and a good overall discriminating ability (AUC by almost 71% in both models).

When external objective prior information on the prevalence of peristomal complications was included, the model showed higher values of sensitivity and PPV, suggesting an improved ability to classify persons with complications correctly. Overall, the use of prior information does not seem to strongly enhance the model’s predictive ability. However, the moderate performances of the 2 models suggest that a good decision-support tool can still be implemented using only information from data without the integration of external sources.

To the authors’ knowledge, this is the first ostomy registry aimed at developing an algorithm tailored to predict the onset of ostomy complications. It has been documented in the literature that registry data collection permits the gathering of a considerable amount of data for monitoring purposes, risk profiling activities, and cost estimates.¹⁴

The SYSTEMIC registry has a modular design, allowing the collection and management of data and expert opinions useful for clinical research. The authors believe that the registry will provide more meaningful information on ostomy management with larger cohorts and with longer follow-up. In the future, these potentials may be enhanced by extending the gathering of data to secondary databases, improving their quality,³⁷ and facilitating comparisons between centers.³⁸

Several challenges have been faced in registry development and management, such as the lack of resources to enter data from patient visits in the registry. Further efforts are needed to develop an automatic data transfer program from the hospital information system to the online platform.

This is a single-center study conducted in a hospital with 5 experienced WOC nurses able to assess the appropriateness of the choice of pouching system. Thus, external study validation needs to be conducted.

The low sample size may limit the performance of the developed algorithm. A larger sample size will likely improve the predictive ability of the model and, thus, the reliability of the results.

This ostomy registry can still be improved by cooperating with both the patients and caregivers. Because the care of a patient with a new ostomy requires multiple follow-up visits,³⁹ such visits should be included in future studies.

The SYSTEMIC project aims to define a new concept of the ostomy registry that is tailored not only to an etiological investigation, but also to prospectively improve patient-oriented outcomes (eg, a reduction in ostomy complications). The present work aims to present the SYSTEMIC initiative by describing the 1) setup of the registry, 2) preliminary data collected since the start of the project, and 3) development and implementation of an algorithm that predicts the presence of ostomy complications based on the registry data.

In this pilot study, the data collected in the SYSTEMIC registry were useful in creating an algorithm that predicts the risk of developing ostomy complications. Radiation therapy before surgery and pouching system convexity showed significant differences between patients with or without complications (P = .012 and P < .001, respectively). Patients using convex pouching systems experienced fewer complications (1 patient, 3%) than those who did not. These results may be influenced by the unbalanced numbers in the 2 groups (convex vs flat).

The results also showed that the ability to see the stoma does not exclude the possibility of developing complications: 30 patients developed complications among 45 patients who were able to see the stoma. This result seems counterintuitive but could be biased by the fact that there were only 7 patients who were not able to see the stoma.

The algorithm does not replace the skills of WOC nurses. Instead, it helps nurses without WOC training to identify the appropriate pouching system through an evidence-based process. This research, with subsequent implementation and external validation on a new dataset, may be the basis for the development of an easy-to-use algorithm to help nonspecialized nurses promptly evaluate the onset of complications. The implementation and enhancement of this registry may improve the quality of patient management and constitute a tool for standard surveillance and treatment. In addition, this registry could be used for comparison and benchmarking among care centers.

Regarding the implementation of the algorithm, it is necessary to collect more data to achieve the sample size presented in the Methods section. Furthermore, the use of expert opinion, together with objective data, may potentially improve the implementation of the algorithm. This could potentially be done by deriving a prior distribution via a prior elicitation process from the WOC opinion (subjective prior) and combining it with the likelihood derived from the observed data.⁴⁰

In clinical practice, the model, with proper implementation, may be used as a screening tool for detecting patients at risk of ostomy complications, directing them to different follow-up pathways. Patients with a higher risk of complications can be followed by WOC nurses or general practitioners. This differentiation in pathways can help reduce the number of visits at the ostomy ambulatory site.

Ms. Ocagli and Mr. Bottigliengo are PhD candidates in biostatistics and clinical epidemiology; Dr. Lorenzoni is a clinical epidemiologist; Dr. Gregori is a full professor, Unit of Biostatistics, Epidemiology and Public Health, Department of Cardiac, Thoracic, Vascular Sciences, and Public Health, University of Padova, Padova, Italy. Mr. Giorato and Ms. Barbierato are wound, ostomy, continence nurses; Ms. Stivanello and Mr. Degan are nurse managers; and Ms. Turra is a nurse, Health Professional Management Service of the University Hospital of Padova, Padova, Italy. Dr. Azzolina is a biostatistician, Department of Translational Medicine, University of Piemonte Orientale, Italy. Address all correspondence to: Dario Gregori, PhD, Unit of Biostatistics, Epidemiology and Public Health, Department of Cardiac, Thoracic, Vascular Sciences, and Public Health, Via Loredan, 18, 35131, Padova, Italy; tel: +39 049 8275384; fax: +39 02 700445089; email: dario.gregori@unipd.it.