Construction and validation of a risk assessment model for acute kidney injury in patients with acute pancreatitis in the intensive care unit

Abstract

Background

To construct and validate a risk assessment model for acute kidney injury (AKI) in patients with acute pancreatitis (AP) in the intensive care unit (ICU).

Methods

A total of 963 patients diagnosed with acute pancreatitis (AP) from the Medical Information Mart for Intensive Care IV (MIMIC-IV) database was included. These patients were randomly divided into training set (N = 674) and validation set (N = 289) at a ratio of 7:3. Clinical characteristics were utilized to establish a nomogram for the prediction of AKI during ICU stay. These variables were selected by the least absolute shrinkage and selection operation (LASSO) regression and included in multivariate logistic regression analysis. Variables with P-values less than 0.05 were included in the final model. A nomogram was constructed based on the final model. The predicted accuracy of the nomogram was assessed by calculating the receiver operating characteristic curve (ROC) and the area under the curve (AUC). Moreover, calibration curves and Hosmer-Lemeshow goodness-of-fit test (HL test) were performed to evaluate model performance. Decision curve analysis (DCA) evaluated the clinical net benefit of the model.

Results

A multivariable model that included 6 variables: weight, SOFA score, white blood cell count, albumin, chronic heart failure, and sepsis. The C-index of the nomogram was 0.82, and the area under the receiver operating characteristic curve (AUC) of the training set and validation set were 0.82 (95% confidence interval:0.79–0.86) and 0.76 (95% confidence interval: 0.70–0.82), respectively. Calibration plots showed good consistency between predicted and observed outcomes in both the training and validation sets. DCA confirmed the clinical value of the model and its good impact on actual decision-making.

Conclusion

We identified risk factors associated with the development of AKI in patients with AP. A risk prediction model for AKI in ICU patients with AP was constructed, and improving the treatment strategy of relevant factors in the model can reduce the risk of AKI in AP patients.

Background

Acute pancreatitis (AP) is a form of acute abdominal distress that originates from the abnormal activation of pancreatic enzymes, resulting in digestive effects on the pancreas and surrounding organs. This condition is marked by local inflammatory reactions within the pancreas and can lead to organ dysfunction. AP is one of the most prevalent gastrointestinal diseases, causing significant physical, psychological, and economic burdens on patients [1]. Global incidence rates of acute pancreatitis range from 4.9 to 73.9 per 100,000 individuals, and the annual incidence rate is on the rise [2]. However, overall mortality rates have remained unchanged [1, 3, 4]. The diagnostic criteria for AP [5] include persistent upper abdominal pain, serum amylase and/or lipase concentrations at least three times higher than the upper limit of normal, and imaging results indicative of acute pancreatitis. Meeting two of these three criteria is sufficient for diagnosis. Most cases of AP are mild and self-limiting, requiring only brief hospitalization [6, 7]. Severe pancreatitis occurs in approximately 15–20% of patients [8] and is defined as the presence of organ failure lasting more than 48 h and/or death [1].

Acute kidney injury (AKI) is a prevalent complication observed in hospitalized patients, especially those who are critically ill. AKI is associated with high mortality and morbidity rates [9]. Moreover, AKI is considered a common and significant complication of acute pancreatitis, due to abdominal hypertension or abdominal compartment syndrome [10, 11], This condition is characterized by a sudden deterioration of renal function, reduced urine output, electrolyte and acid-base imbalances, fluid overload, and changes in the internal environment that negatively affect other organs [12].The occurrence of AKI in patients with acute pancreatitis may be linked to systemic inflammatory responses accompanied by increased vascular permeability, resulting in fluid exudation and accumulation in the peritoneal cavity and retroperitoneum, decreased blood supply to abdominal organs, and insufficient renal perfusion [13]. Patients with acute pancreatitis complicated by AKI have a higher mortality rate compared to those without AKI [14].Therefore, early recognition and risk assessment of AKI in patients with acute pancreatitis are crucial for improving disease prognosis.

Nomograms are graphical tools that utilize statistical prediction model to present and enhance the comprehension of clinical prediction models [15]. Despite their wide-ranging utilization, there are limited nomograms currently accessible for predicting the risk of AKI in patients suffering from acute pancreatitis. To address this research gap, the aim of our study is to develop a robust nomogram capable of accurately predicting the risk of AKI in critically ill patients with acute pancreatitis admitted to the ICU.

Methods

Data

The Medical Information Mart for Intensive Care(MIMIC) database is an expansive, open-access clinical repository that caters to critical care and emergency medicine domains. It is sourced from the intensive care system of Beth Israel Deaconess Medical Center(BIDMC) and archives hospitalization data concerning patients admitted to Massachusetts General Hospital between 2008 and 2019 [16, 17]. MIMIC-IV (version 2.0), which was released in June 2022, encompasses a range of improvements vis-à-vis its predecessor, MIMIC-III database, such as laboratory markers, medication records, recording of vital sign, SOFA score, SAPS II score, among others [18]. The authors have satisfactorily completed the requisite training for navigating the database and been conferred with the requisite certification (11,326,344).

Study population

Patients with AP were identified using the International Classification of Diseases, 9th Revision (ICD-9) code 5770 and the International Classification of Diseases, 10th Revision (ICD-10) code K85. Sepsis is defined as suspected or documented infection and an acute increase in SOFA score of ≥ 2 [19].Exclusion criteria encompassed patients who hadn’t been admitted to the ICU, stayed in ICU for less than 24 h, were below 18 years of age, had developed AKI before admission to the ICU, or had a clinical data missing rate in excess of 20%. The corresponding flowchart outlining the inclusion and exclusion criteria is depicted in Fig. 1.

Fig. 1

The flowchart of the study. Abbreviations: AP: Acute pancreatitis, ICU: Intensive care unit

Definition and clinic variables

The primary outcome of our investigation was the incidence of AKI during ICU stay. To meet the criteria established by the Kidney Disease Improving Global Outcomes(KDIGO) classification system [20], AKI diagnosis required a rise in serum creatinine(Scr) by a minimum of 0.3 mg/dL within a period of 48 h, or an increase equal to or greater than 1.5 times from baseline within the previous seven days, the baseline creatinine was defined as the lowest creatinine level measured during the 7-day period prior to each AKI assessment, or urinary output less than 0.5 ml/kg/h for at least six consecutive hours. We collected detailed demographic information (Table 1), including age, gender, weight, as well as comorbidities such as hypertension, alcohol abuse, calculus gallbladder, pleural effusion, sepsis, myocardial infarction, chronic heart failure, chronic cerebrovascular disease, chronic lung disease, diabetes, chronic kidney disease, severe liver disease. We also documented scores pertaining to disease severity, including Sequential Organ Failure Assessment (SOFA), Glasgow Coma Scale (GCS), Systemic Inflammatory Response Syndrome (SIRS), and Charlson Comorbidity Index (CCI). Further, we analyzed clinical and laboratory indicators obtained within first 24 h after ICU admission. For variables measured multiple times, we included both the maximum and minimum values. This included measurements of heart rate, mean arterial pressure, respiratory rate, temperature, lactate, pH, partial pressure of oxygen, partial pressure of carbon dioxide, hemoglobin, mean corpuscular volume, platelet count, white blood cell count, mean corpuscular hemoglobin content, mean hemoglobin concentration, red blood cells, red cell distribution width, hematocrit, anion gap, bicarbonate, blood urea nitrogen, serum calcium, serum glucose, serum sodium, serum potassium, serum chloride, creatinine, albumin, international normalized ratio(INR), prothrombin time(PT), alanine aminotransferase(ALT), alkaline phosphatase(ALP), aspartate aminotransferase(AST), and total bilirubin. In addition, we considered urine output and fluid intake within 24 h, invasive mechanical ventilation within 24 h, use of vasopressors within 24 h, antibiotic use within 48 h, hospital length of stay, and ICU length of stay. Variables with a missing value greater than 20% were exclude from the analysis.

Table 1 Baseline characteristics of the patients with acute pancreatitis

Statistical analysis

The R statistical software (v4.2.3) was utilized to analyze the data in this study. To minimize information bias, variables with a missing rate of less than 20% were included. The MICE package [21] was employed to manage missing values of variables. Normally distributed continuous variables were presented as mean ± standard deviation (𝑥̄±𝑠), whilst non-normally distributed continuous variables were presented as median and quartiles [M(Q1, Q3)]. Categorical variables, on the other hand, were reported as percentages (%). The chi-square test or Fisher’s exact test was applied for analyzing categorical variables in intergroup comparisons, whereas continuous variables were assessed with paired t-test or Mann-Whitney U test, depending on the particular circumstance.

In the present investigation, the training set was employed to select important predictors for forecasting AKI risk using a suitable regression technique for high-dimensional data regression known as the least absolute and selection operator (LASSO) regression. The optimal penalty parameter λ value was determined by preforming a 10-fold cross-validation. Next, a multivariate logistic regression analysis was carried out based on the variables obtained from the LASSO regression, and variables with a P value of less than 0.05 were included in the model, resulting in six variables in the final model, namely weight, SOFA, white blood cell count, albumin, chronic heart failure, sepsis. To evaluate the performance of the model, the area under the receiver operating characteristic curve (AUC) was calculated for both the training and validation sets. Calibration curves and the Hosmer-Lemeshow goodness-of-fit test (HL test) were utilized to appraise the accuracy of the model. Furthermore, decision curve analysis was employed to assess the clinical utility of the prediction model.

Results

Patient characteristics

This study included a total of 963 patients, among whom 695 patients experienced AKI. Of these AKI patients, 599 were diagnosed based on urine output, 487 were diagnosed based on creatinine, and 391 were diagnosed based on both urine output and creatinine. In the training set, the cohort consisted of a total of 674 patients diagnosed with AP, of which more than half were also diagnosed with AKI(n = 491). The median weight of the patients was established as 83.33 kg, with a recorded history of CHF in 18.6% of the cases and 65.4% of the patients exhibit concomitant sepsis. The median SOFA score amounted to 4 points, while the median white blood cell count was measured at 13.8 × 10^9/L, and the median albumin level was 2.8 g/dL. Moreover, 32.05% of the subjects exhibited reduced urine output (defined as less than 0.5ml/kg/h), and 29.4% of the cases involved the use of vasopressors. Additionally, compared with patients without AKI, patients with AKI exhibited a prolonged length of hospital stay (19.10 days vs. 8.68 days) and ICU stay (4.70 days vs.2.03 days)(Table 2).

Table 2 Comparison of clinical outcomes in AKI patients and NO-AKI patients

Characteristics selection and development of a nomogram

A LASSO regression analysis was conducted on a total of 62 variables, ultimately selecting 11 variables for inclusion in the subsequent multivariable logistic regression analysis (Fig. 2). Of these 11 variables, six predictors were identified as significant contributors to the model(Table 3): weight (OR: 1.03; 95%CI 1.02–1.04), sepsis (OR: 1.89; 95%CI 1.23–2.92), chronic heart failure (OR: 2.53; 95%CI 1.35–4.73), SOFA (OR: 1.24; 95%CI 1.14–1.34), white blood cell count (OR: 1.07; 95%CI 1.04–1.1), albumin (OR: 0.48; 95%CI 0.35–0.66). Considering that urine output is one of the diagnostic criteria for AKI, we ultimately decided not to include it in the model. Based on these findings, a nomogram was constructed to predicted the probability of AKI incidence during the ICU stay of AP patients (Fig. 3).

Fig. 2

Demographic and clinical feature selection using the LASSO logistic regression model. A Tuning parameter (λ) selection using LASSO penalized logistic regression with 10-fold cross-validation. B LASSO coefficient profiles of the radiomic features. A coefficient profile plot was plotted versus the log (λ). Each colored line represents the coefficient of each feature

Table 3 Multivariate logistic regression model of AKI in the training cohort

Fig. 3

Nomogram to identify the risk of AKI following AP, based on logistic regression analysis. To assign scores to each variable, av vertical line is drawn upward to the “Point” axis, and the corresponding value is recorded. The sum of all predicted scores is then located on the “Total Point” axis. By drawing a line downward to the “AKI Possibility” axis, the likelihood of development AKI can be determined. The abbreviations used are as followings: SOFA: Sequential organ failure assessment, WBC: White blood cells, Alb: albumin, CHF: Chronic heart failure

Apparent performance of the Nomogram

The AUC of the proposed model was evaluated in both the training and validation sets, yielding an AUC of 0.82 (95%CI:0.79–0.86) and 0.76(95%CI: 0.70–0.82), respectively (Fig. 4). Notably, the AUC values for both the training and validation sets were indicative of favorable diagnostic performance. Additionally, visual inspection of the calibration plot demonstrated satisfactory agreement between the predicted and observed AKI incidence rates, providing further support for the reliability of the model (Fig. 5).

Fig. 4

Receiver operating characteristic curve of the nomogram in the training set and validation set (A). Receiver operating characteristic curve of SOFA and SIRS (B) for predicting AKI in AP patients during the intensive care admission. Abbreviations: AUC: Area under the receiver operating curve, SOFA: Sequential organ failure assessment, SIRS: Systemic inflammatory response syndrome

Fig. 5

Calibration curves of the predicted nomogram in the training set (A) and validation set (B). The x-axis represents the predicted probability calculated by the nomogram, and the y-axis is the observed actual probability of AKI. The clino diagonal represents a perfect prediction by an ideal model. The solid curve represents the initial cohort and the dotted curve is bias corrected by bootstrapping (B = 100 repetitions), which demonstrates the performance of the predicted model

Clinical practice

Decision curve analysis (DCA) was conducted using both the training and validation datasets to evaluate the clinical usefulness of the proposed model (Fig. 6). The x-axis of the plot represents the proportion of patients not receiving intervention, which results in a net benefit of 0. Conversely, the diagonal line indicates that all patients receive intervention, resulting in a maximal net benefit. By setting a predicted probability threshold of 25–100% and 20–100% in the training and validation sets, respectively, the net benefit ranged from 0 to 62% and 0–62%, suggesting that the proposed model has the potential to provide clinical benefits by supporting individualized decision-making regarding AKI management.

Fig. 6

Decision curve analysis (DCA) of the nomogram in the training set (A) and the validation set (B). The horizontal line indicates no patients develop acute kidney injury (AKI), and the green oblique line indicates patients develop AKI. The red solid line represents the AKI risk nomogram. In DCA, the nomogram shows a more net benefit than full or no treatment across a threshold probability range. Abbreviations: DCA: decision curve analysis, AKI: acute kidney injury

Discussion

This present study aimed to develop a nomogram for predicting the occurrence of AKI in the short term after AP by utilizing route information available in the ICU. Previous studies on AKI in AP patients have been limited by small sample sizes, and have not recorded data well [10].In a multicenter study conducted by Eric A J Hoste et al. [9], among 1802 ICU patients, 1032 patients developed AKI on the first day of ICU admission(57.3, 95%CI:55.0-59.6). Moreover, in another retrospective observational study, Devani et al. [14]reported an overall incidence of AKI of 7.9% among 3,466,493 hospitalized patients with acute pancreatitis. The main focus of this study was to investigate the development of AKI among patients in the ICU, specifically those diagnosed with AP. The findings revealed that a significant proportion (72.1%) of AP patients develop AKI in the ICU, which is notably higher than the incidence rate observed among hospitalized AP patients. Given the severity of the disease and the high incidence of AKI in the ICU, it was considered an acceptable finding. Furthermore, the study investigated the association between AKI and several outcome measures. It was observed that patients with AP complicated by AKI had longer hospital stays, and longer ICU stays compared to those who did not develop AKI(P < 0.01). These results suggest that the treatment efficacy for patients with AP who develop AKI is poorer and that the resulting economic burden is greater. To mitigate the negative outcomes associated with AP, it is necessary to develop a nomogram that can identify high-risk patients early on and understand the impact of relevant factors on outcomes. This study reinforces the importance of such an approach and highlights the need for further research in this area to improve treatment outcomes and reduce the economic burden associated with AP.

The nomogram development in this study incorporated six predictive factors, namely weight, sepsis, chronic heart failure, SOFA score, white blood cell count, albumin. The model achieved an AUC of 0.82 in the training set and 0.76 in the validation set, and calibration curve results indicated that the predicted values were consistent with the actual values. DCA revealed that utilizing the model to forecast AKI offered more benefits than either treating all patients.

Previous studies have linked obesity, a risk factor for AP [22], to increased likelihood of AKI among ICU patients [23]. This study confirmed such finding by suggesting that weight elevation is associated with higher chances of developing AKI in AP patients. Sepsis is defined as organ dysfunction resulting from the host’s deleterious response to infection. One of the most common organs affected is the kidneys [24]. The late-stage secondary infection of the pancreas and abdominal cavity is prone to sepsis following acute pancreatitis. Additionally, patients with chronic heart failure are predisposed to AKI due to inadequate renal functional reserve, alteration in hemodynamic status, low cardiac output or congestion, and drug effects, notably diuretics and rein-angiotensin system blockers [25]. The SOFA score, commonly used to evaluate organ dysfunction in critically ill patients, was identified as another predictive factor for AKI [26], alongside an increase in WBC count, which is a metric utilized in assessing the severity of AP [27]. The heightened risk of AKI seen with elevated WBC count could stem from the generation of reactive oxygen species by neutrophils, leading to the destruction of normal cells in inflamed tissues [28]. Hypoalbuminemia, a reflection of the severity of AP in hospitalized patients according to prior meta-analyses [29], was also found to be relevant with AKI in this case.

Several limitations of this study merit consideration. Firstly, the dataset was derived from a single center spanning the period from 2008 to 2019; thus, external validation of the model is necessary across diverse medical institutions to ascertain its generalizability. Secondly, sub-classification of AP was not conducted, and certain pertinent data such as triglycerides, lipase, amylase, and neutrophil-to-lymphocyte ratio were unavailable due to missing data. Finally given that this study relied on ICU patients with AP, its applicability in the regular ward setting may be limited. Inclusion of data from patients with pancreatitis in ordinary wards has the potential to enhance the model’s accuracy and overall utility.

Conclusions

We identified risk factors associated with the development of AKI in patients with AP. This study developed a novel nomogram for predicting risk of AKI in ICU patients with AP, comprising weight, SOFA score, sepsis, chronic heart failure, white blood cell count, albumin. The proposed nomogram demonstrated favorable performance with respect to discrimination, calibration, and clinical applicability.