Assessment of acute kidney injury risk using a machine-learning guided generalized structural equation model: a cohort study
BMC Nephrology volume 22, Article number: 63 (2021)
Acute kidney injury is common in the surgical intensive care unit (ICU). It is associated with poor patient outcomes and high healthcare resource usage. This study’s primary objective is to help identify which ICU patients are at high risk for acute kidney injury. Its secondary objective is to examine the effect of acute kidney injury on a patient’s prognosis during and after the ICU admission.
A retrospective cohort of patients admitted to a Singaporean surgical ICU between 2015 to 2017 was collated. Patients undergoing chronic dialysis were excluded. The outcomes were occurrence of ICU acute kidney injury, hospital mortality and one-year mortality. Predictors were identified using decision tree algorithms. Confirmatory analysis was performed using a generalized structural equation model.
A total of 201/940 (21.4%) patients suffered acute kidney injury in the ICU. Low ICU haemoglobin levels, low ICU bicarbonate levels, ICU sepsis, low pre-ICU estimated glomerular filtration rate (eGFR) and congestive heart failure was associated with the occurrence of ICU acute kidney injury. Acute kidney injury, together with old age (> 70 years), and low pre-ICU eGFR, was associated with hospital mortality, and one-year mortality. ICU haemoglobin level was discretized into 3 risk categories for acute kidney injury: high risk (haemoglobin ≤9.7 g/dL), moderate risk (haemoglobin between 9.8–12 g/dL), and low risk (haemoglobin > 12 g/dL).
The occurrence of acute kidney injury is common in the surgical ICU. It is associated with a higher risk for hospital and one-year mortality. These results, in particular the identified haemoglobin thresholds, are relevant for stratifying a patient’s acute kidney injury risk.
Acute kidney injury (AKI) is common within surgical intensive care units, affecting 22 to 65% of patients. Moreover, it is associated with a higher mortality risk, lengthened intensive care unit (ICU) stay, unfavorable long-term outcomes and high healthcare resource usage [1,2,3]. Thus, it is important to identify patients at high risk of developing AKI, so as to initiate early intervention to prevent this complication and the associated adverse outcomes.
The risk factors and incidence of AKI can vary considerably, depending on the nature of the ICU population. Previous studies have identified individual risk factors such as age, diabetes, emergency admission, major surgery and sepsis to be independent predictors of AKI [4, 5]. However, the combined effect of multiple risk factors on the risk of AKI remains unclear. To shed light on this issue, a machine learning guided generalized structural equation model was used to identify risk factors for acute kidney injury. This involved the use of decision trees which have brought more clarity to medical research in recent years .
The primary objective of this study was to identify patients at high risk of developing AKI in the surgical ICU population. The secondary objective was to identify the impact of AKI on both hospital and one-year mortality.
Patient population and study design
This was a single-centre retrospective cohort study involving all adult patients who were admitted to the surgical ICU of Singapore General Hospital between 2015 to 2017. Patients undergoing chronic haemodialysis/peritoneal dialyis prior to admission, and those admitted to a separate ICU after cardiac surgery were excluded. This study was approved by the SingHealth centralised institutional review board (Reference Number 2016/2138), with the requirement for informed consent waived for the review of clinical records.
The collated data includes age, gender, ethnicity, body mass index, co-morbidities, baseline serum creatinine levels, ICU serum creatinine levels, pre-ICU and within-ICU haemoglobin levels, vital signs in the ICU (including mean arterial pressure and heart rate), bicarbonate levels, use of blood transfusions, type of surgery, and the presence of sepsis. The occurrence of AKI, hospital mortality, and mortality within 1 year of ICU entry was also recorded.
According to the Acute Kidney Injury Network (AKIN) criteria , AKI is defined as a rise in serum creatinine by ≥26.4 μmol/L or a 50% or greater increase in serum creatinine within a 48 h interval. Baseline levels of serum creatinine were determined using a previously recommended method : The lowest creatinine level prior to ICU entry was selected. This baseline creatinine level had to be within 90 days prior to ICU entry. This was compared with the highest serum creatinine within the first 48 h of ICU entry. A diagnosis of AKI was made if the calculated change in serum creatinine fulfilled the AKIN criteria during this initial period. Urine output was not used to diagnose AKI in our study. Hospital mortality was defined as any mortality from ICU entry to discharge from the hospital. One year mortality was defined as any mortality within 1 year of ICU entry, and was inclusive of hospital mortality.
Haemoglobin levels recorded included the most recent haemoglobin level prior to ICU entry and the lowest haemoglobin level within the first 48 h of ICU stay. ICU bicarbonate levels recorded included the lowest level within the 1st 24 h of ICU stay. Mean arterial pressure (MAP) recorded was categorised based on the MAP categories of the Acute Physiology and Chronic Health Evaluation II score . Blood transfusion was defined as any blood transfusion within a 96-h time period, 48 h before to 48 h after ICU admission. Sepsis was defined as systemic inflammatory response syndrome and the presence of infection . Baseline levels of serum creatinine (as defined earlier) were used to calculate pre-ICU estimated glomerular filtration rate (eGFR). The Chronic Kidney Disease Epidemiology equation  was used to compute the eGFR of patients. Our study assessed pre-ICU eGFR, rather than chronic kidney disease status, as the incidence of undiagnosed chronic kidney disease in our population was expected to be high [11, 12].
Exploratory analyses were performed using the Chi-square test, Fisher’s exact test, Mann-Whitney test, independent t-test and the Kaplan-Meier curve. Variables which satisfied the pre-determined criterion of p < 0.10 were included in the subsequent model selection. Two decision trees, namely Chi-Square Automatic Interaction Detector (CHAID)  and Classification & Regression Tree (CART)  were applied to ascertain the importance of these predictors impact on the occurrence of AKI, and to identify the optimal cut-off(s) of the quantitative predictors. Unlike CART, CHAID is able to identify multiple cut-offs, if they exist, for quantitative predictors. This is a desired feature of CHAID, as all bivariate-splitting decision trees, and the popular Receiver Operating Characteristic (ROC) Curve, are only able to identify one single cut-point. The cut-offs for predictors of AKI are not universally established, and multiple cut-offs may exist.
The findings of the above machine-learning algorithms could help to improve the predictive power of subsequent risk assessment of crucial post-operative outcomes. Analysis of these outcomes had to be done with a suitable statistical model for confirmatory analysis. The generalized structural equation model (gSEM)  was deemed to be most appropriate in view of its specific features and the nature of this study. It is a network of multiple equations which allowed the study outcomes to be sequentially arranged in a meaningful way. The occurrence of AKI, hospital mortality and one-year mortality were analysed with an underling Binomial distribution with logit, while the time to mortality with a Weibull distribution and a log link. The final model was identified with a backward elimination procedure (removal p ≥ 0.05). Collectively, the combination of decision trees and gSEM could unearth any hidden interactions among predictors, and provide a more comprehensive picture about how they were associated with the outcomes.
Thereafter, the final gSEM model’s external validity and predictive accuracy were ascertained with a 5-fold cross validation. Auxiliary models were built with the training sub-samples and their properties examined with the validation sub-samples. The predictive accuracies of the models were ascertained with the area under receiver operating characteristics curve (AUROC).
The proposed analyses were found to be adequate in terms of statistical power (> 95%) given the modelling framework of gSEM, overall sample size, number of predictors and the selected level of significance .
Analysis was performed using SPSS Statistics v20.0, SPSS Modeler v18 (IBM Co., Illinois, USA) and Stata MP v16 (Stata Corp, Texas, USA). All statistical tests were conducted at 5% level of significance.
A total of 1045 first-visit patients were admitted to the surgical ICU between January 2015 to January 2017. Eighty four patients were undergoing dialysis for pre-existing end stage renal failure and 21 patients had incomplete data. The final sample comprised 940 multi-ethnic patients (mean age: 61.3 ± 15.9 years, 56.7% males) whose records were assessed up to 1 year after ICU entry.
The occurrence of ICU AKI was 21.4%, of which 77.6% were AKIN stage 1, 11.4% were stage 2 and 11.0% were stage 3. Patients who developed AKI were generally older, diabetic, more likely to have ischemic heart disease, congestive heart failure, lower pre-ICU eGFR, low pre-ICU haemoglobin (Hb) levels, emergency surgery and require blood transfusion (Table 1). Within the ICU, patients who had low bicarbonate levels, low Hb levels, low mean arterial pressure (MAP ≤70 mmHg) and sepsis were more likely to develop AKI.
CHAID identified low haemoglobin levels, sepsis, low bicarbonate levels and low MAP as predictors for ICU AKI (Fig. 1). It identified ICU haemoglobin levels as the most important predictor for AKI, and discretized it into 3 categories (i.e., ≤ 9.7 g/dL, 9.8—12.0 g/dL and > 12 g/dL). The risk of developing AKI increased as haemoglobin levels fell.
Among those in the high-risk haemoglobin category, patients with low ICU bicarbonate (i.e., bicarbonate ≤18.9 mEq/L) had a higher chance of suffering from AKI (Fig. 1). For those in the moderate-risk category, patients with a lower MAP (≤ 70 mmHg) had a higher risk for AKI. Patients in the low-risk category were more likely to have AKI if they suffered ICU sepsis. By navigating along the branches of the CHAID decision tree one could generate the decision rules concerning the risk of AKI.
Further confirmatory analysis with gSEM (Fig. 2) revealed that the odds of having AKI in the ICU was significantly higher with low ICU haemoglobin levels, low ICU bicarbonate levels, ICU sepsis, low pre-ICU eGFR (defined as ≤48 ml/min/1.73m2, the cut-off derived via CHAID) and congestive heart failure (CHF) as depicted in Table 2.
The mortality status was examined next. The hospital mortality rate was 18.6% (i.e., 175/940). The median time to death was 17.1 days (interquartile range: 6.3–40.4 days). AKIN stage 3 patients were at the highest risk of hospital mortality (45.5%), while stage-1 patients had a risk of 30.1%. An additional 209 deaths occurred within the first year after discharge from ICU. The total one-year mortality rate was 40.9%. The median time to one-year death was about 125 days (interquartile range: 20.6–365 days; inclusive of time in ICU).
Confirmatory analysis with gSEM (Fig. 2) identified ICU AKI to be a strong predictor of hospital mortality (AOR: 1.76, p = 0.004). The results were adjusted with age > 70 years and pre-ICU eGFR (Table 3). As an independent predictor, low pre-ICU eGFR had both a significant direct effect on hospital mortality, and a significant indirect effect through its impact on ICU AKI.
As revealed in the Kaplan-Meier curve (Fig. 3), patients with ICU AKI had a significantly lower survival probability (p < 0.001). Confirmatory analysis with gSEM showed that the hazard of death was 43.3% higher with the onset of ICU AKI (Table 3). As depicted in Table 3, the predictors for occurrence of hospital mortality were also associated with time to 1-year mortality. Hence, the predictors of ICU AKI, age > 70 years, and low pre-ICU eGFR could explain both the occurrence of hospital mortality and 1 year mortality.
The results of 5-fold cross validation for the ICU AKI, hospital mortality and 1-year mortality are shown in the Additional file 1. The auxiliary models built were found to be stable and reliable, as their estimates (odds ratios, hazard ratios, 95% C.I.s and p-values) were numerically similar to that of the original model based on the entire sample (n = 940). The 95% C.I.s of the effect sizes overlapped. The AUROCs of the auxillary models for ICU AKI were 0.7537, 0.7589, 0.7950, 0.7333 and 0.7654.
Affecting 1 in 5 patients, AKI is common in the Singaporean surgical ICU population. The odds of hospital death is 1.76 times higher among patients with ICU AKI. It is therefore crucial to identify the potential causes of ICU AKI.
This study highlights the importance of ICU haemoglobin in stratifying AKI risk. Haemoglobin plays a key role in transporting oxygen to vital organs. Low concentrations of haemoglobin decrease arterial oxygen content, and thus oxygen delivery to the kidney. As red blood cells possess important anti-oxidative functions , anemia may also worsen oxidative stress in the kidneys. Thus, anemia can result in both ischemic and inflammatory organ damage, which can contribute to the occurrence of AKI . The impact of anaemia has also been demonstrated in the Korean population, in which anaemia was associated with higher AKI risk and long-term mortality .
Two key thresholds were identified, beyond which the risk of AKI rose. When ICU haemoglobin levels were below 9.8 g/dL, the occurrence of AKI was 31.1%. This is twice as high compared with patients with haemoglobin levels between 9.8–12 g/dL and nearly 5 times higher compared to patients with haemoglobin levels above 12 g/dL. The CHAID algorithm suggested that at different haemoglobin levels, different pathologies are associated with higher AKI risk. Among those with high risk anaemia (Hb ≤ 9.7 g/dL), low bicarbonate levels have a strong association with AKI risk. Among those with moderate risk anaemia (Hb between 9.8–12.0 g/dL), MAP is strongly associated with AKI risk. Amongst those without anaemia, sepsis was associated with increased risk.
Many critically-ill patients have low bicarbonate levels. Previous studies have shown that low bicarbonate levels could predict the incidence of AKI independently [19, 20]. Our findings of how low bicarbonate levels and anaemia is associated with particularly high rates of AKI has also been shown in the rat model. Rats exposed to metabolic acidosis and induced ischemia reperfusion injury had higher levels of NF-κB and poorer glomerular filtration rate. This was in comparison to rats exposed to induced ischemia reperfusion injury alone . It is postulated in literature that bicarbonate is necessary to reduce iron-mediated free radical formation, and can also help to improve oxygen delivery to the renal medulla [19, 20]. The management of low bicarbonate levels remains controversial. Its routine use in critically ill patients with metabolic acidemia has not been shown to be beneficial, and may result in deleterious side effects . However, in select groups of patients, such as those at high risk of or ongoing acute kidney injury, providing bicarbonate ions via sodium bicarbonate infusion may result in improved outcomes and mortality [23, 24]. Regardless, our study highlights the importance of preventing and managing metabolic acidosis and low bicarbonate levels, particularly amongst patients with anaemia ≤9.7 g/dL, due to its strong association with complications of AKI and subsequent hospital and 1-year mortality.
In patients with haemoglobin levels of 9.8 to 12 g/dL, a low MAP ≤70 mmHg was strongly associated with AKI risk. The combination of anaemia and hypotension can compromise oxygen delivery to the kidneys. Oxygen delivery to end organs is dependent on arterial oxygen content and cardiac output . Cardiac output is determined by mean arterial pressure and systemic vascular resistance. Arterial oxygen content is influenced mainly by the concentration of haemoglobin in the blood. The combination of a persistently poor mean arterial pressure and low haemoglobin concentration within the ICU may potentially overwhelm the compensatory ability of the kidney. As a result insufficient perfusion, ischemic injury and AKI can occur. Thus, in agreement with other studies, maintaining mean arterial pressure at least above 65 mmHg in patients is prudent , and particularly so among patients with anaemia. This study used the cut-off of 70 mmHg following the MAP risk categorization of the Acute Physiology and Chronic Health Evaluation II classification system . It is not meant to weight into the debate of the exact MAP required to maintain adequate tissue perfusion.
The CHAID algorithm demonstrated the association between sepsis and AKI amongst patients with haemoglobin > 12 g/dL. This is expected, as sepsis is known to be a strong risk factor for AKI. This highlights the importance of the optimisation of sepsis management for the prevention of AKI in non-anaemic patients.
Our study had several limitations. Firstly, serum creatinine levels were not analyzed beyond the first 48 h of ICU entry. Thus patients who developed AKI at a later point in time would have been missed. The decision not to analyze beyond 48 h was made because the majority of stays in this ICU were short ones, with a median stay of 2 days. Secondly, the numbers of patients with AKI in the CHAID subgroup with hemoglobin levels > 12.0 g/dL were small, which may impact the reliability of the sepsis vs non sepsis arm. However, the model conclusion that sepsis is associated with AKI is likely sound, as it has been shown in previous studies. Thirdly, as our study population involved multidisciplinary surgical patients, its findings may not be applicable to non-surgical patients. Fourthly, our study was retrospective. Thus, predictors and outcomes may not have been assessed at the same time point or frequency in all included patients. This may result in a higher risk of misclassification bias compared to a prospective study with standardized evaluations of predictors and outcomes.
AKI is a common and serious morbidity within the surgical ICU. It is associated with a higher hospital and one-year mortality risk. Both the CHAID and gSEM identified ICU haemoglobin as a key risk factor for ICU AKI. Key haemoglobin thresholds were identified, beyond which the risk of acute kidney injury rose. Furthermore, potentially synergistic risk factors for acute kidney injury were identified for each haemoglobin threshold. This is important for risk stratification and management of patients. It may also hold the key for determining the mechanism of AKI in future studies. This study is therefore the first but important step in designing patient-specific care pathways based on a patient’s specific risk profile. This is important to lower their risk for ICU AKI and subsequently their mortality risk.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Acute kidney injury
Acute kidney Injury Network
Adjusted odds ratio
Area under receiver operating characteristics curve
Classification & regression tree
Chi-square automatic interaction detector
Congestive heart failure
Estimated glomerular filtration rate
Generalized structural equation model
Intensive care unit
Mean arterial pressure
Nuclear factor kappa-light-chain-enhancer of activated B cells
Kerr M, Bedford M, Matthews B, O'Donoghue D. The economic impact of acute kidney injury in England. Nephrol Dial Transplant. 2014;29(7):1362–8.
Horkan CM, Purtle SW, Mendu ML, Moromizato T, Gibbons FK, Christopher KB. The association of acute kidney injury in the critically ill and postdischarge outcomes: a cohort study*. Crit Care Med. 2015;43(2):354–64.
Trongtrakul K, Sawawiboon C, Wang AY, Chitsomkasem A, Limphunudom P, Kurathong S, et al. Acute kidney injury in critically ill surgical patients: Epidemiology, risk factors and outcomes. Nephrology (Carlton, Vic). 2019;24(1):39–46.
Harris DG, McCrone MP, Koo G, Weltz AS, Chiu WC, Scalea TM, et al. Epidemiology and outcomes of acute kidney injury in critically ill surgical patients. J Crit Care. 2015;30(1):102–6.
Medve L, Gondos T. Epidemiology of postoperative acute kidney injury in Hungarian intensive care units: an exploratory analysis. Ren Fail. 2012;34(9):1074–8.
Bellomo R, Ronco C, Mehta RL, Asfar P, Boisramé-Helms J, Darmon M, et al. Acute kidney injury in the ICU: from injury to recovery: reports from the 5th Paris International Conference. Ann Intensive Care. 2017;7(1):49. https://pubmed.ncbi.nlm.nih.gov/28474317/.
Mehta RL, Kellum JA, Shah SV, Molitoris BA, Ronco C, Warnock DG, et al. Acute kidney injury network: report of an initiative to improve outcomes in acute kidney injury. Crit Care. 2007;11(2):R31.
Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818–29.
Bone RC, Balk RA, Cerra FB, Dellinger RP, Fein AM, Knaus WA, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest. 1992;101(6):1644–55. https://pubmed.ncbi.nlm.nih.gov/1303622/.
Levey AS, Stevens LA, Schmid CH, Zhang YL, Castro AF 3rd, Feldman HI, et al. A new equation to estimate glomerular filtration rate. Ann Intern Med. 2009;150(9):604–12.
Wong LY, Liew AST, Weng WT, Lim CK, Vathsala A, Toh M. Projecting the burden of chronic kidney disease in a developed country and its implications on public health. Int J Nephrol. 2018;2018:5196285.
Low SK, Sum CF, Yeoh LY, Tavintharan S, Ng XW, Lee SB, et al. Prevalence of chronic kidney disease in adults with type 2 diabetes mellitus. Ann Acad Med Singap. 2015;44(5):164–71.
Breiman L, Friedman J, Stone CJ, Olshen RA. Classification and Regression Trees: Taylor & Francis; 1984.
Rabe-Hesketh S, Skrondal A, Pickles A. Generalized multilevel structural equation modeling. Psychometrika. 2004;69(2):167–90.
Westland JC. Lower bounds on sample size in structural equation modeling. Electron Commer Res Appl. 2010;9(6):476–87.
Kuhn V, Diederich L, Keller TCS, Kramer CM, Luckstadt W, Panknin C, et al. Red blood cell function and dysfunction: redox regulation, nitric oxide metabolism, Anemia. Antioxid Redox Signal. 2017;26(13):718–42.
Kanagasundaram NS. Pathophysiology of ischaemic acute kidney injury. Ann Clin Biochem. 2015;52(Pt 2):193–205.
Han SS, Baek SH, Ahn SY, Chin HJ, Na KY, Chae DW, et al. Anemia is a risk factor for acute kidney injury and long-term mortality in critically ill patients. Tohoku J Exp Med. 2015;237(4):287–95.
Jung SY, Park JT, Kwon YE, Kim HW, Ryu GW, Lee SA, et al. Preoperative Low Serum Bicarbonate Levels Predict Acute Kidney Injury After Cardiac Surgery. Medicine (Baltimore). 2016;95(13):e3216. https://journals.lww.com/md-journal/toc/2016/03290#1177512515.
Gujadhur A, Tiruvoipati R, Cole E, Malouf S, Ansari ES, Wong K. Serum bicarbonate may independently predict acute kidney injury in critically ill patients: an observational study. World J Crit Care Med. 2015;4(1):71–6.
Magalhaes PA, de Brito TS, Freire RS, da Silva MT, dos Santos AA, Vale ML, et al. Metabolic acidosis aggravates experimental acute kidney injury. Life Sci. 2016;146:58–65.
Lo KB, Garvia V, Stempel JM, Ram P, Rangaswami J. Bicarbonate use and mortality outcome among critically ill patients with metabolic acidosis: a meta analysis. Heart Lung J Crit Care. 2020;49(2):167–74.
Haase M, Haase-Fielitz A, Bellomo R, Devarajan P, Story D, Matalanis G, et al. Sodium bicarbonate to prevent increases in serum creatinine after cardiac surgery: a pilot double-blind, randomized controlled trial. Crit Care Med. 2009;37(1):39–47.
Ghauri SK, Javaeed A, Mustafa KJ, Podlasek A, Khan AS. Bicarbonate therapy for critically ill patients with metabolic acidosis: a systematic review. Cureus. 2019;11(3):e4297.
Dunn JO, Mythen MG, Grocott MP. Physiology of oxygen transport. BJA Educ. 2016;16(10):341–8.
Mohsenin V. Practical approach to detection and management of acute kidney injury in critically ill patient. J Intensive Care. 2017;5:57.
The authors appreciate the supervisory support of Associate Professor Edwin Seet Chuen Ping, a participating investigator in this study.
No external funding was received for this study.
Ethics approval and consent to participate
This study was approved by the SingHealth centralised institutional review board (Reference Number 2016/2138), with the requirement for informed consent waived for the review of clinical records.
Consent for publication
The authors declare that they have no competing interests relevant to the paper.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Wong, W.E.J., Chan, S.P., Yong, J.K. et al. Assessment of acute kidney injury risk using a machine-learning guided generalized structural equation model: a cohort study. BMC Nephrol 22, 63 (2021). https://doi.org/10.1186/s12882-021-02238-9
- Critically ill
- Acute kidney injury