Predictors of shorter- and longer-term mortality after COVID-19 presentation among dialysis patients: parallel use of machine learning models in Latin and North American countries
BMC Nephrology volume 23, Article number: 340 (2022)
We developed machine learning models to understand the predictors of shorter-, intermediate-, and longer-term mortality among hemodialysis (HD) patients affected by COVID-19 in four countries in the Americas.
We used data from adult HD patients treated at regional institutions of a global provider in Latin America (LatAm) and North America who contracted COVID-19 in 2020 before SARS-CoV-2 vaccines were available. Using 93 commonly captured variables, we developed machine learning models that predicted the likelihood of death overall, as well as during 0–14, 15–30, > 30 days after COVID-19 presentation and identified the importance of predictors. XGBoost models were built in parallel using the same programming with a 60%:20%:20% random split for training, validation, & testing data for the datasets from LatAm (Argentina, Columbia, Ecuador) and North America (United States) countries.
Among HD patients with COVID-19, 28.8% (1,001/3,473) died in LatAm and 20.5% (4,426/21,624) died in North America. Mortality occurred earlier in LatAm versus North America; 15.0% and 7.3% of patients died within 0–14 days, 7.9% and 4.6% of patients died within 15–30 days, and 5.9% and 8.6% of patients died > 30 days after COVID-19 presentation, respectively. Area under curve ranged from 0.73 to 0.83 across prediction models in both regions. Top predictors of death after COVID-19 consistently included older age, longer vintage, markers of poor nutrition and more inflammation in both regions at all timepoints. Unique patient attributes (higher BMI, male sex) were top predictors of mortality during 0–14 and 15–30 days after COVID-19, yet not mortality > 30 days after presentation.
Findings showed distinct profiles of mortality in COVID-19 in LatAm and North America throughout 2020. Mortality rate was higher within 0–14 and 15–30 days after COVID-19 in LatAm, while mortality rate was higher in North America > 30 days after presentation. Nonetheless, a remarkable proportion of HD patients died > 30 days after COVID-19 presentation in both regions. We were able to develop a series of suitable prognostic prediction models and establish the top predictors of death in COVID-19 during shorter-, intermediate-, and longer-term follow up periods.
People with kidney failure treated by dialysis are at a high risk of experiencing serious complications if affected by COVID-19. Reports have estimated 40% to 70% of dialysis patients who contracted COVID-19 were hospitalized and 11% to 34% died during timeframes before SARS-CoV-2 vaccines were available [1,2,3,4,5,6,7,8,9]. During 2020, the mortality rate in the United States dialysis population was estimated to have increased by 18% compared to 2019 . The majority assessments of mortality in dialysis patients with COVID-19 have limited follow up timeframes for assessing outcomes. Although useful, the timeframes generally investigated do not provide an understating of outcomes overall, as well as in distinct shorter and longer periods after COVID-19. Ultimately, this might be limiting our understanding of the profiles and predictors of outcomes in this special population.
In many countries, SARS-CoV-2 vaccines have fortunately become readily available and are being rolled out to the communities [11, 12]. Nonetheless, SARS-CoV-2 vaccines have been shown to create a smaller antibody response among dialysis patients [13, 14], and a proportion of the population has not been vaccinated for SARS-CoV-2, and may never be due to various reasons (e.g. medical/religious contraindications, vaccine hesitancy) [11, 12, 15, 16]. Further establishment of models to identify the predictors of outcomes in unvaccinated dialysis patients continues to be warranted, and as sufficient follow up data becomes available, investigations determining the profiles and predictors of mortality in vaccinated dialysis patients will also be needed.
Through experiences in direct patient care in the pandemic, the physician authors made anecdotal observations that dialysis patients with COVID-19 generally experienced the outcome of death either very quickly (within 14 days), or after prolonged periods of intensive care (often > 30 days). Ultimately, it was hypothesized this might be signaling distinct causes and predictors of early or prolonged mortality in COVID-19. This investigation aimed to evaluate the profiles and predictors of mortality in hemodialysis (HD) patients with COVID-19, overall, as well as considering shorter-, intermediate-, and longer-term follow up periods.
We used real-world data from HD patients treated at regional institutions of a global provider in Latin America (LatAm; Fresenius Medical Care Latin America, Rio de Janeiro, Brazil) and North America (Fresenius Medical Care North America, Waltham, United States) from 01-July-2019 to 31-December-2020 to conduct side-by-side analyses of the profiles and predictors of mortality overall, as well as within 0–14, 15–30, > 30 days after COVID-19 presentation.
In LatAm and North America, we used data collected during the provision of routine medical care in dialysis patients. All data was de-identified for the purposes of the parallel analyses. The EuCliD database was used for capturing data in the Latin America cohort as part of Fresenius Medical Care's quality improvement and management programs in all NephroCare clinics utilizing EuCLiD . EuCLiD governance has established protocols and procedures for use of clinical data from NephroCare clinics for secondary research purposes. Data was only collected from patients who provided informed consent for their data to be collected into EuCliD and the data was de-identified by the LatAm investigator. The Fresenius Medical Care North America Knowledge Center Data Warehouse was used for capturing data in the North America cohort from clinics in the Fresenius Kidney Care network. In North America, data was collected from patients treated in the United States under a protocol approved by New England Independent Review Board (NEIRB; Needham Heights, MA, United States); NEIRB determined the analysis of the North America cohort was exempt due to use of data de-identified by the North America investigator that no longer contained protected health information and consent was not required per title 45 of the United States Code of Federal Regulations part 46.104(d)(4) (NEIRB# 1–1439054-1). The analysis in each region was conducted in accordance with the Declaration of Helsinki.
We included data from adult (age ≥ 18 years) patients with kidney failure who were suspected to have COVID-19 before 02-Dec-2020 and received ≥ 1 outpatient HD treatment (inclusive of hemodiafiltration) before COVID-19 presentation and did not change to a home dialysis modality during the observation period. We excluded data from patients under investigation who were found to have a negative SARS-CoV-2 test result, or patients who were in close contact to someone with known COVID-19, never presented with symptoms, and were not tested. We also excluded data from patients who received outpatient HD for acute kidney failure, as well as patients who were known to be pregnant during the observation period.
The primary outcome (dependent variable) was all-cause death any time after COVID-19 presentation. The time at risk started on the first date of COVID-19 suspicion where patients presented with signs and symptoms. We defined a 30-day minimum follow up period for evaluation of outcomes across the observation period (i.e. COVID-19 suspicion date before 02 Dec 2020).
Further sub-analyses of the primary outcome considered all-cause death within 0–14, 15–30, > 30 days after COVID-19 presentation. We used the same logic for time at risk and minimum follow up as with death any time after COVID-19 presentation. Patients who had a death event in a preceding period were censored from the dataset for analysis performed in the subsequent predefined follow-up period. Therefore, patients who died within 0–14 days after COVID-19 presentation were removed from analyses of outcomes 15–30 and > 30 days after COVID-19 presentation. Consistently, patients who died during 15–30 days after COVID-19 presentation were removed from analyses of outcomes > 30 days after COVID-19 presentation.
We used various patient characteristics, clinical parameters, and laboratories (independent variables; n = 93) to define the characteristics of the cohorts and investigate the predictors of death after COVID-19 presentation. Patient characteristics included age, sex, body mass index (BMI), dialysis vintage, etiology of kidney failure (diabetic nephropathy, hypertensive nephrosclerosis, or other), comorbidities (diabetes, hypertension, heart failure, ischemic heart disease, liver disease, cancer, and chronic obstructive pulmonary disease (COPD)), and country as of the first date of suspicion/presentation with COVID-19, as well as continuous dialysis catheter exposure ≤ 90, ≤ 120, and ≤ 180 days before COVID-19 presentation.
Clinical parameters included pre-/post-HD systolic blood pressure (SBP), diastolic blood pressure (DBP), pulse, body temperature, and weight, as well as the prescribed dry weight and interdialytic weight gain (IDWG). Laboratories included pre-HD albumin, calcium, corrected calcium, creatinine, phosphate, intact parathyroid hormone (iPTH), hemoglobin, ferritin, transferrin saturation (TSAT), white blood cell (WBC) count, and WBC differential (% platelets, % lymphocytes, % neutrophils). All independent variables considered were captured and available for patients treated in the Latin and North America countries included in the parallel analyses. The clinical parameters (e.g., vital signs and weight measures) were universally collected before and after HD for all patients in both regional cohorts. There were some differences in the frequencies of select laboratories with some being measured less frequently in Latin versus North America countries. For instance, pre-HD albumin was measured on a quarterly basis in Latin America and a monthly basis in North America.
All clinical parameters and laboratory values considered the most recent value within 14 days before COVID-19 presentation, the most recent value > 14 days prior to COVID-19 presentation, and the change between the values within 14 days and > 14 days prior to COVID-19 presentation (Fig. 1). These timepoints were selected based off expert knowledge in the domain of medicine and physiology and a prior investigation that estimated the timing of physiological disturbances during the onset of COVID-19 .
A past work identified disturbances in physiology start about 14 days before COVID-19, with the most meaningful changes in clinical and laboratory values being seen at presentation with the first signs and symptoms of the disease . Therefore, the most recent value within 14 days of COVID-19 was chosen to provide a representation of the patient’s clinical status at presentation with signs of symptoms that led to identification of COVID-19. This prior analysis also showed that clinical and laboratory values > 14 days before COVID-19 presentation were representative of each patient’s “normal” physiology before the onset of COVID-19 . Ultimately, this design for predictor variable timing was chosen to show the extent that disturbances in clinical and laboratory values during COVID-19 onset associate with a death event, as well as show the extent that the historic clinical status associates with a death event.
The patient characteristics in the LatAm and North America cohorts were tabulated by region, as well as stratified by the groups who died or survived after COVID-19. We reported the count and proportion of categorical variables and mean and standard deviation (SD) of continuous variables.
Machine learning model development
Given the knowledge on risk factors for mortality in the dialysis population is sparse, has not included continuous data on laboratories and HD treatments, and has not assessed temporal changes in predictors before and longer follow up times after COVID-19 presentation, we decided to use an advanced data driven approach to establish the predictors of mortality after COVID-19. This included developing a series of machine learning prediction models using Python software (Python Software Foundation, Delaware, United States) with the XGBoost package  to predict the likelihood of death after COVID-19 presentation and identify the importance of predictor variables.
For parallel model development in LatAm and North America, we used a 60%:20%:20% random split of the data on patients who died anytime during follow-up (i.e. positive outcome group) for the training, validation, and testing datasets respectively. Data from survivors throughout follow-up (i.e. negative outcome group) were randomly split between the datasets. Down sampling methods in the negative outcome cases were investigated to optimize the models’ ability to learn to identify the outcome from predictor variables considering a 1:1 through 1:6 ratio in the training and validation datasets. Based on our assessments, we chose to down sample the negative outcome cases in only the training dataset to provide a 1:2 ratio of positive to negative outcome cases (i.e. for each patient who died we randomly included 2 patients who survived in the training dataset). The validation and testing datasets were not down sampled and represented the incidence of COVID-19 death observed in the overall HD population in each world region.
The same methods for data splits and sampling were performed for the sub-analysis models developed to predict the likelihood of death in 0–14, 15–30, > 30 days after COVID-19 presentation. In these sub-analysis models, we removed patients who died within 14 days after COVID-19 from the positive outcome group for creating the datasets for the models developed to determine the risk of death during 15–30 or > 30 days after COVID-19 presentation. Furthermore, we removed patients who died within 30 days after COVID-19 from the positive outcome group for creating the datasets for the models developed to determine the probability of death > 30 days after COVID-19 presentation. The negative outcome groups consisted of data from survivors of the predefined follow-up period, and they were randomly split between the training, validation, and testing datasets for each model. All models were developed in a side-by-side manner and used same programming for datasets in LatAm and North America.
For an overview of the XGBoost logic, this non-linear machine learning model used the input (independent) variables in the training dataset to construct an array of decision trees in every possible combination to establish a series of thresholds that split variables to maximize the information gain. Decision trees were constructed iteratively by the model, and new decision trees were added to predict prior errors. The decision trees were inherently able to handle and account for missing values and imputation of null data points was not required. The model determined the presence or missingness of each variable when establishing variable splits in the decision trees for each patient. Therefore, the influence of a missing value was used for information gain in the predictions made for each patient. After the ensemble of decision trees was created using the training datasets, it was assessed using the validation datasets and hyperparameter tuning was evaluated for the overall models using a grid search and 5-fold cross validation method. After no more improvements were achieved in performance, the final ensemble of decision trees produced in the models were used for performance assessments using unseen data in the testing dataset. Hyperparameter tuning and the selection of the final hyperparameter settings in each region was based on the models that predicted mortality any time after COVID-19, and these settings were universally applied to the sub-analysis models that predicted mortality within 0–14 days, 15–30 days, and > 30 days after COVID-19. The details on the initial and final hyperparameters and tuning ranges considered are shown in Additional File 1; Supplementary Table 1.
Assessment of model performance
The performance of the prediction models was measured by the area under the receiver operating characteristic curve (AUC) and balanced accuracy in the training, validation, and testing datasets. The area under the precision-recall curve (AUPRC) was further evaluated in the testing dataset.
The AUC measures the rate of true and false positives classified across probability thresholds (Table 1). AUC scores are represented on a scale of 0 (lowest) to 1 (highest) with chance being a value of 0.5.
Balanced accuracy is a measure of the accuracy of the prediction that is represented as a percent and considers both the sensitivity and specificity at cutoff threshold of 0.50. This metric can reasonably estimate model performance in data with imbalanced positive and negative outcomes, and is calculated as follows:
The AUPRC measures the ratio of precision for corresponding sensitivity values across probability thresholds . AUPRC scores are represented on a scale of 0 (lowest) to 1 (highest) with chance equaling the fraction of positive cases in each regional group for each model (i.e., the number of patients who died in each group divided by the total number of patients in each group).
The definitions for sensitivity, specificity, and precision are provided below since these metrics are used in the calculation of balanced accuracy and the AUPRC.
Sensitivity (also known as recall) shows the rate of true positives classified by the model at a specified threshold, and the equation for this metric is as follows:
Specificity shows the rate of true negatives classified by the model at a specified threshold, and the equation for this metric is as follows:
Precision shows the positive predictive value for the model at a specified threshold, and the equation for this metric is as follows:
The final model performance is represented by the AUC, balanced accuracy, and AUPRC for the testing dataset.
Assessment of the importance of predictors
We assessed the importance of individual predictor variables using Shapley (SHAP) values [21, 22] that were calculated using the SHAP Python package [23, 24]. The SHAP value determined the feature importance for each input variable by calculating the predictors influence on prediction of the outcome considering the influence of the overall combination of variables in the model.
For an overview of the logic, SHAP values were calculated for each predictor variable at each observation, representing the positive or negative impact of the observed value on the prediction of the outcome for each individual patient. The SHAP methods included and withheld the individual variables in all possible combinations. To attribute feature importance, the SHAP method calculated the mean value of all possible combinations considering differences between included and withheld variables. Notably, SHAP values show additive explanations of feature importance and are reported in log odds (i.e. the logarithm of the odds ratio). To calculate the prediction for each individual patient, the model summed the SHAP values for each variable and converted it from log odds to the probability for the occurrence of the outcome. Therefore, larger positive SHAP values increase the probability for the predicted outcome for a given patient, and larger negative SHAP values decrease the probability. The overall feature importance for each predictor variable was determined by calculating the mean absolute SHAP value across all the individual patients’ observations.
Patient characteristics and profiles of mortality after COVID-19
We identified a cohort of 3,473 HD patients who presented with COVID-19 any time before 02 Dec 2020 in three LatAm countries (Argentina, Colombia, Ecuador), as well as a cohort of 21,624 HD patients who presented with COVID-19 during the same time in North America from the United States (Fig. 2). The demographics of patients with COVID-19 by survival status are shown in Table 2 for the LatAm and North America cohorts. On average, patients in LatAm countries had trends for being a few years younger, more often male, had a lower BMI, longer dialysis vintage, with a lower prevalence of diabetes, hypertension, and heart failure.
In the LatAm cohort, 28.8% (1,001/3,473) patients died any time after COVID-19 during the observation period. A lower proportion of 20.5% (4,426/21,624) patients died any time after COVID-19 in the North America cohort (Table 2). There were regional differences in the timing of mortality after COVID-19, with shorter-term outcomes being more frequent in LatAm and vice versa in North America. Among HD patients with COVID-19 in LatAm and North America, 15.0% and 7.3% died within 0–14 days, 7.9% and 4.6% died within 15–30 days, and 5.9% and 8.6% died > 30 days after presentation, respectively (Fig. 3). Univariate analyses showed most demographic (Tables 2 & 3) and clinical (Tables 4, 5, 6, & 7) parameters were related to mortality in COVID-19, especially in the North America cohort.
The machine learning models constructed to establish the predictors of mortality in COVID-19 were found to have suitable performance in prediction of the outcome of death in both regions overall, as well as in the predefined shorter timeframe after COVID-19 presentation (Table 8). The AUC for the model’s classification of death at any time after COVID-19 presentation was 0.76 in LatAm cohort and 0.79 in North America cohort, the balanced accuracy was 71% in the LatAm cohort and 70% in North America cohort, and the AUPRC was 0.21 in LatAm cohort and 0.52 in North America cohort.
Relatively consistent AUCs (ranging from 0.73 to 0.83) and balanced accuracy (ranging from 66 to 75%) were found across models in predefined timeframes 0 to 14, 15 to 30, and > 30 days after COVID-19 for both regions. Considering the AUPRC, the model was found to have suitable performance in classification of shorter-term death events within 0 to 14 days after COVID-19 presentation in both the LatAm cohort (AUPRC = 0.38) and North America cohort (AUPRC = 0.30). Although the AUPRC showed suitable performance in the North America cohort for classification of the risk of death 15 to 30 days (AUPRC = 0.23) and > 30 days (AUPRC = 0.36) after COVID-19, it showed poor performance in prediction of intermediate- (AUPRC = 0.06) and longer-term (AUPRC = 0.04) outcomes in the LatAm cohort.
Predictors of death any time after COVID-19
We estimated the importance of each predictor variable with SHAP values and found the top three predictors of death any time after COVID-19 presentation in the LatAm cohort were older age, higher WBC counts historically (i.e. > 14 days prior to COVID-19 presentation), and lower albumin levels historically; in North America, the top three predictors included older age, lower albumin levels historically, and longer dialysis vintage. In Fig. 4, the bar charts on the left side of each panel show the mean absolute SHAP values that represent the magnitude of importance for each variable in log odds; these are shown in descending order of importance for the top 15 predictors. The SHAP value plots on the right of each panel further show the degree and direction of the effect for each variable on each unique patient’s prediction. The SHAP value plots denote a dot that corresponds to each patient and the dot’s position on the x-axis (positive or negative) represents the magnitude of that variable’s effect on the risk prediction for that unique patient. The color of each dot on the SHAP value plots indicate how large/high or small/low the value is for that variable in that unique patient’s prediction. For an example with the top predictor of age, the mean SHAP values show age has a high magnitude of importance as compared to other variables and the SHAP value plots show more positive SHAP values for dots that had warmer colors (representing increasing age with the warmer the color and increasing risk based on how positive the value is), and more negative SHAP values for dots that had cooler colors (representing younger age with the cooler the color and decreasing risk based on how negative the value is). Age showed the largest contribution to the risk of death after COVID-19; however, many variables had a high magnitude considering the log odds values and the distributions of risks in SHAP value plots.
Albeit distinctions exist between world regions in the predictors of mortality any time after COVID-19 presentation, the trends in the top 15 predictors showed many consistent findings with older age, poorer nutrition (lower albumin and creatinine historically), longer vintage, lower TSAT levels historically, more inflammation (seen in LatAm by higher WBC counts historically and a change to a higher % of neutrophils and in North America by lower % of lymphocytes historically and at presentation) increasing the risk of death (Fig. 4). Some regional differences in the top predictors of mortality any time after COVID-19 included lower or missing iPTH historically and presence of diabetes being among the top 15 risk factors in only LatAm, while being male and higher post-HD pulse at presentation were only in the top 15 predictors in North America. Figure 5 shows a further regional comparison of the mean absolute SHAP values for the top 15 predictors of death any time after COVID-19 presentation from both regional cohorts, and Additional File 1; Supplementary Table 2 shows the SHAP values for all the predictors of death any time after COVID-19.
Predictors of Shorter, Intermediate, and Longer-Term Death after COVID-19
Assessment of the top predictors of shorter-term death within specifically 0 to 14 days after COVID-19 presentation showed older age, higher WBC counts historically, longer vintage, lower albumin historically, higher BMI, and higher creatinine historically were among the top 15 risk factors for shorter-term mortality in both regions (Figs. 6 & 7, Additional File 1; Supplementary Table 3). Mineral bone disorder markers (lower or missing iPTH, higher calcium, higher corrected calcium) historically, higher ferritin levels historically, and having diabetes were found to only be in the top 15 predictors for short-term mortality in LatAm, while a higher post-HD pulse at presentation, a change to a higher pulse, and being male were only in the top 15 predictors in North America, among other distinctions.
The evaluation of the risk factors for intermediate-term mortality during 15 to 30 days after COVID-19 presentation identified consistencies in many of the top 15 predictors of death in between regions (older age, being male, higher TSAT and % of neutrophils historically, and hemoglobin at presentation), along with some regional heterogeneity in some factors (Figs. 6 & 8, Additional File 1; Supplementary Table 4). A surprising contrast in the predictors of mortality 15 to 30 days after COVID-19 between regions included higher WBC counts historically being a top predictor of death in LatAm, while this was opposite with lower WBC counts historically being a top predictor in North America. There was also an inverse association seen with shorter vintage being a top predictor of intermediate-term death in LatAm and vice versa in North America. BMI and diabetes were not among the top 15 predictors of intermediate-term mortality in either region.
The examination of the predictors of longer-term mortality > 30 days after COVID-19 presentation found consistency in risk factors between regions for older age, longer dialysis vintage, lower hemoglobin levels historically, more inflammation (higher % of neutrophils and lower % of lymphocytes historically), poorer nutrition (lower albumin and creatinine historically), and higher ferritin levels being in the top 15 predictors (Fig. 6 & 9, Additional File 1; Supplementary Table 5). Interestingly, we found an inverse association between regions for pre-HD SBP at presentation with a higher SBP being a risk factor in LatAm and vice versa in North America. Catheter exposure for > 90 days, diabetes, lower PTH, and lower BMI were uniquely among the top 15 predictors of longer-term death in the LatAm cohort, as well as other factors. The demographic factor of sex was no longer among the top 15 predictors of a long-term death after COVID-19 in either region.
Among two regional cohorts of HD patients who presented with COVID-19 before SARS-CoV-2 vaccines were available, mortality any time after presentation was 8.3 percentage points higher in LatAm countries compared to the North American country of the United States. Shorter-term mortality after COVID-19 was more common in LatAm as compared to North America cohort, with the mortality rate being 7.7 and 3.3 percentage points higher within 14 days and during 15 to 30 days after presentation respectively. Conversely, longer-term mortality after COVID-19 was more frequent in North America, with the mortality rate being 2.7 percentage points higher than in the LatAm cohort. The series of machine learning models developed in parallel in each region were found to have suitable performance in prediction of death any time after COVID-19, as well as in the prespecified shorter-term follow up timeframes. Albeit we found suitable performance in the prediction of death events in prespecified intermediate- and longer-term periods in North America, the models did not perform as well in LatAm when considering AUPRC. This finding may be related to differences in the timing of outcomes and the number of patients used in the model development. We found some consistencies in top predictors of mortality after COVID-19 in LatAm and North America. In both regions, age and vintage were top predictors of death in all timeframes and the nutrition markers of albumin and creatinine were top predictors for every timeframe except 15–30 days after presentation. The top predictors of shorter-and intermediate-term mortality after COVID-19 appeared to include unique patient attributes (e.g. higher BMI and/or male sex) that were not top predictors for longer-term mortality. Despite the consistencies, there were several regional distinctions identified. Ultimately, the results showed patients who survived COVID-19 had a better clinical status historically and at presentation, which was clearly seen for markers of nutrition in all models at all follow up time points, and further included markers of anemia and mineral bone disorders. Achievement of quality targets before and throughout the recovery process may be of high importance to survival in COVID-19. Furthermore, markers of higher inflammation appeared to remarkably contribute to the risk of death and may be important to consider when determining a patient’s prognosis in COVID-19.
Our study is unique in that it used underexplored follow-up timeframes, included a wide variety of commonly reported variables in the world, assessed temporal patterns in clinical factors before COVID-19 presentation, and utilized machine learning techniques that can account for collinearity and missingness. Other efforts assessing the predictors of mortality in COVID-19 typically assessed outcomes about 30 to 90 days after presentation, and used traditional modeling techniques (e.g. regression methods)  that cannot handle a larger number of input variables and are prone to bias through confounding interactions [3, 4, 8]. These studies provided critical early insights to the nephrology community, yet further investigations with more follow up time and more generalizable patient numbers are sparse. In our study, we observed marked differences in most clinical and demographic factors between the groups who died or survived, which made the selection of meaningful predictors for traditional modeling efforts complex. Initial investigations of correlations and collinearity in our datasets found unacceptable interactions between most variables, and this led us to select machine learning techniques that can account for these issues and limit bias.
Previous studies investigating the risk factors for mortality in dialysis patients with COVID-19 have consistently found older age categories are one of the most important risk factors for death considering follow up timeframes of 28 to 90 days [3, 4, 8, 26]. Our findings in two regional cohorts of adult HD patients further substantiate these observations. In contrast with prior studies that commonly found presence of heart failure or ischemic heart disease to be a key predictor of mortality [3, 4, 27], we never found these to be in the top 15 predictors, in any model at any follow up period in either region. We presume this is reflective of the high importance of clinical variables (e.g. laboratories and vital signs) on the prediction of death after COVID-19, factors that were not included in other reports. The results of this study build upon insights from other studies in dialysis patients and ultimately provide unique results on clinical parameters, show important considerations in temporal associations, and used models that can avoid bias resulting from collinearity. Nonetheless, further analysis is needed to differentiate parameters that are attributable to risks in COVID-19, which would include comparing the predictors of mortality in patients with and without COVID-19.
Considering reports specifically from LatAm countries with longer follow up periods, a study of 741 HD patients with COVID-19 in Brazil showed 18.8% of patients died within 90 days of diagnosis in 2020, and the majority of death events were found to have occurred within 15 days . Using a stepwise regression model, this study found the significant predictors of 90-day mortality in COVID-19 were diabetes and dialysis catheter use, in addition to increasing age in years . We also observed diabetes was in the top 15 predictors of mortality any time, and during shorter- and longer-term follow up periods, after COVID-19 in LatAm. However, we only found catheter exposure was a risk factor for longer-term mortality, ultimately clarifying the that the risk factor is the most meaningful in the subset of patients who survive at least 30 days after COVID-19 in LatAm and may be specific to the region. Notably, we never found catheter exposure to be a top predictor of mortality after COVID-19 in the North America cohort. Given the Brazilian study only evaluated a limited number of predictors and did not include any laboratories or HD treatment variables, it may have inadvertently elevated associations with catheter use to appear more meaningful than they truly are considering the majority of the routinely captured clinical information .
Looking at reports specifically from North America with longer follow up periods, an analysis of data from 60,090 prevalent dialysis patients with COVID-19 in the United States who had Medicare insurance found 26.0% of patients died throughout 2020 . This study used a Cox regression model to determine the risk factors related to mortality after COVID-19 diagnosis, and found the significant predictors of death included older age, longer dialysis vintage, being male, higher BMI categories, being of a white race, presence of congestive heart failure or ischemic heart disease along with other parameters (e.g. modality, population density, nursing home utilization). We showed consistent findings for increased risks of death in COVID-19 with older age and longer vintage for all follow up timepoints in our North America cohort. Further, we also found being male was a top predictor of mortality in COVID-19, especially for shorter- and intermediate-term outcomes. However, we did not observe male sex to be a top predictor of longer-term outcomes occurring > 30 days after presentation. We also found higher BMI to be a top predictor, yet only for shorter-term mortality within 14 days of presentation. Although BMI was not a top predictor of longer-term death in North America, it is noteworthy to mention that the association became inversed with lower BMI being associated with a higher risk of death coming in as the 34th predictor in the region. Remarkably, this observation was more clearly seen in the LatAm cohort where higher BMI was among the top 15 predictors of shorter-term death and lower BMI was among the top 15 predictors of longer-term death after COVID-19 (Fig. 6). As mentioned earlier, we did not find heart failure or ischemic heart disease to be top predictors. We did not include race in our models since we focused on variables that are universally captured in both world regions; data on race is not captured in some LatAm countries, which is a limitation.
Traditional regression modelling techniques can provide a simpler interpretation on a population level due to the requirement for establishing a reference, with categories or successive changes in the measure, of which the former considers everyone in a group to be the same and the latter requires the assumption of linear relationships in effects . This process allows a hazard ratio or odds ratio to be produced and provides an average probability of an outcome in one group or another, or by a specified increase/decrease. Although traditional techniques can provide a simple interpretation for a population, information gain is often lost, and unacceptable generalization can occur. Non-linear modeling, such as the machine learning techniques we utilized, can consider the effects for continuous variables without categorization and do not require arbitrary assumptions in linear relationships . It is worthwhile to mention there have been advancements in predictive modeling techniques in recent years, and deep learning methods might have the potential to perform even better than the machine learning methods chosen by us due to the XGBoost model’s ability to account for collinearity and missingness [28, 29]. A limitation of these machine and deep learning models are that the outputs can be less intuitive on a population level. In our case, we report the SHAP values in log odds (i.e. the logarithm of the odds ratio) with average population risks being provided in absolute values that only show relative importance of a factor, yet not the direction of the association. Nonetheless, the individual predictions can provide more interpretable information for any given individual patient, in a more personalized manner, including each individual patient’s probability of experiencing an outcome, as well as the probability and direction of the association for each individual predictor variable for each individual patient. Importantly, the top predictors established consider the average risk for patients in each regional cohort and the top predictors for individuals will likely differ some since every affected patient may not have the same physiological disturbances in the same factors.
Although we observed consistencies in the top predictors of mortality in COVID-19 in HD patients between the world regions, we did find some contrasts in the top predictors as well as inverse associations. These could be in part reflective of the differences in the timing of death events after COVID, which occurred earlier in LatAm and later in North America. Supporting this, we did find some the top predictors of mortality changed from shorter to longer survival times, such as in the case of BMI. Also, these contrasts could be attributable to differences in the regional cohorts related to patient characteristics, practice patterns, and resource limitations. Some select laboratories were measured less frequently in Latin versus North America countries, which is a potential limitation. However, we did not qualitatively observe any concerning differences in the descriptive statistics for the cohorts.
Our findings highlight how machine learning techniques can provide personalized insights for individual patients to understand the specific risk factors of death in COVID-19 for each patient, as well as provide a better generalization of the most important risk factors for a cohort/population. We found most the models constructed had suitable performance in providing individualized prognosis for HD patients with COVID-19. These modeling techniques can be adopted by providers with analytical resources to assist care teams and enhance treatment paradigms. We recommend using an array of variables and including modifiable factors to provide potential ways to intervene. In the development of models, fewer variables could be considered, and data driven selection of variables is recommended. If models are adapted considering fewer variables (e.g. the top 15, 25, or 50), they would likely perform acceptably with the most information gain being attributable to the top predictors, yet a reasonable proportion of the top predictors should be included to maintain the ability to provide personalized predictions, especially for modifiable factors that can be intervened upon. Notably, we used a default cutoff threshold for calculation the balanced accuracy performance metric. It may be prudent to evaluate adjustments in this cutoff threshold for prospective efforts to optimize model performance for a specific use case and intervention.
Prior efforts have leveraged machine learning modeling to assist with early detection of SARS-CoV-2 infection in HD patients , and these models add another set of resources to be considered in the clinician’s toolbox by providing a method to suitably assist with the prognosis of HD patients who contract COVID-19. Amidst the time of SARS-CoV-2 vaccines being more and more of an option in the world, the predictors of mortality will need to be established specifically in vaccinated dialysis patients considering regional differences in the world in patient populations and vaccine types. Given some countries continue to have limitations in access to SARS-CoV-2 vaccines , these models and the established predictors of mortality in HD patients before vaccines were available will be of high importance to the global nephrology community and can be leveraged for the development of models in vaccinated cohorts.
In summary, our findings show the profiles of mortality in HD patients with COVID-19 were distinct in LatAm and North America throughout the year 2020. There was a higher mortality rate within 0–14 or 15–30 days after COVID-19 in LatAm, while the mortality rate was higher in North America > 30 days after presentation. Irrespective of these differences, a marked proportion of HD patients died > 30 days after presentation with COVID-19 (6% in LatAm and 9% in North America cohorts). We were able to successfully construct a series of prediction models with suitable performance in both regions for determining the risk of death in an HD patient any time after COVID-19 presentation, as well as within 0–14, 15–30, and > 30 days after COVID-19 presentation. Results showed older age, longer vintage, poor nutrition, and higher inflammation were consistently top predictors of death in COVID-19 in both world regions at all timepoints after COVID-19 presentation. Unique patient attributes including higher BMI and male sex were top predictors of shorter-and intermediate-term mortality, yet not longer-term mortality. These insights further expand our understanding of the profiles and predictors of mortality and provide modeling techniques that can be considered for use by dialysis providers internationally.
Availability of data and materials
The datasets generated and/or analysed during the current study are not publicly available due to the datasets being captured from private electronic medical record systems that are restricted to use by only authorized employees of Fresenius Medical Care, but are available from the corresponding author on reasonable request. A reasonable request to access the datasets would include and require agreements to be established between Fresenius Medical Care and an external individual(s) institution.
Area under the curve
Body mass index
Diastolic blood pressure
Interdialytic weight gain
Intact parathyroid hormone
Severe Acute Respiratory Syndrome Coronavirus 2
Systolic blood pressure
White blood cell
Keller N, Chantrel F, Krummel T, Bazin-Kara D, Faller AL, Muller C, Nussbaumer T, Ismer M, Benmoussa A, Brahim-Bouna M, et al. Impact of first-wave COronaVIrus disease 2019 infection in patients on haemoDIALysis in Alsace: the observational COVIDIAL study. Nephrol Dial Transplant. 2020;35(8):1338–411.
Creput C, Fumeron C, Toledano D, Diaconita M, Izzedine H. COVID-19 in Patients Undergoing Hemodialysis: Prevalence and Asymptomatic Screening During a Period of High Community Prevalence in a Large Paris Center. Kidney Med. 2020;2(6):716-723 e711.
Haarhaus M, Santos C, Haase M, MotaVeiga P, Lucas C, Macario F. Risk prediction of COVID-19 incidence and mortality in a large multi-national hemodialysis cohort: implications for management of the pandemic in outpatient hemodialysis settings. Clin Kidney J. 2021;14(3):805–13.
Hsu CM, Weiner DE, Aweh G, Miskulin DC, Manley HJ, Stewart C, Ladik V, Hosford J, Lacson EC, Johnson DS, et al. COVID-19 Among US Dialysis Patients: Risk Factors and Outcomes From a National Dialysis Provider. Am J Kidney Dis. 2021;77(5):748-756 e741.
Neumann ME. Latest data show 305 dialysis patient deaths due to COVID-19 in the US. Nephrology News & Issues 2020, (Accessed 22 Apr 2020) https://www.healio.com/nephrology/infection-control/news/online/%7B3a263aa9-ad59-4c3f-aab7-07b8395508e5%7D/latest-data-show-305-dialysis-patient-deaths-due-to-covid-19-in-the-us.
Taji L, Thomas D, Oliver MJ, Ip J, Tang Y, Yeung A, Cooper R, House AA, McFarlane P, Blake PG. COVID-19 in patients undergoing long-term dialysis in Ontario. CMAJ. 2021;193(8):E278–84.
Quintaliani G, Reboldi G, Di Napoli A, Nordio M, Limido A, Aucella F, Messa P, Brunori G, Italian Society of Nephrology C-RG. Exposure to novel coronavirus in patients on renal replacement therapy during the exponential phase of COVID-19 pandemic: survey of the Italian Society of Nephrology. J Nephrol. 2020;33(4):725–36.
Jager KJ, Kramer A, Chesnaye NC, Couchoud C, Sanchez-Alvarez JE, Garneata L, Collart F, Hemmelder MH, Ambuhl P, Kerschbaum J, et al. Results from the ERA-EDTA Registry indicate a high mortality due to COVID-19 in dialysis patients and kidney transplant recipients across Europe. Kidney Int. 2020;98(6):1540–8.
Robinson BM, Guedes M, Alghonaim M, Cases A, Dasgupta I, Gan L, Jacobson SH, Kanjanabuch T, Kim YL, Kleophas W, et al. Worldwide Early Impact of COVID-19 on Dialysis Patients and Staff and Lessons Learned: A DOPPS Roundtable Discussion. Kidney Med. 2021;3(4):619–34.
United States Renal Data System. 2021 USRDS Annual Data Report: Epidemiology of kidney disease in the United States. In: National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases. Bethesda; 2021.
Dialysis COVID-19 Vaccination Data Dashboard. Centers for Disease Control and Prevention: National Healthcare Safety Network (Accessed 15 Nov 2021):https://www.cdc.gov/nhsn/covid19/dial-vaccination-dashboard.html.
Coronavirus Resource Center: Understanding Vaccination Progress by Country. Johns Hopkins University School of Medicine (Accessed 15 Nov 2021):https://coronavirus.jhu.edu/vaccines/international.
Chen JJ, Lee TH, Tian YC, Lee CC, Fan PC, Chang CH. Immunogenicity rates after SARS-CoV-2 vaccination in people with end-stage kidney disease: a systematic review and meta-analysis. JAMA Netw Open. 2021;4(10):e2131749.
Mulhern JG, Fadia A, Patel R, Ficociello LH, Willetts J, Dahne-Steuber IA, Pollan MC, Mullon C, DeLisi J, Johnson C, et al. Humoral Response to mRNA versus an Adenovirus Vector-Based SARS-CoV-2 Vaccine in Dialysis Patients. Clin J Am Soc Nephrol. 2021;16(11):1720–2.
Pamplona GM, Sullivan T, Kotanko P. COVID-19 vaccination acceptance and hesitancy in dialysis staff: first results from New York City. Kidney Int Rep. 2021;6(4):1192–3.
Bhandari S. Reasons for COVID-19 vaccination hesitancy in hemodialysis patients. Kidney Int. 2021;100(3):702.
Marcelli D, Kirchgessner J, Amato C, Steil H, Mitteregger A, Moscardo V, Carioni C, Orlandini G, Gatti E. EuCliD (European Clinical Database): a database comparing different realities. J Nephrol. 2001;14(Suppl 4):S94-100.
Chaudhuri S, Lasky R, Jiao Y, Larkin J, Monaghan C, Winter A, Neri L, Kotanko P, Hymes J, Lee S, et al. Trajectories of clinical and laboratory characteristics associated with COVID-19 in hemodialysis patients by survival. Hemodial Int. 2022;26(1):94–107.
Chen T, Guestrin C. XGBoost: A Scalable Tree Boosting System. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. San Francisco, California, USA: Association for Computing Machinery; 2016. p. 785–94.
Saito T, Rehmsmeier M. The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS One. 2015;10(3):e0118432.
Shapley LS. A Value for n-Person Games. In: Kuhn HW, Tucker AW, Eds., Contributions to the Theory of Games II. Annals of Mathematics Studies, Princeton University Press, Princeton 1953, 28:307–317.
Štrumbelj E, Kononenko I. Explaining prediction models and individual predictions with feature contributions. J Knowl Inf Syst. 2013;41:647–65.
Lundberg S, Lee SI. A Unified Approach to Interpreting Model Predictions. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R, editors. Advances in Neural Information Processing Systems 30. Curran Associates, Inc. 2017. p. 4765–74.
Lundberg SM, Erion G, Chen H, DeGrave A, Prutkin JM, Nair B, Katz R, Himmelfarb J, Bansal N, Lee SI. From local explanations to global understanding with explainable AI for trees. Nat Mach Intell. 2020;2(1):56–67.
Chaudhuri S, Long A, Zhang H, Monaghan C, Larkin JW, Kotanko P, Kalaskar S, Kooman JP, van der Sande FM, Maddux FW, et al. Artificial intelligence enabled applications in kidney disease. Semin Dial. 2021;34(1):5–16.
Lugon JR, Neves P, Pio-Abreu A, do Nascimento MM, Sesso R; Investigators C-H-B. Evaluation of central venous catheter and other risk factors for mortality in chronic hemodialysis patients with COVID-19 in Brazil. Int Urol Nephrol. 2022;54(1):193–9.
Salerno S, Messana JM, Gremel GW, Dahlerus C, Hirth RA, Han P, Segal JH, Xu T, Shaffer D, Jiao A, et al. COVID-19 risk factors and mortality outcomes among medicare patients receiving long-term dialysis. JAMA Netw Open. 2021;4(11):e2135379–e2135379.
Kivrak M, Guldogan E, Colak C. Prediction of death status on the course of treatment in SARS-COV-2 patients with deep learning and machine learning methods. Comput Methods Programs Biomed. 2021;201:105951.
Pettit RW, Fullem R, Cheng C, Amos CI. Artificial intelligence, machine learning, and deep learning for clinical outcome prediction. Emerg Top Life Sci. 2021;5(6):729–45.
Monaghan CK, Larkin JW, Chaudhuri S, Han H, Jiao Y, Bermudez KM, Weinhandl ED, Dahne-Steuber IA, Belmonte K, Neri L et al. Machine Learning for Prediction of Hemodialysis Patients with an Undetected SARS-CoV-2 Infection. Kidney360 2021:https://doi.org/10.34067/KID.0003802020.
We would like to thank all the direct patient care teams at Fresenius Medical Care who captured the data used in this analysis during the provision of standard medical care, and who have and continue to heroically serve the vulnerable dialysis community during this ongoing COVID-19 pandemic.
No external funding was provided for the conduct of the investigations. The analyses and manuscript composition were internally supported by Fresenius Medical Care, which included employee salaries and company infrastructure.
Ethics approval and consent to participate
All experimental protocols were approved by a named institutional entity and/or licensing committee (Fresenius Medical Care and New England Independent Review Board). In LatAm and North America, de-identified data was used for the purposes of the parallel analyses. The EuCLiD database was used for capturing data in the Latin America cohort as part of Fresenius Medical Care's quality improvement and management programs in all NephroCare clinics utilizing EuCLiD . EuCLiD governance has established protocols and procedures for use of clinical data from NephroCare clinics for secondary research purposes, and granted approval for the extraction of data for this secondary analysis in the Latin America cohort. Data was only collected from patients who provided informed consent for their data to be collected into EuCliD and the data was de-identified by the LatAm investigator. The Fresenius Medical Care North America Knowledge Center Data Warehouse was used for capturing data in the North America cohort from clinics in the Fresenius Kidney Care network. In North America, data was collected from patients treated in the United States under a protocol approved by New England Independent Review Board (NEIRB; Needham Heights, MA, United States); NEIRB determined the analysis of the North America cohort was exempt due to use of data de-identified by the North America investigator that no longer contained protected health information and consent was not required per title 45 of the United States Code of Federal Regulations part 46.104(d)(4) (NEIRB# 1–1439054-1). The analysis in each region was conducted in accordance with the Declaration of Helsinki.
Consent for publication
All authors are employees of Fresenius Medical Care, or its wholly owned subsidiary Renal Research Institute. J.H., L.A.U., P.K., F.W.M. have share options/ownership in Fresenius Medical Care. P.K. receives honorarium from Up-To-Date and is on the Editorial Board of Blood Purification and Kidney and Blood Pressure Research. C.K.M., J.H., J.W.L., L.A.U., P.K., F.W.M. are an inventor on patent(s) in the field of dialysis. J.W.L. is a guest editor on the Editorial Board of Frontiers in Physiology. F.W.M. has directorships in Fresenius Medical Care Management Board, Goldfinch Bio, and Vifor Fresenius Medical Care Renal Pharma.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Initial, Tuning Range, and Final Hyperparameter Settings for Models. Supplementary Table 2. Mean SHAP values for all predictors of death any time after COVID-19 presentation, by region. Supplementary Table 3. Mean SHAP values for all predictors of death 0-14 days after COVID-19 presentation, by region. Supplementary Table 4. Mean SHAP values for all predictors of death 15-30 days after COVID-19 presentation, by region. Supplementary Table 5. Mean SHAP values for all predictors of death >30 days after COVID-19 presentation, by region.
About this article
Cite this article
Guinsburg, A.M., Jiao, Y., Bessone, M.I.D. et al. Predictors of shorter- and longer-term mortality after COVID-19 presentation among dialysis patients: parallel use of machine learning models in Latin and North American countries. BMC Nephrol 23, 340 (2022). https://doi.org/10.1186/s12882-022-02961-x
- Mortality Risk
- Machine Learning
- Prediction Model