Skip to main content
  • Research article
  • Open access
  • Published:

External Validation of a risk stratification model to assist shared decision making for patients starting renal replacement therapy

Abstract

Background

Shared decision making is nowadays acknowledged as an essential step when deciding on starting renal replacement therapy. Valid risk stratification of prognosis is, besides discussing quality of life, crucial in this regard. We intended to validate a recently published risk stratification model in a large cohort of incident patients starting renal replacement therapy in Flanders.

Methods

During 3 years (2001–2003), the data set collected for the Nederlandstalige Belgische Vereniging voor Nefrologie (NBVN) registry was expanded with parameters of comorbidity. For all incident patients, the abbreviated REIN score(aREIN), being the REIN score without the parameter “mobility”, was calculated, and prognostication of mortality at 3, 6 and 12 month after start of renal replacement therapy (RRT) was evaluated.

Results

Three thousand four hundred seventy-two patients started RRT in Flanders during the observation period (mean age 67.6 ± 14.3, 56.7 % men, 33.6 % diabetes). The mean aREIN score was 4.1 ± 2.8, and 56.8, 23.1, 12.6 and 7.4 % of patients had a score of ≤4, 5–6, 7–8 or ≥9 respectively. Mortality at 3, 6 and 12 months was 8.6, 14.1 and 19.6 % in the overall and 13.2, 21.5 and 31.9 % in the group with age >75 respectively. In RoC analysis, the aREIN score had an AUC of 0.74 for prediction of survival at 3, 6 and 12 months. There was an incremental increase in mortality with the aREIN score from 5.6 to 45.8 % mortality at 6 months for those with a score ≤4 or ≥9 respectively.

Conclusion

The aREIN score is a useful tool to predict short term prognosis of patients starting renal replacement therapy as based on comorbidity and age, and delivers meaningful discrimination between low and high risk populations. As such, it can be a useful instrument to be incorporated in shared decision making on whether or not start of dialysis is worthwhile.

Peer Review reports

Background

Worldwide, age at inception of renal replacement therapy is increasing. In Europe, the median age at start of RRT has increased from 64 to 73 years over the last decade [1]. Whereas start of renal replacement therapy can be lifesaving, it is associated with a high short term mortality and a substantial decrease in quality of life for some patients [2, 3]. In this setting, there is an increasing interest for the concept of conservative care [46], as it is accepted that for some patients the benefits of starting renal replacement therapy do not outweigh the drawbacks. Existing literature indicates that mere age on itself is insufficient to prognosticate outcome after start of RRT, and that rather presence of comorbidities should be taken into account [79].

Studies indicate that physicians tend to be overly optimistic on the prognosis of their patients [10]. Such a failure to recognise a poor prognosis might lead to perseveration of therapy and overemphasis of cure rather than care [5, 11] and deprive patients from achieving a good and serene death [4].

In order to discuss the option of conservative care with patients, there is a need for risk stratification models that can more objectively quantify prognosis. Such models should be valid from the statistical point of view, not only within the test population, but also in other external populations. Recently, a prognostic model was developed for 5 year mortality versus need for RRT in patients with CKD [12]. This model can be useful for longer term planning of patients with CKD, but is not intended to predict survival in patients with CKD stage 5, and cannot assist in decision making on whether to start RRT or opt for conservative care.

Observational studies consistently demonstrate that as comorbidity adds up, the survival advantage of starting renal replacement goes down, and survival between patients started on RRT and those with conservative care become similar [9, 13, 14]. In this regard, risk stratification models based on survival of patients who have actually started RRT can provide valid information in the decision making of whether or not starting RRT is warranted. Recently, such a model was published, based on data from the Renal Epidemiology Information Network (REIN) registry in France [15].

We intended to check the validity of this model in a large registry cohort of patients starting RRT in the Flanders region of Belgium, coordinated by the Nederlandstalige Vereniging voor Nefrologie.

Methods

Patients

The registry of the NBVN is a voluntary database which covers 100 % of all patients starting renal replacement therapy in Flanders, Belgium. As in the REIN database, patients with acute kidney injury are excluded.

The registry captures a basic set of baseline data through a web based system consisting of the following items : date of birth, gender, city/zip code, primary renal disease, date of first contact at the nephrology department, weight, length, serum creatinine at start of RRT, change of RTT modality (type and date), and mortality (cause and date).

Between 1/1/2001 until 31/12/2003, a more expanded set of data of comorbidities was collected at baseline, one and two years, with the intention to allow prognostic modelling. Baseline information at dialysis initiation included age, gender, eGFR based on creatinine and the MDRD formula, body mass index (BMI), serum albumin the month preceding dialysis start, diabetes (type 1 or 2), congestive heart failure (New York Heart Association stages I to IV), ischaemic heart disease (including history of myocardial infarction, coronary vascular disease, coronary artery bypass surgery, angioplasty or abnormal angiography), peripheral vascular disease (Leriche classification stages I to IV), cerebrovascular disease, arrhythmia, chronic obstructive pulmonary disease (COPD), malignancy, liver cirrhosis, mental disorders (defined to include dementia and psychosis), initial dialysis modality, and late referral (defined as starting dialysis less than 3 months after first contact with the nephrology department.

Patient data were entered by the individual centres and are validated on an ongoing basis by the registry data manager.

Analysis

For each patient, the abbreviated REIN (aREIN) score was calculated based on the baseline comorbidity data as depicted in the original paper [15]. As the NBVN registry does not capture data on mobility, this item was omitted from the REIN score, and therefore, we designated this as the “abbreviated REIN score” (aREIN score) (Table 1).

Table 1 Parameters of the abbreviated REIN score

Demographic data of patients were represented as mean ± standard deviation for continuous variables or n out of N (n/N) and percentages for categorical variables. Between group comparisons were done with One Way Analysis of Variance and post hoc testing with Least Square Difference for continuous variables, or Chi-square analysis for categorical variables.

Mortality rates at 3, 6 and 12 months were calculated, and this for four different severity stages of the aREIN score (low risk:≤4; moderate low risk: 5–6; moderate high risk: 7–8; and high risk: ≥9 points). Those four groups were selected as they represented a reasonable division of our cohort and resulted in subgroups with comparable mortality risk (see Additional file 1: Table S1).

Areas under the curve (AUC) were calculated for prediction of 3, 6 and 12 months survival based on the aREIN score.

Cox proportional hazards analysis was used to model survival in the different severity stages of the aREIN score.

All statistical analysis were performed in SPSS®version 22.

Results

During the observation period, 3472 patients started renal replacement therapy. The demographic data of these patients are depicted in Table 2 according to their aREIN stage. For 793 patients (22.8 %) information on one parameter of the REIN score was missing, making aREIN score calculation impossible, leaving 2679 patients available for analysis. There was no difference in gender or age between those with versus without missing data.

Table 2 Demographic data of included patients, stratified according to the abbreviated REIN score

More than half (56.4 %) and almost three quarters (70.3 %) of those older than 85 and 90 years of age respectively at start of dialysis had an aREIN stage of 3 or 4.

We registered 276 (8.6 %), 453 (14.1 %) and 681 (19.6 %) deaths at 3, 6 and 12 months respectively.

Patients who died during the first 3 months were older (74.3 ± 9.9 vs 67.0 ± 14.5 years, p < 0.001), had a higher aREIN score at start (6.4 ± 2.7 vs 3.9 ± 2.7, p < 0.001), and a lower serum creatinine (6.1 ± 3.8 vs 6.7 ± 3.1 mg/dl, p < 0.01) and body weight (69.4 ± 15.5 vs 71.7 ± 15.7 kg, p = 0.03) as compared to those who survived (Table 3). Similar results were found for mortality at 6 and 12 months (results not shown).

Table 3 Comorbidities in patients in surviving vs dying within the first 3 months after start of RRT

Mortality at 3, 6 and 12 months according to 4 stages of aREIN score is graphically presented in Fig. 1, showing a good discrimination in mortality risk between the different aREIN stages. The results for the separate scores are presented in Additional file 1: Table S1.

Fig. 1
figure 1

Mortality at 3, 6 and 12 months after start of RRT according to the stages of the abbreviated REIN score

Relative risk of death increased with each increasing aREIN stage with a factor 1.80 (95 % CI: 1.67–1.92), 1.78 (95 % CI: 1.64–1.96) and 1.80 (95 % CI: 1.61–1.99) at 3, 6 and 12 months respectively.

Figure 2 presents these survival data in the different aREIN stages in a format that facilitates shared decision making with patients.

Fig. 2
figure 2

Survival according to aREIN stage panel A: 3 month survival; panel B: 12 month survival; Blue dots: patients who survived this period; Red dots: patients who did not survive this period

Area under the curves for 3, 6 and 12 months mortality are 0.74 (95 % CI: 0.70–0.77), 0.74 (95%CI: 0.71–0.76) and 0.74 (95 % CI: 0.72–0.76).

Long term survival according to different stages of the aREIN score is depicted in Fig. 3, confirming that also longer term mortality risk goes up with aREIN stage.

Fig. 3
figure 3

Long term survival after start of RRT in function of aREIN stage

Discussion

This paper provides an external validation for a recently proposed risk stratification score for short term mortality in patients starting with renal replacement therapy [15]. In the NBVN registry, 3 month mortality risk ranged from 3 % in the lowest risk group to 24.6 % in the highest risk group. The score also performed well in discriminating risk at 6 and 12 months. In the highest risk stage, mortality at one year was higher than 50 %, whereas in the lowest risk stage, this was less than 10 %. With each increase in the aREIN stage, the relative risk for death at 3, 6 or 12 month increased with a factor 1.8. The aREIN score is based on easily available items and can be calculated by hand during outpatient consultation. Items included in the score are all well-acknowledged risk factors that are generally registered in the patient file under one format or another, but are rarely considered together to construct a global picture of the patient. Therefore, the calculation of the risk stratification score can help to increase a more holistic awareness on the prognostic risk of a patient and to identify patients at risk for a dismal outcome. The visual representation of the risk of mortality after start of RRT as depicted in Fig. 2 can help in the shared decision making on whether to start renal replacement therapy or to opt for conservative care.

Over the last decade, the age and comorbidity of patients starting renal replacement therapy has steadily increased [1]. In Flanders for example, the median age at start increased from 67.7 years in 1997 to 74.7 years in 2013. Over this period, the percentage of patients >75 years of age increased from 22.8 to 36.7 %, while over the same time period, comorbidity of patients starting RTT went up [16]. At the same time, withdrawal of dialysis as reason for death increased from 12.9 to 20.8 % [16], and many patients indicate they regret having started RRT [17].

As reported in many other studies, also in the current cohort, short term survival is abysmal especially in the older age group (>85). Several authors have indicated that not age per se, but rather underlying comorbidity, might be a predictor of poor outcome, and that outcome can be good even in the elderly, provided there are no other comorbidities present [9, 13]. Indeed, comorbidity and functional and cognitive function can vary substantially amongst older patients, rendering age itself less relevant as a discriminating prognostic factor. Also in the risk stratification score of Couchoud et al [15], comorbidity is much more important for prognosis than age, which only contributes an additional 1 or 2 points for those older than 85 or 90 respectively. Nevertheless, in our current validation cohort, it was clear that age and comorbidity were linked, with more than half of the patients older than 85 being in the highest risk stages of the REIN score. However, vice versa, it also indicates that a substantial part of the very old does not have comorbidity, and might thus have a good prognosis on dialysis. Thus, our data confirm that age cannot be the sole criterion to decide to start or withhold dialysis, though older age is often associated with more comorbidities.

The use of a risk stratification score can be very helpful to better illustrate the odds of a good or bad outcome in an objective manner. Results of the score and the associated risk, if presented in a format the patient can understand, can be of real value in guiding a shared decision making by creating realistic expectations about the treatment [1820]. Our external validation of the REIN risk stratification score corroborates its use for such purposes.

Shared decision making comprises 3 stages: informing the patient, elicit his/her preferences and values, and assist decision making [21]. The REIN score [15] appears to be sufficiently accurate to provide the necessary prognostic information to the patient. Care should be taken however on the way this information is presented to the patient, as most patients and even physicians struggle in interpreting relative risks and statistical data [22, 23]. Easy pictographic presentations as presented in Fig. 2 can be helpful to convey the underlying messages of the data and their implications to the patient. However, patient’s response to and conclusions drawn from the presented data might be different, based on their background beliefs, values and expectations of how their life (or death) should look like. For an 85 year old, even a 50 % probability of mortality at 3 months might still justify start of dialysis when an important event, for example a marriage of a grandchild, is planned in the near future, whereas for another individual, even a 15 % mortality probability at 6 months might not suffice to choose starting or continuing dialysis if the accompanying quality of life is too poor. The elicitation of the patient’s preferences and values is an important part of the shared decision making process, and it is important that the involved professionals avoid mixing up their own beliefs, values and expectations with those of the patient and/or his next of kin. A parallel can be drawn with the area of dialysis withdrawal, where there is evidence that there is substantial peer pressure amongst nephrologists on whether or not they believe that withdrawal of dialysis is ethically defendable [24]. More important, there is a strong correlation between the actual occurrence of dialysis withdrawal in a centre and the beliefs of the treating physician [24]. These observations underscore the need for easy and neutral pictographic presentation of relative risk, minimizing external bias and maximizing patient autonomy.

It can be argued that we, as did the original REIN study, only included patients that actually did start dialysis, and that we therefore do not know the fate of patients that did not start dialysis. Several studies underscore however that in patients with one or more comorbidities, survival is equal in those who did or did not start dialysis. Most patients who opt not to start dialysis most likely have multiple comorbidities, and would thus end up in the higher risk scores. As such, we believe the use of a cohort that actually started dialysis does not jeopardize the conclusion of its validity to identify patients at high risk of poor outcome.

A limitation of our validation might be that the REIN cohort has similar properties in terms of patient characteristics as the NBVN cohort. In addition, REIN and NBVN operate in a comparable healthcare environment in terms of organisation and cultural background. As a consequence, it can still be that the REIN score does not discriminate sufficiently well in other healthcare settings or regions.

The NBVN registry does not capture mobility and need for assistance for transfer, parameters of functional capacity. By consequence, we were not able to capture frailty in our patients. Frailty is a relatively new emerging concept indicating a lack of physiological reserve. It is difficult to define in strict terms, but its presence can easily be assessed from clinical observations. In the REIN risk stratification tool, it is mainly reflected in the “mobility” issue, and it contributes substantially to the overall risk, with 9 points for the maximal score. Different geriatric assessment scores for frailty exist, but most of them are laborious and/or time consuming, and cannot be performed on a regular or repetitive basis. It has been demonstrated before that frailty and classical comorbidity do not always completely overlap [25]. A simple assessment of the REIN score can discriminate those patients who might benefit from a more extensive geriatric assessment by a geriatrician.

It could be argued that the score could be further improved by adding extra elements. However, it has been advocated before that the search for a highly accurate score for use in clinical practice is probably not justified [26]. Indeed, patients rather interpret information provided by the scores in more general qualitative rather than in absolute numerical terms. A more complex score might be cumbersome and time consuming to use in clinical practice. Therefore, absolute accuracy should not be strived for, and ease of use is probably more of importance. One factor that might improve the score is to include mode of vascular access (catheter vs graft vs fistula), as this has been demonstrated to be associated with outcome. However, most likely, there is a high collinearity between the items already included in the score, and the propensity to have a catheter as first access.

Conclusions

Our paper provides an external validation of a clinically applicable risk stratification tool. Such a tool is crucial to assist evidence based shared decision making on whether to start dialysis or opt for conservative care.

Ethics approval and consent to participate

This study was approved by the ethical committee of the Ghent University Hospital.

Consent for publication

As only summary congregated data are presented, consent for publication of patients is not applicable.

Availability of data and materials

Supplementary data on prognostication at 3, 6 and 12 months are available in “Additional file 1: Table S1”.

A synopsis of the NBVN registry data is available at the NBVN website (www.nbvn.be) under the heading “jaarverslag”.

As far as not already presented in the manuscript or on the website, more detailed data can be obtained after permission to do so is granted by the NBVN registry steering committee. Please send a request to the corresponding author at wim.vanbiesen@ugent.be.

Abbreviations

aREIN:

abbreviated REIN score

AUC:

area under the curve

BMI:

body mass index

CKD:

chronic kidney disease

COPD:

chronic obstructive pulmonary disease

eGFR:

estimated glomerular filtration rate

HD:

haemodialysis

MDRD:

modification of diet in renal disease

NBVN:

Nederlandstalige Vereniging voor Nefrologie

NYHA:

New York Heart Association

PCI:

Percutaneous Coronary Intervention

PD:

Peritoneal Dialysis

REIN:

Renal Epidemiology Information Network

RRT:

Renal Replacement Therapy

References

  1. Noordzij M, Kramer A, Abad Diez JM, et al. Renal replacement therapy in Europe: a summary of the 2011 ERA-EDTA Registry Annual Report. Clin Kidney J. 2014;7(2):227–38.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Kurella Tamura M, Covinsky KE, Chertow GM, et al. Functional status of elderly adults before and after initiation of dialysis. N Engl J Med. 2009;361(16):1539–47.

    Article  PubMed  Google Scholar 

  3. Berger JR, Hedayati SS. Renal replacement therapy in the elderly population. Clin J Am Soc Nephrol. 2012;7(6):1039–46.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Muthalagappan S, Johansson L, Kong WM, et al. Dialysis or conservative care for frail older patients: ethics of shared decision-making. Nephrol Dial Transplant. 2013;28(11):2717–22.

    Article  PubMed  Google Scholar 

  5. Jassal SV, Watson D. Doc, don’t procrastinate…Rehabilitate, palliate, and advocate. Am J Kidney Dis. 2010;55(2):209–12.

    Article  PubMed  Google Scholar 

  6. van de Luijtgaarden MW, Noordzij M, van Biesen W, et al. Conservative care in Europe--nephrologists’ experience with the decision not to start renal replacement therapy. Nephrol Dial Transplant. 2013;28(10):2604–12.

    Article  PubMed  Google Scholar 

  7. Murtagh FE, Marsh JE, Donohoe P, et al. Dialysis or not? A comparative survival study of patients over 75 years with chronic kidney disease stage 5. Nephrol Dial Transplant. 2007;22(7):1955–62.

    Article  PubMed  Google Scholar 

  8. Kurella M, Covinsky KE, Collins AJ, et al. Octogenarians and nonagenarians starting dialysis in the United States. Ann Intern Med. 2007;146(3):177–83.

    Article  PubMed  Google Scholar 

  9. Chandna SM, Da Silva-Gane M, Marshall C, et al. Survival of elderly patients with stage 5 CKD: comparison of conservative management and renal replacement therapy. Nephrol Dial Transplant. 2011;26(5):1608–14.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Glare P, Virik K, Jones M, et al. A systematic review of physicians’ survival predictions in terminally ill cancer patients. BMJ. 2003;327(7408):195–8.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Van Biesen W, Lameire N, Veys N, et al. From curing to caring: one character change makes a world of difference. Issues related to withholding/withdrawing renal replacement therapy (RRT) from patients with important co-morbidities. Nephrol Dial Transplant. 2004;19(3):536–40.

    Article  PubMed  Google Scholar 

  12. Bansal N, Katz R, De Boer IH, et al. Development and validation of a model to predict 5-year risk of death without ESRD among older adults with CKD. Clin J Am Soc Nephrol. 2015;10(3):363–71.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Joly D, Anglicheau D, Alberti C, et al. Octogenarians reaching end-stage renal disease: cohort study of decision-making and clinical outcomes. J Am Soc Nephrol. 2003;14(4):1012–21.

    Article  PubMed  Google Scholar 

  14. Kan WC, Wang JJ, Wang SY, et al. The new comorbidity index for predicting survival in elderly dialysis patients: a long-term population-based study. PLoS One. 2013;8(8):e68748.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Couchoud CG, Beuscart JR, Aldigier JC, et al. Development of a risk stratification algorithm to improve patient-centered care and decision making for incident elderly patients with end-stage renal disease. Kidney Int. 2015.

  16. De Meester J. NBVN jaarverslag. www.nbvn.be.

  17. Davison SN. End-of-life care preferences and needs: perceptions of patients with chronic kidney disease. Clin J Am Soc Nephrol. 2010;5(2):195–204.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Winterbottom A, Conner M, Mooney A, et al. Evaluating the quality of patient leaflets about renal replacement therapy across UK renal units. Nephrol Dial Transplant. 2007;22(8):2291–6.

    Article  PubMed  Google Scholar 

  19. van der Veer SN, Bekker HL, van Biesen W. How to enhance patient experiences of decision-making about kidney transplantation? Nephrol Dial Transplant. 2014;29(11):1991–3.

    Article  PubMed  Google Scholar 

  20. Spiegelhalter D, Pearson M, Short I. Visualizing uncertainty about the future. Science. 2011;333(6048):1393–400.

    Article  CAS  PubMed  Google Scholar 

  21. Winterbottom A, Mooney. Patient acceptability of the yorkshire dialysis decision aid (yodda) booklet: a prospective non-randomized comparison study across 6 predialysis services. Perit Dial Int. [e-publication ahead of print].

  22. Johnson TV, Abbasi A, Schoenberg ED, et al. Numeracy among trainees: are we preparing physicians for evidence-based medicine? J Surg Educ. 2014;71(2):211–5.

    Article  PubMed  Google Scholar 

  23. Wegwarth O, Schwartz LM, Woloshin S, et al. Do physicians understand cancer screening statistics? A national survey of primary care physicians in the United States. Ann Intern Med. 2012;156(5):340–9.

    Article  PubMed  Google Scholar 

  24. van Biesen W, van de Luijtgaarden MW, Brown EA, et al. Nephrologists’ perceptions regarding dialysis withdrawal and palliative care in Europe: lessons from a European Renal Best Practice survey. Nephrol Dial Transplant. 2015;30(12):1951–58.

  25. Lemahieu W, Naessens M, De Meester J. Functional impairment in Flemish dialysis patients: impact on survival. JASN. 2013, p465A FRPO436.

  26. Noble D, Dent T, Greenhalgh T. Time to compare impact and feasibility of prediction models in real life. BMJ. 2012;345:e4357. author reply e4360.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors wish to thank all the participating centres that contributed data to the registry.

Funding

This study was supported by the NBVN registry, which itself is financially supported by the participating renal replacement units on a per capita basis. There was no additional external funding.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Wim Van Biesen.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

PP analysed and cleaned the database and contributed to writing and correcting the paper. WVB conceived the study, analysed the data and contributed to writing and correcting the paper. NV contributed to writing and correcting the paper and setting up the registry. WL analysed the data and contributed to writing and correcting the paper. BDM analysed the data and contributed to writing and correcting the paper. JDM was responsible for setting up the database, analysed the data and contributed to writing and correcting the paper. All authors read and approved the final manuscript.

Additional file

Additional file 1: Table S1.

Survival at 3, 6 and 12 months according to aREIN score. Survival (in %) according to aREIN score at 3, 6 and 12 months. (DOCX 10 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peeters, P., Van Biesen, W., Veys, N. et al. External Validation of a risk stratification model to assist shared decision making for patients starting renal replacement therapy. BMC Nephrol 17, 41 (2016). https://doi.org/10.1186/s12882-016-0253-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12882-016-0253-3

Keywords