We have developed and validated a simplified clinical prediction score for classifying populations into mild, intermediate-low, intermediate-high, and high risks of developing CKD. The score was made easier and simplified using factors that were readily available and simply assessed in clinical practice. The score shows good calibration and discrimination, as it can be seen from a similarity between observed and predicted values, as well as between the C statistic in the derivative and validation phases, respectively.
We mainly focus on estimating risk of developing CKD, not CKD progression. Our study has strengths in research methods as commented in detail in Ingsathit et al[10]. It is derived from a large number of subjects who were stratified-cluster randomly sampling form areas across Thailand. In addition, we have validated the clinical prediction score using a 200-repetition bootstrap technique, which is considered a good technique for internal validation [15–17]. The C statistic of the simplified score was fair in both derivative and bootstrap data (i.e., 0.77 and 0.74), indicating that the score can well discriminate between CKD and non-CKD subjects. Our model is simplified and should be easily to apply in clinical practice because of required variables are routinely measured. The scoring scheme should be able to apply manually by clinicians and then the score is classified into 4 groups without requiring any calculation.
We however have some limitations. Our design was a cross-sectional survey study and thus the temporal sequence between risk factors and CKD was questionable[18]. External validation has not been performed and generalizability of our prediction model is still needed to determine. Given good represent samples across the country, the model might work well in outside the studied areas in Thailand, or in other countries where prevalence of diabetes (~11.9%), hypertension (27.5%), and kidney stone (5.0%) are similar to Thai population. The overly-optimistic predictions in other populations might be less likely because of an optimistic rate from the bootstrap is only 2.9%.
To our knowledge, only a few previous prediction scores [3–5, 19] for kidney diseases were available in prior literature. The study by Hemmelgarn et al [4] had developed a clinical index for rapid progression of kidney function, which was defined as a decline in eGFR of 25% or greater. Five predictors were included in the clinical index, which were age, heart disease, diabetes, gout, and the use of anti-emetic drugs. Comparatively, in our study, only two predictors (i.e., age and diabetes) were similar, but the others were not significantly associated with CKD and thus were not considered. The ability of the score to correctly discriminate those individuals with and without CKD from our study was fair (area under ROC = 0.77) but it was low for this study (area under ROC = 0.59). Bang et al[3] had well developed scores based on a cross-sectional NHANES 1999-2000 & 2001-2002 data. The performance of their scores had also been both internally and externally tested with good and fair performances (i.e., the C statistic were 0.88 and 0.71, respectively). A suggestion of using the score in clinical practice was also discussed. However, applying a score with a required 9 input variables (i.e., age, female, anemia, hypertension, diabetes, history of cardiovascular disease, congestive heart failure, peripheral vascular disease, and proteinuria) in clinical practice might not be as simple as suggested. Data for a history of cardiovascular disease, congestive heart failure, and peripheral vascular disease may not be valid in developing countries where populations have a limited understanding of the diseases they have been diagnosed with. The ability of discrimination for this score in Asians may not be valid since the prevalence of CKD in Caucasian vs. Asian populations is quite different [20], and also lifestyle factors that directly or indirectly influence CKD may be markedly different. Kshirsagar et al[5] conducted a study on a community-based cohorts with ages of 45 years or older in order to create the best fitting and simplified scores for predicting the incidence of CKD (GFR < 60 ml/min/1.73 m2). The study design was better than previous studies in terms of the predictor-CKD causal relationship, which could be assessed for the cohort study. Two prediction score models were proposed, one with, the full score of 10 predictors (i.e., age, white ethnicity, female, anemia, hypertension, diabetes, history of cardiovascular disease, history of heart failure, low HDL, and peripheral vascular disease), and a simplified score with 8 predictors, omitting two variables, ethnicity and HDL. Again, applying either the best fitting or the reduced model may not so simple since so many variables are required to input in the models. Creating a score by rounding up a coefficient to the nearest integer (higher than estimated) was not clearly described in the paper. For instance, estimated coefficients for ages 60-69 and ≥ 70 years of 1.31 and 1.46 respectively were rounded up to be 2 and 3 instead of rounding down to 2 and 2. In addition, coefficients of other variables that were less than 1 (e.g., 0.2-0.6) were rounded up to be equal to a score of 1 and thus given an equal weight of contribution to the total scores. This would raise the question, for example, of whether gender played the same role as hypertension. The clinical prediction scores for diabetes have also been used for predicting CKD[6]. As expected, the discrimination was low, ranging from 0.60 to 0.71, and generalizability was questionable.
Using the clinical prediction score in practice
Using only one cut-off (e.g., according to Yuden's index (sensitivity + specificity - 1)[21, 22]) classifies subjects too broadly and thus does not work for this prediction score. For instance, applying Yuden's index resulted in a cut-off of 5 which provided the highest sensitivity and specificity (i.e., 76% and 69%, respectively). This would also result in a very large screening of serum creatinine across the country if a suggestion were based on this cutoff. In a country with limited resources, scoring should be prioritized with meaningful and clinical relevance. The prediction scores are thus classified into four groups according to the LR+ which are: low (4-5); intermediate-low (6-8); intermediate-high (9-11); and high (≥12). The variables used are easily obtained and measurable in general practice; hence, a general practitioner, an internist, or even a nephrologist should be able to manually apply prediction scores in routine bedside-practice. Interpreting LR+ is more informative using a nomogram, which is widely used in diagnostic test results [23]. For instance, in data from a medical record or physical examination, if a subject is younger than 40 years with high blood pressure, no history of diabetes and history of kidney stones, and normal blood sugar, this would give the patient a score of 5, corresponding to a LR+ of 1. Assuming a baseline prevalence (pre-test probability) of CKD ranging from 17.5% (95% CI: 14.6%- 20.2%)[10], the post-test probability of this person is ~17.5% (95% CI: 15%-20%). This patient is in a low risk group and thus can be checked for kidney function once a year, if and only if he/she does not develop other risks, e.g., diabetes or history of kidney stones. If the patient later develops diabetes and/or kidney stones, this would give him/her a score of 6-8, corresponding to a LR+ of 2.5 and the post-test odds would be 0.530 (i.e., a pre-test odds × LR+ = 0.175/(1-0.175) × 2.5). This resulted in the post-test probability of ~35% (95% CI: 29%-40%, which could be estimated by post-test odds/(1+post-test odds). Monitoring kidney function by measuring both serum creatinine and urine albumin should be more frequent than a score of 5, say twice a year. Subject ages < 40 to 59 years having all 3 risks would have scores of 9-11, giving a LR+ of 4.9 which would result in a post-test probability of ~50% (95% CI: 45%-55%). If he/she is getting older, say age ≥ 60 years, the score is 12 with LR+ of ~7, resulting in a post-test probability of ~60% (95% CI: 55%-65%). These two scores give intermediate-high and high probabilities of having CKD, and thus indicating more frequent follow-ups, say 3-4 times a year. Treatments for diabetes and hypertension should be also intensified according to guideline of treatments to control disease conditions. As a result, risk of developing CKD is lowering or once it occurs, delay progression of CKD will be targeted.
