Skip to main content

Advertisement

We’d like to understand how you use our websites in order to improve them. Register your interest.

Problems with analyses and interpretation of data in “use of the KDQOL-36™ for assessment of health-related quality of life among dialysis patients in the United States”

Abstract

A recent article in the journal reported analyses of KDQOL-36™ survey data collected from 240,343 adults (330,412 surveys) dialyzed at a large dialysis organization in the United States during 2014–2016. The authors concluded that the KDQOL-36™ Symptoms and Problems of Kidney Disease scale had the highest mean score of the KDQOL-36™ scales. We note that this inference was erroneous because the scales are not scored on the same numeric scale. In addition, the authors found that responses to a general health perceptions item (“In general, would you say your health is excellent, very good, good, fair, or poor”) was not significantly associated with any of the 5 KDQOL-36 scale scores. In contrast, we find significant and noteworthy correlations in two other datasets. These analytic issues call into question the accuracy and validity of the conclusions of this paper.

Peer Review reports

Several recent studies have provided new and more extensive evidence of the psychometric soundness of the Kidney Disease Quality of Life (KDQOL™)-36 instrument including further support of its reliability and validity [1, 2] beyond what was reported in the original peer reviewed manuscript [3, 4]. In addition, the KDQOL has been shown to be predictive of healthcare utilization and mortality [5], outcomes that are important to patients who suffer from this end stage kidney disease (ESKD), their families, and health care providers who treat ESKD.

Cohen et al. analyzed KDQOL-36™ survey data collected from 240,343 adults (330,412 surveys) dialyzed at a large dialysis organization in the United States during 2014–2016. The authors reported the following mean scores on the 5 KDQOL-36 scales: Short-form (SF)-12 Physical Component Summary (PCS): 36.6; SF-12 Mental Component Summary (MCS): 49.0; Burden of Kidney Disease (BKD): 51.3; Symptoms and Problems of Kidney Disease (SPKD): 78.1; and Effects of Kidney Disease (EKD): 73.0. They concluded that the SPKD scale “had the highest mean score (78.1) of the 5 subscales on the KDQOL-36™” [6] (p. 4) and suggested that it exceeded the SF-12 PCS mean score by approximately 40 points. “Thus, the two scores convey very different messages about patient health: a PCS score in the 30’s is suggestive of extremely poor overall health, whereas an SPKD score of 70 or higher suggests a relatively low symptom burden. This pattern is suggestive, although not proof positive, that the SPKD subscale may be topped out” [6] (p. 7).

Cohen et al. [6] do not account for the fact that the SF-12 PCS and MCS are not on the same numeric scale as the BKD, SPKD, and EKD. The SF-12 PCS and MCS are scored on a T-score metric, which has a mean of 50 and standard deviation of 10 in the U.S. general population, while the kidney-targeted scales are scored on a 0–100 possible range and have variable means and standard deviations. Therefore, scores on the SF-12 scales and the kidney-targeted scales are not directly comparable to justify their conclusion. A PCS score of 36.6 does indicate a physical health related quality of life score nearly a standard deviation and a half below the US general population, but it cannot be directly compared with the SPKD scale that has a 0–100 possible range.

In fact, each of the means reported by Cohen et al. [6] are very similar to those recently reported for the KDQOL-36 United States dialysis population [7], as shown in Table 1 below. To aid in score interpretation, users of KDQOL instruments are encouraged to refer to [7] and the publicly available and free scoring guide: https://www.rand.org/health-care/surveys_tools/kdqol.html.

Table 1 Mean KDQOL-36 Scale Scores in Cohen et al. [6] and Peipert et al. [7]

Consistent use of norm-based scoring in the future is preferred because it would help make the direct comparisons desired in Cohen et al.’s [6] paper and other, similar applications. It facilitates comparisons because the referent norm (e.g., general population) is built into the scoring algorithm. For example, T-scores above 50 are better than, and those below 50 are worse than, a referent population for measures scored in a positive direction (higher score is better). Furthermore, since the standard deviation for each scale is standardized to be at 10, it is easy to see exactly how far above or below a score is from the norm in standard deviation units.

One of the questions in the KDQOL-36 is the often-used general health rating item [8]. Cohen et al. [6] stated that the response to that item (“In general, would you say your health is”) “was not correlated with any of the 5 subscale scores” [6] (p. 7). These results are improbable and inconsistent with prior research. For example, correlations based on the Medical Education Institute dataset [7] and another dataset (Peipert JD, Caicedo JC, Friedewald JJ, et al: Trends and predictors of multidimensional health-related quality of life after living donor kidney transplantation, submitted) that included 506 patients who completed the KDQOL-36 at the time of evaluation for transplant (before transplant surgery) show highly significant and noteworthy associations. Table 2 shows product-moment correlations (Spearman rank-order correlations were similar).

Table 2 Product-moment correlations of the general health item with the KDQOL-36 scales

It is possible that there was an error in Cohen et al.’s [6] scoring or analysis of the KDQOL-36. However, these suspect correlations and the misinterpretation of KDQOL-36 scale scores noted above call into question the validity of all the analyses and conclusions reached in this manuscript. While errors such as these are sometimes made by researchers unfamiliar with the instrument being employed, the field relies on peer review to discover these flaws prior to publication.

Finally, Cohen et al. suggested that “new or revised HRQOL assessment tools may be designed to addressed those factors that are most important to dialysis patients” and that “improved instruments may in turn provide a more robust foundation to guide interventions aimed at improving HRQOL in patients with ESRD” [6] (p. 8). Previous published articles provide concrete ways to improve the KDQOL-36. For example, Peipert and colleagues [9,10,11] suggested replacing the SF-12 PCS and MCS with the Patient-Reported Outcomes Measurement Information System (PROMIS®) measures [12,13,14].

We agree that improvements to patient-reported outcome measures like the KDQOL-36 are worthwhile. These efforts often occur iteratively, across multiple analyses of diverse datasets like the one analyzed by Cohen, et al. [6]. However, recommendations for improvement to the KDQOL-36 need to be based on accurate and appropriate statistical analyses and interpretation of the scores. The KDQOL Working Group (http://www.kidney.org/sites/default/files/docs/cnsw_webinar_kdqol-36_final_1.pdf) is available to analyze datasets and work with those with access to datasets to ensure the accuracy of results and validity of the interpretation of the scale scores. A third-party analysis of KDQOL data held by private organizations provides a counterbalance to the perceived or actual financial conflict of interest and lack of transparency: “The datasets generated and/or analyzed during the current study are not publicly available due fact (sic) that they are derived from the proprietary database of a large dialysis organization” [6] (p. 8). Working with researchers from academia with high levels of training and experience in patient-reported outcomes research and psychometrics should be embraced to both raise the quality of all research work and assist in the training of future professionals in the field. Such collaborations between industry and academia are often fruitful and stand to make a significant, positive impact on ESKD patients’ lives.

Availability of data and materials

Requests for the datasets analyzed in the current study can be made to the authors except by those who restrict access to their data.

Abbreviations

BKD:

Burden of kidney disease

KDQOL™:

Kidney Disease Quality of Life

MCS:

Mental Component Summary

PCS:

Physical Component Summary

PROMIS®:

Patient-Reported Outcomes Measurement Information System

SPKD:

Symptoms/problems of kidney disease

References

  1. 1.

    Peipert JD, Bentler PM, Klicko K, Hays RD. Psychometric properties of the kidney disease quality of life 36-item short-form survey (KDQOL™-36) in the United States. Am J Kidney Dis. 2018;71(4):461–8.

    Article  Google Scholar 

  2. 2.

    Peipert JD, Bentler P, Klicko K, Hays RD. Neglible impact of differential item functioning between black and white dialysis patients on the kidney disease quality of life 36-item short-form survey (KDQOL™-36). Qual Life Res. 2018;27(10):2699–707.

    Article  Google Scholar 

  3. 3.

    Hays RD, Kallich JD, Mapes DL, Coons SL, Carter WB. Development of the kidney disease quality of life (KDQOL™) instrument. Qual Life Res. 1994;3:329–38.

    CAS  Article  Google Scholar 

  4. 4.

    Rao S, Carter WB, Mapes DL, Kallich JD, Kamberg CJ, Spritzer KL, Hays RD. Development of subscales from the symptom/problems and effects of kidney-disease items in the kidney disease quality of life (KDQOL™) instrument. Clin Ther. 2000;22:1099–111.

    CAS  Article  Google Scholar 

  5. 5.

    Mapes DL, Lopes AA, Satayathum S, McCullough KP, Goodkin DA, Locatelli F, et al. Health-related quality of life as a predictor of mortality and hospitalization: the Dialysis outcomes and practice patterns study (DOPPS). Kidney Int. 2003;64:339–49.

    Article  Google Scholar 

  6. 6.

    Cohen DE, Lee A, Sibbel S, Benner D, Brunelli SM, Tentori F. Use of the KDQOL-36™ for assessment of health-related quality of life among dialysis patients in the United States. BMC Nephrol. 2019;20(1):112.

    Article  Google Scholar 

  7. 7.

    Peipert J, Nair D, Klicko K, Schatell D, Hays RD. Kidney disease quality of life 36-item short form survey (KDQOL-36™) normative values for the United States dialysis population and new single summary score. J Am Soc Nephrol. 2019;30(4):654–63.

    Article  Google Scholar 

  8. 8.

    Hays RD, Spritzer KL, Thompson WW, Cella D. U.S. general population estimate for “excellent” to “poor” self-rated health item. J Gen Intern Med. 2015;30(10):1511–6.

    Article  Google Scholar 

  9. 9.

    Peipert JD, Hays RD. Methodological considerations in using patient reported measures in dialysis clinics. JPRO. 2017;1:11.

    PubMed  Google Scholar 

  10. 10.

    Peipert JD, Hays RD. Using patient reported measures in dialysis clinics. Clin J Am Soc Nephrol. 2017;12:1889–91.

    Article  Google Scholar 

  11. 11.

    Peipert JD, Hays RD. Expanding the patient’s voice in nephrology with patient-reported outcomes. J Am Soc Nephrol. 2019;30:526–32.

    Article  Google Scholar 

  12. 12.

    Cella D, Choi SW, Condon DM, Schalet B, Hays RD, Rothrock NE, et al. PROMIS® adult health profiles: efficient short-form measures of seven health domains. Value Health. 2019;22(5):537–44.

    Article  Google Scholar 

  13. 13.

    Hays RD, Spritzer KL, Schalet BD, Cella D. PROMS®-29 v2.0 profile physical and mental health summary scores. Qual Life Res. 2018;27:1885–91.

    Article  Google Scholar 

  14. 14.

    DeWitt B, Feeny D, Fischoff B, Cella D, Hays RD, Hess R, et al. Estimation of a preference-based summary score for the patient-reported outcomes measurement information system: the PROMIS®-preference (PROPr) scoring system. Med Decis Mak. 2018;38(6):683–98.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable

Funding

RDH received support from the University of California, Los Angeles (UCLA), Resource Centers for Minority Aging Research Center for Health Improvement of Minority Elderly (RCMAR/CHIME) under NIH/NIA Grant P30-AG021684.

Author information

Affiliations

Authors

Contributions

RDH drafted the article, and JDP and JDK provided edits. JDP analyzed the two datasets. All authors read and approved the final manuscript.”

Corresponding author

Correspondence to Ron D. Hays.

Ethics declarations

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hays, R.D., Peipert, J.D. & Kallich, J.D. Problems with analyses and interpretation of data in “use of the KDQOL-36™ for assessment of health-related quality of life among dialysis patients in the United States”. BMC Nephrol 20, 447 (2019). https://doi.org/10.1186/s12882-019-1609-2

Download citation

Keywords

  • Renal disease
  • Health-related quality of life, KDQOL-36™