Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Identification of volume overload hospitalizations among hemodialysis patients using administrative claims: a validation study

  • Magdalene M. Assimon1, 2,
  • Thuy Nguyen3,
  • Suzanne L. Katsanos1,
  • Steven M. Brunelli4 and
  • Jennifer E. Flythe1, 5Email author
BMC NephrologyBMC series – open, inclusive and trusted201617:173

https://doi.org/10.1186/s12882-016-0384-6

Received: 21 July 2016

Accepted: 30 October 2016

Published: 11 November 2016

Abstract

Background

High rates of volume overload hospitalizations may indicate inadequate dialysis facility fluid management. Administrative claims databases are often used to study such outcomes, but these data are generated for billing purposes and may not capture clinical nuance. It is unknown if volume overload admissions can be correctly identified in administrative data and if a single claims-based definition for volume overload can be used across epidemiologic surveillance studies, observational studies of exposure-outcome associations and quality assessments. We conducted a validation study to assess the accuracy of claims-based definitions for volume overload hospitalizations among hemodialysis patients.

Methods

Data were taken from a random sample of 315 adult hemodialysis patients admitted to University of North Carolina Hospitals from January 2010 through June 2013. Standardized chart reviews were conducted to clinically adjudicate the presence or absence of volume overload at hospital admission. Claims-based definitions were constructed from varying combinations of fluid-related ICD-9 discharge diagnosis codes including fluid overload, pulmonary edema, pleural effusion, and heart failure. Using clinically adjudicated volume overload hospitalizations as the reference standard, validity metrics and their 95 % confidence intervals (CIs) were estimated for each definition.

Results

Of the 315 hospital admissions, 77 (24.4 %) were clinically adjudicated as volume overload hospitalizations. The prevalence of claims-identified volume overload admissions varied across definitions, ranging from 1.6 to 37.1 %. When definitions were constructed with discharge diagnosis codes present in any billing position, volume overload hospitalizations defined by fluid overload, pleural effusion or heart failure diagnosis codes had the highest sensitivity, 81.8 % (95 % CI: 71.4 %, 89.7 %). Volume overload hospitalizations defined by pulmonary edema diagnosis codes had the highest specificity, 98.3 % (95 % CI: 95.8 %, 99.5 %). Definitions constructed with discharge diagnosis codes present in any billing position (versus the primary position) captured more false positive events.

Conclusions

Prevalence and validity estimates of volume overload hospitalizations vary across claims-based definitions. A universal claims-based definition for volume overload hospitalizations may not apply to all clinical and research scenarios. Investigators and regulators need to consider the implications of misclassifying events when evaluating and monitoring hemodialysis patient volume overload admissions with administrative data. Claims-based definitions should be selected accordingly.

Keywords

Administrative claims Hemodialysis Hospitalization ICD-9 Volume overload

Background

The over 400,000 individuals receiving hemodialysis in the United States (U.S.) have exceedingly high rates of cardiovascular morbidity and mortality, with 30 % of hospitalizations and nearly 50 % of deaths attributed to cardiovascular causes [1]. Care of this complex population is expensive. In 2011, persons with end-stage kidney disease represented just 1.4 % of Medicare enrollees but consumed 6.3 % of the total Medicare budget [2]. Inadequate volume control is associated with both adverse cardiovascular outcomes and substantial healthcare costs among hemodialysis patients [35]. Volume-related hospital admissions are a significant driver of the cardiovascular hospitalization rate in the hemodialysis population, and estimated annual costs related to these encounters total over $250 million [1, 6].

Some volume overload hospitalizations may be preventable with better dialysis facility fluid management practices. For example, close attention to prescribed target (“estimated dry”) weight achievement at the end of each dialysis treatment as well as delivery of effective dietary salt and fluid restriction counseling by dialysis unit personnel may prevent some volume-related complications [4, 7, 8]. Tracking volume overload hospitalizations represents one potential strategy to measure and assess dialysis facility fluid management practices. The Medicare-based United States Renal Data System (USRDS), a national registry of end-stage kidney disease patients, is a readily available and cost effective data source often used to monitor and study cause-specific hospitalizations in the U.S. hemodialysis population.

Administrative claims data, such as that housed in the USRDS, are primarily generated for reimbursement and billing purposes. These data may not always capture clinical subtleties, potentially affecting the accuracy of claims-identified, cause-specific hospital admissions. For example, general population validation studies suggest that ~25 % of true heart failure hospitalizations are not captured by administrative claims data [9]. Prior evaluations of volume overload hospitalizations among hemodialysis patients were performed using USRDS data, each relying upon distinct combinations of discharge diagnosis and/or procedure codes to define events [6, 10, 11]. However, the validity of these claims-based definitions is unknown. In the medically complex hemodialysis population, restrictions on the number of diagnosis and procedure codes that can be billed per inpatient encounter, among other factors, may influence the ability of investigators to accurately identify cause-specific hospitalizations in administrative data. As such, when choosing claims-based volume overload definitions for observational studies, investigators must consider the implications of outcome misclassification and appropriately prioritize validity metrics (e.g. sensitivity and specificity) to optimize study accuracy. Study objectives and corresponding study design should guide claims-based outcome definition selection.

We undertook this study to evaluate the validity of several claims-based definitions for volume overload hospital admissions in the hemodialysis population using rigorous medical record reviews and medical center billing data.

Methods

Study population

This study included a random sample of 315 unique, adult maintenance hemodialysis patients admitted to University of North Carolina (UNC) Hospitals (Chapel Hill, NC) between January 1, 2010 and June 30, 2013. UNC Hospitals is a public academic medical center with over 800 inpatient beds and over 35,000 annual discharges. The study cohort consisted of hemodialysis patients who were: 1) ≥18 years of age and 2) admitted to medical or surgical services. We excluded patients who were: 1) receiving home hemodialysis or peritoneal dialysis or 2) newly designated as end-stage renal disease during the sampled admission. We selected 2010 as the study start year in order to exclude hospital admissions occurring before the Medicare policy change expanding the maximum number of billable discharge diagnoses per inpatient claim from 9 to 25 [12].

We performed a priori sample size calculations. Assuming a 20 % prevalence of volume overload admissions [1], a sample size of 298 patients would be needed to estimate a minimum specificity of 70 % with an acceptable lower 95 % confidence interval (CI) of at least 60 % [13]. This study was approved by the UNC Chapel Hill Institutional Review Board.

Data sources

Overview

Data were obtained from the Carolina Data Warehouse for Health (CDW-H), a central data repository containing administrative healthcare data sourced from the UNC Health Care System. Detailed clinical data are not captured in this database and were abstracted from the electronic medical record by three clinicians (M.M.A., T.N. and S.L.K.).

Clinical adjudication of volume overload hospital admissions

For each sampled hospitalization, we conducted detailed clinical chart reviews to adjudicate the presence or absence of volume overload at the time of admission. We sought to identify hospitalizations of patients admitted with volume overload. Unlike other cardiovascular conditions, such as myocardial infarction, there is no established, objective definition for the clinical diagnosis of volume overload. Volume overloaded hemodialysis patients often present with a constellation of signs and symptoms indicative of fluid retention (e.g. shortness of breath, rales on lung auscultation, pulmonary edema on chest imaging, etc.) that may vary from individual to individual. Thus, an in-depth review of the medical record was necessary to capture all volume-related clinical findings associated with each sampled hospitalization. Medical chart notes (e.g. emergency department, admitting team and consult notes), chest and abdominal imaging reports, and cardiac procedure reports occurring within 24 h of admission were evaluated. Abstractors utilized a standardized data collection form (Additional file 1) to record symptoms, physical exam findings, imaging results, and clinical impressions. Each medical record was independently abstracted by two clinical reviewers who were blinded to the hospitalization’s billed diagnosis and procedure codes and to the abstraction results of the other reviewer. Inter-abstractor discrepancies in individual data elements were resolved by a board-certified nephrologist (J.E.F) Initial agreement between abstractors was high across all data elements, ranging 98.1 % (κ = 0.91) for subjective dyspnea to 100 % (κ = 1.00) for central venous pressure. After review and error resolution, consensus was reached on all abstracted charts.

We created a standardized diagnostic algorithm to determine the presence or absence of volume overload at the time of admission based on the American College of Cardiology Foundation/American Heart Association and European Society of Cardiology guidelines [14, 15], and input from local nephrologists and cardiologists (Fig. 1). Diagnostic algorithm pre-testing revealed that a clinical criteria-based algorithm (e.g. symptoms, physical exam findings, and imaging results) failed to capture emergent presentations of volume overload requiring immediate treatment. In severe cases, imaging was not always performed prior to treatment (e.g. ultrafiltration). To capture such events, we expanded the algorithm to include clinical criteria or physician impression of volume overload. Furthermore, our diagnostic algorithm was developed to capture a range of volume overload severities. By design, our clinical definition does not distinguish patients admitted for the indication of volume overload from patients admitted with volume overload. Both scenarios were identified by our algorithm as clinically adjudicated volume overload events.
Fig. 1

Reference standard criteria for adjudication of volume overload hospital admissions. Documented clinical assessment and physician impression criteria occurring within 24 h of admission were considered. Hospitalization events were adjudicated as volume overload admissions if either the clinical assessment criteria or physician assessment criteria for volume overload were met. Hospitalization events were adjudicated as non-volume overload admissions if they did not meet the physician impression criteria and did not meet the clinical criteria for volume overload

We applied the diagnostic algorithm to the abstracted data. Hospitalizations were adjudicated as volume overload admissions if either the clinical assessment criteria or the physician impression criteria for volume overload were met. Otherwise, hospitalizations were adjudicated as non-volume overload admissions. Agreement between volume overload admissions identified by clinical assessment criteria and admissions identified by physician impression criteria was high, 91.4 % (κ = 0.73).

Administrative claims-based definitions for volume overload hospital admissions

We obtained administrative data including demographics, billed hospital discharge diagnoses and procedure codes from the CDW-H for each sampled admission. We evaluated a range of administrative claims definitions for hospitalized volume overload. Definitions were constructed based upon literature precedent using various combinations of fluid overload, pulmonary edema, pleural effusion and heart failure International Classification of Diseases, Ninth Revision (ICD-9) discharge diagnosis codes (Table 1 and Additional file 2: Table S1) [6, 10, 11]. Primary validation analyses considered discharge diagnosis codes present in any position. Secondary analyses considered discharge diagnosis codes present in the: 1) primary billing position only and 2) primary or leading secondary billing position (separately). In additional secondary analyses, we evaluated the validity of claims-based volume overload definitions that included both fluid-related discharge diagnosis codes and the presence of a dialysis Current Procedural Terminology (CPT) procedure codes billed on the day of admission or the following day (Additional file 2: Table S2) [6]. The Medicare requirement of attending presence during in-hospital dialysis treatments for the billing of dialysis CPT codes may lead to inaccurate estimates of inpatient dialysis procedures in administrative data sources [16]. In our cohort, of the 59 clinically adjudicated cases who received dialysis within 24 h of admission (per medical chart documentation), only 39 (66.1 %) had a corresponding billed dialysis CPT procedure code. To avoid misclassification resulting from non-billed in-hospital dialysis treatments, validation analyses of claims-based definitions containing of discharge diagnosis codes were considered secondary, and results are presented in the supplemental material.
Table 1

Administrative claims-based definitions for volume overload hospital admissions

Definition number and description

Lead author (year of publication)a

ICD-9 discharge diagnosis codesb

1. Fluid overloadc

Banerjee (2007) [10]

276.6, 276.69

2. Pulmonary edema

--

514, 518.4

3. Heart failure

--

398.91, 402.x1, 404.x1, 404.x3, 428d

4. Fluid overloadc or pulmonary edema

--

276.6, 276.69, 514, 518.4

5. Fluid overloadc or pleural effusion

Weinhandl (2015) [11]

276.6, 276.69, 511.9

6. Fluid overloadc or heart failure

--

276.6, 276.69, 398.91, 402.x1, 404.x1, 404.x3, 428d

7. Fluid overloadc, pulmonary edema or heart failure

Arneson (2010) [6]

276.6, 276.69, 514, 518.4, 402.x1, 404.x1, 404.x3, 428d

Abbreviations: ICD-9 International Classification of Diseases, Ninth Revision

aDenotes prior use of the ICD-9 diagnosis code combination to define volume overload hospitalizations

bSeparate analyses evaluating definition validity considered ICD-9 diagnosis codes present in: 1) any billing order position, 2) the primary billing position only and 3) the primary and leading secondary billing positions

cPrior to October 1, 2010 the ICD-9 discharge diagnosis code 276.6 (fluid overload) was the only applicable code in existence. On October 1, 2010, ICD-9 diagnosis code 276.6 (fluid overload) became invalid and was replaced by more granular codes: 276.61 (transfusion associated circulatory overload) and 276.69 (other fluid overload). For hospitalizations with a discharge date prior to October 1, 2010 the ICD-9 code 276.6 was used to construct claims-based volume overload definitions. For hospitalizations with a discharge date on or after October 1, 2010 the ICD-9 code 276.69 was used to construct claims-based volume overload definitions

dSpecified three digit ICD-9 diagnosis categories included all existing 4th and 5th digit diagnosis codes

Statistical analyses

Analyses were performed using SAS version 9.3 (SAS Institute, Cary, NC). Data are presented as means and standard deviations or medians and interquartile ranges for continuous variables, and as frequencies and percentages for categorical variables. We computed the prevalence of clinically-adjudicated volume overload admissions and claims-identified volume overload admissions in the study cohort. Using clinically adjudicated volume overload hospitalization as the reference standard, we computed sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) and their exact binomial 95 % CIs for each claims-based volume overload definition (Fig. 2).
Fig. 2

Computation and interpretation of validity measures

We conducted sensitivity analyses to assess the robustness of our findings. On October 1, 2010, the ICD-9 diagnosis code for fluid overload, 276.6, became invalid and was replaced by more granular codes: 276.61 (transfusion associated circulatory overload) and 276.69 (other fluid overload). Thus, we repeated validity assessments in a cohort restricted to patients admitted on or after October 1, 2010. Second, since most administrative claims analyses of U.S. hemodialysis patients are conducted in Medicare-based databases, we repeated validity assessments in a cohort restricted to patients with Medicare as their primary insurer.

Results

Cohort characteristics

Study cohort characteristics are displayed in Table 2. In the 315 patient cohort, the mean age was 57 ± 14 years, 172 (54.6 %) were male, 195 (61.9 %) were black, 119 (37.8 %) had diabetes and 143 (45.4 %) had heart failure. The study cohort was similar to the U.S hemodialysis population in terms of age and sex. Consistent with regional demographics, the study cohort had a higher proportion of black patients compared to a broader, national cohort [1]. The median (quartile 1 – quartile 3) length of admission was 4 (2–8) days. Of the 315 sampled admissions, 77 (24.4 %) were clinically adjudicated as volume overload hospitalizations. Compared to patients without volume overload at admission, patients with volume overload were more likely to have a history of hypertension, coronary artery disease and heart failure.
Table 2

Study cohort characteristics

Characteristics

All N = 315a

Volume overloaded at admissionb n = 77 (24.4 %)

Not volume overloaded at admissionb n = 238 (75.6 %)

Age (years)

57 ± 14

57 ± 16

57 ± 14

Female

143 (45.4)

37 (48.1)

106 (44.5)

Race

 Black

195 (61.9)

45 (58.4)

150 (63.0)

 White

82 (26.0)

22 (28.6)

60 (25.2)

 Other

38 (12.1)

10 (13.0)

28 (11.8)

Medicare as primary payer

266 (84.4)

62 (80.5)

204 (85.7)

History of diabetesc

119 (37.8)

30 (39.0)

89 (37.4)

History of hypertensionc

217 (68.9)

64 (83.1)

153 (64.3)

History of arterial diseasec

150 (47.6)

43 (55.8)

107 (45.0)

History of heart failurec

143 (45.4)

60 (77.9)

83 (34.9)

Length of hospital stay (days)d

4 (2–8)

3 (2–6)

4 (2–8)

Admitting service

 Medicine

257 (81.6)

74 (96.1)

183 (76.9)

 Surgery

58 (18.4)

3 (3.9)

55 (23.1)

# of billed ICD-9 discharge diagnosis codes

14 (11–19)

16 (12–21)

13 (10–18)

Dialysis CPT procedure code billed on day of admission or the following daye

150 (48.5) [N = 309]

42 (56.0) [n = 75]

108 (46.2) [n = 234]

Recent TTE ejection fractionf

  < 35 %

4 (6.1)

3 (21.4)

1 (1.9)

 35–54 %

30 (45.5)

4 (28.6)

27 (50.0)

  ≥ 55 %

32 (48.5) [N = 66]

7 (50.0) [n = 14]

25 (48.1) [n = 52]

Values presented as mean ± standard deviation, median (quartile 1 – quartile 3) or n (%)

Abbreviations: CPT Current Procedural Terminology, ICD-9 International Classification of Diseases, Ninth Revision, TTE transthoracic echocardiogram

aExcept where noted

bVolume overload status at hospital admission based on clinical adjudication

cComorbid conditions were captured using all available administrative data occurring before the sampled hospitalization. ICD-9 codes for: diabetes included 250.xx; hypertension included 401.xx-405.xx (except 402.11), 402.91, 404.11, 404.13, 404.91, 404.93; arterial disease included 410.xx, 414.0x, 429.2x, 429.5x, 429.7x, 440.x, 440.2x, 440.3x, 440.8x, 440.9x, 443.9x; and heart failure included 398.91, 402.x1, 404.x1, 404.x3, 428.xx

dLength of hospital stay was computed as the discharge date minus the admission date. Patients admitted and discharged on the same day were assigned a length of stay = 0.5 days

eSix patients were admitted and discharged on the same day and were excluded from this computation because we were unable to determine if they received dialysis on the day following admission using inpatient administrative claims data. Dialysis CPT procedure codes used to identify inpatient dialysis treatments include: 90935, 90937, 90945 and 90947

fTTE conducted ≤1 year before sampled hospital admission

Prevalence of volume overload admissions identified by administrative claims

The prevalence of volume overload hospitalizations differed across administrative claims-based definitions (Fig. 3). In primary analyses, when definitions were constructed considering diagnosis codes in any billing position, volume overload admission prevalence ranged from 4.1 % (definition 2, pulmonary edema) to 37.1 % (definition 7, fluid overload, pulmonary edema or heart failure) (Table 3). Definitions containing heart failure diagnosis codes (definitions 3, 6, and 7) overestimated volume overload admission prevalence, whereas definitions without heart failure codes (definitions 1, 2, 4, and 5) underestimated prevalence. Narrower definitions (i.e. those constructed with diagnosis codes billed in the primary position or in the primary or leading secondary positions) grossly underestimated volume overload admission prevalence. All claims-based definitions for volume overload hospitalizations comprised of discharge diagnosis and dialysis procedure codes underestimated volume overload prevalence (Additional file 2: Figure S1).
Fig. 3

Prevalence of volume overload admissions identified by administrative claims definitions. The claims-based volume overload definition numbers on the x-axis correspond to the following definitions: 1) fluid overload; 2) pulmonary edema; 3) heart failure; 4) fluid overload or pulmonary edema; 5) fluid overload or pleural effusion, 6) fluid overload or heart failure; and 7) fluid overload, pulmonary edema or heart failure. The total bar height (light gray portion + dark gray portion) represents the prevalence of volume overload admissions in the study cohort identified using the specified claims-based definitions. Panel a depicts prevalence estimates when claims-based definitions were constructed considering ICD-9 discharge diagnosis codes in any position. Panel b depicts prevalence estimates when claims-based definitions were constructed considering discharge diagnosis codes in the primary position only. Panel c depicts prevalence estimates when claims-based definitions were constructed considering discharge diagnosis codes in the primary or leading secondary position. Abbreviations: ICD-9, International Classification of Diseases, Ninth Revision

Table 3

Validity of administrative claims definitions for volume overload hospital admissions

Claims-based definition

n (%)a

SENS (95 % CI)b

SPEC (95 % CI)b

PPV (95 % CI)b

NPV (95 % CI)b

ICD-9 discharge codes could be in any position

 1. Fluid overload

30 (9.5)

32.5 (22.2, 44.1)

97.9 (95.2, 99.3)

83.3 (65.3, 94.4)

81.8 (76.8, 86.1)

 2. Pulmonary edema

13 (4.1)

11.7 (5.5, 21.0)

98.3 (95.8, 99.5)

69.2 (38.6, 90.9)

77.5 (72.3, 82.1)

 3. Heart failure

87 (27.6)

51.9 (40.3, 63.5)

80.3 (74.6, 85.1)

46.0 (35.2, 57.0)

83.8 (78.3, 88.3)

 4. Fluid overload or pulmonary edema

39 (12.4)

40.3 (29.2, 52.1)

96.6 (93.5, 98.5)

79.5 (63.5, 90.7)

83.3 (78.4, 87.5)

 5. Fluid overload or pleural effusion

39 (12.4)

41.6 (30.4, 53.4)

97.1 (94.0, 98.8)

82.1 (66.5, 92.5)

83.7 (78.8, 87.9)

 6. Fluid overload or heart failure

111 (35.2)

76.6 (65.6, 85.5)

78.2 (72.4, 83.2)

53.2 (43.4, 62.7)

91.2 (86.4, 94.7)

 7. Fluid overload, pulmonary edema or heart failure

117 (37.1)

81.8 (71.4, 89.7)

77.3 (71.5, 82.5)

53.8 (44.4, 63.1)

92.9 (88.4, 96.1)

ICD-9 discharge codes could be in primary position only

 1. Fluid overload

5 (1.6)

6.5 (2.1, 14.5)

100.0 (98.5, 100.0)

100.0 (47.8, 100.0)

76.8 (71.7, 81.4)

 2. Pulmonary edema

5 (1.6)

6.5 (2.1, 14.5)

100.0 (98.5, 100.0)

100.0 (47.8, 100.0)

76.8 (71.7, 81.4)

 3. Heart failure

10 (3.2)

11.7 (5.5, 21.0)

99.6 (97.7, 100.0)

90.0 (55.5, 99.7)

77.7 (72.6, 82.3)

 4. Fluid overload or pulmonary edema

10 (3.2)

13.0 (6.4, 22.6)

100.0 (98.5, 100.0)

100.0 (69.2, 100.0)

78.0 (73.0, 82.6)

 5. Fluid overload or pleural effusion

6 (1.9)

6.5 (2.1, 14.5)

99.6 (97.7, 100.0)

83.3 (35.9, 99.6)

76.7 (71.6, 81.3)

 6. Fluid overload or heart failure

15 (4.8)

18.2 (10.3, 28.6)

99.6 (97.7, 100.0)

93.3 (68.1, 99.8)

79.0 (73.9, 83.5)

 7. Fluid overload, pulmonary edema or heart failure

20 (6.3)

24.7 (15.6, 35.8)

99.6 (97.7, 100.0)

95.0 (75.1, 99.9)

80.3 (75.3, 84.7)

ICD-9 discharge codes could be in primary or leading secondary positions

 1. Fluid overload

5 (1.6)

6.5 (2.1, 14.5)

100.0 (98.5, 100.0)

100.0 (47.8, 100.0)

76.8 (71.7, 81.4)

 2. Pulmonary edema

7 (2.2)

9.1 (3.7, 17.8)

100.0 (98.5, 100.0)

100.0 (59.0, 100.0)

77.3 (72.2, 81.8)

 3. Heart failure

15 (4.8)

16.9 (9.3, 27.1)

99.2 (97.0, 99.9)

86.7 (59.5, 98.3)

78.7 (73.6, 83.2)

 4. Fluid overload or pulmonary edema

12 (3.8)

15.6 (8.3, 25.6)

100.0 (98.5, 100.0)

100.0 (73.5, 100.0)

78.5 (73.5, 83.0)

 5. Fluid overload or pleural effusion

6 (1.9)

6.5 (2.1, 14.5)

99.6 (97.7, 100.0)

83.3 (35.9, 99.6)

76.7 (71.6, 81.3)

 6. Fluid overload or heart failure

20 (6.3)

23.4 (14.5, 34.4)

99.2 (97.0, 99.9)

90.0 (68.3, 98.8)

80.0 (75.0, 84.4)

 7. Fluid overload, pulmonary edema or heart failure

27 (8.6)

32.5 (22.2, 44.1)

99.2 (97.0, 99.9)

92.6 (75.7, 99.1)

81.9 (77.0, 86.2)

Abbreviations: 95 % CI 95 % confidence interval, ICD-9 International Classification of Diseases, Ninth Revision, NPV negative predictive value, PPV positive predictive value, SENS sensitivity, SPEC specificity

aCount (prevalence) of volume overload admissions identified by each administrative claims definition in the study cohort (N = 315)

bValidity estimates and 95 % CIs are expressed as percentages. Clinically adjudicated volume overload events, as outlined in Fig. 1, served as the reference standard. In the study cohort there were 77 adjudicated volume overload admissions

Validity of claims-based definitions for volume overload admissions

Table 3 displays the number of events, sensitivity, specificity, PPV and NPV for diagnosis code-based definitions for volume overload hospitalizations. In primary analyses considering definitions with diagnosis codes in any position, validity estimates varied across claims-based definitions. Sensitivity ranged from 11.7 % (definition 2, pulmonary edema) to 81.8 % (definition 7, fluid overload, pulmonary edema or heart failure). Specificity ranged from 77.3 % (definition 7 fluid overload, pulmonary edema or heart failure) to 98.3 % (definition 2, pulmonary edema). PPV ranged from 46.0 % (definition 3, heart failure) to 83.3 % (definition 1, fluid overload). NPV ranged from 77.5 % (definition 2, pulmonary edema) to 92.9 % (definition 7, fluid overload, pulmonary edema or heart failure). Compared to definitions considering ICD-9 codes in any position, definitions considering diagnosis codes in the primary position only, or in the primary or leading secondary positions had higher specificity and PPV and lower sensitivity and NPV.

Additional file 2: Table S3 displays validity estimates from secondary analyses considering claims-based volume definitions comprised of discharge diagnosis and dialysis procedure codes. In general, specificity and PPV were modestly higher, but sensitivity and NPV were commensurately lower when a dialysis procedure code was added to diagnosis code-based definitions. Validity results from cohorts restricted to patients admitted on or after October 1, 2010 (n = 287) and, separately, to patients with Medicare as the primary insurer (n = 266) were analogous to full cohort results (Additional file 2: Tables S4 and S5).

Discussion

To our knowledge, this is the first study evaluating the accuracy of administrative claims definitions for volume overload hospitalizations in a hemodialysis population. Our study demonstrated that clinically adjudicated volume overload hospitalization prevalence differed from claims-derived prevalence estimates. In general, claims-based definitions had high specificity and low sensitivity. Our data suggest that certain claims-based definitions for volume overload hospitalizations could: 1) generate inaccurate estimates of temporal trends in disease surveillance programs; 2) misestimate the contribution of volume-related admissions to overall hemodialysis population health care utilization costs; or 3) render inaccurate estimates in observational studies seeking to understand how exposures impact rates of volume overload hospitalizations.

Existing data reveal that volume related-factors such as chronic volume expansion, interdialytic weight gain, and ultrafiltration rate contribute to the high hospitalization and mortality rates experienced by hemodialysis patients [3, 5, 1720]. Thus, there is growing interest in identifying, quantifying and monitoring associated outcomes such as volume overload hospitalizations. To detect cause-specific hospitalizations, investigators and regulators typically rely on diagnosis and procedure codes in administrative healthcare databases such as the USRDS. However, administrative healthcare data may be inaccurate or incomplete for a variety of reasons. First, available diagnosis and procedure codes may not accurately identify the clinical condition of interest [21, 22]. Second, medical record documentation, coding and billing practices may vary across healthcare providers or institutions, creating data inconsistencies [23, 24]. Third, only a limited number of discharge diagnosis codes per hospitalization can be billed to insurers, possibly reducing clinical event ascertainment. Fourth, patients could receive treatment at a hospital or clinic without insurance filing, rendering administrative data sources incomplete [22, 23, 25]. While administrative databases are often the most accessible data sources, they may not be the most accurate. Potential data shortcomings must be considered when defining clinical outcomes.

In claims-based studies of hemodialysis patients, investigators have defined volume overload hospitalizations using a variety of fluid-related discharge diagnosis code combinations (e.g. fluid overload, pulmonary edema, pleural effusion, and heart failure) in varying billing positions (Additional file 2: Table S1) [6, 10, 11]. Banerjee et al. defined volume overload hospitalizations as the presence of a fluid overload or pulmonary edema discharge diagnosis code (separately) in any billing position [10]. Others have employed more restrictive definitions. Arneson and colleagues considered several fluid-related diagnosis codes (e.g. fluid overload, pulmonary edema, heart failure) present in the primary billing position only [6]. Whereas Weinhandl et al. defined volume overload hospital admissions as the presence of a fluid overload or pleural effusion discharge diagnosis code in the primary position only, or in the primary or leading secondary positions (separately) [11]. Not surprisingly, we found that broader (versus narrower) definitions identified more true positive volume overload admission events, but did so at the expense of capturing more false positive events. Most notably, we observed that claims-based definitions containing heart failure diagnosis codes (definitions 3, 6 and 7 with codes considered in any position) had the greatest tendency to identify false positive events. This finding may be attributable to the fact that some ICD-9 codes can be used to bill for both chronic stable heart failure and acute heart failure events.

Some investigators have identified volume overload admissions using discharge diagnosis codes in conjunction with dialysis procedure codes. For example, the claims-based definition used by Arneson et al. included fluid-related discharge diagnosis codes and also required the presence of a dialysis procedure code billed on the day of admission or the following day [6]. Inclusion of disease-specific procedure codes often increases definition specificity [23]. As anticipated, when we added dialysis procedure codes to diagnosis code-based definitions, we observed gains in specificity paired with reductions in sensitivity. However, the overall impact on validity estimates was minimal. This finding may, in part, be attributable to a hospital’s tendency to adhere to a patient’s outpatient hemodialysis schedule. Based solely on schedule, regardless of clinical presentation, greater than a third of all patients would be expected to receive dialysis within 24 to 36 h of admission. Furthermore, Medicare billing rules may impact the accuracy of claims-based definitions relying on dialysis procedure codes. Hospitals cannot bill dialysis CPT codes for treatments provided without the physical presence of the attending physician during the dialysis session [16]. In administrative data, this billing rule may lead to underestimation of dialysis procedures in academic environments where trainees supervise emergent overnight or weekend dialysis without in-hospital attending presence and in community hospitals where remote nephrology coverage is common. Thus, to maximize definition stability across clinical practice environments and to avoid outcome misclassification related to billing rules, it may be prudent to omit dialysis procedure CPT codes from claims-based definitions for volume overload hospital admissions.

Dialysis patient clinical complexity may also impact accuracy of claims-defined, cause-specific hospitalizations. A limited number of diagnosis codes can be billed for each hospital encounter. Most often, payers reimburse hospitals for inpatient services based upon billed Medicare Severity Diagnosis Related Groups (MS-DRGs). The patient’s primary (or principal) discharge diagnosis in combination with other factors such as patient sex, discharge status, complications and/or comorbidities documented as secondary discharge diagnoses, medical procedures performed, and length of stay, determine the assigned MS-DRG and corresponding level of reimbursement. Hemodialysis patients often have multiple comorbidities and are treated for numerous clinical conditions during hospitalizations, resulting in a wide range of potential discharge diagnoses from which to choose for coding and billing purposes. Medicare policies allow hospitals to preferentially select discharge diagnosis codes to maximize payment as long as they are supported by adequate medical record documentation [26]. The tendency of heart failure-based definitions to identify false positive volume overload admissions may be explained, in part, by a health system’s preference for coding more resource intensive conditions or comorbidities such as heart failure. Such practices likely vary across healthcare and reimbursement settings.

The ideal claims-based definition for volume overload hospital admissions would have perfect sensitivity (i.e. it would not capture any false negative events) and perfect specificity (i.e. it would not capture any false positive events). As claims-based definitions for clinical event identification are often imperfect, investigators must weigh the advantages and disadvantages of employing more sensitive versus more specific outcome definitions. Study objectives should drive this decision. More sensitive outcome definitions may be preferred in scenarios where enhanced inclusiveness is desired (e.g. epidemiologic surveillance studies) and when generalizability is important (e.g. quality assessment initiatives) [27, 28]. We demonstrated that claims-based volume overload admission definitions with poor sensitivity (<50.0 %) led to systematic underestimation of the volume-related hospital admission burden. This finding suggests that existing national prevalence estimates and temporal trends of volume-related hospitalizations may be conservative. On the other hand, more specific outcome definitions may be favored in observational studies examining exposure–outcome associations via relative effect measures. In the setting of non-differential outcome misclassification, implementation of claims-based volume overload admission outcome definitions with perfect specificity will generate unbiased risk ratio estimates [29]. Consideration of definition PPV and NPV is also important. Positive predictive value, like specificity, is an indicator of false positive event ascertainment, whereas NPV, like sensitivity, is an indicator of false negative event capture. However, unlike specificity and sensitivity, generalizability of PPV and NPV to a population other than the validation cohort depends on the prevalence of the outcome of interest in that population.

Strengths of our study include random selection of hospital admissions enabling estimation of the full spectrum of validity metrics, rigorous data abstraction by two independent reviewers, and utilization of standardized procedures for volume overload admission adjudication. Our study also has limitations. First, we used data from a single academic medical center. Validity estimates may not generalize to administrative data from hospitals with different billing and coding practices. Reassuringly, our study had a similar frequency of billed volume-related discharge diagnosis codes to prior investigations using USRDS data [6, 10, 11]. Second, an established, universal definition for the clinical diagnosis of volume overload does not exist. To address this limitation, we developed a standardized algorithm for clinical adjudication based on guideline body-accepted clinical and radiologic evidence of volume overload [14, 15]. Third, we investigated inpatient volume overload hospital admissions. Our validity estimates may not generalize to other hospital-based encounters such as observation stays or emergency department visits. Given that reimbursement rules and billing mechanisms differ across hospital encounter type, optimal volume overload definitions may vary across inpatient admissions, observation stays and emergency department visits [30, 31]. Future studies should assess the validity of claims-based definitions for volume-related observation and emergency department visits. Fourth, we evaluated inpatient admissions from January 2010 through June 2013. Our validity estimates may not generalize to periods outside of the study timeframe. Our modest sample size prevented evaluation of potential temporal coding trends on claims-based definition validity during the study period. Finally, we studied in-center hemodialysis patients. Results should not be extrapolated to excluded populations such as peritoneal dialysis or home hemodialysis patients or those with non-dialysis dependent chronic kidney disease.

Conclusions

In conclusion, we investigated the validity of administrative claims-based definitions for volume overload hospital admissions in a cohort of maintenance hemodialysis patients. While administrative claims databases are efficient and cost-effective data sources, investigators and regulators must consider the implications of misclassifying volume overload admissions when studying, evaluating and monitoring such events. Our results suggest that a single, universal claims-based definition for volume overload hospitalizations may not be appropriate for all clinical and research scenarios.

Abbreviations

CDW-H: 

Carolina Data Warehouse for Health

CI: 

Confidence interval

CPT: 

Current Procedural Terminology

ICD-9: 

International Classification of Diseases, Ninth Revision

MS-DRG: 

Medicare Severity Diagnosis Related Group

NPV: 

Negative predictive value

PPV: 

Positive predictive value

SENS: 

Sensitivity

SPEC: 

Specificity

U.S.: 

United States

UNC: 

University of North Carolina

USRDS: 

United States Renal Data System

Declarations

Acknowledgments

None.

Funding

This project was supported by a grant from the Renal Research Institute, a subsidiary of Fresenius Medical Care Renal Therapies Group (awarded to J.E.F. and M.M.A). Neither the Renal Research Institute nor Fresenius Medical Care played any role in the design, analysis or reporting of study results. This project was also made possible by the North Carolina Translational and Clinical Sciences Institute funded by National Center for Advancing Translational Sciences, National Institutes of Health grant UL1TR001111. M.M.A. was supported by training grant T32 DK007750 from the National Institute of Diabetes and Digestive and Kidney Diseases of the National Institutes of Health.

Availability of data and material

The data collected and analyzed for this study are available from the corresponding author upon reasonable request.

Authors’ contributions

MMA and JEF conceived the study and its design. MMA, TN, SLK and JEF collected and acquired the study data. MMA preformed the data analyses. All authors participated in manuscript development by interpreting study data and contributing to manuscript drafts and revisions. All authors read and approved the final manuscript.

Authors’ information

Not applicable.

Competing interests

S.M.B. is an employee of DaVita Clinical Research, a subsidiary of DaVita HealthCare Partners, Inc. and his spouse is employed by AstraZeneca. J.E.F. has received speaking honoraria from Dialysis Clinic, Incorporated, Renal Ventures, American Renal Associates, the American Society of Nephrology, and Baxter. M.M.A., T.N., and S.L.K. have no relevant disclosures.

Consent for publication

Not applicable.

Ethics approval and consent to participate

This study was approved by the UNC Chapel Hill Institutional Review Board (#15-1869) and a waiver of consent was granted due to the retrospective nature of the study.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Division of Nephrology and Hypertension, Department of Medicine, University of North Carolina Kidney Center, UNC School of Medicine
(2)
Department of Epidemiology, UNC Gillings School of Global Public Health
(3)
Department of Medicine, UNC School of Medicine
(4)
DaVita Clinical Research
(5)
Cecil G. Sheps Center for Health Services Research, University of North Carolina

References

  1. Saran R, Li Y, Robinson B, Abbott KC, Agodoa LY, Ayanian J, Bragg-Gresham J, Balkrishnan R, Chen JL, Cope E, et al. US Renal Data System 2015 annual data report: epidemiology of kidney disease in the United States. Am J Kidney Dis. 2016;67(3 Suppl 1):A7–8.View ArticleGoogle Scholar
  2. U.S. Renal Data System. USRDS 2013 annual data report: atlas of chronic kidney disease and end-stage renal disease in the United States. Bethesda: National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases; 2013.Google Scholar
  3. Cabrera C, Brunelli SM, Rosenbaum D, Anum E, Ramakrishnan K, Jensen DE, Stalhammar NO, Stefansson BV. A retrospective, longitudinal study estimating the association between interdialytic weight gain and cardiovascular events and death in hemodialysis patients. BMC Nephrol. 2015;16:113.View ArticlePubMedPubMed CentralGoogle Scholar
  4. Flythe JE, Kshirsagar AV, Falk RJ, Brunelli SM. Associations of posthemodialysis weights above and below target weight with all-cause and cardiovascular mortality. Clin J Am Soc Nephrol. 2015;10(5):808–16.View ArticlePubMedPubMed CentralGoogle Scholar
  5. Kalantar-Zadeh K, Regidor DL, Kovesdy CP, Van Wyck D, Bunnapradist S, Horwich TB, Fonarow GC. Fluid retention is associated with cardiovascular mortality in patients undergoing long-term hemodialysis. Circulation. 2009;119(5):671–9.View ArticlePubMedPubMed CentralGoogle Scholar
  6. Arneson TJ, Liu J, Qiu Y, Gilbertson DT, Foley RN, Collins AJ. Hospital treatment for fluid overload in the Medicare hemodialysis population. Clin J Am Soc Nephrol. 2010;5(6):1054–63.View ArticlePubMedPubMed CentralGoogle Scholar
  7. Chazot C. Can chronic volume overload be recognized and prevented in hemodialysis patients? Use of a restricted-salt diet. Semin Dial. 2009;22(5):482–6.View ArticlePubMedGoogle Scholar
  8. Kayikcioglu M, Tumuklu M, Ozkahya M, Ozdogan O, Asci G, Duman S, Toz H, Can LH, Basci A, Ok E. The benefit of salt restriction in the treatment of end-stage renal disease by haemodialysis. Nephrol Dial Transplant. 2009;24(3):956–62.View ArticlePubMedGoogle Scholar
  9. McCormick N, Lacaille D, Bhole V, Avina-Zubieta JA. Validity of heart failure diagnoses in administrative databases: a systematic review and meta-analysis. PLoS One. 2014;9(8):e104519.View ArticlePubMedPubMed CentralGoogle Scholar
  10. Banerjee D, Ma JZ, Collins AJ, Herzog CA. Long-term survival of incident hemodialysis patients who are hospitalized for congestive heart failure, pulmonary edema, or fluid overload. Clin J Am Soc Nephrol. 2007;2(6):1186–90.View ArticlePubMedGoogle Scholar
  11. Weinhandl E, Constantini E, Everson S, Gilbertson D, Li S, Solid C, Anger M, Bhat JG, DeOreo P, Krishnan M, et al. Peer kidney care initiative 2014 report: dialysis care and outcomes in the United States. Am J Kidney Dis. 2015;65(6 Suppl 1):Svi, S1–140.PubMedGoogle Scholar
  12. 2015 Researcher’s Guide to the USRDS Database, Appendix B: Data File Descriptions. [https://www.usrds.org/2015/rg/USRDS_Res_Guide_App_B_DataFileDescrip_15.pdf]. Accessed 11 Sept 2016.
  13. Flahault A, Cadilhac M, Thomas G. Sample size calculation should be performed for design accuracy in diagnostic test studies. J Clin Epidemiol. 2005;58(8):859–62.View ArticlePubMedGoogle Scholar
  14. Yancy CW, Jessup M, Bozkurt B, Butler J, Casey Jr DE, Drazner MH, Fonarow GC, Geraci SA, Horwich T, Januzzi JL, et al. 2013 ACCF/AHA guideline for the management of heart failure: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2013;62(16):e147–239.View ArticlePubMedGoogle Scholar
  15. Nieminen MS, Bohm M, Cowie MR, Drexler H, Filippatos GS, Jondeau G, Hasin Y, Lopez-Sendon J, Mebazaa A, Metra M, et al. Executive summary of the guidelines on the diagnosis and treatment of acute heart failure: the Task Force on Acute Heart Failure of the European Society of Cardiology. Eur Heart J. 2005;26(4):384–416.View ArticlePubMedGoogle Scholar
  16. Section 15062.1, Payment for Physician Services Furnished to Dialysis Inpatients. [https://www.cms.gov/Regulations-and-Guidance/Guidance/Transmittals/downloads/R1810B3.pdf]. Accessed 5 June 2016.
  17. Movilli E, Gaggia P, Zubani R, Camerini C, Vizzardi V, Parrinello G, Savoldi S, Fischer MS, Londrino F, Cancarini G. Association between high ultrafiltration rates and mortality in uraemic patients on regular haemodialysis. A 5-year prospective observational multicentre study. Nephrol Dial Transplant. 2007;22(12):3547–52.View ArticlePubMedGoogle Scholar
  18. Flythe JE, Kimmel SE, Brunelli SM. Rapid fluid removal during dialysis is associated with cardiovascular morbidity and mortality. Kidney Int. 2011;79(2):250–7.View ArticlePubMedGoogle Scholar
  19. Chazot C, Wabel P, Chamney P, Moissl U, Wieskotten S, Wizemann V. Importance of normohydration for the long-term survival of haemodialysis patients. Nephrol Dial Transplant. 2012;27(6):2404–10.View ArticlePubMedGoogle Scholar
  20. Wizemann V, Wabel P, Chamney P, Zaluska W, Moissl U, Rode C, Malecka-Masalska T, Marcelli D. The mortality risk of overhydration in haemodialysis patients. Nephrol Dial Transplant. 2009;24(5):1574–9.View ArticlePubMedPubMed CentralGoogle Scholar
  21. Roos LL, Mustard CA, Nicol JP, McLerran DF, Malenka DJ, Young TK, Cohen MM. Registries and administrative data: organization and accuracy. Med Care. 1993;31(3):201–12.View ArticlePubMedGoogle Scholar
  22. Iezzoni LI. Assessing quality using administrative data. Ann Intern Med. 1997;127(8 Pt 2):666–74.View ArticlePubMedGoogle Scholar
  23. Schneeweiss S, Avorn J. A review of uses of health care utilization databases for epidemiologic research on therapeutics. J Clin Epidemiol. 2005;58(4):323–37.View ArticlePubMedGoogle Scholar
  24. Peabody JW, Luck J, Jain S, Bertenthal D, Glassman P. Assessing the accuracy of administrative data in health information systems. Med Care. 2004;42(11):1066–72.View ArticlePubMedGoogle Scholar
  25. Ray WA, Griffin MR. Use of Medicaid data for pharmacoepidemiology. Am J Epidemiol. 1989;129(4):837–49.View ArticlePubMedGoogle Scholar
  26. Changes to the Hospital Inpatient Prospective Payment Systems and Fiscal Year 2008 Rates. [https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/downloads/CMS-1533-FC.pdf]. Accessed 5 June 2016.
  27. Winkelmayer WC, Schneeweiss S, Mogun H, Patrick AR, Avorn J, Solomon DH. Identification of individuals with CKD from Medicare claims data: a validation study. Am J Kidney Dis. 2005;46(2):225–32.View ArticlePubMedGoogle Scholar
  28. Chubak J, Pocobelli G, Weiss NS. Tradeoffs between accuracy measures for electronic health care data algorithms. J Clin Epidemiol. 2012;65(3):343–9. e342.View ArticlePubMedGoogle Scholar
  29. Rothman KJ, Greenland S, Lash TL. Modern epidemiology. 3rd ed. Philadelphia: Wolters Kluwer Health/Lippincott Williams & Wilkins; 2008. p. 758.Google Scholar
  30. Transmittal 2455, Hospital Dialysis Services for Patients with and without End Stage Renal Disease (ESRD). [https://www.cms.gov/Regulations-and-Guidance/Guidance/Transmittals/downloads/R2455CP.pdf]. Accessed 11 Sept 2016.
  31. Medicare Benefit Policy Manual: Chapter 6 - Hospital Services Covered Under Part B. [https://www.cms.gov/Regulations-and-Guidance/Guidance/Manuals/downloads/bp102c06.pdf]. Accessed 11 Sept 2016.

Copyright

© The Author(s). 2016

Advertisement