Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Validity and reliability of a novel immunosuppressive adverse effects scoring system in renal transplant recipients

  • Calvin J Meaney1, 2Email author,
  • Ziad Arabi3,
  • Rocco C Venuto3, 5,
  • Joseph D Consiglio4,
  • Gregory E Wilding4 and
  • Kathleen M Tornatore1, 2, 3
BMC Nephrology201415:88

DOI: 10.1186/1471-2369-15-88

Received: 27 November 2013

Accepted: 29 May 2014

Published: 12 June 2014

Abstract

Background

After renal transplantation, many patients experience adverse effects from maintenance immunosuppressive drugs. When these adverse effects occur, patient adherence with immunosuppression may be reduced and impact allograft survival. If these adverse effects could be prospectively monitored in an objective manner and possibly prevented, adherence to immunosuppressive regimens could be optimized and allograft survival improved. Prospective, standardized clinical approaches to assess immunosuppressive adverse effects by health care providers are limited. Therefore, we developed and evaluated the application, reliability and validity of a novel adverse effects scoring system in renal transplant recipients receiving calcineurin inhibitor (cyclosporine or tacrolimus) and mycophenolic acid based immunosuppressive therapy.

Methods

The scoring system included 18 non-renal adverse effects organized into gastrointestinal, central nervous system and aesthetic domains developed by a multidisciplinary physician group. Nephrologists employed this standardized adverse effect evaluation in stable renal transplant patients using physical exam, review of systems, recent laboratory results, and medication adherence assessment during a clinic visit. Stable renal transplant recipients in two clinical studies were evaluated and received immunosuppressive regimens comprised of either cyclosporine or tacrolimus with mycophenolic acid. Face, content, and construct validity were assessed to document these adverse effect evaluations. Inter-rater reliability was determined using the Kappa statistic and intra-class correlation.

Results

A total of 58 renal transplant recipients were assessed using the adverse effects scoring system confirming face validity. Nephrologists (subject matter experts) rated the 18 adverse effects as: 3.1 ± 0.75 out of 4 (maximum) regarding clinical importance to verify content validity. The adverse effects scoring system distinguished 1.75-fold increased gastrointestinal adverse effects (p = 0.008) in renal transplant recipients receiving tacrolimus and mycophenolic acid compared to the cyclosporine regimen. This finding demonstrated construct validity. Intra-class correlation was 0.81 (95% confidence interval: 0.65-0.90) and Kappa statistic of 0.68 ± 0.25 for all 18 adverse effects and verified substantial inter-rater reliability.

Conclusions

This immunosuppressive adverse effects scoring system in stable renal transplant recipients was evaluated and substantiated face, content and construct validity with inter-rater reliability. The scoring system may facilitate prospective, standardized clinical monitoring of immunosuppressive adverse drug effects in stable renal transplant recipients and improve medication adherence.

Keywords

Immunosuppressive agents Adverse effects Renal transplantation Calcineurin inhibitors Mycophenolic acid Tacrolimus Cyclosporine

Background

Renal transplantation is the preferred intervention for patients with end-stage renal disease due to decreased patient morbidity and mortality compared to dialysis [13]. Renal transplant recipients are treated with combination immunosuppression consisting of calcineurin inhibitors such as cyclosporine or tacrolimus and mycophenolic acid to prevent allograft rejection [4]. However, these medications are associated with significant adverse effects which increase patient morbidity and decrease medication adherence [414]. Patient non-adherence with immunosuppressive medications has increased as a result of mild to moderate adverse effects which are not consistently documented during outpatient clinic appointments. This may impact acute rejection and long-term allograft survival post-transplant [1517]. Prospective clinical monitoring of immunosuppressive adverse effects is critical to prevent post-transplant complications and improve patient safety outcomes.

Routine clinical assessment of immunosuppressive adverse effects has not included standardized, objective rating guides with quantitation of severity. Therefore, adverse effect evaluation post-transplant is variable among clinicians resulting in inconsistent reporting and interventions. Assessment is the first fundamental step in the identification, prevention and management of adverse effects. Despite routine use of prospective, standardized clinical approaches to appraise immunosuppressive drug efficacy by health care providers, most adverse effects are not assessed in a systematic, objective manner. Therefore, we developed and evaluated the application, reliability and validity of a novel non-renal adverse effects assessment system in stable renal transplant recipients receiving maintenance immunosuppression consisting of a calcineurin inhibitor (cyclosporine or tacrolimus) and mycophenolic acid with or without prednisone.

Methods

A novel immunosuppressive adverse effects assessment for 18 individual adverse effects (Tables 1 and 2) were developed by a medical sub-specialist group within the University at Buffalo Nephrology/Transplant Program. This adverse effect scoring system was evaluated in two prospective studies with the enrollment of 58 stable renal transplant recipients who participated in clinical pharmacology studies from 2007 to 2013. Stable patients at least 12 months post-renal transplant receiving maintenance immunosuppression consisting of either tacrolimus (Prograf) with enteric coated mycophenolate sodium (EC-MPS; Myfortic) or cyclosporine (Neoral) with mycophenolate mofetil (MMF; CellCept) were enrolled. Studies were approved by the University of Buffalo Human Subjects Institutional Review Board. All patients provided informed consent upon enrollment.
Table 1

Immunosuppressive adverse effects scoring system with gastrointestinal and aesthetic adverse effects

Gastrointestinal adverse effectsa

Vomiting

 

    0:

None

    1+:

Minimal vomiting

    2+:

Excessive vomiting requiring symptomatic treatment

Diarrhea

 

    0:

None

    1+:

One loose bowel movement per day

    2+:

Two to five loose bowel movements per day

Dyspepsia

 

    0:

None

    1+:

Episode of indigestion within one hour after taking immunosuppressive medication

    2+:

Indigestion for at least half the day

    3+:

Indigestion the majority of the day requiring symptomatic treatment

Acid suppressive therapy

 

    0:

None

    1+:

Daily use of a histamine-2 receptor antagonist OR a proton pump inhibitor

    2+:

Daily use of a histamine-2 receptor antagonist AND a proton pump inhibitor

Aesthetic adverse effects a

Acne

 

    0:

None

    1+:

Lesion restricted to face, no specific treatment

    2+:

Numerous facial lesions; lesions of upper trunk

    3+:

Grade II and topical treatment

    4+:

Grade II and systemic treatment

Skin changes

 

    0:

No change

    1+:

No gross change but easy bruising

    2+:

Obvious thinning multiple or frequent ecchymoses

    3+:

Grade II and easy sloughing and/or striae; lacerations

Hirsutism

 

    0:

None

    1+:

New hair growth anywhere

    2+:

Marked increase in hair growth with noticeable change in appearance

    3+:

Grade II and use of depilatories or shaves

Moon facies

 

    0:

No change

    1+:

Rounding of jaws barely detected

    2+:

Marked rounding with noticeable change in appearance

    3+:

Grade II with plethora

Gingival hyperplasia

 

    0:

Absent

    1+:

Some hyperplasia

    2+:

Severe hyperplasia and bleeding

Buffalo hump

 

    0:

Absent

    1+:

Present

a The scoring system presents a quantitative severity rating of the most common adverse effects of immunosuppressive therapy. Each adverse effect was assessed as a change from pre-transplant status to document that the manifestation was attributed to immunosuppressive therapy or a change related therapy in the post-transplant period.

Table 2

Immunosuppressive adverse effects scoring system with central nervous system and miscellaneous adverse effects

Central nervous system adverse effectsa

Tremor

 

    0:

None

    1+:

Noticeable movement of paper suspended over outstretched hands with fingers spread

    2+:

Prominent movement of suspended paper

    3+:

Resting tremor and/or tremor that impairs function

Headache

 

    0:

Absent

    1+:

Present

Insomnia

 

    0:

None

    1+:

Difficulty sleeping

    2+:

Noticeable loss of sleep; requires pharmacotherapy to achieve adequate sleep

    3+:

Severe sleep deprivation; difficulty sleeping despite pharmacotherapy

Miscellaneous adverse effects a

Myopathy

 

    0:

None

    1+:

Complaints of muscle weakness or decreased strength

    2+:

Clinically apparent weakness difficulty rising from chair or squatting position

    3+:

Muscle wasting + severe limitation of exercise capacity

Ophthalmic changes

 

    0:

No cataract formation

    1+:

Cataracts present; or surgical removal of cataracts has occurred

Mania/Excitable behavior

 

    0:

None

    1+:

Excitable behavior; rapid speech and activity level

    2+:

Very excitable behavior, actions and speech

Depression

 

    0:

None

    1+:

Depressed thoughts with normal level of activity for patient ( no antidepressants)

    2+:

Depressed thoughts with normal level of activity for patient ( with antidepressants)

    3+:

Depressed thoughts without normal level of activity for patient ( does not dress, poor hygiene, etc.) and receives anti-depressants

Post-transplant Diabetes mellitus

Symptoms of diabetes plus casual plasma glucose ≥200 mg/dl

OR

 

Fasting plasma glucose ≥126 mg/dl

OR

(two of the criteria are required)

2 hour plasma glucose ≥200 mg/dl during an oral glucose tolerance test

OR

    0: Absent

Glycated Hemoglobin A1C > 7%

OR

    1+: Present

Use of anti-diabetic agents

 

General laboratory results

Renal function – Serum Creatinine (mg/dl)

_____________________

Blood pressure

_____________________

Glucose (mg/dl)

_____________________

Glycated hemoglobin A1C (%)

_____________________

Total cholesterol (mg/dl)

_____________________

High density lipoprotein (mg/dl)

_____________________

Low density lipoprotein (mg/dl)

_____________________

Triglycerides (mg/dl)

_____________________

Total white blood cells (cell/mm3)

_____________________

Neutrophils (%)

_____________________

Lymphocytes (%)

_____________________

Platelets (cells/mm3)

_____________________

Hemoglobin (g/dl)

_____________________

Hematocrit (%)

_____________________

a The scoring system presents a quantitative severity rating of the most common adverse effects of immunosuppressive therapy. Each adverse effect was assessed as a change from pre-transplant status to document that the manifestation was attributed to immunosuppressive therapy or a change related therapy in the post-transplant period.

Validity assessment

Face validity, or appearance of the scoring system to measure the defined adverse effect parameter, was developed through nephrologist consultation with medical sub-specialists (i.e. gastroenterologist consultation for dyspepsia, vomiting and diarrhea) and then applied to the target population of renal transplant recipients [18].

Content validity, or measurement of essential components of immunosuppressive adverse effects (Tables 1 and 2) was determined by a group of nephrologists who were subject matter experts using standard recommendations [18]. Fourteen nephrologists were invited to respond to a survey using the standardized rating scales and document the clinical importance of each adverse effect for assessment in the renal transplant population. Nephrologists ranked individual adverse effects for clinical importance post-transplant using a scale of: 1 = no importance through 4 = significant importance. Descriptive statistics were determined with an a priori critical value of 3 (moderate importance) to achieve content validation [18].

The adverse effects scoring system was applied to each renal transplant recipient enrolled in two clinical studies using similar inclusion and exclusion criteria to assess construct validity, or the extent the scoring system measures each adverse effect described [18]. Adverse effect ratios (gastrointestinal, central nervous system [CNS], aesthetic and cumulative adverse effects) were compared between renal transplant recipients stabilized on either cyclosporine and MMF or tacrolimus and EC-MPS regimens since pre-established differences in adverse effect profiles have been reported [7, 1922]. Construct validity was determined by analyzing adverse effect differences between renal transplant recipients receiving these two immunosuppressive regimens using general linear modeling.

Reliability assessment

Inter-rater reliability was evaluated in 32 stable renal transplant recipients treated with pre-established immunosuppressive regimens [23]. Nephrologists separately assessed each patient using the adverse effects scoring system during the same clinic visit. Intra-class correlation was generated from a pooled adverse effect score including the 18 adverse effects. The Kappa statistic was determined for each adverse effect to document inter-rater reliability and reported as an overall mean with standard deviation. Intra-class correlation ≥ 0.7 and Kappa statistic ≥ 0.6 were established a priori as desired endpoints since these scores represent substantial agreement between physician raters [23, 24].

Implementation of adverse effects scoring system

Nephrologists evaluated 18 adverse effects for each renal transplant patient during a scheduled clinic visit using objective clinical documentation including physical exam, review of systems and recent laboratory results. Medication adherence was assessed by the physician and pharmacist through patient interviews. Evaluation of the immunosuppressive adverse effects using the standardized scoring system was completed once (or twice during inter-rater reliability phase) during a single clinic visit for each patient in this cross-sectional study. Adverse effects were assessed as a change from pre-transplant status in an effort to identify the post-transplant development of immunosuppressive adverse effects. Participating nephrologists received training prior to utilizing this adverse effect scoring system. The scoring system focused on common non-renal adverse effects associated with immunosuppressive regimens of either cyclosporine or tacrolimus, mycophenolic acid and low dose glucocorticoids. After development of the adverse effects scoring system, nephrologists evaluated individual stable renal transplant recipients using these pre-established criteria with documentation of severity (i.e. 0 = no adverse effect; 1+ = mild to 3+ = severe manifestations). An overall adverse effect total was determined for each patient using the sum of individual adverse effect scores. The cumulative adverse effect ratio was then calculated as the quotient of each patient’s total score divided by the maximum score of all possible manifestations. This cumulative adverse effect ratio represented the number of adverse effects with corresponding severity rating. The gastrointestinal, CNS, and aesthetic ratios were generated a posteriori to further evaluate the organ system specific adverse effects. For example, if a patient received the following adverse effect evaluations: vomiting 0, diarrhea 1+, dyspepsia 1+, acid suppressive therapy 1+, the gastrointestinal adverse effect ratio would be 3/9 = 0.33 or 33% (9 is maximal possible score for gastrointestinal adverse effects). The denominator for each ratio provides normalization for the different severity rating among patients to allow comparison.

Statistical analysis

Continuous data was presented as mean ± standard deviation and categorical data as frequency (percentage). Demographic and clinical parameters were evaluated using student’s T-test for parametric data, Wilcoxon for non-parametric data, or Chi-Square for categorical data. All statistical analysis was performed using SAS version 9.3 (SAS Institute, Cary NC).

Results

A total of 58 stable renal transplant recipients were included in this study. Maintenance immunosuppressive regimens consisted of either cyclosporine and MMF (n = 30 males) or tacrolimus plus EC-MPS (n = 28 females and males). Table 3 provides a comparison of patient demographics for these two groups.
Table 3

Demographic characteristics

Parametera

Cyclosporine + MMF

Tacrolimus + EC-MPS

P value

 

(n = 30)

(n = 28)

 

Age (years)

52.3 ± 9.29

54.1 ± 11.5

0.527b

Time post-transplant (years)

6.93 ± 4.27

5.15 ± 3.34

0.084b

Male

30 (100%)

13 (46.4%)

<0.001*

Caucasian

17 (56.7%)

13 (46.4%)

0.221c

BMI (kg/m2)

33.1 ± 6.73

30.4 ± 6.94

0.151b

eGFR (ml/min/1.73 m2)

51.7 ± 13.5

52.2 ± 18.7

0.921b

Albumin

4.24 ± 0.37

4.26 ± 0.41

0.839b

WBC (cells/mm3)

6.81 ± 2.26

6.38 ± 1.52

0.401b

MPA dosee

1417 ± 407

1242 ± 319

0.048d*

Prednisone use

12 (40.0%)

9 (32.1%)

0.585 c

Calcineurin inhibitor trough concentration (ng/ml)

124 ± 47.1

6.80 ± 1.98

NA

aData presented as mean ± standard deviation or frequency (percentage); bT test; c Chi-square test; dWilcoxon test; eExpressed as active mycophenolic acid.

*p ≤ 0.05: statistical significance.

BMI, body mass index; eGFR, estimated glomerular filtration rate; MMF, mycophenolate mofetil; MPA, mycophenolic acid; EC-MPS, enteric coated mycophenolate sodium; NA, not applicable; WBC, white blood cells.

Content validity was verified by 9 nephrologists out of the 14 requests for completion of the rating survey. The nephrologists’ survey confirmed that the adverse effects scoring system provided moderate-to-significant clinical importance (3.1 ± 0.75 with maximum score of 4.0) for the assessment of the 18 adverse effects in renal transplant recipients.

Table 4 provides severity scores and frequencies of individual adverse effects for the patients receiving the two immunosuppressive regimens. Post-transplant diabetes mellitus was observed in 30.0% of patients receiving cyclosporine and MMF compared to 32.1% of patient receiving tacrolimus (P = 0.860). Patients treated with tacrolimus and EC-MPS experienced higher gastrointestinal adverse effect ratio (0.21 ± 0.15) compared to cyclosporine and MMF, (0.12 ± 0.11; P = 0.008). Greater CNS adverse effect ratio was found in patients treated with tacrolimus and EC-MPS (0.26 ± 0.18) compared to cyclosporine and MMF (0.16 ± 0.17; P = 0.035). There was no difference between patients treated with tacrolimus and EC-MPS or cyclosporine and MMF for the cumulative (0.154 ± 0.069 vs. 0.128 ± 0.080; P = 0.197) or aesthetic (0.101 ± 0.089 vs. 0.133 ± 0.084; P = 0.161) adverse effect ratios, respectively. For individual adverse effects, 76.7% of patients treated with cyclosporine and MMF experienced gingival hyperplasia compared to 21.4% of those treated with tacrolimus and EC-MPS (P < 0.0001). Of the 316 individual adverse effects scored in all patients, 84.5% were rated as 1+; 14.9% were rated as 2+; and less than 1% rated as 3+ (Table 4).
Table 4

Frequency of severity scores for immunosuppressive adverse effects a

Adverse effects

0

1+

2+

3+

Overall frequency (as %)

Vomiting

30 / 27

0 / 1

0 / 0

NA

0 / 3.57

Diarrhea

28 / 20

2 / 8

0 / 0

NA

6.67 / 28.6

Dyspepsia

19 / 14

9 / 5

2 / 9

0 / 0

36.7 / 50.0

Acid suppressive therapy

12 / 9

18 / 16

0 / 3

NA

60.0 / 67.9

Acne b

26 / 21

4 / 4

0 / 3

0 / 0

13.3 / 25.0

Skin changes

21 / 21

8 / 5

1 / 2

0 / 0

30.0 / 25.0

Hirsutism

22 / 22

7 / 3

1 / 3

0 / 0

26.7 / 21.4

Moon facies

13 / 17

15 / 11

2 / 0

0 / 0

56.7 / 39.3

Gingival hyperplasia

7 / 22

21 / 6

2 / 0

NA

76.7 / 21.4

Buffalo humpc

21 / 22

9 / 6

NA

NA

30.0 / 21.4

Tremor

13 / 7

15 / 16

2 / 4

0 / 1

56.7 / 75.0

Headachec

28 / 22

2 / 6

NA

NA

6.67 / 21.4

Insomnia

22 / 15

5 / 9

2 / 4

1 / 0

26.7 / 46.4

Myopathy

27 / 24

2 / 4

1 / 0

0 / 0

10.0 / 14.3

Ophthalmic changesc

17 / 18

13 / 10

NA

NA

43.3 / 35.7

Mania

29 / 28

1 / 0

0 / 0

NA

3.33 / 0

Depression

26 / 18

4 / 4

0 / 6

0 / 0

13.3 / 35.7

Post-transplant diabetes mellitusc

21 / 19

9 / 9

NA

NA

30.0 / 32.1

Data displayed as cyclosporine and MMF regimen (n = 30) / tacrolimus and EC-MPS regimen (n = 28).

aSeverity score determined using Tables 1 and 2; bNo patients received a 4+ for acne; c rated as present or absent.

NA, non-applicable since no rating; MMF, mycophenolate mofetil; EC-MPS, enteric coated mycophenolate sodium.

The mean Kappa statistic was 0.68 ± 0.25. The intra-class correlation was 0.81 (95% confidence interval [CI]: 0.65-0.90) to represent the reliability of summary scores for the 18 adverse effects. The intra-class correlations for adverse effect ratios exceeded the a priori designation as follows: 0.75 (CI: 0.55-0.87) for cumulative; 0.78 (CI: 0.60-0.88) for gastrointestinal; 0.81 (CI: 0.65-0.90) for CNS; and 0.84 (CI: 0.70-0.92) for aesthetic adverse effect ratios. Inspection of the Bland-Altman graph (Figure 1) revealed limited variability between raters without systematic bias.
https://static-content.springer.com/image/art%3A10.1186%2F1471-2369-15-88/MediaObjects/12882_2013_Article_780_Fig1_HTML.jpg
Figure 1

Bland-Altman Plot for Immunosuppressive Adverse Effects Scoring System. This figure presents the inter-rater agreement of n = 32 renal transplant recipients for the adverse effects scoring system with the difference between raters on the y-axis and the mean total adverse effect score for each patient on the x-axis. Each solid dot represents one patient (Note: there are 7 overlapping data points). The dotted lines represent the mean difference between the two raters (−0.15; middle line) with the upper and lower bounds of 2 standard deviations around the mean (−3.59, +3.28). There is little systematic bias or variability between raters.

Discussion

This study presents a novel standardized immunosuppressive adverse effects scoring system for renal transplant recipients with validation and inter-rater reliability demonstrated. Multidisciplinary development of this scoring system with successful application to the intended population provides face validity [18]. The heterogeneous nature of the individual adverse effects depicted in Table 3 emphasizes varying degrees of severity and frequency among stable renal transplant recipients receiving calcineurin inhibitor based immunosuppressive regimens. Severity assessment of a 3+ score for any adverse effect was less than 1% of all adverse effects during this study since clinically stable patients were targeted for enrollment. Identifying transplant patients with mild to moderate adverse effects may have important clinical impact since drug regimen adjustments during maintenance immunosuppression may improve or eliminate these manifestations. Renal transplant recipients with numerous mild to moderate adverse effects also have increased risk for medication non-adherence as time post-transplant increases which may impact long-term allograft survival [16, 17]. Therefore, prospective, standardized monitoring of immunosuppressive adverse effects may reduce medication non-adherence, improve patient tolerability of immunosuppressive regimens and may improve renal allograft survival [16, 17]. In addition, this adverse effects evaluation system provides an objective approach to document and rank severity of medication related effects in patients during clinic visits. Differences in the rating scales for the individual adverse effects are present (i.e. 0-2+, 0-3+) based on the smallest detectable incremental change in manifestation clinically observed by the physician. This rating is adjusted by using the cumulative and organ system specific adverse effect ratios which incorporates internal normalization in order to compare patients. This approach documents adverse effect frequency and severity which is rarely verified in clinical studies [21].

Nephrologists as subject matter experts verified moderate to significant clinical importance for the individual adverse effects to be used in the scoring system and substantiated content validity [18]. Some limitations exist with this evaluation including physician bias with current prescribing trends for low glucocorticoid doses or steroid-free immunosuppressive regimens post-transplant. These prescribing trends may account for lower clinical importance scores of steroid associated adverse effects. Since interpatient variation in clinical response, drug exposure, and adverse effect manifestations to steroid therapy post-transplant exists, these adverse effects will remain in the scoring system to guide maintenance immunosuppression [2527]. Tailoring the specific adverse effects scoring system to reflect the inclusion or exclusion of glucocorticoids (i.e. prednisone) in the immunosuppressive regimen provides a practical and flexible objective monitoring alternative for transplant recipients.

The adverse effect ratios documented that renal transplant recipients receiving tacrolimus and EC-MPS regimen had greater gastrointestinal and CNS adverse effects compared to patients stabilized on the cyclosporine and MMF regimen. These findings are consistent with previous clinical reports [7, 22]. Aesthetic adverse effects such as gingival hyperplasia, hirsutism, acne, and skin changes are more frequent in cyclosporine treated patients, but was not observed in this study for the aesthetic adverse effect ratio [7, 21]. This finding may be attributed to sample size or the male predominance in the cyclosporine group. Additionally, individual aesthetic adverse effects such as gingival hyperplasia was more frequent in the cyclosporine treated patients and confirms adverse effect patterns observed in renal transplant recipients participating in comparative efficacy studies between these two immunosuppressive regimens. This finding further verifies construct validity [19, 20, 22]. The cumulative adverse effect ratio was created a priori with incorporation of the normalization for purpose of interpatient comparison. No differences were noted in the cumulative ratio between regimens. However, gastrointestinal, central nervous system (CNS), and aesthetic adverse effect ratios were generated a posteriori to further quantitate organ system specific adverse effects and interpatient differences of these manifestations were successfully demonstrated. Potential construct limitations include confounding factors such as concomitant medications with overlapping adverse effect profiles and the inability to distinguish causative relationships to individual medications. Since this scoring system provides a composite of adverse effects manifested by common immunosuppression regimens, additional factors such as concomitant medications or co-morbidities should also be considered during routine clinical assessment.

Inter-rater reliability of the scoring system was substantial based on intra-class correlation and Kappa analyses documenting agreement between raters [23, 24]. The overall scoring system and each organ system adverse effect ratio achieved the a priori endpoint for intra-class correlation ≥ 0.7. Qualitative assessment of specificity was observed through rigorous patient assessment methodologies to characterize the adverse effects, the significant inter-rater reliability, and calcineurin inhibitor specific differences in adverse effect profiles observed.

The adverse effects scoring system administered by a clinician provides a time efficient, standardized clinical approach to assess the frequency and severity of these manifestations in renal transplant recipients. In contrast, the Memphis survey provides a patient completed questionnaire with comprehensive evaluation of symptoms and quality of life post-transplant, but does not provide succinct, efficient, objective adverse effects evaluation completed by the practitioner [28]. In addition, the patient completed gastrointestinal symptom rating scale validated for renal transplant recipients does not include physician verification or other organ-specific adverse effects [29]. Therefore, routine clinical application of these patient rating scales may be limited for objective immunosuppressive monitoring.

This is a single-center experience in stable renal transplant recipients receiving maintenance immunosuppression according to pre-established protocols. Therefore, variability observed in adverse effect manifestations as detected by the scoring system supports internal validity. As a single center experience, consideration of scoring bias by participating nephrologists who may provide routine medical care to some of these patients was possible. Due to the high inter-rater reliability, this type of bias was minimized. The novelty of the scoring system is the utilization of clinician directed evaluation of common adverse effects using a systematic, objective approach instead of a reactive response to these manifestations. A few infrequent adverse effects (e.g. alopecia) were not included and may be considered a study limitation.

Future research using this validated adverse effect rating system may include longitudinal evaluation of immunosuppressive adverse effects during the acute and chronic post-transplant periods and relationships to medication adherence, physician evaluation and allograft outcomes. This preliminary report provides a novel standardized adverse effect assessment which may benefit other solid organ transplant populations through prospective evaluations.

Conclusions

This validation study presents a clinician administered immunosuppressive adverse effects scoring system for stable renal transplant recipients which provides face, content and construct validity with inter-rater reliability. The scoring system provides a systematic, standardized, objective assessment of immunosuppressive adverse effects that could be utilized for periodic evaluation (i.e. quarterly) post-transplant to enhance individualization of immunosuppression. This approach to clinical adverse effects monitoring may improve patient safety and immunosuppressive adherence post-transplant.

Abbreviations

BMI: 

Body mass index

CI: 

95% Confidence interval

CNS: 

Central nervous system

EC-MPS: 

Enteric-coated mycophenolate sodium

eGFR: 

Estimated glomerular filtration rate

MMF: 

Mycophenolate mofetil

MPA: 

Mycophenolic acid

NA: 

Not applicable

WBC: 

White blood cells.

Declarations

Acknowledgements

Dr. Meaney was an Immunosuppressive Pharmacology Research Fellow in the Department of Pharmacy Practice/Translational Pharmacology Research Core at the NYS Center of Excellence for Bioinformatics and Life Sciences and UB School of Pharmacy & Pharmaceutical Sciences. Dr. Arabi was a NYS ERCIP Transplant Fellow at Erie County Medical Center; Renal Division, Department of Medicine, UB School of Medicine. Joseph Consiglio, MA is completing the PhD program in Biostatistics, UB School of Public Health.

The assistance of the following individuals is greatly appreciated: Anne Case, BS; Vanessa Gray, RN; Lisa Venuto, PA; Ellen Jackson, RN and Kris Reed, RN from Renal Division at ECMC. The assistance of Louise Cooper, RPh, MS from the UB Research Pharmacy and Immunosuppressive Pharmacology Research Program is also appreciated.

Authors’ Affiliations

(1)
Immunosuppressive Pharmacology Research Program, Translational Pharmacology Core, NYS Center of Excellence in Bioinformatics and Life Sciences
(2)
Pharmacy Practice, School of Pharmacy and Pharmaceutical Sciences, University at Buffalo
(3)
Medicine, Nephrology Division; School of Medicine and Biomedical Sciences, University at Buffalo
(4)
Biostatistics; School of Public Health, University at Buffalo
(5)
Erie County Medical Center

References

  1. United States Renal Data System, USRDS: Annual Data Report: Atlas of Chronic Kidney Disease and End-Stage Renal Disease in the United States. 2012, Bethesda, MD: National Institutes of Health, National Institute of Diabetes and Digestive and Kidney DiseasesGoogle Scholar
  2. United States Renal Data System, USRDS: Incident and Prevalent Counts by Quarter. 2013, Bethesda, MD: National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases, http://www.usrds.org/qtr/default.aspx,Google Scholar
  3. Tonelli M, Wiebe N, Knoll G, Bello A, Browne S, Jadhav D, Klarenbach S, Gill J: Systematic review: kidney transplantation compared with dialysis in clinically relevant outcomes. Am J Transplant. 2011, 11 (10): 2093-2109. 10.1111/j.1600-6143.2011.03686.x.View ArticlePubMedGoogle Scholar
  4. Halloran PF: Immunosuppressive drugs for kidney transplantation. N Engl J Med. 2004, 351 (26): 2715-2729. 10.1056/NEJMra033540.View ArticlePubMedGoogle Scholar
  5. Kidney Disease: Improving Global Outcomes (KDIGO) Transplant Work Group: KDIGO clinical practice guideline for the care of kidney transplant recipients. Am J Transplant. 2009, 9 (Suppl 3): S1-S157.Google Scholar
  6. Dobbels F, Moons P, Abraham I, Larsen CP, Dupont L, De Geest S: Measuring symptom experience of side-effects of immunosuppressive drugs: the Modified Transplant Symptom Occurrence and Distress Scale. Transpl Int. 2008, 21 (8): 764-773. 10.1111/j.1432-2277.2008.00674.x.View ArticlePubMedGoogle Scholar
  7. Jose M: Calcineurin Inhibitors in Renal Transplantation: Adverse Effects. Nephrology (Carlton). 2007, 12: S66-S74. 10.1111/j.1440-1797.2007.00731.x.View ArticleGoogle Scholar
  8. Moons P, Vanrenterghem Y, Van Hooff JP, Squifflet JP, Margodt D, Mullens M, Thevissen I, De Geest S: Health-related quality of life and symptom experience in tacrolimus-based regimens after renal transplantation: a multicentre study. Transpl Int. 2003, 16 (9): 653-664. 10.1111/j.1432-2277.2003.tb00366.x.View ArticlePubMedGoogle Scholar
  9. Boots JM, Christiaans MH, van Hooff JP: Effect of immunosuppressive agents on long-term survival of renal transplant recipients: focus on the cardiovascular risk. Drugs. 2004, 64 (18): 2047-2073. 10.2165/00003495-200464180-00004.View ArticlePubMedGoogle Scholar
  10. Fiebiger W, Mitterbauer C, Oberbauer R: Health-related quality of life outcomes after kidney transplantation. Health Qual Life Outcomes. 2004, 2 (2): ePub, doi:10.1186/1477-7525-2-2Google Scholar
  11. Olyaei AJ, Demattos AM, Bennett WM: Cardiovascular complications of immunosuppressive agents in renal transplant recipients. Expert Opin Drug Saf. 2005, 4 (1): 29-44. 10.1517/14740338.4.1.29.View ArticlePubMedGoogle Scholar
  12. Rosenberger J, Geckova AM, van Dijk JP, Roland R, van den Heuvel WJA, Groothoff JW: Factors modifying stress from adverse effects of immunosuppressive medication in kidney transplant recipients. Clin Transplant. 2005, 19 (1): 70-76. 10.1111/j.1399-0012.2004.00300.x.View ArticlePubMedGoogle Scholar
  13. Miller LW: Cardiovascular toxicities of immunosuppressive agents. Am J Transplant. 2002, 2 (9): 807-818. 10.1034/j.1600-6143.2002.20902.x.View ArticlePubMedGoogle Scholar
  14. Wissing KM, Abramowicz D, Broeders N, Vereerstraeten P: Hypercholesterolemia is associated with increased kidney graft loss caused by chronic rejection in male patients with previous acute rejection. Transplantation. 2000, 70 (3): 464-472. 10.1097/00007890-200008150-00012.View ArticlePubMedGoogle Scholar
  15. Chisholm MA, Lance CE, Mulloy LL: Patient factors associated with adherence to immunosuppressant therapy in renal transplant recipients. Am J Health Syst Pharm. 2005, 62 (17): 1775-1781. 10.2146/ajhp040541.View ArticlePubMedGoogle Scholar
  16. Pinsky BW, Takemoto SK, Lentine KL, Burroughs TE, Schnitzler MA, Salvalaggio PR: Transplant outcomes and economic costs associated with patient noncompliance to immunosuppression. Am J Transplant. 2009, 9 (11): 2597-2606. 10.1111/j.1600-6143.2009.02798.x.View ArticlePubMedGoogle Scholar
  17. Vlaminck H, Maes B, Evers G, Verbeke G, Lerut E, Van Damme B, Vanrenterghem Y: Prospective study on late consequences of subclinical non-compliance with immunosuppressive therapy in renal transplant patients. Am J Transplant. 2004, 4 (9): 1509-1513. 10.1111/j.1600-6143.2004.00537.x.View ArticlePubMedGoogle Scholar
  18. Abell N, Springer D, Kamata A: Developing and Validating Rapid Assessment Instruments. 2009, New York: Oxford University PressView ArticleGoogle Scholar
  19. Package Insert: Neoral (cyclosporine). Novartis Pharmaceuticals. 2012, East Hanover, NJ, https://www.pharma.us.novartis.com/product/pi/pdf/neoral.pdf,
  20. Package Insert: Prograf (tacrolimus). Astellas Pharma US, Inc. 2012, Northbrook, IL, http://www.astellas.us/docs/prograf.pdf,
  21. Ekberg H, Bernasconi C, Noldeke J, Yussim A, Mjornstedt L, Erken U, Ketteler M, Navratil P: Cyclosporine, tacrolimus and sirolimus retain their distinct toxicity profiles despite low doses in the Symphony study. Nephrol Dial Transpl. 2010, 25 (6): 2004-2010. 10.1093/ndt/gfp778.View ArticleGoogle Scholar
  22. Henry ML: Cyclosporine and tacrolimus (FK506): a comparison of efficacy and safety profiles. Clin Transplant. 1999, 13 (3): 209-220. 10.1034/j.1399-0012.1999.130301.x.View ArticlePubMedGoogle Scholar
  23. Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977, 33 (1): 159-174. 10.2307/2529310.View ArticlePubMedGoogle Scholar
  24. Shrout P, Fleiss J: Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979, 86 (2): 420-428.View ArticlePubMedGoogle Scholar
  25. Steiner R, Awdishu L: Steroids in kidney transplant recipients. Semin Immunopathol. 2011, 33 (2): 157-167. 10.1007/s00281-011-0259-7.View ArticlePubMedPubMed CentralGoogle Scholar
  26. Tornatore KM, Gilliland-Johnson KK, Farooqui M, Reed KA, Venuto RC: Pharmacokinetics and pharmacodynamic response of methylprednisolone in premenopausal renal transplant recipients. J Clin Pharmacol. 2004, 44 (9): 1003-1011. 10.1177/0091270004268130.View ArticlePubMedGoogle Scholar
  27. Tornatore KM, Reed KA, Venuto RC: Assessment of methylprednisolone pharmacokinetics and cortisol response during the early and chronic postrenal transplant periods. Transplantation. 1995, 60 (12): 1607-1611. 10.1097/00007890-199560120-00039.View ArticlePubMedGoogle Scholar
  28. Winsett RP, Arheart K, Stratta RJ, Alloway R, Wicks MN, Gaber AO, Hathaway DK: Evaluation of an immunosuppressant side effect instrument. Prog Transplant. 2004, 14 (3): 210-216+240-View ArticleGoogle Scholar
  29. Kleinman L, Kilburg A, Machnicki G, Faull R, Walker R, Prasad R, Ambuehl P, Bahner U, Margolis MK: Using GI-specific patient outcome measures in renal transplant patients: validation of the GSRS and GIQLI. Qual Life Res. 2006, 15 (7): 1223-1232. 10.1007/s11136-006-0053-5.View ArticlePubMedGoogle Scholar
  30. Pre-publication history

    1. The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2369/15/88/prepub

Copyright

© Meaney et al.; licensee BioMed Central Ltd. 2014

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Advertisement