Assessing concordance between patient-reported and investigator-reported CTCAE after proton beam therapy for prostate cancer

Highlights • Acute patient-reported outcomes were compared with investigator-reported outcomes.• We identified low agreement between IR-CTCAE and PRO-CTCAE for prostate cancer.• Physicians underestimated the frequency and severity of GU symptoms and diarrhea.• The end of treatment was a key timepoint showing high levels of discordance.


Introduction
The benefits of using patient-reported outcomes (PROs) are well established, with increased importance being placed on the patient perspective [1]. There are significant variations in the use of PROs in clinical trials, and novel ways to validate PROs and implement them into clinical workflows are needed [2][3][4].
PROs have been utilized to assess the impact of radiotherapy on the quality of life for a variety of diseases, and they have helped to guide the selection of an appropriate dose-fractionation regimen in some clinical settings [5][6][7][8]. For example, PROs have been used to identify dosimetric factors associated with decreased quality of life for head and neck cancer patients undergoing radiotherapy; however, it has been also recognized that PROs may demonstrate significant differences compared with those recorded by physicians [9,10].
There is strong interest among clinicians to incorporate validated PROs into clinical practice for prostate cancer patients to characterize genitourinary (GU) and gastrointestinal (GI) side effects [11,12]. In a phase III clinical study of 1643 men with clinically localized prostate cancer, the results of PROs demonstrated that radical prostatectomy (RP) was associated with a greater detriment to urinary continence and sexual function than radiotherapy therapy (RT). However, RT resulted in a greater detriment to urinary voiding and nocturia than RP or active surveillance at 6 months post-treatment, although these symptoms recovered and were similar to RP or active surveillance at 12 months. Bowel function was worse with RT at 6 months than RP or active surveillance, but then partially recovered at 12 months [13].
There are several instruments available to measure PROs in prostate cancer. For example, EPIC (Expanded Prostate Cancer Index Composite) is a widely utilized tool to assess patient-reported quality of life. Recently, Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) has been developed as a novel, validated, and reliable method for reporting patient-reported adverse events [14][15][16][17]. Because investigator-reported CTCAE (IR-CTCAE) often under-detects symptomatic adverse events, PRO-CTCAE can be used to enhance the precision and patientcenteredness of adverse event reporting in cancer clinical research [18,19]. In this study, we report acute PRO-CTCAE of proton beam therapy for patients with high or unfavorable-intermediate risk prostate cancer who received radiotherapy to the prostate/seminal vesicles and the regional pelvic lymph nodes in a prospective clinical trial. In addition, the correlation of PRO-CTCAE with IR-CTCAE is evaluated for *Patients were also given the option to select "Not sexually active" or "Prefer not to answer." discordance.

Patient cohort
A prospective study was designed to assess the outcomes of moderately hypofractionated proton radiotherapy for high-risk or unfavorable intermediate-risk prostate cancer. This trial was approved by our institutional review board, and registered at clinicaltrials.gov (NCT02874014). Target accrual of 56 patients was attained in December 2018. Details concerning the inclusion criteria, dose-fractionation regimen, CT simulation, volume guidelines, treatment planning, and radiotherapy delivery are described in the supplemental documentation. Of note, patients also received androgen-deprivation therapy (ADT) for 4-36 months as part of the treatment paradigm of high or unfavorable intermediate risk prostate cancer.

Investigator-reported CTCAE (IR-CTCAE)
IR-CTCAE were collected using the National Cancer Institute Common Terminology Criteria for Adverse Events, version 4 (CTCAE v4.0). Adverse events were assessed at baseline (prior to RT), weekly during RT, at the end of RT, 3-, 6-, and 12-months post-RT, and every 6 months up to 60 months post-RT.
In the study, 7 GI categories of CTCAE v4.0 were used to assess GI adverse events: diarrhea, proctitis, fecal incontinence, rectal stenosis, rectal ulcer, rectal hemorrhage, and small intestinal obstruction. For the evaluation of GU adverse events, 9 GU categories were used: urinary frequency, urinary urgency, urinary tract obstruction, urinary tract pain, urinary incontinence, hematuria, non-infective cystitis, urinary retention, and bladder spasm. For the assessment of erectile dysfunction adverse event, erectile function category was used. Adverse event grades were assigned by an attending physician or a clinical assistant, who also administered the CTCAE questionnaire.

Patient-Reported outcomes version of the CTCAE (PRO-CTCAE)
The PRO-CTCAE questions used in the study were selected as the most relevant to prostate cancer patients undergoing radiotherapy and androgen deprivation therapy. They included 7 questions in the GI domain, 7 questions in the GU domain, 10 questions in the endocrine and erectile dysfunction domains, and 1 concerning skin toxicity. The selection process of these questions was subjective and based on the expected side effects from RT and/or ADT. These PRO-CTCAE questions were administered at baseline, at the end of RT, 3-, 6-, and 12-month post-RT, and every 6 months up to 60 months post-RT.

Correlating PRO-CTCAE with IR-CTCAE
Few attempts have been made to correlate PRO-CTCAE with IR-CTCAE, and the most robust effort involved a panel of clinical investigators developing a general correlation [20]. In our study, there were 11 PRO-CTCAE questions that were deemed very closely corresponding to the IR-CTCAE definitions of a GI, GU, or erectile function adverse event. This correlation between PRO-CTCAE and IR-CTCAE in our study was conducted by two physicians independently, and the results demonstrated negligible disagreement. Correlating PRO-CTCAE descriptive grades to IR-CTCAE grading scores was conducted for each category of GI, GU and erectile function domains. This effort involved equating the descriptive term of each PRO-CTCAE grade to that of the IR-CTCAE grading score to have the most appropriate comparableness between the two instruments. In IR-CTCAE, some categories do not have a grade score 3 or higher designation (e.g. urinary frequency and urgency). Therefore, in such instances, an IR-CTCAE grading score that could be assigned for the worst descriptive PRO-CTCAE was limited by the maximum grade score available in a given IR-CTCAE category.
The corresponding adverse events and grades between the PRO-CTCAE questions and the IR-CTCAE categories used in our study are depicted in Table 1. This novel correlative effort between patientreported vs. investigator-reported CTCAE has not been previously validated; thus, the analyses of the concordance and discordance between PRO-CTCAE and IR-CTCAE were conducted using both descriptive terms, as well as the correlated grade scores, per Table 1.
All patients were provided with the list of PRO-CTCAE questions. One patient routinely failed to complete these questions. For erectile dysfunction, many patients chose the "Prefer not to answer" or "Not sexually active" options, which made the correlation with IR-CTCAE v4.0 very difficult.

IR-CTCAE
IR-CTCAE are demonstrated in Fig. 1a and 1b, for GI and GU toxicities, respectively. At baseline, most patients (94%) had no diarrhea. Fecal incontinence was similarly uncommon. A small portion of patients (9%) had baseline grade 1 proctitis. Urinary symptoms at baseline were more common: 74% with ≥ grade 1 urinary frequency and 49% with ≥ grade 1 urgency. Most patients (71%) had some degree of erectile dysfunction at baseline.
At the end of RT, IR-CTCAE registered an increase in diarrhea (31% with grade 1) but no other significant changes in GI adverse events. GU adverse events also increased at the end of RT, with most patients suffering from either grade 1 (70%) or grade 2 (28%) urinary frequency. Most patients (72%) had grade 1 urinary urgency. Urinary tract pain (≥grade 1) increased from 11% at baseline to 31% at the end of RT. Some degree of erectile dysfunction was registered in the majority of patients (79%).
Most GI and GU symptoms normalized 3 months after radiation therapy. Urinary frequency, however, remained increased from baseline (94% ≥ grade 1, and 15% grade 2). Erectile dysfunction also remained worse than baseline, with only 14% of patients with grade 0 and 46% with grade 2.

PRO-CTCAE
PRO-CTCAE were assessed in their raw, non-converted forms, as well as with the correlations to IR-CTCAE v4.0 outlined in Table 1. Raw PRO-CTCAE revealed increased "frequent" diarrhea at the end of treatment (18%) versus baseline (2%) (Fig. 1a). The converted values similarly reported an increase in grade 2 diarrhea at the end of treatment (18%) versus baseline (2%). Proctitis showed an increase in "mild" pain at the end of treatment (34%) versus baseline (13%), which directly mirrored the converted results. GI symptoms recovered 3 months after treatment. Fecal incontinence was uncommon (Fig. 2).
Urinary frequency and urgency were addressed by two different PRO-CTCAE questions. First, frequency was queried (ranging from "never" to "almost constantly"). In the non-converted form, increases were noted in urinary frequency "occasionally," frequently," and "almost constantly" at the end of treatment, with rates of 36% vs. 24%, 38% vs. 20%, and 10% vs. 0%, compared to baseline, respectively. Mild increases in urinary urgency were also common at the end of treatment (converted grade 2: 24% vs. 9% at baseline). Both frequency and urgency improved three months after radiotherapy, but there remained a mild increase in urinary frequency "occasionally," compared to baseline (37% vs. 24%). Using the converted values, grade 2 urinary frequency showed a similar pattern: it increased to 48% at the end of treatment (vs. 20% at baseline). It subsequently declined, but it remained slightly elevated (24% 3 months after radiotherapy) (Fig. 3).
When patients were asked about how urinary symptoms "interfered" with "usual or daily activities," much lower rates of adverse events were reported. Only 8% and 6% of patients reported that urinary frequency and urgency, respectively, interfered "quite a bit" at the end of treatment. No patients reported "quite a bit" of interference from urinary frequency at 3 months after radiotherapy, and only 1 patient reported such interference from urinary urgency. Converted values showed comparable results.
Finally, erectile function steadily declined from baseline to 3 months after radiotherapy. At 3 months after radiotherapy, fewer patients had normal erectile function (27% vs. 58%), and a majority of patients reported "severe" or "very severe" (grade 3) erectile dysfunction (57% vs. 30%), compared to baseline. Of note, all patients received ADT concurrently with radiation.

Comparison of IR-CTCAE and PRO-CTCAE
The direct comparison of the IR-CTCAE with PRO-CTCAE is shown in Fig. 2 (GI toxicity) and Fig. 3 (GU toxicity). The degree of concordance is explored in Table 3, where the rates of exact match, investigator underestimates, and investigator over-estimates are shown in comparison with PRO-CTCAE. Uncommon adverse events (fecal incontinence, urinary incontinence) were not included. Proctitis in IR-CTCAE was excluded because of its imperfect correlation with abdominal pain in PRO-CTCAE (Table 1).
Investigator-reported diarrhea had relatively good concordance with patient-reported diarrhea at baseline and 3 months after radiotherapy, but there was still a trend towards underestimation in both cases (15% in each time point). The underestimation was worse at the end of treatment, with a 28% rate of underestimation and an 11% rate of discordance by ≥ 2 toxicity grades (all of which were underestimates). Urinary tract pain demonstrated a similar overall pattern. Concordance was high at baseline (over 80% of cases). At EOT, however, only 53% of investigator-reported pain matched patient-reported pain, with a 45% rate of underestimation and a 17% rate of discordance by ≥ 2 toxicity grades (all of which were underestimates). The Kappa statistic generally demonstrated fair concordance for these adverse events.
Urinary urgency and frequency were both considered through two questions. In these domains, IR-CTCAE matched exactly with PRO-CTCAE in only 15 -62% of cases across all time points. The frequency of symptoms (question 1) tended be overestimated prior to treatment (36% and 24% for urinary frequency and urgency, respectively), but they were generally underestimated at the end of treatment (35% and 31% for urinary frequency and urgency, respectively). The Kappa statistic also fell significantly for both of these adverse events from a slight to fair concordance to no concordance at the end of treatment. The degree of interference with daily activities from these symptoms (question 2) was consistently overestimated by investigators (45% − 85%) across all time points. Poor concordance between IR-CTCAE and PRO-CTCAE for the questions concerning interference with daily activities was noted with low Kappa statistic (Kappa: − 0.03-0.13).

Discussion
We present a detailed analysis comparing IR-CTCAE to PRO-CTCAE, and multiple important differences were noted. Overall, there was low agreement between IR-CTCAE and PRO-CTCAE, and investigators tended to underestimate the frequency and severity of urinary symptoms and diarrhea at the end of treatment. Even so, our findings suggest that IR-CTCAE and PRO-CTCAE are complementary, and consideration of both tools may allow for a more complete understanding of patient side effect burden and its impact on quality-of-life [21]. While IR-CTCAE is a Fig. 2. The rates and corresponding grades of diarrhea and fecal incontinence via IR-CTCAE and PRO-CTCAE are shown at baseline, during radiotherapy, and 3 months after radiotherapy.
useful endpoint, the differences identified via PRO-CTCAE suggest the presence of additional information that can be analyzed through the use of both assessments. Finally, our findings point towards an overall favorable toxicity profile for high-risk or unfavorable intermediate-risk prostate cancer treated with proton beam therapy [22].
There is a dearth of validated tools and literature to correlate patientreported outcomes with investigator-reported outcomes [23,24]. Such studies generally reveal low agreement between patient-and investigator-reported outcomes, with investigators often underreporting symptom severity [18,19]. Other studies have found that clinicians are best able to assess patient toxicities with more severe side effects [25]. In our study, a novel correlation between PRO-CTCAE and IR-CTCAE for many important GI and GU adverse events was developed to examine the concordance between the two assessment tools. Correlating PRO-CTCAE with IR-CTCAE involved both the non-converted, raw data, of PRO-CTCAE and the converted grade of PRO-CTCAE (by equating a descriptive term for each PRO-CTCAE raw data to that of the IR-CTCAE grading score, Table 1). The degree of concordance between PRO-CTCAE and IR-CTCAE was similar with either using the nonconverted, raw data, of PRO-CTCAE or using the converted grade of PRO-CTCAE. This result suggests that our correlation effort yielded an appropriate comparison between the two instruments for those GI and GU domains examined in this study. We strongly encourage further study and validation of this new method for correlating PRO-CTCAE with IR-CTCAE.
PROs have shown the potential to guide treatment decisions, and their use has increased in recent clinical trials [17,26]. Our results indicate that urinary symptoms were underestimated by investigators compared to PRO-CTCAE in about one-third of cases. This high rate of discordance at the end of treatment suggests that this time period represents an important phase of treatment during which a clinician needs to carefully assess for side effects of radiotherapy [27]. These results suggest that patients may not always volunteer this information at the last management visit. Instead, it may be that direct questioning regarding the presence or absence of these symptoms (e.g. via PRO-CTCAE) is required, and we encourage investigators to directly ask patients regarding the development of these side effects. It is also noteworthy that rates of symptom overestimation by investigators were high at baseline, potentially clouding accurate assessment of changes in those symptoms throughout the treatment course. Medications can be prescribed to address urinary frequency and urgency, and our results suggest that additional intervention may prove beneficial.
Significant differences were also noted regarding the interference of urinary symptoms with daily activities. Patients reported low rates of interference while investigators overestimated the impact of urinary symptoms on daily activities. It may be that interference with daily activities is too distinct a question to directly correlate with CTCAE. Even so, the striking differences between PRO-CTCAE and IR-CTCAE in this setting suggest that this area is inadequately addressed by our current practices of toxicity reporting. Ideally, investigators would seek to capture both the frequency and degree of interference with quality of life caused by a particular side effect. Current investigator-report outcomes likely inadequately address this aspect of treatment toxicity.
An analysis of over 15,000 patients in the Danish Prostate Cancer Registry suggests that overall quality-of-life was most adversely affected by sexual function, regardless of treatment modality [28]. Furthermore, another study found that regret about the treatment choice was more common among those who experienced more treatment-related symptoms during the year after treatment [29]. Though the reason for the decline in erectile function is likely multi-factorial (e.g. increasing age, ADT, and radiotherapy), the striking differences between IR-CTCAE and PRO-CTCAE regarding erectile dysfunction in our study suggests a need for close evaluation and frank conversations between patients and healthcare providers.
Limitations of this study chiefly relate to a relatively small number of patients analyzed at each time point. Since only 55 patients answered PRO assessments, more rigorous statistical analysis was not performed. Additionally, it is likely that the correlations drawn between PRO-CTCAE and IR-CTCAE were imperfect and may have influenced our results. Even so, results were quite similar between the converted and non-converted PROs, suggesting that any loss through this process was minimal. Finally, proctitis in IR-CTCAE was difficult to address robustly in this analysis because it correlated poorly with 'abdominal pain' in PRO-CTCAE. There is no domain in the current PRO-CTCAE that can be correlated well with proctitis in IR-CTCAE. Future studies should aim to more completely address this adverse event.

Conclusions
Our study shows low agreement between IR-CTCAE and PRO-CTCAE in the setting of proton therapy for prostate cancer. Compared to patientreported outcomes, physicians underestimated the frequency and severity of urinary symptoms and diarrhea at the end of treatment. Continued use of PROs should be strongly encouraged.

Declaration of Competing Interest
The wife of Dr. Kowalchuk is a senior technical product manager for GE Healthcare. No other authors have relevant conflicts of interest to report.

Table 3
Correlations between investigator-reported outcomes and PROs are considered. EOT stands for end of treatment; 3 months refers to the time point 3 months after the completion of radiotherapy; and Q1 and Q2 refer to questions 1 and 2 for a specific toxicity (question 2 addresses the degree to which symptoms interfere with a patient's quality of life).