Patient-reported outcomes for monitoring symptomatic toxicities in cancer patients treated with immune-checkpoint inhibitors: A Delphi study

Citation for pulished version (APA): Da Silva Lopes, A. M., Colomer-Lahiguera, S., Mederos Alfonso, N., Aedo-Lopez, V., Spurrier-Bernard, G., Tolstrup, L. K., Pappot, H., Aspeslagh, S., Rogiers, A., Neyns, B., Haanen, J. B., Mitchell, S. A., Addeo, A., Michielin, O., & Eicher, M. (2021). Patient-reported outcomes for monitoring symptomatic toxicities in cancer patients treated with immune-checkpoint inhibitors: A Delphi study. European Journal of Cancer, 157, 225-237. https://doi.org/10.1016/j.ejca.2021.08.026


Introduction
The growing complexity of cancer care motivates efforts to improve the safety, effectiveness and tolerability of cancer treatments. With the recent widespread adoption of immune checkpoint inhibitors (ICIs) for an expanding number of disease indications, a wide range of new immune-related adverse events (IrAEs) has been reported [1]. While detection and monitoring of treatment toxicity is a priority across cancer care, it is particularly important during immunotherapy treatment. IrAEs are thought to be effects of an over-activated immune system that can affect almost any organ ('off-target' effects), varying in frequency and severity, with the most severe leading to hospitalisation, treatment discontinuation, long-term or permanent conditions or even death [2e6]. Despite frequent patient follow-up visits while on treatment, IrAEs can rapidly progress in severity [2], underlining the need to empower patients with the means to self-monitor and self-report their symptoms [1].
The Common Terminology Criteria for Adverse Events (CTCAE) are standardised criteria used by clinicians to identify, grade and report adverse events (AEs) experienced by patients receiving cancer therapies, including ICIs [7] (p0) . However, accurately and reliably reporting AEs can be challenging, prompting the United States Food and Drug Administration (FDA) to call for the inclusion of the patient's perspective when describing symptomatic AEs through the collection of patient-reported outcomes (PROs) [8] [e] [11]. PROs are defined as 'any report of the status of a patient's health condition that comes directly from the patient, without interpretation of the patient's response by a clinician or anyone else' [11]. Their use in clinical trials to track symptomatic toxicities of cancer treatments can improve the management of those symptoms, thereby preserving the health-related quality of life, and allowing patients to remain in treatment for longer, and decreasing emergency department visits [12,13]. Moreover, using PROs can enhance patient-clinician communication, allowing for a complete discussion of therapy side effects during office visits [13,14].
The PRO version of the CTCAE (PRO-CTCAEä) was developed by the US National Cancer Institute to address the need to capture through self-reporting the symptomatic toxicities experienced by patients participating in cancer clinical trials [15]. The PRO-CTCAEä Item Library is comprised of 124 items representing 78 symptomatic adverse events drawn from the CTCAE [16]. For each of these symptomatic AEs, PRO items were created to evaluate attributes of presence or absence, amount, frequency, severity and interference with usual activities. For a given AE, one to three attributes were selected depending on the content of the CTCAE criteria and the nature of the symptom. The PRO-CTCAEä has demonstrated favourable validity, reliability and responsiveness in a large, heterogeneous sample of United States patients undergoing cancer treatment [15]. Researchers select the relevant symptom terms for prospective surveillance, considering the agent under study, trial goals and the patient population [15]. Regarding the use of the PRO-CTCAEä items to declare symptomatic adverse events, FDA recommends selecting a set of the most important symptomatic AEs that are expected to occur [17]. However, research on methods to select appropriate symptom-related PROs is still limited [18,19].
Using the PRO-CTCAEä to describe symptomatic toxicities of ICIs poses some challenges. PRO-CTCAEä development, like that of other PRO measures (PROMs), has to date focused on the symptomatic toxicities of chemotherapy, radiotherapy and targeted therapy across multiple tumour types [15]. As such, the anticipated symptomatic toxicities associated with the use of ICIs, like vitiligo and xerophthalmia, may not be fully addressed by the current version of the PRO-CTCAEä Item Library. The uniqueness of IrAEs associated with ICIs raises questions about the suitability of existing PROMs to capture ICI-related symptomatic toxicities, and a recent review has identified gaps in the content validity of existing PROMs, including the PRO-CTCAEä [20] (p), [21,22]. Consequentially, several clinical trials have reported the use of multiple PROMs, combining cancer-specific and disease-specific instruments, to address the large spectrum of IrAEs [20]. The highly variable and heterogeneous profile of symptomatic IrAEs experienced by patients receiving ICI treatment also presents a challenge in defining a parsimonious and acceptable PRO strategy that both limits patient burden and is sufficiently comprehensive [19]. This underscores the need to systematically appraise the content validity of the symptom terms included in the PRO-CTCAEä Item Library with respect to the toxicities commonly associated with ICIs, identify candidate symptom terms for expansion of the library, and derive consensus among experts on core domains to be addressed when monitoring for symptomatic IrAEs.
The aim of this study was to reach a consensus on the PRO-CTCAEä Symptom Terms relevant for cancer patients treated with ICIs and gather preliminary expert opinions on additional PRO symptom terms that could be related to symptomatic ICI toxicity. Additionally, we gathered expert consensus on the importance of each symptom term when monitoring patients receiving ICI therapy, thereby identifying a core set of symptoms to be evaluated in that population.

Material and methods
We applied a Delphi technique [23,24] as part of a larger study on the use of electronic patient-reported outcomes monitoring of melanoma and lung cancer patients treated with ICIs.

Expert recruitment
When recruiting an expert panel, we aimed to represent European physicians, nurses and patients, experienced in at least two of four fields of expertise: immunooncology, lung cancer, melanoma and PROs. For physicians and nurses, we reviewed relevant publications and presentations in the medical field across these domains and contacted the experts directly. In particular we aimed to recruit clinically active staff in university hospitals with at least two years of experience and renowned researchers in their field. We identified patient advocates serving in leadership roles of national and international patient advocacy groups related to the aforementioned fields of expertise and with experience in dealing with ICIs and their side effects.
A total of 15 experts (N Z 8 physicians; N Z 6 nurses; N Z 2 patient advocates) were identified through convenience sampling and contacted by e-mail. Experts were sent a plan of the Delphi study that included its background and goals, the number of rounds planned, and how the data would be used.

Delphi planning
This Delphi included four rounds during which experts replied to an online questionnaire. They were e-mailed a secure link and required to log in to a personal account in order to view and reply to the questionnaires. Google Forms and LimeSurvey were used to develop the online questionnaires. STATA â 14 and Microsoft Excel 2016 were used to analyse the data. Through the duration of the Delphi, experts were able to contact the investigators for any questions regarding the online questionnaires, including technical support.
Four investigators collected, reviewed and anonymised expert's answers before sharing them with the other experts that were blinded. Results from each round were presented in a word document and shared via e-mail with all experts. The four investigators were not blinded as they were required to contact the experts to follow up on replies needing further clarification. The overall process is illustrated in Fig. 1.

Delphi round 1
The first round of the Delphi aimed to identify relevant PRO-CTCAEä Symptom Terms and collect experts' suggestions on additional symptoms to monitor in the aforementioned population. The 80 symptom terms were grouped according to the categories defined in the PRO-CTCAEä Item Library Quick Guide [25].
Experts were asked to classify each term as 'Relevant', 'Not relevant' or 'Do not know'. A free-text option to add comments to their answers was provided. The consensus was set at 75% agreement, in accordance to the European Society of Medical Oncology's consensus meeting the guidelines [26]. If a term was considered not relevant to monitor in patients receiving ICIs by 75% of all experts, it was excluded from the following round. Experts were also asked to add any additional symptoms not covered by the PRO-CTCAEä that they deemed relevant for monitoring adverse events in this patient population. Suggested additional symptom terms were assessed by the investigators according to predefined requirements (evidence that the symptom had been observed in the patient population and that it was likely related to ICIs; no redundancy with existing PRO-CTCAEä terms; a clear description of the symptom; and amenable to self-reporting) and submitted to the following round. If a Symptom Term did not meet these criteria, the suggesting expert was approached by e-mail or phone to clarify what was intended to be addressed. Investigators would draft an assessment to be reviewed separately by the remaining experts in the following round.

Delphi round 2
In the second round, experts were asked to assess the importance of monitoring symptoms represented by the PRO-CTCAEä items found relevant in the previous round and new suggested ones. For assessing their importance, experts were advised to consider: (i) the likelihood that the symptom can be meaningfully selfreported by the patient; (ii) the likelihood that the symptom is related to an IrAE and (iii) how consequential the resulting IrAE would be to the patient. Importance was rated on a 5-point Likert scale ranging from 1-'not important' to 5 'very important'.
Three levels of importance were defined by grouping ratings together: level 1 included ratings 4 'rather important' and 5 'very important'; level 2 included rating 2 'slightly important' and 3 'moderately important'; and the remainder 'not important' were level 3. The consensus was defined as 75% agreement in one of the three levels of importance.
Furthermore, as part of this second round, experts were asked to review and validate the investigators' decision of rejection or validation of each of the new symptoms proposed in the previous round. Experts could choose 'Agree', 'Disagree' or 'Undecided'. If experts expressed disagreement or were undecided, they were encouraged to provide a rationale for their opinion using a free text field.

Delphi round 3
The third round of the Delphi shared the same goal and was structurally similar to the second round, featuring the same 5-point Likert scale with an added ability to comment on each of the answers. Experts were able to see the overall results of the previous round as they replied to each question and were encouraged to express their views on the previous results. The intent was to understand why there was no consensus in certain PRO terms.

Delphi round 4
The fourth round of the Delphi featured a questionnaire with the same structure as that of rounds 2 and 3. Experts were invited to a real-time online discussion after they consented to being unmasked to other experts. Experts who were unavailable for the online discussion were given the option to reply to the questionnaire, with the written comments of the live discussion, at a later date.
The live discussion was moderated by the investigators. Each of the participating experts was able to access the same questionnaire and reply to it at the same time. In addition to expressing their opinions verbally during the live discussion, experts were encouraged to write them down in the questionnaire.

Ethical considerations
Since no medical data were collected, this study is not covered by the Human Research Act and did not require ethical approval. All experts consented to participate in all expert rounds in written form.

Expert panel
The Delphi process took place between July 2019 and May 2020. Eleven experts were available and consented to participate in the Delphi by e-mail. All experts participated in rounds one to three, and nine experts participated in the final round (n Z 1 physician and n Z 1 nurse were unavailable) due to decreased availability during the SARS-CoV-2 pandemic. All experts had training and experience relevant to at least two fields of expertise, as described in Table 1.

PRO-CTCAEä symptom terms
In round one, all (n Z 80) PRO-CTCAEä Symptom Terms were considered relevant to the target population. With respect to the importance to monitor, a consensus was reached for 65/80 (81%) of the PRO-CTCAEä Symptom Terms. Among the Symptom Terms considered rather or very important (n Z 30), 23% belonged to the gastro-intestinal subgroup, followed by pain (13%), respiratory (10%), cutaneous terms (10%). In the slightly or moderately important category (n Z 33), 24% of the terms were cutaneous symptoms, followed by gynecologic/urinary, sexual and miscellaneous terms at 15% each. Two terms were considered 'not important'. Overall results are portrayed in Fig. 1. The percentage of agreement by the level of importance for each symptom term is presented in Table 2. An infographic listing the terms ordered by level of importance is available for PRO-CTCAEä Symptom Terms (Fig. 2) and the terms suggested by experts (Fig. 3).
Oral, cutaneous and gynecologic/urinary terms, each make up 20% of the 15 PRO-CTCAEä Symptom Terms where no consensus on importance was achieved. For the gynecologic/urinary terms, in particular, experts expressed difficulty in relating the occurrence of these symptoms to the immune-checkpoint blockade. They also noted that several terms in this subgroup and the sexual terms subgroup were likely underreported in the literature, as they may not often be discussed with patients.

New PRO symptom terms
In round one, experts suggested 60 new symptom terms, of which six were rejected by the investigators for the following round, with unanimous agreement from the experts. These included five symptom terms that could not be meaningfully captured by patient self-report ('Arrhythmia', 'Arthritis', 'Asthenia', 'Cellulitis' and 'Sudden increase in caries') and one ('Symptom-related Fatigue') that was considered difficult to differentiate from the existing PRO-CTCAEä Symptom Term 'Fatigue'. To address 'Arthritis' and 'Cellulitis', experts suggested and validated two new terms: 'Swelling of the joints' and 'Heat or burning sensation in an area of the body', respectively. Thus, 56 new symptom terms were rated on the importance to monitor.
Expert consensus was reached in 47 of the 56 new symptom terms. Of these, 62% (n Z 29) were considered 'rather' or 'very important', 36% (n Z 17) were classed as 'slightly' or 'moderately important', and one term 'not important'. The number of items per Delphi round is illustrated in Fig. 1. Expert consensus for each term is described in Table 3.
Consensus on importance was not achieved in nine (14%) of the new terms. These were among the most discussed. Abdominal cramps were among the terms where experts considered that complete contextual information was crucial to determine its importance. Specifically, it would be considered increasingly important as other symptoms were manifested, like diarrhoea or abdominal pain, or if confounding variables like menstrual pain were present.
Other terms like 'Infusion-related reaction' were considered either too broad to be meaningfully assessed by patient self-report or were more amenable to direct observation by clinicians during infusion. Experts also noted that some of the suggested PRO terms, like 'overalertness', were more likely related to the corticosteroid treatment for the IrAEs than a symptom of ICI toxicity. Additional comments from experts on symptom terms can be found in Appendix Table A1.

Discussion
Experts reached a consensus on the salience of all (n Z 80) terms in the PRO-CTCAEä Item Library for surveillance for symptomatic adverse events in cancer patients being treated with ICIs. A consensus was also reached on the importance of these terms, with 30 terms endorsed as very important by 75% or more of the Delphi panellists. Among the new terms suggested by experts, 56 new PRO terms were proposed as potentially salient in capturing side effects of ICIs, and a consensus was reached that 45 of these terms are candidates for item development to expand the PRO-CTCAEä Item Library for patients treated with ICI therapy.
Several caveats should be considered in interpreting these study findings. While the international expert panel reflected a diversity of professional experiences and disciplinary perspectives, the panel was small and drawn predominantly from Switzerland (five out of 11 experts). Expert roles were not equally represented, with only one patient advocate participating. While differences in expertise may increase the challenge of reaching    consensus, there were no clear associations between expert background and deviation from consensus, although this can be due to the small sample size. Our findings should be replicated and extended with a larger, more balanced and more geographically diverse panel, including patients that are receiving or have received immune checkpoint inhibitors. We nevertheless maintain that diversity in expertise enriched the discussion, bringing together multiple perspectives and decreasing the likelihood of an authority bias. The number of additional symptom terms experts identified for inclusion extends results of a prior systematic review [20] and provides preliminary evidence that the current PRO-CTCAEä Item Library should be expanded in order to capture the full spectrum of symptomatic toxicities associated with ICIs. The toxicity profile of ICIs has been described as heterogeneous, pleomorphic and more variable than that of radiotherapy, chemotherapy, molecularly targeted and combination regimens [2]. This challenged experts in the interpretation of what existing PROs represent -symptomatic adverse events experienced in association with ICI treatment can be indicators of off-target effects rather than being related to anti-tumour immunity. The current PRO-CTCAEä version was conceived with symptoms related to chemotherapy, radiotherapy and targeted therapies in mind, which may explain how more complex IrAEs elude existing symptom terms [15]. It is important to consider how the PRO-CTCAEä is derived from the constantly evolving CTCAE, which has been updated to reflect some IrAEs. Some of the newly suggested PRO items do, in fact, reflect CTCAE terms included in version 5.0, such as photophobia. While updates to the current PRO-CTCAEä item library are inbound, the use of some of the existing symptom terms will remain challenging in the context of ICIs. This is illustrated by some unexpected results on specific symptoms, such as the unanimous assessment of 'radiation skin reaction' as level 2 importance. Experts argued such a symptom could signal a broader autoimmune reaction. While there have been reports of ICIinduced radiation recall dermatitis [27e29], it can be questioned if this item would retain its original meaning to patients who were not treated with radiotherapy. A large item library can pose important feasibility challenges as the patient burden is increased. The defined levels of importance may inform new ways to present patients with a large library of symptom terms, particularly when paired with computer-adaptive questionnaires and artificial intelligence. Level 1 terms could be used as a standard starting point, and terms from other levels could be called upon according to potential symptom associations or clusters. As item libraries are expanded to account for the diversity of ICI-related symptomatic IrAEs, these tools will become essential to balance the patient burden and the exhaustiveness of symptom-related PROMs.
The aforementioned heterogeneity of the adverse effects that may be experienced by patients receiving ICI therapy makes self-reporting of symptomatic IrAEs complex, as illustrated by new terms such as 'depressive mood', 'impaired distance assessment' and 'walking difficulties'. Experts' comments on these and other terms can be found in Supplementary Table A1. While these examples require further refinement to better clarify what they intend to assess, they raise questions perpetrating the use of highly specific symptom terms as the most comprehensive approach to best reflect the patient experience regarding IrAEs. Experts were challenged to identify symptomatic components of clinical syndromes (e.g. pneumonitis, myasthenia gravis, iritis) that may have aspects that can be captured through a PRO (e.g. cough, changes in voice quality, visual disturbance) but which can only be identified precisely by the inclusion of clinician adverse event reports or information derived from diagnostic or laboratory testing.
Some new suggested PRO terms could be interpreted as redundant when considering existing PRO-CTCAEä Symptom Terms, as is the case between 'Sad' (PRO-CTCAEä) and the expert suggestion 'Depressive mood', or 'anxious' (PRO-CTCAEä) and 'worries' (expert suggestion). This further illustrates the aforementioned complexity of symptoms, as experts appeared to have different representations of the same term. While these results provided some support for the content validity of the PRO CTCAEä and resulted in a preliminary set of salient symptomatic adverse events related to the use of ICIs, broader international agreement and further validation, including patient involvement, is needed to continue to validate our initial findings. Further mixed methods studies examining the experiences of adverse effects of ICI are needed to develop and test additional PRO-CTCAEä items and to identify efficient, interpretable and meaningful approaches to profile symptomatic adverse effects of ICI therapies.

Funding
This project was supported by the Swiss Institute for Experimental Cancer Research (ISREC) Foundation. from MPNE, WECAN, outside the submitted work; Dr. Tolstrup has nothing to disclose; Dr. Pappot has nothing to disclose; Dr. Aspeslagh reports and has received speaker fees from Roche, BMS, Novartis, Merck and Pfizer in the last 36 months; Dr. Rogiers has nothing to disclose; Dr. Neyns reports grants and personal fees from Novartis, grants and personal fees from Pfizer, personal fees from BMS, personal fees from MSD, outside the submitted work; Dr. Haanen reports grants and other from BMS, grants and other from MSD, grants and other from Novartis, grants and other from BioNTech, other from Achilles Tx, grants and other from Amgen, other from GSK, other from Immunocore, other from Ipsen, other from Merck Serono, other from Molecular Partners, personal fees from Neogene Tx, other from Pfizer, other from Roche/ Genentech, other from Sanofi, other from Seattle Genetics, other from Third Rock Ventures, other from Vaximm, outside the submitted work; Dr. Mitchell has nothing to disclose; Dr. Addeo reports personal fees from BMS, personal fees from MSD, personal fees from AstraZeneca, personal fees from Pfizer, personal fees from Roche, personal fees from Boehringer, personal fees from Ely-Lilly, outside the submitted work; Dr. Michielin reports grants and personal fees from BMS, grants and personal fees from MSD, personal fees from Roche, personal fees from Novartis, grants and personal fees from Pierre-Fabre, grants and personal fees from Amgen, personal fees from GSL, grants from Merck, outside the submitted work; Dr. Eicher reports grants from Fondation Recherche Cancer ISREC, during the conduct of the study; grants and personal fees from Roche, grants and personal fees from BMS, grants from Kaiku Health, personal fees from VIFOR, outside the submitted work.