Advertisement

Artificial intelligence–powered programmed death ligand 1 analyser reduces interobserver variation in tumour proportion score for non–small cell lung cancer with better prediction of immunotherapy response

      Highlights

      • Inter-observer bias of PD-L1 reading limits prediction of clinical outcome of NSCLC.
      • We developed an AI model to detect PD-L1 expression in tumour and to calculate TPS.
      • Discrepancy among pathologists on TPS interpretation was reduced by AI assistance.
      • AI-assisted TPS reading leads to better prediction of therapeutic outcome of ICI.

      Abstract

      Background

      Manual evaluation of programmed death ligand 1 (PD-L1) tumour proportion score (TPS) by pathologists is associated with interobserver bias.

      Objective

      This study explored the role of artificial intelligence (AI)-powered TPS analyser in minimisation of interobserver variation and enhancement of therapeutic response prediction.

      Methods

      A prototype model of an AI-powered TPS analyser was developed with a total of 802 non–small cell lung cancer (NSCLC) whole-slide images. Three independent board-certified pathologists labelled PD-L1 TPS in an external cohort of 479 NSCLC slides. For cases of disagreement between each pathologist and the AI model, the pathologists were asked to revise the TPS grade (<1%, 1%–49% and ≥50%) with AI assistance. The concordance rates among the pathologists with or without AI assistance and the effect of the AI-assisted revision on clinical outcome upon immune checkpoint inhibitor (ICI) treatment were evaluated.

      Results

      Without AI assistance, pathologists concordantly classified TPS in 81.4% of the cases. They revised their initial interpretation by using the AI model for the disagreement cases between the pathologist and the AI model (N = 91, 93 and 107 for each pathologist). The overall concordance rate among the pathologists was increased to 90.2% after the AI assistance (P < 0.001). A reduction in hazard ratio for overall survival and progression-free survival upon ICI treatment was identified in the TPS subgroups after the AI-assisted TPS revision.

      Conclusion

      The AI-powered TPS analyser assistance improves the pathologists’ consensus of reading and prediction of the therapeutic response, raising a possibility of standardised approach for the accurate interpretation.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to European Journal of Cancer
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Park K.
        • Vansteenkiste J.
        • Lee K.H.
        • et al.
        Pan-Asian adapted ESMO clinical practice guidelines for the management of patients with locally-advanced unresectable non-small-cell lung cancer: a KSMO-ESMO initiative endorsed by CSCO, ISMPO, JSMO, MOS, SSO and TOS.
        Ann Oncol. 2020; 31: 191-201https://doi.org/10.1016/j.annonc.2019.10.026
        • Herbst R.S.
        • Baas P.
        • Kim D.W.
        • et al.
        Pembrolizumab versus docetaxel for previously treated, PD-L1-positive, advanced non-small-cell lung cancer (KEYNOTE-010): a randomised controlled trial.
        Lancet. 2016; 387: 1540-1550https://doi.org/10.1016/S0140-6736(15)01281-7
        • Horn L.
        • Spigel D.R.
        • Vokes E.E.
        • et al.
        Nivolumab versus docetaxel in previously treated patients with advanced non-small-cell lung cancer: two-year outcomes from two randomized, open-label, phase III trials (CheckMate 017 and CheckMate 057).
        J Clin Oncol. 2017; 35: 3924-3933https://doi.org/10.1200/JCO.2017.74.3062
        • Reck M.
        • Rodriguez-Abreu D.
        • Robinson A.G.
        • et al.
        Pembrolizumab versus chemotherapy for PD-L1-positive non-small-cell lung cancer.
        N Engl J Med. 2016; 375: 1823-1833https://doi.org/10.1056/NEJMoa1606774
        • Rittmeyer A.
        • Barlesi F.
        • Waterkamp D.
        • et al.
        Atezolizumab versus docetaxel in patients with previously treated non-small-cell lung cancer (OAK): a phase 3, open-label, multicentre randomised controlled trial.
        Lancet. 2017; 389: 255-265https://doi.org/10.1016/S0140-6736(16)32517-X
        • Gibney G.T.
        • Weiner L.M.
        • Atkins M.B.
        Predictive biomarkers for checkpoint inhibitor-based immunotherapy.
        Lancet Oncol. 2016; 17: e542-e551https://doi.org/10.1016/S1470-2045(16)30406-5
        • Aguiar P.N.
        • De Mello R.A.
        • Hall P.
        • Tadokoro H.
        • de Lima G.
        PD-L1 expression as a predictive biomarker in advanced non-small-cell lung cancer: updated survival data.
        Immunotherapy. 2017; 9: 499-506https://doi.org/10.2217/imt-2016-0150
        • Borghaei H.
        • Paz-Ares L.
        • Horn L.
        • et al.
        Nivolumab versus docetaxel in advanced nonsquamous non-small-cell lung cancer.
        N Engl J Med. 2015; 373: 1627-1639https://doi.org/10.1056/NEJMoa1507643
        • Fehrenbacher L.
        • Spira A.
        • Ballinger M.
        • et al.
        Atezolizumab versus docetaxel for patients with previously treated non-small-cell lung cancer (POPLAR): a multicentre, open-label, phase 2 randomised controlled trial.
        Lancet. 2016; 387: 1837-1846https://doi.org/10.1016/S0140-6736(16)00587-0
        • Garon E.B.
        • Rizvi N.A.
        • Hui R.
        • et al.
        Pembrolizumab for the treatment of non-small-cell lung cancer.
        N Engl J Med. 2015; 372: 2018-2028https://doi.org/10.1056/NEJMoa1501824
        • Brunnstrom H.
        • Johansson A.
        • Westbom-Fremer S.
        • et al.
        PD-L1 immunohistochemistry in clinical diagnostics of lung cancer: inter-pathologist variability is higher than assay variability.
        Mod Pathol. 2017; 30: 1411-1421https://doi.org/10.1038/modpathol.2017.59
        • Chang S.
        • Park H.K.
        • Choi Y.L.
        • Jang S.J.
        Interobserver reproducibility of PD-L1 biomarker in non-small cell lung cancer: a multi-institutional study by 27 pathologists.
        J Pathol Transl Med. 2019; 53: 347-353https://doi.org/10.4132/jptm.2019.09.29
        • Cooper W.A.
        • Russell P.A.
        • Cherian M.
        • et al.
        Intra- and interobserver reproducibility assessment of PD-L1 biomarker in non-small cell lung cancer.
        Clin Cancer Res. 2017; 23: 4569-4577https://doi.org/10.1158/1078-0432.CCR-17-0151
        • He J.
        • Baxter S.L.
        • Xu J.
        • Xu J.
        • Zhou X.
        • Zhang K.
        The practical implementation of artificial intelligence technologies in medicine.
        Nat Med. 2019; 25: 30-36https://doi.org/10.1038/s41591-018-0307-0
        • Skrede O.J.
        • De Raedt S.
        • Kleppe A.
        • et al.
        Deep learning for prediction of colorectal cancer outcome: a discovery and validation study.
        Lancet. 2020; 395: 350-360https://doi.org/10.1016/S0140-6736(19)32998-8
        • Lu M.Y.
        • Chen T.Y.
        • Williamson D.F.
        • et al.
        AI-based pathology predicts origins for cancers of unknown primary.
        Nature. 2021; 594: 106-110https://doi.org/10.1038/s41586-021-03512-4
        • Acs B.
        • Rantalainen M.
        • Hartman J.
        Artificial intelligence as the next step towards precision pathology.
        J Intern Med. 2020; 288: 62-81https://doi.org/10.1111/joim.13030
        • Niazi M.K.K.
        • Parwani A.V.
        • Gurcan M.N.
        Digital pathology and artificial intelligence.
        Lancet Oncol. 2019; 20: e253-e261https://doi.org/10.1016/S1470-2045(19)30154-8
        • Bulten W.
        • Balkenhol M.
        • Belinga J.A.
        • et al.
        Artificial intelligence assistance significantly improves Gleason grading of prostate biopsies by pathologists.
        Mod Pathol. 2021; 34: 660-671https://doi.org/10.1038/s41379-020-0640-y
        • Foersch S.
        • Eckstein M.
        • Wagner D.C.
        • et al.
        Deep learning for diagnosis and survival prediction in soft tissue sarcoma.
        Ann Oncol. 2021; 32: 1178-1187https://doi.org/10.1016/j.annonc.2021.06.007
        • Park J.
        • Jang B.G.
        • Kim Y.W.
        • et al.
        A prospective validation and observer performance study of a deep learning algorithm for pathologic diagnosis of gastric tumors in endoscopic biopsies.
        Clin Cancer Res. 2021; 27: 719-728https://doi.org/10.1158/1078-0432.CCR-20-3159
        • Steiner D.F.
        • MacDonald R.
        • Liu Y.
        • et al.
        Impact of deep learning assistance on the histopathologic review of lymph nodes for metastatic breast cancer.
        Am J Surg Pathol. 2018; 42: 1636-1646https://doi.org/10.1097/PAS.0000000000001151
        • Inge L.
        • Dennis E.
        Development and applications of computer image analysis algorithms for scoring of PD-L1 immunohistochemistry.
        Immuno-Oncol Technol. 2020; 6: 2-8https://doi.org/10.1016/j.iotech.2020.04.001
        • Humphries M.P.
        • McQuaid S.
        • Craig S.G.
        • et al.
        Critical appraisal of programmed death ligand 1 reflex diagnostic testing: current standards and future opportunities.
        J Thorac Oncol. 2019; 14: 45-53https://doi.org/10.1016/j.jtho.2018.09.025
        • Koelzer V.H.
        • Gisler A.
        • Hanhart J.C.
        • et al.
        Digital image analysis improves precision of PD-L1 scoring in cutaneous melanoma.
        Histopathology. 2018; 73: 397-406https://doi.org/10.1111/his.13528
        • Kim H.N.
        • Jang J.
        • Heo Y.J.
        • et al.
        PD-L1 expression in gastric cancer determined by digital image analyses: pitfalls and correlation with pathologist interpretation.
        Virchows Arch. 2020; 476: 243-250https://doi.org/10.1007/s00428-019-02653-2
        • Pan B.
        • Kang Y.
        • Jin Y.
        • et al.
        Automated tumor proportion scoring for PD-L1 expression based on multistage ensemble strategy in non-small cell lung cancer.
        J Transl Med. 2021; 19: 249https://doi.org/10.1186/s12967-021-02898-z
        • Wang X.
        • Wang L.
        • Bu H.
        • et al.
        How can artificial intelligence models assist PD-L1 expression scoring in breast cancer: results of multi-institutional ring studies.
        NPJ breast cancer. 2021; 7: 61https://doi.org/10.1038/s41523-021-00268-y
        • Kapil A.
        • Meier A.
        • Zuraw A.
        • et al.
        Deep semi supervised generative learning for automated tumor proportion scoring on NSCLC tissue needle biopsies.
        Sci Rep. 2018; 8: 17343https://doi.org/10.1038/s41598-018-35501-5
        • Widmaier M.
        • Wiestler T.
        • Walker J.
        • et al.
        Comparison of continuous measures across diagnostic PD-L1 assays in non-small cell lung cancer using automated image analysis.
        Mod Pathol. 2020; 33: 380-390https://doi.org/10.1038/s41379-019-0349-y
        • Taylor C.R.
        • Jadhav A.P.
        • Gholap A.
        • et al.
        A multi-institutional study to evaluate automated whole slide scoring of immunohistochemistry for assessment of programmed death-ligand 1 (PD-L1) expression in non-small cell lung cancer.
        Appl Immunohistochem Mol Morphol. 2019; 27: 263-269https://doi.org/10.1097/PAI.0000000000000737
        • Liu J.
        • Zheng Q.
        • Mu X.
        • et al.
        Automated tumor proportion score analysis for PD-L1 (22C3) expression in lung squamous cell carcinoma.
        Sci Rep. 2021; 11: 15907https://doi.org/10.1038/s41598-021-95372-1
        • Wu J.
        • Liu C.
        • Liu X.
        • et al.
        Artificial intelligence-assisted system for precision diagnosis of PD-L1 expression in non-small cell lung cancer.
        Mod Pathol. 2021; (Article in Press. https://doi.org/10.1038/s41379-021-00904-9)
        • Ren S.
        • He K.
        • Girshick R.
        • Sun J.
        • Faster R.-C.N.N.
        Towards real-time object detection with region proposal networks.
        IEEE Trans Pattern Anal Mach Intell. 2017; 39: 1137-1149https://doi.org/10.1109/TPAMI.2016.2577031
      1. Agilent technologies, Inc. PD-L1 IHC 22C3 pharmDx interpretation manual – NSCLC. Accessed December 30, 2021. https://www.agilent.com/cs/library/usermanuals/public/29158_pd-l1-ihc-22C3-pharmdx-nsclc-interpretation-manual.pdf.

        • Eisenhauer E.A.
        • Therasse P.
        • Bogaerts J.
        • et al.
        New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1).
        Eur J Cancer. 2009; 45: 228-247https://doi.org/10.1016/j.ejca.2008.10.026
        • Topol E.J.
        High-performance medicine: the convergence of human and artificial intelligence.
        Nat Med. 2019; 25: 44-56https://doi.org/10.1038/s41591-018-0300-7
        • Reyes M.
        • Meier R.
        • Pereira S.
        • et al.
        On the interpretability of artificial intelligence in radiology: challenges and opportunities.
        Radiol Artif Intell. 2020; 2: e190043https://doi.org/10.1148/ryai.2020190043
        • Pagni F.
        • Malapelle U.
        • Doglioni C.
        • et al.
        Digital pathology and PD-L1 testing in non small cell lung cancer: a workshop record.
        Cancers. 2020; 12: 1800https://doi.org/10.3390/cancers12071800