Artificial intelligence–powered programmed death ligand 1 analyser reduces interobserver variation in tumour proportion score for non–small cell lung cancer with better prediction of immunotherapy response


      • Inter-observer bias of PD-L1 reading limits prediction of clinical outcome of NSCLC.
      • We developed an AI model to detect PD-L1 expression in tumour and to calculate TPS.
      • Discrepancy among pathologists on TPS interpretation was reduced by AI assistance.
      • AI-assisted TPS reading leads to better prediction of therapeutic outcome of ICI.



      Manual evaluation of programmed death ligand 1 (PD-L1) tumour proportion score (TPS) by pathologists is associated with interobserver bias.


      This study explored the role of artificial intelligence (AI)-powered TPS analyser in minimisation of interobserver variation and enhancement of therapeutic response prediction.


      A prototype model of an AI-powered TPS analyser was developed with a total of 802 non–small cell lung cancer (NSCLC) whole-slide images. Three independent board-certified pathologists labelled PD-L1 TPS in an external cohort of 479 NSCLC slides. For cases of disagreement between each pathologist and the AI model, the pathologists were asked to revise the TPS grade (<1%, 1%–49% and ≥50%) with AI assistance. The concordance rates among the pathologists with or without AI assistance and the effect of the AI-assisted revision on clinical outcome upon immune checkpoint inhibitor (ICI) treatment were evaluated.


      Without AI assistance, pathologists concordantly classified TPS in 81.4% of the cases. They revised their initial interpretation by using the AI model for the disagreement cases between the pathologist and the AI model (N = 91, 93 and 107 for each pathologist). The overall concordance rate among the pathologists was increased to 90.2% after the AI assistance (P < 0.001). A reduction in hazard ratio for overall survival and progression-free survival upon ICI treatment was identified in the TPS subgroups after the AI-assisted TPS revision.


      The AI-powered TPS analyser assistance improves the pathologists’ consensus of reading and prediction of the therapeutic response, raising a possibility of standardised approach for the accurate interpretation.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to European Journal of Cancer
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Park K.
        • Vansteenkiste J.
        • Lee K.H.
        • et al.
        Pan-Asian adapted ESMO clinical practice guidelines for the management of patients with locally-advanced unresectable non-small-cell lung cancer: a KSMO-ESMO initiative endorsed by CSCO, ISMPO, JSMO, MOS, SSO and TOS.
        Ann Oncol. 2020; 31: 191-201
        • Herbst R.S.
        • Baas P.
        • Kim D.W.
        • et al.
        Pembrolizumab versus docetaxel for previously treated, PD-L1-positive, advanced non-small-cell lung cancer (KEYNOTE-010): a randomised controlled trial.
        Lancet. 2016; 387: 1540-1550
        • Horn L.
        • Spigel D.R.
        • Vokes E.E.
        • et al.
        Nivolumab versus docetaxel in previously treated patients with advanced non-small-cell lung cancer: two-year outcomes from two randomized, open-label, phase III trials (CheckMate 017 and CheckMate 057).
        J Clin Oncol. 2017; 35: 3924-3933
        • Reck M.
        • Rodriguez-Abreu D.
        • Robinson A.G.
        • et al.
        Pembrolizumab versus chemotherapy for PD-L1-positive non-small-cell lung cancer.
        N Engl J Med. 2016; 375: 1823-1833
        • Rittmeyer A.
        • Barlesi F.
        • Waterkamp D.
        • et al.
        Atezolizumab versus docetaxel in patients with previously treated non-small-cell lung cancer (OAK): a phase 3, open-label, multicentre randomised controlled trial.
        Lancet. 2017; 389: 255-265
        • Gibney G.T.
        • Weiner L.M.
        • Atkins M.B.
        Predictive biomarkers for checkpoint inhibitor-based immunotherapy.
        Lancet Oncol. 2016; 17: e542-e551
        • Aguiar P.N.
        • De Mello R.A.
        • Hall P.
        • Tadokoro H.
        • de Lima G.
        PD-L1 expression as a predictive biomarker in advanced non-small-cell lung cancer: updated survival data.
        Immunotherapy. 2017; 9: 499-506
        • Borghaei H.
        • Paz-Ares L.
        • Horn L.
        • et al.
        Nivolumab versus docetaxel in advanced nonsquamous non-small-cell lung cancer.
        N Engl J Med. 2015; 373: 1627-1639
        • Fehrenbacher L.
        • Spira A.
        • Ballinger M.
        • et al.
        Atezolizumab versus docetaxel for patients with previously treated non-small-cell lung cancer (POPLAR): a multicentre, open-label, phase 2 randomised controlled trial.
        Lancet. 2016; 387: 1837-1846
        • Garon E.B.
        • Rizvi N.A.
        • Hui R.
        • et al.
        Pembrolizumab for the treatment of non-small-cell lung cancer.
        N Engl J Med. 2015; 372: 2018-2028
        • Brunnstrom H.
        • Johansson A.
        • Westbom-Fremer S.
        • et al.
        PD-L1 immunohistochemistry in clinical diagnostics of lung cancer: inter-pathologist variability is higher than assay variability.
        Mod Pathol. 2017; 30: 1411-1421
        • Chang S.
        • Park H.K.
        • Choi Y.L.
        • Jang S.J.
        Interobserver reproducibility of PD-L1 biomarker in non-small cell lung cancer: a multi-institutional study by 27 pathologists.
        J Pathol Transl Med. 2019; 53: 347-353
        • Cooper W.A.
        • Russell P.A.
        • Cherian M.
        • et al.
        Intra- and interobserver reproducibility assessment of PD-L1 biomarker in non-small cell lung cancer.
        Clin Cancer Res. 2017; 23: 4569-4577
        • He J.
        • Baxter S.L.
        • Xu J.
        • Xu J.
        • Zhou X.
        • Zhang K.
        The practical implementation of artificial intelligence technologies in medicine.
        Nat Med. 2019; 25: 30-36
        • Skrede O.J.
        • De Raedt S.
        • Kleppe A.
        • et al.
        Deep learning for prediction of colorectal cancer outcome: a discovery and validation study.
        Lancet. 2020; 395: 350-360
        • Lu M.Y.
        • Chen T.Y.
        • Williamson D.F.
        • et al.
        AI-based pathology predicts origins for cancers of unknown primary.
        Nature. 2021; 594: 106-110
        • Acs B.
        • Rantalainen M.
        • Hartman J.
        Artificial intelligence as the next step towards precision pathology.
        J Intern Med. 2020; 288: 62-81
        • Niazi M.K.K.
        • Parwani A.V.
        • Gurcan M.N.
        Digital pathology and artificial intelligence.
        Lancet Oncol. 2019; 20: e253-e261
        • Bulten W.
        • Balkenhol M.
        • Belinga J.A.
        • et al.
        Artificial intelligence assistance significantly improves Gleason grading of prostate biopsies by pathologists.
        Mod Pathol. 2021; 34: 660-671
        • Foersch S.
        • Eckstein M.
        • Wagner D.C.
        • et al.
        Deep learning for diagnosis and survival prediction in soft tissue sarcoma.
        Ann Oncol. 2021; 32: 1178-1187
        • Park J.
        • Jang B.G.
        • Kim Y.W.
        • et al.
        A prospective validation and observer performance study of a deep learning algorithm for pathologic diagnosis of gastric tumors in endoscopic biopsies.
        Clin Cancer Res. 2021; 27: 719-728
        • Steiner D.F.
        • MacDonald R.
        • Liu Y.
        • et al.
        Impact of deep learning assistance on the histopathologic review of lymph nodes for metastatic breast cancer.
        Am J Surg Pathol. 2018; 42: 1636-1646
        • Inge L.
        • Dennis E.
        Development and applications of computer image analysis algorithms for scoring of PD-L1 immunohistochemistry.
        Immuno-Oncol Technol. 2020; 6: 2-8
        • Humphries M.P.
        • McQuaid S.
        • Craig S.G.
        • et al.
        Critical appraisal of programmed death ligand 1 reflex diagnostic testing: current standards and future opportunities.
        J Thorac Oncol. 2019; 14: 45-53
        • Koelzer V.H.
        • Gisler A.
        • Hanhart J.C.
        • et al.
        Digital image analysis improves precision of PD-L1 scoring in cutaneous melanoma.
        Histopathology. 2018; 73: 397-406
        • Kim H.N.
        • Jang J.
        • Heo Y.J.
        • et al.
        PD-L1 expression in gastric cancer determined by digital image analyses: pitfalls and correlation with pathologist interpretation.
        Virchows Arch. 2020; 476: 243-250
        • Pan B.
        • Kang Y.
        • Jin Y.
        • et al.
        Automated tumor proportion scoring for PD-L1 expression based on multistage ensemble strategy in non-small cell lung cancer.
        J Transl Med. 2021; 19: 249
        • Wang X.
        • Wang L.
        • Bu H.
        • et al.
        How can artificial intelligence models assist PD-L1 expression scoring in breast cancer: results of multi-institutional ring studies.
        NPJ breast cancer. 2021; 7: 61
        • Kapil A.
        • Meier A.
        • Zuraw A.
        • et al.
        Deep semi supervised generative learning for automated tumor proportion scoring on NSCLC tissue needle biopsies.
        Sci Rep. 2018; 8: 17343
        • Widmaier M.
        • Wiestler T.
        • Walker J.
        • et al.
        Comparison of continuous measures across diagnostic PD-L1 assays in non-small cell lung cancer using automated image analysis.
        Mod Pathol. 2020; 33: 380-390
        • Taylor C.R.
        • Jadhav A.P.
        • Gholap A.
        • et al.
        A multi-institutional study to evaluate automated whole slide scoring of immunohistochemistry for assessment of programmed death-ligand 1 (PD-L1) expression in non-small cell lung cancer.
        Appl Immunohistochem Mol Morphol. 2019; 27: 263-269
        • Liu J.
        • Zheng Q.
        • Mu X.
        • et al.
        Automated tumor proportion score analysis for PD-L1 (22C3) expression in lung squamous cell carcinoma.
        Sci Rep. 2021; 11: 15907
        • Wu J.
        • Liu C.
        • Liu X.
        • et al.
        Artificial intelligence-assisted system for precision diagnosis of PD-L1 expression in non-small cell lung cancer.
        Mod Pathol. 2021; (Article in Press.
        • Ren S.
        • He K.
        • Girshick R.
        • Sun J.
        • Faster R.-C.N.N.
        Towards real-time object detection with region proposal networks.
        IEEE Trans Pattern Anal Mach Intell. 2017; 39: 1137-1149
      1. Agilent technologies, Inc. PD-L1 IHC 22C3 pharmDx interpretation manual – NSCLC. Accessed December 30, 2021.

        • Eisenhauer E.A.
        • Therasse P.
        • Bogaerts J.
        • et al.
        New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1).
        Eur J Cancer. 2009; 45: 228-247
        • Topol E.J.
        High-performance medicine: the convergence of human and artificial intelligence.
        Nat Med. 2019; 25: 44-56
        • Reyes M.
        • Meier R.
        • Pereira S.
        • et al.
        On the interpretability of artificial intelligence in radiology: challenges and opportunities.
        Radiol Artif Intell. 2020; 2: e190043
        • Pagni F.
        • Malapelle U.
        • Doglioni C.
        • et al.
        Digital pathology and PD-L1 testing in non small cell lung cancer: a workshop record.
        Cancers. 2020; 12: 1800