Highlights
- •A multi-class CNN was developed using 25,773 clinical images.
- •An MRMC study was conducted to evaluate performance of CNN-assisted dermatologists.
- •CNN-assisted dermatologists achieved a higher accuracy and kappa than unassisted.
- •Dermatologists with less experience benefited more from CNN assistance.
Abstract
Background
Convolutional neural networks (CNNs) have demonstrated expert-level performance in
cutaneous tumour classification using clinical images, but most previous studies have
focused on dermatologist-versus-CNN comparisons rather than their combination. The
objective of our study was to evaluate the potential impact of CNN assistance on dermatologists
for clinical image interpretation.
Methods
A multi-class CNN was trained and validated using a dataset of 25,773 clinical images
comprising 10 categories of cutaneous tumours. The CNN's performance was tested on
an independent dataset of 2107 images. A total of 400 images (40 per category) were
randomly selected from the test dataset. A fully crossed, self-control, multi-reader
multi-case (MRMC) study was conducted to compare the performance of 18 board-certified
dermatologists (experience: 13/18 ≤ 10 years; 5/18＞10 years) in interpreting the 400
clinical images with or without CNN assistance.
Results
The CNN achieved an overall accuracy of 78.45% and kappa of 0.73 in the classification
of 10 types of cutaneous tumours on 2107 images. CNN-assisted dermatologists achieved
a higher accuracy (76.60% vs. 62.78%, P < 0.001) and kappa (0.74 vs. 0.59, P < 0.001)
than unassisted dermatologists in interpreting the 400 clinical images. Dermatologists
with less experience benefited more from CNN assistance. At the binary classification
level (malignant or benign), the sensitivity (89.56% vs. 83.21%, P < 0.001) and specificity
(87.90% vs. 80.92%, P < 0.001) of dermatologists with CNN assistance were also significantly
improved than those without.
Conclusions
CNN assistance improved dermatologist accuracy in interpreting cutaneous tumours and
could further boost the acceptance of this new technique.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to European Journal of CancerAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Deep convolutional neural networks for image classification: a comprehensive review.Neural Comput. 2017; 29: 2352-2449
- The practical implementation of artificial intelligence technologies in medicine.Nat Med. 2019; 25: 30-36
- A guide to deep learning in healthcare.Nat Med. 2019; 25: 24-29
- Keratinocytic skin cancer detection on the face using region-based convolutional neural network.JAMA Dermatol. 2020; 156: 29-37
- Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists.Ann Oncol. 2018; 29: 1836-1842
- Classification of the clinical images for benign and malignant cutaneous tumors using a deep learning algorithm.J Invest Dermatol. 2018; 138: 1529-1538
- Artificial intelligence in dermatology: a primer.J Invest Dermatol. 2020; 140: 1504-1512
- Deep-learning-based, computer-aided classifier developed with a small dataset of clinical images surpasses board-certified dermatologists in skin tumour diagnosis.Br J Dermatol. 2019; 180: 373-381
- Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: an open, web-based, international, diagnostic study.Lancet Oncol. 2019; 20: 938-947
- A convolutional neural network trained with dermoscopic images performed on par with 145 dermatologists in a clinical melanoma image classification task.Eur J Cancer. 2019; 111: 148-154
- A deep learning system for differential diagnosis of skin diseases.Nat Med. 2020; 26: 900-908
- High-performance medicine: the convergence of human and artificial intelligence.Nat Med. 2019; 25: 44-56
- Human-computer collaboration for skin cancer recognition.Nat Med. 2020; 26: 1229-1234
- Augmented intelligence dermatology: deep neural networks empower medical professionals in diagnosing skin cancer and predicting treatment options for 134 skin disorders.J Invest Dermatol. 2020; 140: 1753-1761
- Superior skin cancer classification by the combination of human and artificial intelligence.Eur J Cancer. 2019; 120: 114-121
- Artificial intelligence and its effect on dermatologists' accuracy in dermoscopic melanoma image classification: web-based survey study.J Med Internet Res. 2020; 22e18091
- Augmented decision-making for acral lentiginous melanoma detection using deep convolutional neural networks.J Eur Acad Dermatol Venereol. 2020; 34: 1842-1850
Clinical Performance Assessment: Considerations for Computer-Assisted Detection Devices Applied to Radiology Images and Radiology Device Data in-Premarket Notification (510(k)) https://www.fda.gov/regulatory-information/search-fda-guidance-documents/clinical-performance-assessment-considerations-computer-assisted-detection-devices-applied-radiology.
- Statistical methods for evidence-based medicine: the diagnostic test. Part I.Minerva Anestesiol. 2008; 74: 431-437
- Statistical methods in diagnostic medicine.in: Biometrics. vol. 59. Wiley, New York2002: 203-204 (2015)
- What's the control in studies measuring the effect of computer-aided detection (CAD) on observer performance?.Acad Radiol. 2010; 17: 761-767
- Epidemiology of skin cancer: update 2019.Adv Exp Med Biol. 2020; 1268: 123-139
- A systematic review of worldwide incidence of nonmelanoma skin cancer.Br J Dermatol. 2012; 166: 1069-1080
- Colorectal cancer statistics.CA Cancer J Clin. 2017; 67 (2017): 177-193
- Incidence estimate of nonmelanoma skin cancer in the United States, 2006.Arch Dermatol. 2010; 146: 283-287
- Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries.CA Cancer J Clin. 2018; 68: 394-424
- Shades of gray and colour constancy. Color and imaging conference.2004
- Image analysis and machine learning in digital pathology: challenges and opportunities.Med Image Anal. 2016; 33: 170-175
- An unsupervised feature learning framework for basal cell carcinoma image analysis.Artif Intell Med. 2015; 64: 131-145
- Computer aided diagnostic support system for skin cancer: a review of techniques and algorithms.Int J Biomed Imag. 2013; 2013: 323268
- Dermatologist-level classification of skin cancer with deep neural networks.Nature. 2017; 542: 115-118
- Artificial intelligence: learning to play Go from scratch.Nature. 2017; 550: 336-337
- Deep echocardiography: data-efficient supervised and semi-supervised deep learning towards automated diagnosis of cardiac disease.NPJ Digit Med. 2018; 1: 59
- Mastering the game of Go with deep neural networks and tree search.Nature. 2016; 529: 484-489
- Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies.BMJ. 2020; 368: m689
- The incidence and body site of skin cancers in the population groups of Astana.Kazakhstan. Health Sci Rep. 2018; 1: e51
- Skin cancer in asians: part 1: nonmelanoma skin cancer.J Clin Aesthet Dermatol. 2009; 2: 39-42
- Cancer risks in Nairobi (2000-2014) by ethnic group.Int J Cancer. 2017; 140: 788-797
- Basal cell carcinoma, squamous cell carcinoma, and cutaneous melanoma in skin of color patients.Dermatol Clin. 2019; 37: 519-526
- Clinical presentations of melanoma in African Americans, Hispanics, and Asians.Dermatol Surg. 2019; 45: 791-801
Article info
Publication history
Published online: May 12, 2022
Accepted:
April 7,
2022
Received in revised form:
March 20,
2022
Received:
February 6,
2022
Identification
Copyright
© 2022 Elsevier Ltd. All rights reserved.
