Assessment of a Smartphone-Based Neural Network Application for the Risk Assessment of Skin Lesions under Real-World Conditions

Teresa Maria Kränke; Philipp Efferl; Katharina Tripolt-Droschl; Rainer Hofmann-Wellenhof

doi:10.5826/dpc.1503a5110

Assessment of a Smartphone-Based Neural Network Application for the Risk Assessment of Skin Lesions under Real-World Conditions

Authors

Teresa Maria Kränke Department of Dermatology, Medical University of Graz, Austria https://orcid.org/0000-0001-8406-6063
Philipp Efferl Department of Dermatology and Venereology, Augenbruggerplatz 8 Medical University of Graz, Austria
Katharina Tripolt-Droschl Department of Dermatology and Venereology, Augenbruggerplatz 8 Medical University of Graz, Austria
Rainer Hofmann-Wellenhof Department of Dermatology and Venereology, Augenbruggerplatz 8 Medical University of Graz, Austria

Keywords:

Dermatology, Artificial Intelligence, Convolutional Neural Networks, skin lesion assessment, skin cancer

Abstract

Introduction: The diagnostic performance of convolutional neural networks (CNNs) in diagnosing different types of skin cancer has been quite promising. Mobile phone applications with integrated artificial intelligence (AI) are an understudied area.

Objective: We evaluated the risk assessment of the SkinScreener (Medaia GmbH, Graz, Austria) AI-based algorithm in comparison to an expert panel of 3 dermatologists.

Methods: In this retrospective single-center study at the Department of Dermatology and Venereology in Graz, photographs of lesions were taken by the users’ mobile phone cameras. The algorithm allocated them to 3 risk classes. Blinded to AI´s results, the images were evaluated by 3 dermatologists—our reference standard. A consensus was defined as at least a two-thirds majority.

Results: A total of 1,428 skin lesions were included. In 902 lesions (63.16%) there was a full agreement, and in 441 lesions (30.88%) a two-thirds majority was reached. Eighty-five lesions (5.69%) had to be discussed in a joint review process. The tested algorithm reached a sensitivity of 76.9% (CI 71.7%-81.5%) and a specificity of 80.9% (CI 78.5%-83.2%). Overall accuracy results were 77.2%.

Conclusions: Our results indicate that the tested mobile phone algorithm is a valuable tool for the correct risk classification of various skin lesions. As expected, its performance is worse than in a professional setting. Nonetheless, the use of these applications on mobile phones should raise awareness of skin cancer and encourage users to deal more intensively with preventive measures. In light of our results, these applications are also reliable for use by laypersons.

References

Siegel RL, Miller KD, Fuchs HE, Jemal A. Cancer Statistics, 2021. CA Cancer J Clin. 2021 Jan;71(1):7–33. DOI: 10.3322/caac.21654. PMID: 33433946.

Nikolaou V, Stratigos AJ. Emerging trends in the epidemiology of melanoma. Br J Dermatol. 2014 Jan;170(1):11–9. DOI: 10.1111/bjd.12492. PMID: 23815297.

Hollestein L, Weinstock M, Le Roux E, Olsen C. More Than Many: How to Manage the Most Frequent Cancer? J Invest Dermatol. 2017 Sep;137(9):1823–6. DOI: 0.1016/j.jid.2017.06.017. PMID 28843292.

Petrie T, Samatham R, Witkowski AM, Esteva A, Leachman SA. Melanoma Early Detection: Big Data, Bigger Picture. J Invest Dermatol. 2019 Jan;139(1):25–30. DOI: 10.1016/j.jid.2018.06.187. PMID 30482597.

Dorrell DN, Strowd LC. Skin Cancer Detection Technology. Dermatol Clin. 2019 Oct;37(4):527–36. DOI: 10.1016/j.det.2019.05.010. PMID: 31466592.

Marka A, Carter JB, Toto E, Hassanpour S. Automated detection of nonmelanoma skin cancer using digital images: A systematic review. BMC Med Imaging. 2019;19(1):21. DOI: 10.1186/s12880-019-0307-7. PMID: 30819133.

Maron RC, Weichenthal M, Utikal JS et al. Systematic outperformance of 112 dermatologists in multiclass skin cancer image classification by convolutional neural networks. Eur J Cancer. 2019;119:57-65. DOI: 10.1016/j.ejca.2019.06.013. PMID: 31419752.

Brinker TJ, Hekler A, Enk AH et al. A convolutional neural network trained with dermoscopic images performed on par with 145 dermatologists in a clinical melanoma image classification task. Eur J Cancer. 2019;111:148-154. DOI: 10.1016/j.ejca.2019.02.005. PMID: 30852421.

Brinker TJ, Hekler A, Enk AH et al. Deep learning outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma image classification task. Eur J Cancer. 2019;113:47-54. DOI: 10.1016/j.ejca.2019.04.001. PMID: 30981091.

Zhao XY, Wu X, Li FF et al. The Application of Deep Learning in the Risk Grading of Skin Tumors for Patients Using Clinical Images. J Med Syst. 2019;43(8):283. DOI: 10.1007/s10916-019-1414-2. PMID: 31300897.

Cui X, Wei R, Gong L et al. Assessing the effectiveness of artificial intelligence methods for melanoma: A retrospective review. J Am Acad Dermatol. 2019;81(5):1176-1180. DOI: 10.1016/j.jaad.2019.06.042. PMID: 31255749.

Fujisawa Y, Otomo Y, Ogata Y et al. Deep-learning-based, computer-aided classifier developed with a small dataset of clinical images surpasses board-certified dermatologists in skin tumour diagnosis. Br J Dermatol. 2019;180(2):373-381. DOI: 10.1111/bjd.16924. PMID: 29953582.

Al-Masni MA, Kim DH, Kim TS. Multiple skin lesions diagnostics via integrated deep convolutional networks for segmentation and classification. Comput Methods Programs Biomed. 2020;190:105351. DOI: 10.1016/j.cmpb.2020.105351. PMID: 32028084.

Han SS, Kim MS, Lim W, Park GH, Park I, Chang SE. Classification of the Clinical Images for Benign and Malignant Cutaneous Tumors Using a Deep Learning Algorithm. J Invest Dermatol. 2018;138(7):1529-1538. DOI: 10.1016/j.jid.2018.01.028. PMID: 29428356.

Udrea A, Mitra GD, Costea D et al. Accuracy of a smartphone application for triage of skin lesions based on machine learning algorithms. J Eur Acad Dermatol Venereol. 2020;34(3):648-655. DOI: 10.1111/jdv.15935. PMID 31494983.

Liu Y, Jain A, Eng C et al. A deep learning system for differential diagnosis of skin diseases. Nat Med. 2020 Jun;26(6):900–8. DOI: 10.1038/s41591-020-0842-3. PMID: 32424212.

Esteva A, Kuprel B, Novoa RA et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017 Feb 2;542(7639):115–8. DOI: 10.1038/nature21056. PMID: 28117445.

Chao E, Meenan CK, Ferris LK. Smartphone-Based Applications for Skin Monitoring and Melanoma Detection. Dermatol Clin. 2017 Oct;35(4):551–7. DOI: 10.1016/j.det.2017.06.014. PMID: 28886812.

Wurm EM, Curchin CE, Soyer HP. Recent advances in diagnosing cutaneous melanomas. F1000 Med Rep. 2010;2:46. DOI: 10.3410/M2-46. PMID: 20948838.

Waweru AK, Ahmed K, Miao Y, Kawan P. Deep Learning in Skin Lesion Analysis Towards Cancer Detection. In: 2020 24th International Conference Information Visualisation (IV) [Internet]. Melbourne, Australia: IEEE; 2020 [cited 2023 Nov 29]. p. 740–5. https://ieeexplore.ieee.org/document/9373111/

Acharya P, Mathur M. Artificial intelligence in dermatology: the ‘unsupervised’ learning. Br J Dermatol. 2020 Jun;182(6):1507–8. DOI: 10.1111/bjd.18933. PMID: 32030726.

Felmingham C, MacNamara S, Cranwell W et al. Improving Skin cancer Management with ARTificial Intelligence (SMARTI): protocol for a preintervention/postintervention trial of an artificial intelligence system used as a diagnostic aid for skin cancer management in a specialist dermatology setting. BMJ Open. 2022 Jan;12(1):e050203. DOI: 10.1136/bmjopen-2021-050203. PMID: 34983756.

Kränke T, Tripolt-Droschl K, Röd L et al. New AI-algorithms on smartphones to detect skin cancer in a clinical setting—A validation study. PLOS ONE. 2023 Feb 15;18(2):e0280670. DOI: 10.1371/journal.pone.0280670. PMID: 36791068.

Tschandl P, Rosendahl C, Akay BN et al. Expert-Level Diagnosis of Nonpigmented Skin Cancer by Combined Convolutional Neural Networks. JAMA Dermatol. 2019 Jan 1;155(1):58. DOI: 10.1001/jamadermatol.2018.4378. PMID: 30484822.

Young AT, Vora NB, Cortez J et al. The role of technology in melanoma screening and diagnosis. Pigment Cell Melanoma Res. 2021 Mar;34(2):288–300. DOI: 10.1111/pcmr.12907. PMID: 32558281.

Jahn AS, Navarini AA, Cerminara SE, et al. Over-Detection of Melanoma-Suspect Lesions by a CE-Certified Smartphone App: Performance in Comparison to Dermatologists, 2D and 3D Convolutional Neural Networks in a Prospective Data Set of 1204 Pigmented Skin Lesions Involving Patients’ Perception. Cancers. 2022 Aug 7;14(15):3829. DOI: 10.3390/cancers14153829. PMID: 35954491.

Freeman K, Dinnes J, Chuchu N et al. Algorithm based smartphone apps to assess risk of skin cancer in adults: systematic review of diagnostic accuracy studies. BMJ. 2020 Feb 10;368:m127. DOI: 10.1136/bmj.m127. PMID: 32041693.

O’Sullivan JW, Banerjee A, Heneghan C, Pluddemann A. Verification bias. BMJ Evid-Based Med. 2018 Apr;23(2):54–5. DOI: 10.1136/bmjebm-2018-110919. PMID 29595130.

Chen X, Lu Q, Chen C, Jiang G. Recent developments in dermoscopy for dermatology. J Cosmet Dermatol. 2021 Jun;20(6):1611–7. DOI: 10.1111/jocd.13846. PMID: 33197276.