Comparing artificial intelligence and human intelligence in diagnosing oral lesions
DOI:
https://doi.org/10.55629/pakjpathol.v36i3.946Keywords:
Artificial intelligence, House officers, Oral lesions, Large language models, DiagnosisAbstract
Objective: To evaluate the diagnostic accuracy of artificial intelligence (AI) tools compared with human clinicians in identifying oral lesions.
Material and Methods: A comparative cross-sectional diagnostic accuracy (quantitative) study research has been conducted at the Department of Oral and Maxillofacial Surgery, Fatima Jinnah Dental College, Jinnah Hospital, Karachi, following approval from the ethical board, from November 2023 to January 2024.involved an examination consisting of 20 MCQs. These questions had options of responses titled A to D. The exam, aligned as multiple-choice questionnaire by an expert of that content field, was administered via Google Forms.
Results: Among the 35 participants, 14.3% were LLMs and 85.7% were house officers, with the mean ages of house officer’s 29.07±4.05 years. The majority of participants were male. In terms of diagnostic accuracy, LLMs and house officers performed similarly, with correct answer rates of 80.0% and 83.3%, respectively, and no statistically significant difference (p=0.855). Additionally, artificial intelligence tools demonstrated varying efficiencies, with OpenAI achieving 91.4% correct answers, PopAI at 77.1%, and Gemini at 80.0%.
Conclusion: The findings notably demonstrate that LLMs exhibit a level of performance that is on par with that of senior dental house officers, thus underlining their promise as a valuable tool in augmenting diagnostic capabilities within clinical settings.
Keywords: Artificial intelligence, Diagnosis, Human intelligence, Large language models, Oral lesions.
References
Khanagar SB, Alfouzan K, Awawdeh M, Alkadi L, Albalawi F, Alfadley A. Application and performance of artificial intelligence technology in detection, diagnosis and prediction of dental caries: A systematic review. Diagnostics. 2022; 12(5): 1083. DOI: https://doi.org/10.3390/diagnostics12051083
Orhan K, Bayrakdar IS, Ezhov M, Kravtsov A, Özyürek T. Evaluation of artificial intelligence for detecting periapical pathosis on cone-beam computed tomography scans. Int Endod J. 2020; 53(5): 680-9. DOI: https://doi.org/10.1111/iej.13265
Cantu AG, Gehrung S, Krois J, Chaurasia A, Rossi JG, Gaudin R, et al. Detecting caries lesions of different radiographic extension on bitewings using deep learning. J Dent. 2020; 100: 103425. DOI: https://doi.org/10.1016/j.jdent.2020.103425
Pauwels R, Brasil DM, Yamasaki MC, Jacobs R, Bosmans H, Freitas DQ, et al. Artificial intelligence for detection of periapical lesions on intraoral radiographs: comparison between convolutional neural networks and human observers. Oral Surg Oral Med Oral Pathol Oral Radiol. 2021; 131(5): 610-6. DOI: https://doi.org/10.1016/j.oooo.2021.01.018
Patil S, Albogami S, Hosmani J, Mujoo S, Kamil MA, Mansour MA, et al. Artificial intelligence in the diagnosis of oral diseases: Applications and pitfalls. Diagnostics. 2022; 12(5): 1029. DOI: https://doi.org/10.3390/diagnostics12051029
Mahmood H, Shaban M, Indave BI, Santos-Silva AR, Rajpoot N, Khurram SA. Use of artificial intelligence in diagnosis of head and neck precancerous and cancerous lesions: A systematic review. Oral Oncol. 2020; 110: 104885.
DOI: https://doi.org/10.1016/j.oraloncology.2020.104885
Revilla-León M, Gómez-Polo M, Barmak AB, Inam W, Kan JY, Kois JC, et al. Artificial intelligence models for diagnosing gingivitis and periodontal disease: a systematic review. J Prosthet Dent. 2023; 130(6): 816-24.
DOI:https://doi.org/10.1016/j.prosdent.2022.01.026
Kühnisch J, Meyer O, Hesenius M, Hickel R, Gruhn V. Caries detection on intraoral images using artificial intelligence. J Dent Res. 2022; 101(2): 158-65. DOI: https://doi.org/10.1177/00220345211032524
Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C, et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLoS Digit Health. 2023; 2(2): e0000198.
DOI: https://doi.org/10.1371/journal.pdig.0000198
Nori H, King N, McKinney SM, Carignan D, Horvitz E. Capabilities of GPT-4 on medical challenge problems. medRxiv. 2023. 13375. DOI: https://doi.org/10.48550/arXiv.2303.13375
Tavakol M, Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ. 2011; 2: 53-5. DOI: https://doi.org/10.5116/ijme.4dfb.8dfd
Fleiss JL, Levin B, Paik MC. Statistical methods for rates and proportions. 3rd ed. Hoboken (NJ): Wiley; 2013. Available from: https://content.e-bookshelf.de/media/reading/L-583786-76ad72e5cb.pdf
Khanagar SB, Naik S, Al Kheraif AA, Vishwanathaiah S, Maganur PC, Alhazmi Y, et al. Application and performance of artificial intelligence technology in oral cancer diagnosis and prediction of prognosis: a systematic review. Diagnostics. 2021; 11(6): 1004. DOI: https://doi.org/10.3390/diagnostics11061004
Mira ES, Sapri AM, Aljehanı RF, Jambi BS, Bashir T, El-Kenawy ES, et al. Early diagnosis of oral cancer using image processing and artificial intelligence. Future Pract Appl. 2024; 14(1): 293-308. DOI: https://doi.org/10.54216/FPA.140122
Alabi RO, Elmusrati M, Sawazaki-Calone I, Kowalski LP, Haglund C, Coletta RD, et al. Comparison of supervised machine learning classification techniques in prediction of locoregional recurrences in early oral tongue cancer. Int J Med Inform. 2020; 136: 104068. DOI: https://doi.org/10.1016/j.ijmedinf.2019.104068
Shams WK, Htike ZZ. Oral cancer prediction using gene expression profiling and machine learning. Int J Appl Eng Res. 2017;12(15):4893-8. Available from: https://www.ripublication.com/ijaer17/ijaerv12n15_%20(23).pdf
Chu CS, Lee NP, Adeoye J, Thomson P, Choi SW. Machine learning and treatment outcome prediction for oral cancer. J Oral Pathol Med. 2020; 49(10): 977-85. DOI: https://doi.org/10.1111/jop.13089
Tseng WT, Chiang WF, Liu SY, Roan J, Lin CN. The application of data mining techniques to oral cancer prognosis. J Med Syst. 2015; 39(5): 59. DOI: https://doi.org/10.1007/s10916-015-0241-3
Dom RBM, Karemm SA, Abodin A, Mazlipah S, Norzaidi MD. The use of artificial intelligence to identify people at risk of oral cancer: Empirical evidence in Malaysian University. Int J Sci Res Educ. 2010;3(1):10-20. Available from: https://www.ijsre.com.ng/assets/vol.%2C-3_1_-rosma-et-al.pdf
Kim DW, Lee S, Kwon S, Nam W, Cha IH, Kim HJ. Deep learning-based survival prediction of oral cancer patients. Sci Rep. 2019; 9(1): 6994. DOI: https://doi.org/10.1038/s41598-019-43372-7
Warnakulasuriya S. Global epidemiology of oral and oropharyngeal cancer. Oral Oncol. 2009; 45(4-5): 309-16.
DOI: https://doi.org/10.1016/j.oraloncology.2008.06.002
García-Pola M, Pons-Fuster E, Suárez-Fernández C, Seoane-Romero J, Romero-Méndez A, López-Jornet P. Role of artificial intelligence in the early diagnosis of oral cancer: A scoping review. Cancers (Basel). 2021; 13(18): 4600. DOI: https://doi.org/10.3390/cancers13184600
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Dr. Aziz Ali ,BDS Khowaja, Dr. Tarwani , BDS Bhart, Dr Raja Muhammad Daniyal, Dr. Naeemullah BDS Awan, Dr. Ayesha, BDS. Abbasi, 6. Dr. Hasan BDS, FCPS. Mehdi

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Readers may “Share-copy and redistribute the material in any medium or format” and “Adapt-remix, transform, and build upon the material”. The readers must give appropriate credit to the source of the material and indicate if changes were made to the material. Readers may not use the material for commercial purpose. The readers may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
