Comparing artificial intelligence and human intelligence in diagnosing oral lesions

Authors

  • Aziz Ali Khowaja Fatima Jinnah Dental College, Karachi Pakistan
  • Tarwani Bhart Fatima Jinnah Dental College, Karachi Pakistan
  • Raja Muhammad Daniyal Fatima Jinnah Dental College, Karachi Pakistan
  • Naeemullah Awan Fatima Jinnah Dental College, Karachi Pakistan
  • Ayesha Abbasi Fatima Jinnah Dental College, Karachi Pakistan
  • Hasan Mehdi Fatima Jinnah Dental College, Karachi Pakistan

DOI:

https://doi.org/10.55629/pakjpathol.v36i3.946

Keywords:

Artificial intelligence, House officers, Oral lesions, Large language models, Diagnosis

Abstract

Objective: To evaluate the diagnostic accuracy of artificial intelligence (AI) tools compared with human clinicians in identifying oral lesions.

Material and Methods: A comparative cross-sectional diagnostic accuracy (quantitative) study research has been conducted at the Department of Oral and Maxillofacial Surgery, Fatima Jinnah Dental College, Jinnah Hospital, Karachi, following approval from the ethical board, from November 2023 to January 2024.involved an examination consisting of 20 MCQs. These questions had options of responses titled A to D. The exam, aligned as multiple-choice questionnaire by an expert of that content field, was administered via Google Forms.

Results: Among the 35 participants, 14.3% were LLMs and 85.7% were house officers, with the mean ages of house officer’s 29.07±4.05 years. The majority of participants were male. In terms of diagnostic accuracy, LLMs and house officers performed similarly, with correct answer rates of 80.0% and 83.3%, respectively, and no statistically significant difference (p=0.855). Additionally, artificial intelligence tools demonstrated varying efficiencies, with OpenAI achieving 91.4% correct answers, PopAI at 77.1%, and Gemini at 80.0%.

Conclusion: The findings notably demonstrate that LLMs exhibit a level of performance that is on par with that of senior dental house officers, thus underlining their promise as a valuable tool in augmenting diagnostic capabilities within clinical settings.

Keywords: Artificial intelligence, Diagnosis, Human intelligence, Large language models, Oral lesions.

References

Khanagar SB, Alfouzan K, Awawdeh M, Alkadi L, Albalawi F, Alfadley A. Application and performance of artificial intelligence technology in detection, diagnosis and prediction of dental caries: A systematic review. Diagnostics. 2022; 12(5): 1083. DOI: https://doi.org/10.3390/diagnostics12051083

Orhan K, Bayrakdar IS, Ezhov M, Kravtsov A, Özyürek T. Evaluation of artificial intelligence for detecting periapical pathosis on cone-beam computed tomography scans. Int Endod J. 2020; 53(5): 680-9. DOI: https://doi.org/10.1111/iej.13265

Cantu AG, Gehrung S, Krois J, Chaurasia A, Rossi JG, Gaudin R, et al. Detecting caries lesions of different radiographic extension on bitewings using deep learning. J Dent. 2020; 100: 103425. DOI: https://doi.org/10.1016/j.jdent.2020.103425

Pauwels R, Brasil DM, Yamasaki MC, Jacobs R, Bosmans H, Freitas DQ, et al. Artificial intelligence for detection of periapical lesions on intraoral radiographs: comparison between convolutional neural networks and human observers. Oral Surg Oral Med Oral Pathol Oral Radiol. 2021; 131(5): 610-6. DOI: https://doi.org/10.1016/j.oooo.2021.01.018

Patil S, Albogami S, Hosmani J, Mujoo S, Kamil MA, Mansour MA, et al. Artificial intelligence in the diagnosis of oral diseases: Applications and pitfalls. Diagnostics. 2022; 12(5): 1029. DOI: https://doi.org/10.3390/diagnostics12051029

Mahmood H, Shaban M, Indave BI, Santos-Silva AR, Rajpoot N, Khurram SA. Use of artificial intelligence in diagnosis of head and neck precancerous and cancerous lesions: A systematic review. Oral Oncol. 2020; 110: 104885.

DOI: https://doi.org/10.1016/j.oraloncology.2020.104885

Revilla-León M, Gómez-Polo M, Barmak AB, Inam W, Kan JY, Kois JC, et al. Artificial intelligence models for diagnosing gingivitis and periodontal disease: a systematic review. J Prosthet Dent. 2023; 130(6): 816-24.

DOI:https://doi.org/10.1016/j.prosdent.2022.01.026

Kühnisch J, Meyer O, Hesenius M, Hickel R, Gruhn V. Caries detection on intraoral images using artificial intelligence. J Dent Res. 2022; 101(2): 158-65. DOI: https://doi.org/10.1177/00220345211032524

Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C, et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLoS Digit Health. 2023; 2(2): e0000198.

DOI: https://doi.org/10.1371/journal.pdig.0000198

Nori H, King N, McKinney SM, Carignan D, Horvitz E. Capabilities of GPT-4 on medical challenge problems. medRxiv. 2023. 13375. DOI: https://doi.org/10.48550/arXiv.2303.13375

Tavakol M, Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ. 2011; 2: 53-5. DOI: https://doi.org/10.5116/ijme.4dfb.8dfd

Fleiss JL, Levin B, Paik MC. Statistical methods for rates and proportions. 3rd ed. Hoboken (NJ): Wiley; 2013. Available from: https://content.e-bookshelf.de/media/reading/L-583786-76ad72e5cb.pdf

Khanagar SB, Naik S, Al Kheraif AA, Vishwanathaiah S, Maganur PC, Alhazmi Y, et al. Application and performance of artificial intelligence technology in oral cancer diagnosis and prediction of prognosis: a systematic review. Diagnostics. 2021; 11(6): 1004. DOI: https://doi.org/10.3390/diagnostics11061004

Mira ES, Sapri AM, Aljehanı RF, Jambi BS, Bashir T, El-Kenawy ES, et al. Early diagnosis of oral cancer using image processing and artificial intelligence. Future Pract Appl. 2024; 14(1): 293-308. DOI: https://doi.org/10.54216/FPA.140122

Alabi RO, Elmusrati M, Sawazaki-Calone I, Kowalski LP, Haglund C, Coletta RD, et al. Comparison of supervised machine learning classification techniques in prediction of locoregional recurrences in early oral tongue cancer. Int J Med Inform. 2020; 136: 104068. DOI: https://doi.org/10.1016/j.ijmedinf.2019.104068

Shams WK, Htike ZZ. Oral cancer prediction using gene expression profiling and machine learning. Int J Appl Eng Res. 2017;12(15):4893-8. Available from: https://www.ripublication.com/ijaer17/ijaerv12n15_%20(23).pdf

Chu CS, Lee NP, Adeoye J, Thomson P, Choi SW. Machine learning and treatment outcome prediction for oral cancer. J Oral Pathol Med. 2020; 49(10): 977-85. DOI: https://doi.org/10.1111/jop.13089

Tseng WT, Chiang WF, Liu SY, Roan J, Lin CN. The application of data mining techniques to oral cancer prognosis. J Med Syst. 2015; 39(5): 59. DOI: https://doi.org/10.1007/s10916-015-0241-3

Dom RBM, Karemm SA, Abodin A, Mazlipah S, Norzaidi MD. The use of artificial intelligence to identify people at risk of oral cancer: Empirical evidence in Malaysian University. Int J Sci Res Educ. 2010;3(1):10-20. Available from: https://www.ijsre.com.ng/assets/vol.%2C-3_1_-rosma-et-al.pdf

Kim DW, Lee S, Kwon S, Nam W, Cha IH, Kim HJ. Deep learning-based survival prediction of oral cancer patients. Sci Rep. 2019; 9(1): 6994. DOI: https://doi.org/10.1038/s41598-019-43372-7

Warnakulasuriya S. Global epidemiology of oral and oropharyngeal cancer. Oral Oncol. 2009; 45(4-5): 309-16.

DOI: https://doi.org/10.1016/j.oraloncology.2008.06.002

García-Pola M, Pons-Fuster E, Suárez-Fernández C, Seoane-Romero J, Romero-Méndez A, López-Jornet P. Role of artificial intelligence in the early diagnosis of oral cancer: A scoping review. Cancers (Basel). 2021; 13(18): 4600. DOI: https://doi.org/10.3390/cancers13184600

Downloads

Published

30-09-2025

How to Cite

1.
Khowaja AA, Bhart T, Daniyal RM, Awan N, Abbasi A, Mehdi H. Comparing artificial intelligence and human intelligence in diagnosing oral lesions. Pak J Pathol [Internet]. 2025 Sep. 30 [cited 2026 Nov. 17];36(3):245-51. Available from: https://pakjpath.com/index.php/Pak-J-Pathol/article/view/946