Assessing Patient Perceptions of Artificial Intelligence versus Human Surgeon Counseling in Facial Plastic Surgery
Journal of Otorhinolaryngology and Facial Plastic Surgery,
Vol. 10 No. 1 (2024),
13 March 2024
,
Page 1-6
https://doi.org/10.22037/orlfps.v10i1.47066
Abstract
Background: The integration of artificial intelligence (AI), particularly generative large language (LLM) models like ChatGPT, promises to enhance patient education and communication in many fields, including facial plastic surgery.
Aim: We herein assess patient perceptions of AI-generated versus human surgeon preoperative and postoperative counseling for septorhinoplasty surgery.
Methods: Responses to hypothetical questions by ChatGPT and a human surgeon were evaluated by 103 blind evaluators from a general audience to assess empathy, accuracy, completeness, and overall quality. Evaluators were also asked to quantify their prior usage of AI LLMs and rate their confidence in discerning the AI and human responses.
Results: ChatGPT's responses were generally preferred, receiving significantly higher scores in accuracy (p<0.001), completeness (p<0.001), and overall quality (p<0.001). ChatGPT scored lower in empathy, but the difference was not significant (p=0.11). No significant differences were found in evaluators' trust in AI or their ability to discern between human and AI responses based on past AI LLM usage (p=0.43; p=0.25)..
Conclusion: AI, specifically ChatGPT, can supplement traditional patient education and communication methods in facial plastic surgery. Further research is necessary to understand the broader implications and optimal integration of AI in clinical practice.
- Artificial intelligence; Large language model; Patient education; Human-AI comparison.
How to Cite
References
Chew HSJ, Achananuparp P. Perceptions and Needs of Artificial Intelligence in Health Care to Increase Adoption: Scoping Review. J Med Internet Res. 2022 Jan 14;24(1):e32939. doi: 10.2196/32939. PMID: 35029538; PMCID: PMC8800095. Link: https://pubmed.ncbi.nlm.nih.gov/35029538/
Park CW, Seo SW, Kang N, Ko B, Choi BW, Park CM, Chang DK, Kim H, Kim H, Lee H, Jang J, Ye JC, Jeon JH, Seo JB, Kim KJ, Jung KH, Kim N, Paek S, Shin SY, Yoo S, Choi YS, Kim Y, Yoon HJ. Artificial Intelligence in Health Care: Current Applications and Issues. J Korean Med Sci. 2020 Nov 2;35(42):e379. doi: 10.3346/jkms.2020.35.e379. Erratum in: J Korean Med Sci. 2020 Dec 14;35(48):e425. PMID: 33140591; PMCID: PMC7606883. Link: https://pubmed.ncbi.nlm.nih.gov/33140591/
Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C, Madriaga M, Aggabao R, Diaz-Candido G, Maningo J, Tseng V. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023 Feb 9;2(2):e0000198. doi: 10.1371/journal.pdig.0000198. PMID: 36812645; PMCID: PMC9931230. Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC9931230/
Durairaj KK, Baker O, Bertossi D, Dayan S, Karimi K, Kim R, Most S, Robotti E, Rosengaus F. Artificial Intelligence Versus Expert Plastic Surgeon: Comparative Study Shows ChatGPT "Wins" Rhinoplasty Consultations: Should We Be Worried? Facial Plast Surg Aesthet Med. 2023 Nov 20. doi: 10.1089/fpsam.2023.0224. Epub ahead of print. PMID: 37982677. Link: https://www.liebertpub.com/doi/10.1089/fpsam.2023.0224
Seth I, Cox A, Xie Y, Bulloch G, Hunter-Smith DJ, Rozen WM, Ross RJ. Evaluating Chatbot Efficacy for Answering Frequently Asked Questions in Plastic Surgery: A ChatGPT Case Study Focused on Breast Augmentation. Aesthet Surg J. 2023 Sep 14;43(10):1126-1135. doi: 10.1093/asj/sjad140. PMID: 37158147. Link: https://pubmed.ncbi.nlm.nih.gov/37158147/
Xie Y, Seth I, Hunter-Smith DJ, Rozen WM, Ross R, Lee M. Aesthetic Surgery Advice and Counseling from Artificial Intelligence: A Rhinoplasty Consultation with ChatGPT. Aesthetic Plast Surg. 2023 Oct;47(5):1985-1993. doi: 10.1007/s00266-023-03338-7. Epub 2023 Apr 24. PMID: 37095384; PMCID: PMC10581928. Link: https://pubmed.ncbi.nlm.nih.gov/37095384/
Ayers JW, Poliak A, Dredze M, Leas EC, Zhu Z, Kelley JB, Faix DJ, Goodman AM, Longhurst CA, Hogarth M, Smith DM. Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum. JAMA Intern Med. 2023 Jun 1;183(6):589-596. doi: 10.1001/jamainternmed.2023.1838. PMID: 37115527; PMCID: PMC10148230. Link: https://pubmed.ncbi.nlm.nih.gov/37115527/
Shanafelt TD, West CP, Dyrbye LN, Trockel M, Tutty M, Wang H, Carlasare LE, Sinsky C. Changes in Burnout and Satisfaction With Work-Life Integration in Physicians During the First 2 Years of the COVID-19 Pandemic. Mayo Clin Proc. 2022 Dec;97(12):2248-2258. doi: 10.1016/j.mayocp.2022.09.002. Epub 2022 Sep 14. PMID: 36229269; PMCID: PMC9472795. Link: https://pubmed.ncbi.nlm.nih.gov/36229269/
Walsh S, O'Neill A, Hannigan A, Harmon D. Patient-rated physician empathy and patient satisfaction during pain clinic consultations. Ir J Med Sci. 2019 Nov;188(4):1379-1384. doi: 10.1007/s11845-019-01999-5. Epub 2019 Mar 27. PMID: 30919198. Link: https://pubmed.ncbi.nlm.nih.gov/30919198/
Bhattacharyya M, Miller VM, Bhattacharyya D, Miller LE. High Rates of Fabricated and Inaccurate References in ChatGPT-Generated Medical Content. Cureus. 2023 May 19;15(5):e39238. doi: 10.7759/cureus.39238. PMID: 37337480; PMCID: PMC10277170. Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC10277170/
B urgess DJ, Warren J, Phelan S, Dovidio J, van Ryn M. Stereotype threat and health disparities: what medical educators and future physicians need to know. J Gen Intern Med. 2010 May;25 Suppl 2(Suppl 2):S169-77. doi: 10.1007/s11606-009-1221-4. PMID: 20352514; PMCID: PMC2847106. Link: https://pubmed.ncbi.nlm.nih.gov/20352514/
- Abstract Viewed: 137 times
- PDF Downloaded: 91 times