Effectiveness of ChatGPT for Clinical Scenario Generation: A Qualitative Study
Archives of Academic Emergency Medicine,
Vol. 13 No. 1 (2025),
6 September 2025
,
Page e49
https://doi.org/10.22037/aaemj.v13i1.2690
Abstract
Introduction: A growing area is the use of ChatGPT in simulation-based learning, a widely recognized methodology in medical education. This study aimed to evaluate ChatGPT’s ability to generate realistic simulation scenarios to assist faculty as a significant challenge in medical education.
Method: This study employs a qualitative research design and thematic analysis to interpret expert opinions. The study was conducted in two phases. Scenario generation via ChatGPT and expert review for validation. We used ChatGPT (GPT-4) to create clinical scenarios on cardiovascular topics, including cardiogenic shock, postoperative cardiac tamponade after heart surgery, and heart failure. A panel of five experts, four nurses with expertise in emergency medicine and critical care and an anesthesia specialist, evaluated the scenarios. The experts' feedback, strengths and weaknesses, and proposed revisions from the expert discussions were analyzed via thematic analysis. Key themes and proposed revisions were identified, recorded, and compiled by the research team.
Results: The clinical scenarios were produced by ChatGPT in less than 5 seconds per case. The thematic analysis identified six recurring themes in the experts' discussions: clinical accuracy, the clarity of learning objectives, the logical flow of patient cases, realism and feasibility, alignment with nursing competencies, and level of difficulty. All the experts agreed that the scenarios were realistic and followed clinical guidelines. However, they also identified several errors and areas that needed improvement. The experts identified and documented specific errors, incorrect recommendations, missing information, and inconsistencies with standard nursing practices.
Conclusion: It seems that, ChatGPT can be a valuable tool for developing clinical scenarios, but expert review and refinement are necessary to ensure the accuracy and alignment of the generated scenarios with clinical and educational standards.
- Generative artificial intelligence
- Artificial intelligence
- Education, medical
- Computer simulation
How to Cite
References
Hwai H, Ho Y-J, Wang C-H, Huang C-H. Large language model application in emergency medicine and critical care. Journal of the Formosan Medical Association. Taiwan yi zhi. 2024.
Zhang K, Meng X, Yan X, Ji J, Liu J, Xu H, et al. Revolutionizing Health Care: The Transformative Impact of Large Language Models in Medicine. J Med Internet Res. 2025;27:e59069.
Gupta MR. ChatGPT-A Generative Pre-Trained Transformer. International Journal of Advanced Research in Science, Communication and Technology. 2024.
Guo AA, Li J. Harnessing the power of ChatGPT in medical education. Medical Teacher. 2023;45:1063 -
Mitra NK, Chitra E. Glimpses of the Use of Generative AI and ChatGPT in Medical Education. Education in Medicine Journal. 2024.
Komasawa N. Transformative Landscape of Anesthesia Education: Simulation, AI Integration, and Learner-Centric Reforms: A Narrative Review. Anesthesia Research. 2024;1(1):34-43.
Rêgo A, Araújo-Filho I. Artificial intelligence and active learning methodologies in the training of future general surgeons: A comprehensive review. World Journal of Advanced Research and Reviews. 2024;23:2870-81.
Sawaya RD, Mrad S, Rajha E, Saleh R, Rice J. Simulation-based curriculum development: lessons learnt in Global Health education. BMC Medical Education. 2021;21(1):33.
Miller CW, Lin Y, Schafer M. Designing Evidence-based Simulation Scenarios for Clinical Practice. The Nursing clinics of North America. 2024;59 3:415-26.
Watts PI, Rossler K, Bowler F, Miller C, Charnetski M, Decker S, et al. Onward and Upward: Introducing the Healthcare Simulation Standards of Best PracticeTM. Clinical Simulation In Nursing. 2021;58:1-4.
Kemelova GS, Saparova AS, Nurekeshova RJ. Drawing up a Clinical Scenario for the Standardized Patient Methodology Using Chatgpt. Virtual Technologies in Medicine. 2024.
Vaughn J, Ford SH, Scott M, Jones C, Lewinski A. Enhancing Healthcare Education: Leveraging ChatGPT for Innovative Simulation Scenarios. Clinical Simulation in Nursing. 2024.
Harrington DW, Simon LV. Designing a Simulation Scenario. StatPearls. Treasure Island (FL): StatPearls Publishing
Copyright © 2025, StatPearls Publishing LLC.; 2025.
Watts PI, McDermott DS, Alinier G, Charnetski M, Ludlow J, Horsley E, et al. Healthcare Simulation Standards of Best PracticeTM Simulation Design. Clinical Simulation In Nursing. 2021;58:14-21.
Reid JA. Building Clinical Simulations With ChatGPT in Nursing Education. The Journal of nursing education. 2024:1-2.
Violato E, Corbett C, Rose B, Rauschning B, Witschen B. The effectiveness and efficiency of using ChatGPT for writing health care simulations. International Journal of Healthcare Simulation. 2023.
Liu J, Liu F, Fang J, Liu S. The application of Chat Generative Pre-trained Transformer in nursing education. Nursing outlook. 2023;71 6:102064.
Rodgers DL, Needler M, Robinson A, Barnes R, Brosche TA, Hernandez J, et al. Artificial Intelligence and the Simulationists. Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare. 2023;18:395 - 9.
Wang J, Molina MD, Sundar SS. When expert recommendation contradicts peer opinion: Relative social influence of valence, group identity and artificial intelligence. Computers in Human Behavior. 2020;107:106278.
Horowitz MC, Kahn L, Macdonald J, Schneider J. Adopting AI: how familiarity breeds both trust and contempt. AI & SOCIETY. 2024;39(4):1721-35.
Comes T, Wijngaards NJE, Allen DK, Schultmann F. Scenario reliability assessment to support decision makers in situations of severe uncertainty. 2012 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support. 2012:30-7.
- Abstract Viewed: 777 times
- pdf Downloaded: 1123 times
Download Statastics
Make a Submission
SJR
COPE
