Evaluation of Artificial Intelligence-Generated Information on Cell Culture and Laboratory Protocols in the Field of Tissue Engineering: A Comparison between GPT-3.5, Claude-Instant, and Microsoft Copilot AI in Cell Lab
Regeneration, Reconstruction & Restoration (Triple R),
Vol. 9 (2024),
1 January 2024
https://doi.org/10.22037/rrr.v9.45423
Abstract
Background and objectives: The current study aimed to evaluate and compare the validity and precision of the information provided by three online artificial intelligence (AI) platforms, including GPT-3.5 (Open AI), Claude-Instant (Anthropic), and Copilot (Microsoft), in the context of laboratory protocols in the field of tissue engineering.
Materials and methods: Three specialists in the field of tissue engineering research and molecular and cellular laboratory methods created a survey with 20 open-ended questions. The questions included a variety of subjects regarding cell culture processes and cell analysis techniques. The chatbots' responses to the open-ended inquiries were assessed according to the modified global quality scale. A P-value less than 0.05 was considered statistically significant.
Results: GPT-3.5, with a median score of 4 out of 5, performed significantly better than Claude-Instant and Copilot. GPT-3.5 showed superior performance compared to Claude-Instant (P < 0.001) and Copilot (P = 0.007). Both Claude-Instant and Copilot chatbots received a median score of 3 out of 5 in total.
Conclusion: In conclusion, AI language models provided satisfactory responses for laboratory procedures in the field of tissue engineering. Specifically, GPT-3.5 outperformed both Claude-Instant and Microsoft Copilot chatbots. Nevertheless, the information offered by all three chatbots was insufficient in terms of the crucial details required for the design and execution of an in vitro analysis.
- Artificial Intelligence
- Cell Culture Techniques
- Dentistry
- Machine Learning
- Tissue Engineering
How to Cite
References
Godbey, W. and A. Atala, In vitro systems for tissue engineering. Annals of the new York Academy of Sciences, 2002. 961(1): p. 10-26.
Hirsch, C. and S. Schildknecht, In vitro research reproducibility: Keeping up high standards. Frontiers in pharmacology, 2019. 10: p. 492837.
Fanni, S.C., et al., Natural language processing, in Introduction to Artificial Intelligence. 2023, Springer. p. 87-99.
Stiennon, N., et al., Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 2020. 33: p. 3008-3021.
Ray, P.P., ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems, 2023.
Haleem, A., M. Javaid, and R.P. Singh, An era of ChatGPT as a significant futuristic support tool: A study on features, abilities, and challenges. BenchCouncil transactions on benchmarks, standards and evaluations, 2022. 2(4): p. 100089.
Chubb, J., P. Cowling, and D. Reed, Speeding up to keep up: exploring the use of AI in the research process. AI & society, 2022. 37(4): p. 1439-1457.
Salvagno, M., F.S. Taccone, and A.G. Gerli, Can artificial intelligence help for scientific writing? Critical Care, 2023. 27(1): p. 75.
Kitamura, F.C., ChatGPT is shaping the future of medical writing but still requires human judgment. 2023, Radiological Society of North America. p. e230171.
Bernard, A., et al., A systematic review of patient inflammatory bowel disease information resources on the World Wide Web. Official journal of the American College of Gastroenterology| ACG, 2007. 102(9): p. 2070-2077.
Thoppilan, R., et al., Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239, 2022.
Borji, A. and M. Mohammadian, Battle of the Wordsmiths: Comparing ChatGPT, GPT-4, Claude, and Bard. GPT-4, Claude, and Bard (June 12, 2023), 2023.
Lozić, E. and B. Štular, ChatGPT v Bard v Bing v Claude 2 v Aria v human-expert. How good are AI chatbots at scientific writing?(ver. 23Q3). arXiv preprint arXiv:2309.08636, 2023.
Ray, P.P., ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems, 2023. 3: p. 121-154.
Rudolph, J., S. Tan, and S. Tan, War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education. Journal of Applied Learning and Teaching, 2023. 6(1).
Alberts, I.L., et al., Large language models (LLM) and ChatGPT: what will the impact on nuclear medicine be? European journal of nuclear medicine and molecular imaging, 2023. 50(6): p. 1549-1552.
Bai, H. A practical three-phase approach to fully automated programming using system decomposition and coding copilots. in Proceedings of the 2022 5th International Conference on Machine Learning and Machine Intelligence. 2022.
Azadi, A., et al., Evaluation of AI-generated responses by different artificial intelligence chatbots to the clinical decision-making case-based questions in oral and maxillofacial surgery. Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology, 2024.
- Abstract Viewed: 319 times
- PDF Downloaded: 106 times