An Attention-Based Residual Connection Convolutional Neural Network for Classification Tasks in Computer Vision
Journal of Dental School,
Vol. 42 No. 1 (2024),
2 June 2024
,
Page 14-25
https://doi.org/10.22037/jds.v42i1.45245
Abstract
Objectives In the field of medical and dental image analysis, the development of advanced deep learning architectures for precise classification tasks has become essential. The present study aims to introduce an innovative Attention-based Residual Connection Convolutional Neural Network (ARN-CNN) designed for accurate classification of medical images using the Med-MNIST (Medical Modified National Institute of Standards and Technology) dataset.
Methods Attention mechanisms and residual connections were integrated into the ARN-CNN model to enhance feature extraction and prediction accuracy. The model's performance was evaluated through a comparative analysis with state-of-the-art CNN architectures on the challenging MNIST medical dataset, based on key metrics, including accuracy, precision, recall, and F1 score.
Results The ARN-CNN model achieves a classification accuracy of 99.96% and a loss of 0.0037. These results showcase the superior performance of ARN-CNN in improving classification accuracy and its potential for enhancing medical image analysis.
Conclusion The study demonstrates the crucial role that residual connections and attention processes play in capturing intricate details and maximizing information flow in the network. It highlights the potential of deep learning techniques for revolutionizing medical image analysis and laying the foundation for future investigation into automated medical and dental diagnosis and treatment in healthcare.
- Attention-Based Residual Connection Convolutional Neural Network (CNN)
- Medical and dental Image Classification
- MNIST Medical Dataset
- Deep Learning
How to Cite
References
References
Hassan E, Shams MY, Hikal NA, Elmougy S. A novel convolutional neural network model for malaria cell images classification. Computers, Materials & Continua. 2022;72(3):5889-907.
Kayalibay B, Jensen G, van der Smagt P. CNN-based segmentation of medical imaging data. arXiv preprint arXiv:170103056. 2017.
Anand R, Sowmya V, Gopalakrishnan E, Soman K, editors. Modified Vgg deep learning architecture for Covid-19 classification using bio-medical images. IOP conference series: materials science and engineering; 2021: IOP Publishing.
Tapaswi S, Joshi RC, editors. Classification of bio-medical images using neuro fuzzy approach. International Conference on Database Systems for Advanced Applications; 2004: Springer.
O'Shea K, Nash R. An introduction to convolutional neural networks. arXiv preprint arXiv:151108458. 2015.
Obeso AM, Benois-Pineau J, Vázquez MSG, Acosta AÁR. Visual vs internal attention mechanisms in deep neural networks for image classification and object detection. Pattern Recognition. 2022;123:108411.
Garcia ACP. Convolutional Neural Networks and Residual Connections for Cow Teat Image Classification. arXiv preprint arXiv:14091556. 2014;1.
Xu L, Huang J, Nitanda A, Asaoka R, Yamanishi K. A novel global spatial attention mechanism in convolutional neural network for medical image classification. arXiv preprint arXiv:200715897. 2020.
Abdi M, Nahavandi S. Multi-residual networks: Improving the speed and accuracy of residual networks. arXiv preprint arXiv:160905672. 2016.
[Available from: https://www.kaggle.com/datasets/andrewmvd/medical-mnist. .
Hassan E, Hossain MS, Saber A, Elmougy S, Ghoneim A, Muhammad G. A quantum convolutional network and ResNet (50)-based classification architecture for the MNIST medical dataset. Biomedical Signal Processing and Control. 2024;87:105560.
Sarvamangala D, Kulkarni RV. Convolutional neural networks in medical image understanding: a survey. Evolutionary intelligence. 2022;15(1):1-22.
Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, Liang J. Convolutional neural networks for medical image analysis: Full training or fine tuning? IEEE transactions on medical imaging. 2016;35(5):1299-312.
Schwendicke F, Golla T, Dreher M, Krois J. Convolutional neural networks for dental image diagnostics: A scoping review. Journal of dentistry. 2019;91:103226.
Gu R, Wang G, Song T, Huang R, Aertsen M, Deprest J, et al. CA-Net: Comprehensive attention convolutional neural networks for explainable medical image segmentation. IEEE transactions on medical imaging. 2020;40(2):699-711.
Anwar SM, Majid M, Qayyum A, Awais M, Alnowami M, Khan MK. Medical image analysis using convolutional neural networks: a review. Journal of medical systems. 2018;42:1-13.
Zhang J, Li D, Wang L, Zhang L. Auto machine learning for medical image analysis by unifying the search on data augmentation and neural architecture. arXiv preprint arXiv:220710351. 2022.
Jiang P, Liu J, Wang L, Ynag Z, Dong H, Feng J. Deeply Supervised Layer Selective Attention Network: Towards Label-Efficient Learning for Medical Image Classification. arXiv preprint arXiv:220913844. 2022.
Rajaraman S, Ganesan P, Antani S. Deep learning model calibration for improving performance in class-imbalanced medical image classification tasks. PloS one. 2022;17(1):e0262838.
Zheng Z, Jia X. Label distribution learning via implicit distribution representation. arXiv preprint arXiv:220913824. 2022.
Valliani AA, Gulamali FF, Kwon YJ, Martini ML, Wang C, Kondziolka D, et al. Deploying deep learning models on unseen medical imaging using adversarial domain adaptation. Plos one. 2022;17(10):e0273262.
Nawaz M, Nazir T, Baili J, Khan MA, Kim YJ, Cha J-H. CXray-EffDet: chest disease detection and classification from X-ray images using the EfficientDet model. Diagnostics. 2023;13(2):248.
Awad FH, Hamad MM, Alzubaidi L. Robust classification and detection of big medical data using advanced parallel K-means clustering, YOLOv4, and logistic regression. Life. 2023;13(3):691.
Lee M. Mathematical analysis and performance evaluation of the gelu activation function in deep learning. Journal of Mathematics. 2023;2023.
Li J, Jin K, Zhou D, Kubota N, Ju Z. Attention mechanism-based CNN for facial expression recognition. Neurocomputing. 2020;411:340-50.
Murugan P, Durairaj S. Regularization and optimization strategies in deep convolutional neural network. arXiv preprint arXiv:171204711. 2017.
Song H, Kim M, Park D, Lee J-G. Prestopping: How does early stopping help generalization against label noise? 2019.
Vani S, Rao TM, editors. An experimental approach towards the performance assessment of various optimizers on convolutional neural network. 2019 3rd international conference on trends in electronics and informatics (ICOEI); 2019: IEEE.
Mao A, Mohri M, Zhong Y. Cross-entropy loss functions: Theoretical analysis and applications. arXiv preprint arXiv:230407288. 2023.
Vajapeyam S. Understanding Shannon's entropy metric for information. arXiv preprint arXiv:14052061. 2014.
- Abstract Viewed: 216 times
- PDF Downloaded: 139 times