Calculation of Sensitivity and Specificity from Partial Data for Meta-Analyses: Introducing Some Practical Methods
学术急诊医学档案,
卷 13 编号 1 (2025),
6 九月 2025
,
第 e56 页
https://doi.org/10.22037/aaemj.v13i1.2678
摘要
Introduction: Meta-analyses of diagnostic/prognostic studies for calculating the pooled sensitivity and specificity require true positive (TP), true negative (TN), false positive (FP), and false negative (FN) counts. However, few studies report these values directly. This study aimed to consolidate practical methods to reconstruct sensitivity and specificity from minimal data.
Methods: Our framework addresses three main situations: (1) algebraic rearrangements to compute specificity given partial metrics; (2) digitization of receiver operating characteristic (ROC) curves to obtain threshold-specific sensitivity and specificity; and (3) application of the binormal model when only AUC and prevalence are available. We tested these methods on a dataset related to mortality prediction in myocardial infarction (MI) using machine learning models, assessing how well they reconstructed sensitivity and specificity.
Results: Algebraic formulas and ROC digitization yielded reliable estimates when partial metrics or graphical curves were sufficiently detailed. However, the binormal model, which assumes equal variances, showed noticeable inaccuracies, especially for sensitivity. Linear regression analyses indicated that higher prevalence and higher AUC reduced estimation errors.
Conclusion: These methods offer practical alternatives for reconstructing diagnostic accuracy measures when data are incomplete. Relying solely on AUC-based estimations may introduce substantial bias, particularly in low-prevalence contexts. We recommend that primary studies report threshold-specific sensitivity and specificity to support more accurate meta-analytic estimations.
- Diagnostic Accuracy
- Prognostic study
- Sensitivity and Specificity
- Meta-Analysis
- Predictive Value of Tests
- specificity
- ROC curve
- Area Under the Curve
- Statistics
##submission.howToCite##
参考
Cleophas TJ, Zwinderman AHJCc, medicine l. Meta-analyses of diagnostic studies. 2009;47(11):1351-4.
Ter Riet G, Bachmann LM, Kessels AG, Khan KSJBE-BM. Individual patient data meta-analysis of diagnostic studies: opportunities and challenges. 2013;18(5):165-9.
Walter S, Sinuff TJJoce. Studies reporting ROC curves of diagnostic and prediction data can be incorporated into meta-analyses using corresponding odds ratios. 2007;60(5):530-4.
Hanley JAJMdm. The robustness of the" binormal" assumptions used in fitting ROC curves. 1988;8(3):197-203.
Hillis SLJAr. Simulation of unequal-variance binormal multireader ROC decision data: an extension of the Roe and Metz simulation model. 2012;19(12):1518-28.
Stapor K, editor Evaluation of classifiers: current methods and future research directions. FedCSIS (Position Papers); 2017.
Zhou X-H, Obuchowski NA, McClish DK. Statistical methods in diagnostic medicine: John Wiley & Sons; 2014.
Carter JV, Pan J, Rai SN, Galandiuk SJS. ROC-ing along: Evaluation and interpretation of receiver operating characteristic curves. 2016;159(6):1638-45.
Huwaldt JA, Steinhorst SJUhpsn. Plot digitizer. 2013.
Kumar R, Indrayan AJIp. Receiver operating characteristic (ROC) curve for medical researchers. 2011;48:277-87.
Wald NJ, Bestwick JPJJoMS. The area under the ROC curve: is it a valid measure of screening performance? : SAGE Publications Sage UK: London, England; 2014. p. 220-.
Hajian-Tilaki KO, Hanley JA, Joseph L, Collet J-PJMDM. A comparison of parametric and nonparametric approaches to ROC analysis of quantitative diagnostic tests. 1997;17(1):94-102.
Jiménez-Valverde AJB, Conservation. Threshold-dependence as a desirable attribute for discrimination assessment: implications for the evaluation of species distribution models. 2014;23:369-85.
Bandos AI, Guo B, Gur DJAR. Estimating the area under ROC curve when the fitted binormal curves demonstrate improper shape. 2017;24(2):209-19.
Hanley JAJSim. The use of the ‘binormal’model for parametric ROC analysis of quantitative diagnostic tests. 1996;15(14):1575-85.
Metz CE, editor Basic principles of ROC analysis. Seminars in nuclear medicine; 1978: Elsevier.
WALSH SJJSim. Limitations to the robustness of binormal ROC curves: effects of model misspecification and location of decision thresholds on bias, precision, size and power. 1997;16(6):669-79.
- 摘要 ##plugins.themes.ojsPlusA.frontend.article.viewed##: 368 ##plugins.themes.ojsPlusA.frontend.article.times##
- pdf (English) ##plugins.themes.ojsPlusA.frontend.article.downloaded##: 833 ##plugins.themes.ojsPlusA.frontend.article.times##
##plugins.themes.ojsPlusA.frontend.article.downloadstatastics##
##plugins.block.makeSubmission.linkLabel##
SJR
COPE
