Novelty in Biomedicine

Abstract

Background and Aim: The rapid expansion of biomedical literature has made the manual identification and extraction of biomarker information increasingly time-consuming. Natural language processing (NLP) tools may facilitate large-scale literature screening and support the generation of research hypotheses. However, their ability to broadly identify diverse biomarker types without predefined target terms remains insufficiently characterized.

Materials and Methods: Fifty original articles on colorectal cancer biomarkers published between 2015 and 2025 were manually reviewed, and reported biomarkers were extracted to establish a reference dataset. The same articles were processed using two biomedical NLP models: d4data/biomedical_ner_all and en_ner_bionlp3cg_md from the SciSpaCy framework. Biomarkers extracted by each model were compared with the manually curated reference set. The proportion of manually identified biomarkers recovered by each model was calculated, and extracted entities not present in the reference set were also assessed. Statistical analyses were performed using an independent-samples t-test.

Results: Both NLP pipelines successfully processed 94% of the 50 articles. The mean matching rates with the manually curated biomarkers were 49.6% for SciSpaCy and 50.7% for BioMedNLP, indicating only a minimal difference between the two models. More than 50% of the manually identified biomarkers were recovered in 46% of the SciSpaCy outputs and 54% of the BioMedNLP outputs. Additional entities not present in the manual reference set accounted for 21.4% and 19.2% of the entities extracted by SciSpaCy and BioMedNLP, respectively. The differences between NLP-derived and manually curated biomarker counts were statistically significant (P<0.0001).

Conclusion: Without predefined biomarker-specific keywords or domain-specific fine-tuning, the evaluated NLP models recovered approximately half of the manually identified biomarkers. Nevertheless, the relatively similar distribution of biomarker categories suggests that these tools may provide useful preliminary overviews of biomarker landscapes. Their most appropriate role may therefore be as screening and ideation-support tools, followed by manual verification for precise biomarker identification and interpretation.