Systematic contextual biases in SegmentNT potentially relevant to other nucleotide transformer models

Authors

• Mark T.W. Ebbert, University of KentuckyFollow
• Anna Ho, University of Kentucky College of Medicine
• Madeline L. Page, University of Kentucky College of Medicine
• Bram Dutch, University of Kentucky
• Blake K. Byer, University of Kentucky College of Medicine
• Kristen L. Hankins, University of Kentucky
• Hady Sabra, University of Kentucky College of Medicine
• Bernardo Aguzzoli Heberle, University of Kentucky College of MedicineFollow
• Mark E. Wadsworth, University of Kentucky College of Medicine
• Grant A. Fox, University of Kentucky College of Medicine
• Bikram Karki, University of Kentucky College of Medicine
• Caylin Hickey, University of Kentucky College of Medicine
• David W. Fardo, University of KentuckyFollow
• Cody Bumgardner, University of KentuckyFollow
• Yasminka A. Jakubek, University of KentuckyFollow
• Cody J. Steely, University of Kentucky
• Justin B. Miller, University of KentuckyFollow

Abstract

Recent advances in large language models have extended to genomic applications, yet model robustness relative to context is unclear. Here, we demonstrate two intrinsic biases (input sequence length and nucleotide position) affecting SegmentNT results, a model included with the Nucleotide Transformer that provides nucleotide-level predictions of biological features. We demonstrate that nucleotide position within the input sequence (beginning, middle, or end) alters the nature of SegmentNT’s raw prediction probabilities, which can be standardized to improve prediction consistency. While longer input sequence length improves model performance, diminishing returns suggest a surprisingly small input length of ∼3072 nucleotides might be sufficient for many applications. We further identify a 24-nucleotide periodic oscillation in SegmentNT’s prediction probabilities, revealing an intrinsic bias potentially linked to the model’s training tokenization (6-mers) and architecture. We identify potential approaches to account for these biases and provide generalizable insights for utilizing nucleotide-resolution functional prediction models.

Document Type

Article

Publication Date

8-12-2026

Notes/Citation Information

Publisher Copyright: © The Author(s) 2026. Published by Oxford University Press. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.

Digital Object Identifier (DOI)

10.1093/nar/gkag704

Archival?

Archival

Repository Citation

Ebbert, Mark T.W.; Ho, Anna; Page, Madeline L.; Dutch, Bram; Byer, Blake K.; Hankins, Kristen L.; Sabra, Hady; Aguzzoli Heberle, Bernardo; Wadsworth, Mark E.; Fox, Grant A.; Karki, Bikram; Hickey, Caylin; Fardo, David W.; Bumgardner, Cody; Jakubek, Yasminka A.; Steely, Cody J.; and Miller, Justin B., "Systematic contextual biases in SegmentNT potentially relevant to other nucleotide transformer models" (2026). Biostatistics Faculty Publications. 77.
https://uknowledge.uky.edu/biostatistics_facpub/77