INVESTIGATING THE PSYCHOMETRIC PROPERTIES OF A SET OF LOCAL STEM LITERACY INSTRUMENTS IN RASCH MODEL
Author ORCID Identifier
https://orcid.org/0000-0003-2576-6419
Date Available
8-7-2025
Year of Publication
2025
Document Type
Doctoral Dissertation
Degree Name
Doctor of Philosophy (PhD)
College
Education
Department/School/Program
Educational Policy Studies and Eval
Faculty
Dr. Kelly Bradley
Abstract
This study investigates the psychometric features of the See Blue STEM literacy inventories (SBSTEMs, including the elementary version SB-E and the middle school version SB-M) to determine their validity in measuring STEM literacy. Using data from elementary and middle school students participating in the See Blue STEM camp in 2019 and 2021-22, this research employed the Rasch Partial Credit Model (PCM) to analyze item-level performance and identify DIF items across various demographic subgroups (race, gender, free-lunch status, and potential first-generation status) and camp delivery modes (online versus in-person). Results indicated that both the elementary (SB-E) and middle school (SB-M) versions of the SBSTEMs demonstrate strong validity, with good model-data fit across all examined dimensions. The elementary assessment (SB-E) showed robustness, exhibiting gender-DIF-free measurement and only one item displaying minor differential item functioning across demographic groups. However, the SB-E exhibited somewhat limited differentiation ability, probably attributable to its shorter test length. The middle school assessment (SB-M) demonstrated excellent differentiation ability, though a small number of items showed statistically significant DIF across certain demographic and delivery mode variables. These findings collectively underscore the utility of Rasch PCM for psychometric evaluation and bias detection. This research contributes to the field by demonstrating that the SBSTEMs are largely unbiased concerning demographic variables and camp delivery modes. It provides a valuable tool for future researchers and educators aiming to ensure psychometric soundness in educational assessments. Further investigation is needed to continue refining items of SBSTEMs and to apply SBSTEMs to larger populations for precise estimation.
Digital Object Identifier (DOI)
https://doi.org/10.13023/etd.2025.312
Recommended Citation
Shen, Lijun, "INVESTIGATING THE PSYCHOMETRIC PROPERTIES OF A SET OF LOCAL STEM LITERACY INSTRUMENTS IN RASCH MODEL" (2025). Theses and Dissertations--Educational Policy Studies and Evaluation. 117.
https://uknowledge.uky.edu/epe_etds/117
