Author ORCID Identifier

https://orcid.org/0009-0008-1516-9948

Date Available

5-1-2025

Year of Publication

2025

Document Type

Doctoral Dissertation

Degree Name

Doctor of Philosophy (PhD)

College

Education

Department/School/Program

Educational Policy Studies and Eval

Faculty

Jungmin Lee

Faculty

Kelly Bradley

Faculty

Neal Hutchens

Abstract

Student engagement is a central variable in higher education research as it influences academic achievement, retention, and the development of problem-solving skills. The Community College Survey of Student Engagement (CCSSE) is a vital tool for assessing student engagement in community colleges. Previous validation studies were mainly conducted in the mid-to-late 2000s and have used Classical Test Theory frameworks despite the known limitations of this approach. The present study seeks to evaluate the psychometric properties of the CCSSE survey using a Rasch Analysis framework. The two CCSSE datasets were obtained from the Center for Community College Student Engagement at The University of Texas at Austin. These datasets include responses from the 2019 and 2023 CCSSE cohort member colleges. Two research questions: a) To what extent do the 2019 and 2023 CCSSE survey benchmarks align with the assumptions and expectations of the Rasch model, considering issues of unidimensionality and fit? and b) How do racking and anchoring methods compare in measuring changes in item difficulty parameters of the CCSSE survey, specifically between the 2019 and 2023 survey administrations? were used to explore the fits and functionality of the survey.

These research questions were addressed using Rasch analysis to examine the psychometric properties of the 2019 and 2023 CCSSE benchmarks, determining how well the items fit the assumptions of the Rasch model. Additionally, racking and anchoring techniques were applied to compare their techniques in measuring changes in item difficulty parameters over time, identifying items that exhibited significant shifts in difficulty between the 2019 and 2023 survey administrations.

The Principal Component Analysis of Rasch residuals suggests that three of the five benchmarks demonstrate unidimensionality, while two show multidimensional traits. The Pearson and Disattenuated correlation coefficients further support this, indicating that two benchmarks are likely multidimensional, while three remain unidimensional. However, after examining the wording of the items that could cause potential dimensionality issues, this study concluded that all five survey benchmarks are unidimensional; however, some items may need to be revisited and reworded.

The survey benchmarks showed strong item reliability and separation but weaker person reliability and separation. Infit and outfit MNSQ statistics revealed acceptable values for nearly all items, supporting the validity of the survey. Wright's maps for the survey benchmarks showed a well-distributed item spread, suggesting that the survey is well-targeted to the student. However, the gaps at both ends of the Wright maps suggest that the items may not capture the range of student engagement traits in each benchmark. The anchoring and racking analyses revealed that the difficulty of most items remained stable between the two survey administrations. Overall, this study found that the CCSSE survey can be effectively used to assess various aspects of student engagement.

Digital Object Identifier (DOI)

https://doi.org/10.13023/etd.2025.114

Download

Share

COinS