Date Available

4-27-2021

Year of Publication

2021

Degree Name

Doctor of Philosophy (PhD)

Document Type

Doctoral Dissertation

College

Education

Department/School/Program

Education Sciences

First Advisor

Dr. Kelly D. Bradley

Second Advisor

Dr. Shannon O. Sampson

Abstract

Since the mid-twentieth century, the U.S. Department of Education has drafted and enacted policies to bridge the research-practice gap—that is, the gap between “what works” according to educational research and what is actually practiced by teachers and their administrators (e.g., Dirkx, 2006; Joyce & Cartwright, 2019; Tseng, 2012). One of the latest manifestations of this “what works” political legacy is the What Works Clearinghouse (WWC), which took shape as part of the Institute of Education Sciences (IES) in 2002. The WWC’s mission is to be a “central and trusted source of scientific evidence for what works in education” (WWC, 2020d, p. 1) while, at the same, helping the IES “…increase [the] use of data and research in education decision-making” (IES, n.d.-a). The purpose of this dissertation is to evaluate the extent to which the WWC has realized its own mission as well as contributed to the IES’s larger goal.

Guided by principles of evaluative thinking (Vo & Archibald, 2018) and premises of the Two-Communities theoretical tradition (Caplan, 1979; Farley-Ripple et al., 2018), this project used a theory-based evaluation approach called contribution analysis (Mayne, 2008, 2012b, 2019) to investigate three guiding questions. Those questions inquired into (a) the extent of the WWC’s impact among educators, (b) the reasons why its impact may be wanting, and (c) the changes it could make to maximize its impact. To investigate these questions, a six-step procedure was used to both articulate and scrutinize the WWC’s theory of change according to available evidence. An array of evidence was considered, including existing publications (e.g., previously published evaluations, literature reviews, and large-scale surveys), analyses of publicly available data (e.g., public data exports, data requested through the Freedom of Information Act, transcripts from congressional hearings), and findings from a preservice teacher survey conducted for this project.

The results of this contribution analysis offered compelling answers to each of the three guiding questions. First, given the WWC’s original benchmark for success (e.g., Baldwin et al., 2008), evidence suggested that it is likely failing to fully reach educators and guide their decision-making. This was especially true for teachers. Second, the evidence suggested that the WWC’s impact may be wanting because its theory of change depends on several unsupported assumptions. Not only were many of the WWC’s causal assumptions refuted by the evidence, but some of its foundational assumptions—such as the belief that systematic research review would be an effective way of bringing educational research to practice—were refuted as well. Finally, because several of its foundational assumptions were refutable, the WWC may only be able to maximize its impact if it fundamentally retools its approach to systematic research review or to educational research more generally. Suggestions for doing so are discussed.

Digital Object Identifier (DOI)

https://doi.org/10.13023/etd.2021.052

Download

Share

COinS