Year of Publication

2008

College

Martin School of Public Policy and Administration

Date Available

8-27-2014

Executive Summary

Questionnaires can be useful tools for evaluating program outcomes in human services programs, in that they provide quick, easy-, and inexpensive-to-gather information about the program's success in meeting its objectives. Questionnaires may be easy to use, but they are not necessarily easy to design. If developed using sound methodology, the data gathered from questionnaires can be valid and of use to administrators, board members, direct-service staff, and funders in improving a program's services without a more resource-intensive program evaluation.

An exit questionnaire was developed to evaluate the Bluegrass Domestic Violence Program's (BDVP) achievement of short-term outcomes. Short-term outcomes for the BDVP were defined using Kentucky Victim Service Standards mandated service provision guidelines for Kentucky Domestic Violence Shelters, as well as through conversations with the Bluegrass Domestic Violence Program's Executive Director, Darlene Thomas. The questionnaire was then pre-tested using cognitive interviewing techniques.

Results of cognitive interviews revealed several classes of problems with the original questionnaire, including lexical, temporal, and computational problems with questionnaire items. As well, specific problems emerged in the single interview where the respondent was primarily Spanish-speaking and an interpreter was used.

Based on these findings, the following recommendations are made:

  • Change the wording of questions to more explicitly elicit the information about short-term outcomes desired, based on interview responses.
  • Determine how important it is to know the specific time frames in which some services were provided. Several of the questions involved very specific time frames (e.g. “within the first 2 days of arriving at shelter…”), and respondents found these questions almost impossible to answer as originally written.
  • Have the questionnaire document translated into other languages needed before engaging in cognitive testing, since this is the form it will be administered in once it is ready for the field.
  • Explore other structural issues that might weaken the instrument. Particular attention should be paid to question sequencing and methods for administering, collecting, and analyzing the questionnaire data.
  • Format the instrument to reflect current changes in programming, and complete additional iterations of the pre-testing exercise.

This study has several limitations, though it provides assessments of the existing exit questionnaire that can produce a stronger questionnaire instrument and that may be more likely to capture how well the BDVP meets its short-term outcomes.

Share

COinS
 
 

To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.