Date Available

4-25-2017

Year of Publication

2017

Document Type

Doctoral Dissertation

Degree Name

Doctor of Philosophy (PhD)

College

Arts and Sciences

Department/School/Program

Psychology

Advisor

Dr. C. Melody Carswell

Abstract

Much of the research on metacognition in human factors has focused on prescriptive, normative strategy training. That is, many researchers have concentrated their efforts on finding ways to improve system users’ prediction, planning, monitoring and evaluation strategies for tasks. However little research has focused on the strategies and heuristics users employ on their own to make usability predictions. Understanding usability prediction methods is critical because users’ predictions inform their expectations about whether they will make errors using a product, how much effort they will need to expend to be successful in using the product, whether they can perform two tasks successfully at the same time, whether the costs of learning to use a device are worth the benefits of using it, which tools will assist in accomplishing goals and which tools will make performing the same task more difficult. The following study aims to identify the specific strategies people use to make usability predictions about product designs. From these strategies a set of guidelines, for designers who wish to ensure users’ expectations meet post hoc usability assessments, were proposed. The study was completed in two phases.

During the first phase of this study, prediction strategies were elicited by 1) asking participants to make routine product usability judgments, from which implicit strategies can be inferred, and by 2) using explicit free-response methods. Judgments were analyzed using multi-dimensional scaling (MDS) methods to establish the number of dimensions that are implicitly used to predict usability. Subject matter experts (SMEs) coded free-response strategies using coding schemes developed in a pilot study. SMEs will also matched user strategies to formal, professional usability standards. The outcome of Phase 1 was usability taxonomy for classifying usability strategies that includes both expert and user language. The procedure was repeated with three different product design classes to determine how strategies differ as a function of the to-be-judged product.

During the second phase of the study, a new group of participants rated specific usability attributes of designs to validate the strategies collected from users’ free-responses in Phase 1. Attributes were selected based on the strategies discovered in Phase 1. These usability attribute ratings helped to inform interpretations of the dimensions of the MDS model generated in Phase 1 and provided input into defining the usability attributes that influenced usability predictions.

Results of this study reveal that the type design class participants evaluated had a significant effect on the type of strategy participants used to make their a priori usability assessments (UAs). Participants reported using “complexity” or “organization” most often to predict the usability of cookbooks. Participants reported using “mental simulation” or “typicality/familiarity” most often for predicting the usability of drinking glasses. Participants reported using “complexity,” “organization,” and to a lesser extent “typicality/familiarity,” and “mental simulation” as strategies for predicting the usability of cooktops. MDS methods were used to uncover the underlying dimension of the UA space. For drinking glasses, the “fanciness” and “holdability” were associated with UAs. For cooktops, “the number of controls” and whether participants believed “it was easy to understand how each burner was controlled” were associated with making UAs. And for cookbooks, “the length of the instructions” and “poor contrast of the text with the background” were associated with UAs. Overall, there is evidence that at least some participants in Phase 2 used terminology that was consistent with the terminology people used to describe the designs during Phase 1 and that these were congruent with the uncovered strategies.

Digital Object Identifier (DOI)

https://doi.org/10.13023/ETD.2017.160

Download

Share

COinS