Do Students Behave Rationally in Multiple Choice Tests? Evidence from a Field Experiment

Abstract

A disadvantage of multiple choice tests is that students have incentives to guess. To discourage guessing, it is common to use scoring rules that either penalize wrong answers or reward omissions. In psychometrics, penalty and reward scoring rules are considered equivalent. However, experimental evidence indicates that students behave differently under penalty or reward scoring rules. These differences have been attributed to the different framing (penalty versus reward). In this paper, we model students’ behavior in multiple choice tests as a choice among lotteries. We show that strategic equivalence among penalty and reward scoring rules holds only under risk neutrality. Therefore, risk aversion could be an alternative explanation to the previously found differences in students’ behavior when confronted with penalty and reward scoring rules. We suggest the use of a modified penalty scoring rule which is equivalent to the reward rule for whatever risk attitudes students might have. To disentangle the effect of framing and risk aversion on students’ behavior we design a field experiment with three treatments, each one with a different scoring rule. Two of these scoring rules are equivalent but have different framing, while the third is not equivalent but has the same framing as one of the other two. The experimental results indicate that differences in students’ behavior are due to risk aversion and not due to different framing.