Gaming the test: When should test threshold be unpredictable?

Abstract

We study the design of thresholds in pass/fail tests. The principal aims for the agent to pass when the agent's natural type (e.g. ability) is sufficiently high. However, the agent can manipulate the perceived natural type at a cost. Randomizing the passing threshold becomes optimal when the principal faces significant uncertainty about the agent's gaming costs. In particular, any positive probability that the agent cannot game (i.e., has infinite gaming costs) makes randomization optimal. Randomization allows high natural types with high gaming costs to pass, at least with some probability, without incentivizing low ability types to game. Moreover, it incentivizes low natural types to limit their gaming, thereby separating them from high natural types. We identify randomization strategies that allow the principal to achieve better outcomes than a deterministic threshold, without requiring precise knowledge of the distribution of the agent's natural type and gaming ability. We thereby provide robust rules for when unpredictable test thresholds are justified.