24  Understanding the hypothesis

Experimental papers are designed to test hypotheses. One of the first tasks in designing a replication is to understand the hypothesis proposed in the original study. Only by understanding the hypothesis can you make sense of the experimental design and critically assess whether the design is an effective test.

The replication report should state the hypotheses and the predictions arising from them. Describe each hypothesis specifically, capturing the direction of the effect and, where possible, the expected magnitude. Avoid phrases such as “can influence”; instead use words like “increase”, “decrease”, “higher”, “lower”, “strengthen” or “weaken”. Be clear about which group should be higher and by how much.

24.1 Example

In (Dietvorst et al., 2015), the hypothesis is that “seeing the model perform, and therefore err, would decrease participants’ tendency to bet on it rather than the human forecaster, despite the fact that the model was more accurate than the human.” Note that:

• The hypothesis is not that people are inherently averse to algorithms(which requires a judgment as to the appropriate level of use). It is that seeing an algorithm err increases (or generates) that aversion.
• There is an implicit hypothesis that seeing a human err does not produce the same relative aversion. The phenomenon is specific to the algorithm.

Dietvorst, B. J., Simmons, J. P., and Massey, C. (2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, 144, 114–126. https://doi.org/10.1037/xge0000033