Hypothesis Testing

If we (a) made a one-tailed prediction, (b) predicted that our distribution (the “seminar” distribution) moved in the correct direction (either up or down the scale), and (c) the result was statistically significant at the level we selected (either 0.05 or 0.01), then we accept the alternative hypothesis. If we were wrong about either the direction of our prediction or the result was not statistically significant at the selected level, then we reject the alternative hypothesis.



If we made a two-tailed prediction, this has a bearing on our significance level because we are unsure in which direction our distribution (the “seminar” distribution) will move (up or down). As a result, both ends of our distribution are relevant. Allowing a 5% or less chance that a score from our distribution (the “seminar” distribution”) came from the comparison distribution (the “lectures only” distribution) would result in a 10% or less chance of making an error because we have to add together the 5% or less significance level from both ends of our distribution (see diagram A). Therefore, we can only allow at 2.5% chance or less (0.025) that a score from our distribution came from the comparison distribution (see diagram B), which added together would ensure that our overall chance of making an error remained at the 0.05 significance level.

On this basis, if we (a) made a two-tailed prediction, and (b) the result was statistically significant at the level we selected (either 0.025 or 0.005), then we accept the alternative hypothesis, that “undertaking seminar class has an effect on students’ performance”, but we would also state the direction of the effect that was indicated by our results. In other words, we would state that undertaking seminar class has a “positive” effect on students’ performance.

Moving forward

Other key aspects of hypothesis testing include a discussion of normality, selecting statistical tests, and running these statistical tests. These are discussed in the statistical guide, Selecting Statistical Tests. However, we would recommend that before you read this guide, you first read the guide on Sampling.