Hypothesis Testing

Operationally defining (measuring) the study

The question arises: What do we mean by performance? Clearly, performance could mean how students score in a piece of coursework, how many times they can answer questions in class, what marks they get in their exams, and so on. There are two obvious reasons why we should be clear about how we operationalise (that is, measure) what we are studying. First, we simply need to be clear so that people reading our work are in no doubt about what we are studying. This makes it easier for them to repeat the study in future to see if they also get the same (or similar) results; something called internal validity. Second, one of the criteria by which quantitative research is assessed, perhaps by an examiner if you are a student, is how you define what your are measuring (in this case, performance) and how you choose to measure it.

Nonetheless, it is worth noting that these choices will sometimes be personal choices and at other times they will be guided by something else. For example, if we were to measure intelligence, there may be a number of characteristics that we could use, such as IQ, emotional intelligence, and so forth. What we choose here will likely be a personal choice because all these variables are proxies for intelligence; that is, they are variables used to infer an individual’s intelligence but not everyone would agree that IQ alone is an accurate measure of intelligence. In comparison, if we were measuring firm performance, there may be an established number of measures in the academic and practitioner literature that determine what we should test, such as Return on Assets, etc… Therefore, to know what you should measure, it is always worth looking at the literature first to see what other studies have done, whether you use the same measures or not. It is then a matter of making an educated decision whether the variables you choose to examine are accurate proxies for what you are trying to study, as well as discussing the potential limitations of these proxies.

In the case of measuring a student’s performance there are a number of proxies that could be used, such as class participation, coursework marks and exam marks, since these are all good measures of performance. However, in this case, we choose exam marks as our measure of performance for two reasons. First, as a statistics tutor, we feel that Sarah’s main job is to help her students get the best grade possible since this will affect her students’ overall grades in their graduate management degree. Second, the assessment for the statistics course is a single 2 hour exam. Since there is no coursework and class participation is not assessed in this course, exam marks seem to be the most appropriate proxy for performance. However, it is worth noting that if the assessment for the statistics course was not only a 2 hour exam but also a piece of coursework, we would probably have chosen to measure both exam marks and coursework marks as proxies of performance.

Variables

The next step is to define the variables that we are using in our study (see the statistical guide, Types of Variable, for more information). Since the study aims to examine the effect that two different teaching methods – providing lectures and seminar classes (Sarah) and providing lectures by themselves (Mike) – had on the performance of Sarah’s 50 students and Mike’s 50 students, the variables being measured are:

Dependent variable:	Exam marks
Independent variable:	Teaching method (“seminar” vs. “lecture only”)

By using a very straightforward example, we have only one dependent variable and one independent variable although studies can examine any number of dependent and independent variables.

One- and two-tailed predictions

Returning to our discussion of the null and alternative hypothesis, it is worth re-stating that if the two distributions (the seminar distribution and the lecture only distribution) are the same, this would mean that the addition of seminars to lectures as a teaching method did not have an effect on students’ performance and we would accept the null hypothesis. Alternatively, if there was a difference in the distributions and this difference was statistically significant, we would reject the null hypothesis. The question then arises: Do we accept the alternative hypothesis?

Alternative Hypothesis (Ha):

Undertaking seminar class has a positive effect on students’ performance.

The alternative hypothesis tells us two things. First, what predictions did we make about the effect of the independent variable(s) on the dependent variable(s)? Second, what was the predicted direction of this effect? Let’s use our example to highlight these two points.

Sarah predicted that her teaching method (independent variable: teaching method), whereby she not only required her students to attend lectures (distribution: lecture only), but also seminars (distribution: seminar), had a positive effect (that is, increased) students’ performance (dependent variable: exam marks). The direction of her prediction is what we call one-tailed, which can be illustrated using the diagrams below.