This page is intended as a resource for undergraduate and postgraduate students embarking on a research dissertation.
When planning your study, I recommend considering the research question, the experiment design, the constructs (and how to measure them), the hypotheses and the statistical analyses all at the same time. Unless you know what you are doing on all fronts, you do not have a thoroughly planned out research project, and this can lead to unhappiness.
One of the best ways to formulate a well-planned research project is to complete a pre-registration form. Ask me about this – I have a modified pre-registration form that I have found useful, and I can point you towards examples of formally pre-registered studies. Completing the pre-registration form will involve thinking about the following key components of a research project…
Research question(s)
These are your high-level questions which you should be able to write in plain English that pretty much anyone can understand. For example, “How does hunger affect impulsive decision making?” or “Does impulsivity affect body composition through the mediating variable of physical activity?”
Experiment design
What kind of experiment design is needed to answer your research question? It is important to remember that there is no one single golden true experimental approach for a given research question – instead there are a variety of choices and compromises to be made. For example, if you are running a lab-based study you might be in the position to answer how hunger affects decision making through experimentally manipulating hunger levels. In which case, you can outline your experimental design in terms of what constructs are being manipulated, how so, and how will you measure the success of the experimental manipulation? In other contexts, you may take a purely observational approach to experiment design.
Participants
Who will be your participants? This answer may well end up being a compromise between what is ideal for your research question and what is practicable. If you are collecting data on students, it is possible that they may have characteristics which are not fully representative of the broader population. If this has an impact on the ability to generalise your findings, that does not necessarily mean the project has no merit, but you should consider how it affects your research conclusions. If you are collecting questionnaire data, how will the dissemination methods influence the participant composition? For example, if you are posting your questionnaire to special interest groups, there is the potential for selection bias which you might want to consider.
Constructs
Once you have a high level research question, you are probably all set with a few constructs that need to be measured. For example, if the research question is “Does impulsivity affect body composition through the mediating variable of physical activity?”, then your constructs are: impulsivity, body composition, and physical activity.
Measures
Your job will be to work out how to quantify your constructs of interest. This is not always an easy thing to do, but various measures have been built which attempt to quantify different constructs. I have a page of measures of interest here. One important point to consider is if you will end up with just a single measurement for each construct, or if there will be multiple. For example, some measures of impulsivity will break down into a number of sub-scales of different forms of impulsivity.
Hypotheses
While research questions are high-level and understandable by anyone, statistical hypotheses are more specific and technical. You are likely going to have many more hypotheses than research questions. For example, if your research question is “How does hunger affect impulsive decision making?” then you can imagine a full answer to this question might involve multiple sub research questions which can be phrased as specific hypotheses such as:
- Is my measure of hunger lower in the hunger condition, compared to the control condition?
- Is my measure of impulsivity higher in the hunger condition, compared to the control condition?
Statistical analyses
The important point is that unless you have thought through each of these steps (research question, experiment design, constructs, measures, hypotheses, and statistical analyses), then this is a recipe for unhappiness. It is beyond the scope of this page to delve into how to choose statistical analyses, but you should think through what statistical tests will enable you to quantitatively evaluate your hypotheses. In thinking about this, it is vital to ask questions such as:
- What is my outcome variable? Is is categorical or continuous?
- What are my predictor variables? Are they all categorical or all continuous, or is there a mixture?
Answering these will go a long way to selecting appropriate analyses.
One point of confusion that I often come across is how to identify the predictor and outcome variables. My answer to this is that while on one level there is probably some kind of causal direction of influence. In such cases, it often makes sense for the predictors to be putatively causal factors driving the outcome variable. That said, there are situations where regardless of any causal direction we want to ask different statistical questions. In this case the research questions are more like “Do these predictor variables predict this outcome variable?” but with no commitment to the causal direction. So is it entirely possible that a measure might be an outcome variable for one research question, but a predictor variable for a different research question. That is, there is nothing inherent about a measure impulsivity that makes it a predictor variable, it depends entirely on what your research questions are.