Student as researcher: how to approach a research project

Step 5: Choose a Research Design

A research design is a plan for answering a research question, a plan for testing the hypothesis. The design researchers choose depends on the research question and hypothesis, and ultimately, their goal for the research. In this section, we will cover each research design and provide examples. As you'll see, this section provides more detail than other sections. This is because choosing the research design is one of the most important steps in the research process.
What research design should I choose if I want to explore, describe or explain a phenomenon?
Three important goals of business research are exploration, description and explanation. In explorative research, researchers attempt to explore phenomena. They explore phenomena which are currently not well understood and often attempt to find possible explanations. The objective of descriptive research is to give a good account of reality. Explanatory research goes one step further by attempting to find causal relationships among the variables that are measured in the study. A correlation exists when two variables are associated (co-vary). Explanatory research tries to identify causal relations. However, establishing the causality of a relationship is difficult and often the research design does not facilitate it.
Often studies do not fall into one of these categories but contain explorative, descriptive and explanatory elements. Usually a good description of the phenomena is the base, from which researchers either try to either explore or explain it.
How do I conduct a study?
< Observe, measure variables >
Typically, researchers start an investigation by selecting a case or a sample and then observe and measure aspects which seem to be important. Observations and measurements vary in how structured they are. In a case study, observations and measurements are often much less structured, as the researcher does not want to limit perceptions by any structure imposed ex-ante. Studies based on surveys of larger sample or experiments are much more structured. The researcher predefines the aspects which are of interest and then measures them. This enhances the comparability of observations but reduces the chances of detecting aspects not covered by the predefined structure.

What research design should I choose if I want to understand causality?
< Experimental research design
Academic researchers are mainly interested in finding explanations for phenomena; we are interested in why things happen. It is, however, often difficult to really establish the causality of a relationship. Two common problems of causality are artefact correlations and reversed causality.
Artefact correlations
Recent research in the Netherlands has shown that there is a positive correlation between exercise and school performance supporting the Latin saying “mens sana in corpore sano” However, is this relation really causal or are there some individual characteristics that affect both exercise and school performance? An even more obvious example is the correlation between the number of storks and the number of newborn babies in Germany between 1950-1990. Data shows that the number of newborn babies followed the decline of the number of storks. Does this support the old belief that babies are brought by storks? Certainly not. A possible explanation for the correlation between storks and babies is industrialization. Industrialization reduced the habitats of storks causing a decline in their number and at the same time increased female labour participation with the consequence that women (and of course their partners) had less children.
Reversed causality
Studies have shown that the number of patents a company holds is positively related to performance. But what is the causation? One explanation is that companies investing more in R&D can obtain more patents which form a competitive advantage resulting in higher performance. Thus the causality runs from patents to performance. An alternative explanation is that firms who perform well have more resources to invest in R&D and consequently file more patents. Thus the causality runs from performance to patents.
To establish causality we need to fulfil three requirements: (1) the two variables need to co-vary. This requirement is met in all examples above. (2) The cause needs to be before the effect, thus there is a time order. In the examples above we cannot establish this time order. (3) No other variable explains the outcome. This last requirement is problematic in the social sciences, because many factors influence social phenomena. The first two examples are, however, cases in which such a third factor explains the phenomenon better.
Often researchers, suggest that longitudinal studies are able to establish causation. Strictly speaking, this is not true, as longitudinal studies only ensure that the first and second requirement is met, but not the third. There is just one research approach that is able to establish causation beyond doubt and that is experiments.
How do I set up an experiment?
Experimental design allows researchers to control (manipulate) one or several variables, while all other variables will not affect the results between the experimental group and the control group, if participants have been randomly assigned to one or other group. An important feature of experimental design is that the researcher compares two (or more) conditions or groups. In one condition, a "treatment" is present in the situation (called the "treatment" condition), and in another condition, the treatment is either absent (the "control" or "comparison" condition) or a different condition is used.
Let us illustrate experimental design with an example from trust research. Participants of the experiment are asked to play a trust game with another participant. The structure of the trust-game is depicted in figure 5.1 and requires the two participants Antonie and Hermine to make sequential decisions and their earnings depend on their choices. First, Antonie has to decide whether she should trust Hermine or not. If she does not trust Hermine both receive € 25 and the game is over. If Antonie trusts Hermine, it is Hermine’s turn. Hermine can decide to honour the given trust and each will receive € 75 or she can decide to dishonour the trust given and then Antonie would receive nothing and Hermine would receive € 150. The problem of the game is of course that if Antonie trusts Hermine, Antonie might get nothing and Hermine can take all. However, for Hermine it would always be better if Antonie trusts her.
The game is played on a computer on which Antonie and Hermine cannot see each other. In the first condition they cannot communicate with each other. In the second condition Hermine can send a message promising Antonie € 30 if she does not honour trust and this promise is credible, i.e. Antonie will receive € 30 if Hermine dishonours trust. The point made here is that the researcher controls the independent variable, he controls whether Hermine can send a message and he can even control which message Hermine sends. You as a researcher might even choose to vary the content of the message Hermine can send by varying the promise from € 30 to € 25, € 20 or € 50 etc. You could even ask Antonie and Hermine to play different games and record their choices in each setting.
This hypothetical research study has two essential ingredients of an experiment: an independent variable and a dependent variable. An independent variable is controlled, or manipulated, by the researcher. In this hypothetical experiment, the variable we controlled is the possibility to communicate before the game, i.e. Hermine can give a commitment. Researchers measure dependent variables to determine the effect of the independent variable.
A second feature that must present in the experiment in order to conclude that commitments increase the chance that people trust each other is called holding conditions constant. Holding conditions constant means that the only thing we allow to vary in the two conditions is the presence or absence of the possibility to give commitments. Everything else for the two groups is the same. Remember that scientists seek to isolate the variables they think impact behaviour. By manipulating only whether a commitment is possible and holding all other potential variables constant, the researcher can test whether commitments influence trust.
So far, it has been shown that experiments are powerful designs, because you as a researcher control the independent variable. Researchers are, however, even more powerful, as they can decide who takes the role of Antonie and who takes the role of Hermine.
The researcher’s power to decide who takes which role, i.e. who is in which group is essential in experimental designs. For a true experiment, you apply random assignment, i.e. participants are randomly assigned to the different groups. Random assignment to groups has the huge advantage that the groups are equivalent. Suppose 80 students are willing to participate in the experiment above and you randomly assign 40 students to take the role of Antonie and 40 students to take the role of Hermine. If you apply random assignment both groups are about equal on all aspects. In both groups the proportion of females will not differ significantly, nor will the proportion of first year students, or the mean age. You can think of anything and it will not differ significantly between the groups. As a consequence the only difference between the two groups will be your manipulation of the independent variable, in our example the possibility to send messages or not. Thus, if you find differences in whether Antonie trusts or not, you are sure that these differences can be ascribed to the variable you manipulated. 
The goal of experimental research is to understand the causes of people's behaviour. When we manipulate an independent variable, randomly assign participants to conditions, and hold conditions constant, we are in a position to state that the independent variable causes any differences in the dependent variable. When we can confidently make this causal inference, we say that an experiment has internal validity.
Experimental designs are the most powerful designs for identifying cause-and-effect relationships (causal inferences) between variables. Thus, if your research question seeks to identify the causes of a relationship between variables, you should use an experimental design.
Why do we use research designs other than experiments?
For establishing sound support for causality, experiments are superior. But what are the limits of experiments? By and large there are two main limitations, namely (i) differences between the experimental and control group and (ii) experiments are artificial.
Often, individuals participate in only one of the conditions. This is called "independent groups design." In our hypothetical experiment, one group of participants would read the apology-present scenario, and a separate group of participants would read the no-apology scenario. We would calculate the mean (average) revenge rating for participants in the apology group and the mean revenge rating for participants in the no-apology group. Suppose the mean revenge rating for the no-apology group is 8.0 on the 10-point scale, and the mean revenge rating for the apology group is 4.0. We would conclude that an apology, compared to no apology, causes people to have less desire for revenge. This would indicate than an apology helps. An alternative explanation for the outcome (i.e. mean revenge ratings of 4.0 and 8.0) is, however, that the people in the two groups differed in terms of whether they are naturally more vengeful or forgiving. That is, the mean revenge ratings might differ because different people participated in the groups of the experiment, not because of the presence or absence of an apology.
The solution to this potential problem, though, is random assignment. Random assignment creates equivalent groups of participants, on average, before participants read the scenarios. Neither group is more vengeful or forgiving; nor do the groups differ, on average, in terms of any other potentially important characteristics. Therefore, we can rule out the alternative explanation that differences in revenge might be due to characteristics of the people who participated in each group. It should, however, be noted that random assignment only creates equal groups if the groups are sufficiently large. How large they need to be depends on the variation expected.
The second limitation of experiments is that they are artificial, because they reduce reality. In the trust game above, communication was one-sided and limited to a few pre-set sentences. Conditions that are rather strict compared to real life situations in which communication is typically two sided and each person can choose from millions of sentences.
What research design should I choose if I want to understand the causes of behaviour or create change in the "real world"?
We've seen that control is an essential aspect of experimental research designs. Sometimes, however, researchers cannot control all aspects of a situation, for example, when they conduct research in the "real world" rather than a lab. When researchers seek to control some aspects of an experimental situation, but cannot control all important aspects, they may conduct a quasi-experiment. Quasi means "almost"; therefore, quasi-experiments are "almost-experiments."
How do quasi-experiments differ from "true" experiments?
When researchers use a quasi-experimental design they seek to compare the effects of a treatment condition to a control condition in which the treatment is not present-just like in a "true" experiment. However, in quasi-experiments, researchers often are unable to assign participants randomly to the conditions. In addition, the researcher may not be able to isolate the effects of the independent variable by holding conditions constant. Thus, participants' behaviour (as measured by the dependent variable) may be affected by factors other than the independent variable.
Although quasi-experiments provide some information about variables, the cause-and-effect relationship (causal inference) may not be clear. The benefit of quasi-experimental designs, however, is that they provide information about variables in the real world. Often researchers conduct quasi-experiments with the goal of creating change. Psychologists have a social responsibility to apply what they know to improve people's lives; quasi-experiments help psychologists to meet this goal.
How do I conduct a quasi-experiment?
An essential feature of an experiment is that the researcher compares at least two conditions. One group receives a "treatment," and the other does not. In quasi-experimental designs, rather than randomly assigning individual participants to treatment and control conditions, we might assign an entire group to receive a treatment and withhold the treatment from another group.
For example, we might test the hypothesis that students who are allowed to choose the type of assignments they complete in a course perform better than students who are not given a choice. The independent variable is whether students are allowed choice. The dependent variable could be their final grade for the course.
You may see that it wouldn't be fair to allow some students in a class to choose their assignments and give other students in the class no choice. Therefore, we might manipulate the independent variable using two different sections of the same course. That is, students in one section of the course would be allowed to make choices and students in another section would not make choices. We would hold constant that students have to do the same number of assignments.
Although this experiment includes an independent variable (choice) and a dependent variable (grade), we have no control over many aspects of this experiment. Most importantly, students in the two sections are likely to be different. Suppose one section meets at 8:00 a.m. and another section meets at 2:00 p.m. Students who enrol in an 8:00 class are likely to be different from students who select a 2:00 class. In addition, class discussions may differ during the academic term, and the instructor may cover slightly different material. All of these potential variables may influence the outcome - students’ final grades in the course.
Quasi-experiments provide some information about variables, but the cause-and-effect relationship between choosing assignments and grades may not be clear at the end of the study. Suppose students who are allowed to choose their assignments earn higher grades than students who are not allowed a choice. Can we confidently say that our independent variable, assignment choice, caused this difference in grades? Researchers who conduct quasi-experiments often face difficult decisions about whether other variables, such as time of day or material covered in the class, could have caused the different grade outcomes.
Thus, if in your research question you seek to examine the causal effect of an independent variable on a dependent variable, but you cannot control other important variables in the research, you should use a quasi-experimental design.

What research design should I choose if I want to understand populations?
Experimental research enables us to see whether the relationship between two variables is causal. Taken the commitment – trust experiment above, the experiment would have enabled us to see whether giving commitments increases the chance that the other person trusts. If participants in the role of Antonie choose more often to trust Hermine, if Hermine places a commitment before, we could conclude that commitments lead to more trust. However, the experiment would not allow us to say anything about what percentage of people would honour trust if they receive a commitment.
The problem is that the participants of an experiment are often not representative for the whole population. For example, students are typical participants in experimental research, because you can find them easily at a university, they have time to participate, a moderate monetary compensation is sufficient and they are usually open to scientific research and motivated to participate. Students are, however, not a good representation of the whole general population. They differ substantially on range of characteristics, such as age (younger than average), education (higher than average), social background (higher than average), available income (lower than average), gender distribution (recently more women) etc. Thus, if in our experiment 65 % of the participants are likely to honour trust if the other places a commitment, that percentage can be very different in the whole population.
How can I learn something about a whole population?
The most common method used to collect data on populations is survey employing probability or random sampling methods. Note random assignment used in experiments and random sampling used in survey research are different. Random assignment refers to you, as a researcher, randomly assigning participants to different groups. Random sampling refers to the fact that you, as a researcher, randomly choose the participants in your survey from a population list. The reason for employing random sampling is that you need to ensure that your sample is not different from the whole population, because then findings in the sample can be generalized to the whole population. Technically speaking random sampling and surveys offer a high external validity of the results.
Can findings from surveys always be generalized to the whole population?
As mentioned above, random sampling is necessary to obtain a representative sample. If the sample is not random, it is likely that the respondents in the sample differ from the general population. Even if a researcher selects a random sample, the sample might not be representative due to non-response errors.
For our research on self-employment in South-Limburg, we approached potential respondents based on the information available at the local chamber of commerce. In this project, we experienced several forms of non-response.
First, people were unreachable because the known phone number was not working anymore and letters were returned as the address was unknown. The main reason for this kind of non-response is problems with the information in the sampling frame. Either some addresses and phone numbers were incorrect or they were out of date. For example, if a company goes out of business or is deregistered it often takes months before the chamber of commerce updates its data bases with this information.
Second, people were unreachable because phone calls on different days and at different times of the day remained unanswered. Thus, the phone number still exists, but nobody or an answering machine answers the phone. The same applies if you try to visit people at home, but each time you ring the door nobody is at home.
Third, people were reachable, but refused to participate, because they had no time, they are not interested in the survey, etc. This type of non-response is called refusal.
The main problem with non-response is that those who do not respond differ from those who respond. Regarding the wrong addresses, it could be that wrong addresses are mainly caused by businesses that have been deregistered. Consequently the responding businesses are on average more successful than the non-responding ones. Regarding the issue that no-one answers the phone is again an indication that the business is either not operating anymore or that it has very limited operations, e.g. just a few hours a week. Finally, those who refuse to cooperate will also differ from those who participate. It could be that business owners that are less successful are less likely to talk about their business. Likewise, very successful business owners might also be reluctant to provide information about their business etc.

Which research design should I use if I want to understand and treat the behaviour of one person?
< Single-case research design
In observational/correlational experiments, and quasi-experimental designs, researchers focus on groups of participants. We use these designs to identify "general laws" of behaviour and describe how people behave and think on average. As the name implies, the researcher who uses a single-case design focuses on a particular individual or organization.
How do I conduct a single-case research design?
< Observe behaviour during baseline and treatment
Similar to quasi-experimental designs, single-case researchers frequently cannot control all the important variables in the research. For example, suppose you want to investigate how entrepreneurs recognize business opportunities and contact a very successful businessman, who has started a series of companies in the last decade. Your research question might be what distinguishes this entrepreneur from other people or even entrepreneurs and why he recognizes certain business opportunities earlier than others. Now suppose that you observe that this entrepreneur is very open-minded, curious about new things etc.
Can you claim that open minds und curiosity causes a quick recognition of business ideas?
< Other explanations for improvement exist
Although it seems easy to determine this relation, many alternative explanations can frustrate the result. For example, the entrepreneur in question might come from a wealthy family that has given her the funds to exploit the first opportunities. Or, you also observe that she is rather risk-taking. Any of these other "factors" rather than the open-mindedness causes opportunity recognition. Single-case research designs require that the researcher investigates such alternative explanations.
How to conduct a case study?
< Multiple sources of evidence >
While experiments and surveys are highly structured, case studies are much less so and that can be turned into an advantage if one uses multiple sources of evidence. That means a sound case study should rely on more than just a few unstructured interviews, but should combine the information obtained through interviews with information from other sources. First, in a case study it is common to interview more than one person regarding a specific event, such as the start of a business. In the survey, we only interviewed one of the business starters; in a case study that would not be sufficient and we would extend the interviewees to other persons next to the business founder, such as the spouse of the business founder, business partners, employees working for her/him etc. This would allow us to look at that specific phenomenon (the starting of business) from multiple perspectives. Interviewing more than one person also allows us to cross validate information we have obtained in each interview.
Multiple sources of evidence do not only refer to multiple interviewees, but also to other sources such as written documentation, observations etc. Thus, in the case of a business start, we would try to collect a lot of additional information, such as clippings from the local newspapers on that new business, advertising and promotion material the new company developed and we would try to understand the atmosphere during our company visits through observational methods etc. The logic behind using multiple sources of evidence is that four eyes see more than two. If an interpretation of a phenomenon fits with the information obtained in interviews and how the company presents itself in their promotional brochures etc., we can be much more confident that the interpretation is probable.
STEP 5: Choose a research design
Think about the following questions before you decide which design is most appropriate for your research.
1.         Is the research question crystal clear and are the concepts used well understood?
If no, an explorative research design based on a case study is most appropriate. If yes, either survey or experiment is useful.
2.         How well is the research question covered in the literature?
If no, an explorative research design based on a case study is usually most appropriate, especially if you already answered no to the previous question.
3.         Are you more interested in the causality of an effect or how strong that effect is in real life conditions, i.e. are you interested in the population?
If you are interested in the causality issue, experiments are more appropriate as long as you do not want to make statements regarding whole populations. If your interest is more in generalizations to the whole population, you should employ a survey design.